Workmates of a labourer who fell to his death at a central Sydney worksite have struck a deal with contractor Lend Lease in an effort to raise $100,000 for the man's family.

The agreement for Lend Lease to match worker contributions dollar for dollar comes amid union concerns about the deceased man’s worker’s compensation cover.

Emergency services were called to the Barangaroo work site on Thursday morning after reports that a man had fallen from scaffolding.

Colleagues and Ambulance NSW paramedics were unable to revive the 23-year-old, who died at the scene.  A police spokesperson said the death was not being treated as suspicious or work-related.

Mates in Construction, a firm running a suicide-prevention program at the project, is providing counselling, Lend Lease says.

The man had been working at the site for a few weeks as part of the Koori Job Ready Program.

About 500 workers attended a stop-work meeting at the development site on Friday morning where some emotions boiled over.

Lend Lease gave workers the option to stay at work or take the day off. Most decided on the latter.  One man fired off a torrent of abuse at journalists outside the site and was led away by police.

Lend Lease said in a statement that WorkCover had not issued any safety notices regarding the site.

In an email to employees, Lend Lease group CEO Steve McCann said the death was a timely reminder for people to look after each other.

He urged staff to ask colleagues, friends and family if they were all right or needed support.

Construction, Forestry, Mining and Energy Union (CFMEU) NSW Secretary Brian Parker said workers and Lend Lease decided to raise money to support the deceased man’s family.

“We think at the end, it could be in excess of $100,000,” he told AAP.

However, the union says employer contributions related to the man’s worker’s compensation cover had not been made.

Mr Parker claimed that the cover included a funeral benefit, which would not be paid out.  “It leaves [the family] considerably out of pocket,” he said.

There is a dispute over which contractor or sub-contractor is responsible for the dead worker’s compensation contributions.

CFMEU has also said Lend Lease was lacking in its supervision and mentoring of workers.  Lend Lease said it was disappointing anyone would use such a tragic incident to point-score.

It has also rejected union claims suggesting that it did not take safety seriously.