A large section of a 15-metre deep pit in Burwood partially collapsed forcing 10 university students from a nearby block of townhouses to flee.
Police say it is the second time in a week that residents have been evacuated near the site.
Once a petrol station, it is being turned into a medical and childcare centre with a double-storey underground car park.
WorkSafe, CFMEU, Monash Council and Yarra Valley Water representatives are assessing the damage, which is believed to have been caused by heavy rains.
No further excavation work will happen until it is deemed safe.
CFMEU safety manager Gerry Ayers said safety measures to prevent a collapse were not in place in the pit.
"Something so deep should have some type of contingency plan to prevent the collapse that occurred here," Dr Ayers told said.
"Whether it's retaining walls, or whether they shotcrete it and put rock anchors in, there are a number of options they can choose to do.
"We certainly don't want another Swanston Street wall collapse."
A nearby resident, who did not want to be named, heard a bang about 10pm.
"I didn't know what it was until I turned on the news this morning," she said.
On Tuesday, a back wall of the backyard of one of the townhouses was brought down.
Sarah Humphrey, who works at the Sphere Cafe, next door to the site, said construction started in January.
"We had no idea it wasn't being done right, but some people had noticed there were no retaining walls," Ms Humphrey told said.
"No one has told us much since. I'm sure if we were at risk, we'd be told."
Local shops have had their sewerage closed off and are using portable toilets while authorities work out how to solve the problem and stop it from occurring again in the future.