Labor leader Anthony Albanese has secured cross-faction support for his bid to oust rogue union boss John Setka.

Mr Albanese, a left-winger, has moved to expel right-aligned Mr Setka from the party following accusations he told colleagues anti-family violence campaigner Rosie Batty’s advocacy has left men with fewer rights.

Mr Setka has rejected the allegations.

Labor’s shadow treasurer Jim Chalmers, from the right faction of the party and aligned with the Australian Workers’ Union, on Wednesday broke silence on the issue to support Mr Albanese “100 per cent”.

“I think that’s a very good decision – I think the community will support that decision,” he told reporters in Mackay.

However, Mr Chalmers says Mr Setka’s role at CFMMEU is a matter for union members.

The national tier of the construction union is yet to say whether it stands with their Victorian branch secretary, with five other unions from the state throwing their support behind him.

Victoria’s CFMMEU has vowed to end all financial support to the ALP if Mr Setka is expelled from the party and to cut ties with unions that have “attacked the branch”.

The union boss has threatened legal action and a long battle if Mr Albanese moves against him.

But the Labor leader is standing his ground and says he doesn’t respond to threats.

“The fact that threats are made perhaps reinforces the fact there’s a problem, if people think that’s the way to engage,” Mr Albanese told 2GB radio.

“I think it says more about the people making the threats than those receiving them.”

Victoria’s manufacturing, electrical trades, rail and transport, plumbers and firefighters unions declared their support for Mr Setka on Tuesday, saying he should stay in his role unless his members decide otherwise.

The unions are calling for the issue and decisions on Mr Setka’s position to be left to members.

Mr Setka faces court on June 26 when he is expected to plead guilty to two criminal charges, including one of using a carriage service to harass a woman.

The ETU’s Victorian state secretary Troy Gray is threatening to “audit” Labor MPs for any history of domestic violence for moving against Mr Setka.

“If people are going to try and do a political hatchet job on a union official then we will be looking to see if people have skeletons in their closet,” he told The Australian.

“If sending a text message is the standard for resignation then there definitely will be some nervous politicians out there because we will audit them.”

The Victorian branch of the CFMMEU wants the national arm to publicly support Mr Setka and to forensically investigate the “cowardly manufactured leaks” from the meeting where the comments about Ms Batty were allegedly made.

ACTU secretary Sally McManus is demanding Mr Setka resign with the leaders of the three largest unions – representing mainly female workforces – following suit.

The AWU backs them but wants proper processes followed.

Source: AAP