Scott Morrison has called for people attending future anti-racism protests to be arrested and charged, warning rallies are blocking eased coronavirus restrictions.
The prime minister ramped up his opposition to Black Lives Matter demonstrations during the pandemic, as one protester who attended last week's Melbourne rally tested positive.
Mr Morrison said the "double standard" displayed by protesters had offended many Australians.
The potential health fallout is impeding decisions around interstate travel, funeral numbers and places of worship.
"It just puts a massive spanner in the works and that's why it's so frustrating," he told 2GB radio on Thursday.
"They have put the whole track back to recovery at risk and certainly any further action on this front would be absolutely unacceptable."
The Australian Health Protection Principal Committee firmed up its health advice warning against mass gatherings at a meeting on Thursday.
Deputy Chief Medical Officer Michael Kidd said the large crowds struggled to maintain social distancing.
"If we had had a person or a number of people with COVID-19 amongst those crowds, then there would have been the risk of significant transmission," he told the ABC.
The prime minister said protesters should be charged if they attend further Black Lives Matter rallies.
"It's a free country and we have our liberty, but the price of liberty is we respect our fellow Australians," he told 3AW radio.
"Turning up to a rally this weekend would show great disrespect to your neighbours."
It could be weeks before authorities determine whether the infected man, who wore a mask to the rally, has spread the disease to others.
Opposition Leader Anthony Albanese told people considering joining further protests to listen to the medical advice.
"We need to not be complacent about coming out of this crisis. We're not through it yet. And that's why we need to be consistent," he told reporters.
Mr Morrison has also declared JobKeeper payments will remain until the end of September for everyone except childcare workers.
"I already said that so I don't know what that other speculation is about," he said.
Treasurer Josh Frydenberg has piled pressure on the states to reopen their borders, declaring "closed borders cost jobs".
"The states should go the whole hog. They should lift those closures immediately," he told reporters in Canberra.
The prime minister - who will meet with premiers for national cabinet on Friday - wants them to nominate a July date, in line with the target for the third stage of eased restrictions.
But WA Premier Mark McGowan said the infected protester vindicated his state keeping its borders closed.
"I think what's occurred in Victoria overnight shows our approach has been correct," he told reporters in Perth.
The national cabinet will also look at infrastructure and red tape issues holding up projects.
As well, rules around indoor gatherings and how airlines operate will be on the table.
Australia has 422 active cases of the virus which has claimed 102 lives.
Australian Associated Press