The men, who are yet to be formally identified, were found unresponsive in the water off Frankston in Melbourne's south east at about 5pm on Wednesday as the state was lashed by intense wind gusts.
They were winched back to shore using a police helicopter but could not be revived, Victoria Police said in a statement.
Frankston City Mayor Kris Bolam said the men were swept from the Frankston Pier during extreme wind conditions.
"This devastating event is a sharp reminder of the power of nature and the real risk posed by severe weather," Mr Bolam said in a statement.
"I ask everyone to stay safe, remain vigilant, and if in doubt, stay off the water and away from exposed coastal structures until conditions improve."
There were about 1500 requests for assistance to Victoria's State Emergency Service on Wednesday, with the majority related to trees down and building damage.
That included tin roofs ripped from houses and trees over roads, SES duty officer Grant Machell said.
"It was actually one of our most widespread wind events that we've had for a long time," Mr Machell told AAP.
He said coastal areas copped the brunt of the wild winds but it was an unusual weather event as winds came from both the northwest and southwest.
"We were saying that wind gusts could be hitting around 110 kilometres and that's certainly enough to blow over people if they're not prepared for it," he said.
"We were encouraging people to stay inside, if necessary."
Wild weather hit much of the the nation's east coast on Wednesday, with scorching conditions in NSW easing by Thursday.
The mercury hit 37 degrees in Sydney and 39 in Bankstown Airport.
"Based on preliminary data, it does appear that records were set across much of the Sydney area for the highest temperatures recorded in October," Bureau of Meteorology senior meteorologist Angus Hines said.
Heatwave warnings remain in place for Queensland and the Northern Territory.
13°C