Green scored the second-fastest century in Australian ODI history after Head and Marsh put on a 250-run opening stand, to set up a huge score of 2-431.
The home side then skittled the Proteas for 155 in 24.5 overs.
After only taking one wicket in his previous four ODIs, spinning allrounder Cooper Connolly starred with the ball by bagging 5-22 - the first five-wicket haul of his professional career at any level or format.
Australia's score fell three runs short of their largest ODI total of 4-434, which came in the classic 2006 game against South Africa in Johannesburg when the Proteas incredibly chased down their target.
Green's first ODI ton came from just 47 balls, which sits only behind Glenn Maxwell's 40-ball hundred in the 2023 World Cup against the Netherlands among Australian centuries.
The 26-year-old finished unbeaten on 118 from 55 balls, pairing with Alex Carey (50no) to capitalise on a high-scoring start.
His late heroics overshadowed openers Head and Marsh putting on the seventh-highest partnership in Australia's ODI history.
Head (142 from 103 balls) made his seventh ODI century - and first in almost a year - in a powerful reminder of why he is one of the most destructive batsmen in world cricket.
Marsh (100) later joined him, posting his fourth ODI ton, before being dismissed on the next ball he faced.
The pair's 250-run partnership was the fifth-highest opening stand for Australia, with Head and former star David Warner responsible for the best two.
Warner was involved in all of the five biggest Aussie partnerships in 50-over cricket.
Marsh's decision to bat first after winning the toss for the first time in his Australian captaincy career seriously paid off.
After electing to field the previous 21 times he has won the toss, Marsh decided to bat.
Australia had only pride to play for in the regional Queensland city after being thrashed in the opening two games.
"A pretty crazy day," Marsh said.
"A shame it was a little bit too late, but full credit to South Africa.
"They played outstanding in the first two games, and were too good for us."
Struggling allrounder Aaron Hardie made way for his West Australian teammate Connolly, while star quick Josh Hazlewood was rested ahead of a busy summer.
Sean Abbott was added to the XI as the replacement for Hazlewood, taking the key wickets of star opener Aiden Markram (two) and captain Temba Bavuma (19).
Out-of-form No.3 Marnus Labuschagne held his spot, despite scoring just two runs in the first two games.
But he wasn't required to bat, with Australia promoting Green and Carey to keep the runs flowing.
Bavuma returned to the line-up after missing Friday's 84-run win as the Proteas manage his workload following a hamstring injury in June.
"We were under the pump from the first ball," Bavuma said.
"They made it look like a very good surface, which I think it was, but they were far better than us."
South Africa rested Matthew Breetzke, despite the middle-order batter top-scoring with 88 two days ago.
After claiming 5-42 in the second ODI, star quick Lungi Ngidi also sat out and he was sorely missed in the weakened bowling attack.
Allrounder Wiaan Mulder was hit all over the Great Barrier Reef Arena, going for 0-93 from just seven overs.
