The boy died after being critically injured in the crash at Auburn South Primary School at Hawthorn East about 2.30pm on Tuesday, Victoria Police confirmed.
Also seriously injured were two girls aged 11-years-old, one 10-year-old girl and a 10-year-old boy who were all taken to hospital.
The 40-year-old driver of the car has been arrested and will be interviewed by police.
Inspector Craig McEvoy said after driving to the school to collect her child, the woman had performed a U-turn and collided with the fence.
The car struck a table where the five children were seated.
"It appears it is a tragic accident," Insp McEvoy told reporters at the scene.
The woman and her child were not injured.
The female driver of the vehicle has been arrested and will be interviewed by police. (Joel Carrett/AAP PHOTOS)
The leafy street remains closed off as officers come and go from the area of impact and Public Order Response crews are also at the scene.
Parent Lucy told radio station 3AW her son was friends with some of the grade five students and possibly a prep student hit by the car during recess.
"We got a text message from the school just saying that Burgess Street closed and then we... actually drove past, and I just saw one police car," she said.
"Then we just started hearing more and more police cars go down so I just raced down there."
Police say the car was travelling along Burgess Street when it left the road and crashed. (Supplied/AAP PHOTOS)
Lucy said she arrived at the scene to find parents hugging their children and each other.
"It's just every person's worst nightmare and I just hope the children are okay," she said.
Resident Don Owen told reporters the road could be busy with traffic, particularly at the start and finish of school, and cars sometimes sped by.
The Department of Education said in a statement it was working closely with staff, students and parents of the school to support them.
16°C