The round-of-32 match ended 1-1 after extra time on Monday (Tuesday AEST). Paraguay had gone in front when Julio Enciso headed home late in the first half, before Kai Havertz equalised in the 52nd minute for the four-time world champions.
Paraguay will next face the winner of France and Sweden on July 4 in Philadelphia. A win in that match would land them back in Foxborough for a quarter-final match on July 9.
"We had to analyze every player, every detail. Thanks to that I was able to only miss two penalties," Gill said.
"This is for all the people of Paraguay."
Germany had won six of seven penalty shootouts in major tournaments, including six straight since losing to Czechoslovakia in the 1976 European Championship final.
In the only previous World Cup match between the teams, Germany beat Paraguay 1-0 in the round of 16 at the 2002 tournament. Nearly a quarter-century later, Paraguay have their revenge.
La Albirroja had appeared in five previous knockout games, but failed to score in each. They advanced only once in those previous occasions, winning on penalties against Japan in the last 16 at the 2010 tournament in South Africa. They fell that year to eventual champions Spain in the quarters.
It was Germany's first knockout game since the 2014 final in Brazil when the Europeans beat Argentina 1-0 to capture their fourth World Cup title.
"We had very big plans for this World Cup," Havertz said.
"It's very difficult to disappoint again. It was difficult to create chances and keep the pace."
Julian Nagelsmann's side appeared to take a 2-1 lead in the 102nd minute when Jonathan Tah headed in a corner kick by Nathaniel Brown, but a video review ruled that Waldemar Anton has pushed Gill to the ground and the goal was disallowed.
"What I want to highlight from our team is how united we are," Canale said.
"Today was a game we really needed to show our true colours."
Paraguay are set to host one game in the six-nation 2030 edition and before that can celebrate a famous victory over a Germany side who suffered a third early World Cup elimination in a row.
With DPA.
