The new pontiff arrived for the unannounced visit in the passenger seat of a Volkswagen vehicle and was greeted to cheers from crowds gathered outside the Shrine of Our Lady of Good Counsel.
Leo, the former US cardinal Robert Prevost, was elected Pope on May 8.
He is a member of the Augustinian religious order, which runs the shrine in Genazzano.
Leo shook hands and offered blessings to a few people in the crowd before entering the shrine.
At the end of the visit, the Pope told those in the shrine that he wanted to come to pray for guidance in the first days of his papacy, according to a Vatican statement.
The late Pope Francis, who died on April 21, made surprise visits to Catholic sites near Rome quite frequently.
He was particularly devoted to Rome's Basilica of St Mary Major, another Marian shrine, and chose to be buried there.
Pope Leo signalled earlier on Saturday he would continue with the vision and reforms of Pope Francis, telling cardinals the late pontiff left a "precious legacy" that must carry on.
In his first meeting with all the cardinals since his election as pontiff on May 8, Leo also asked the senior clerics to renew their commitment to major Church reforms enacted by the landmark Second Vatican Council in the 1960s.
Leo said Francis broadly had a vision of opening the staid 1.4-billion-member Church to the modern world, had left an "example of complete dedication to service".
"Let us take up this precious legacy and continue on the journey," the Pope told the cardinals in a speech.
The pontiff also asked the clerics to "renew together our complete commitment" to the reforms enacted by the Council, which included celebrating the Mass in local languages rather than Latin and pursuing dialogue with other religions.
9°C