During their show on Saturday, the duo chanted "Death, death, to the IDF" in reference to the Israel Defence Forces, the formal name of the Israeli military.
"There is no excuse for this kind of appalling hate speech," Starmer said.
"I said that Kneecap should not be given a platform and that goes for any other performers making threats or inciting violence," he said.
"The BBC needs to explain how these scenes came to be broadcast."
Police said they were considering whether to launch an investigation but did not name Bob Vylan or Irish rap band Kneecap, who appeared on the same stage and also criticised Israel.
"Video evidence will be assessed by officers to determine whether any offences may have been committed that would require a criminal investigation," Avon and Somerset Police, in western England where the festival is held, said on X late on Saturday.
The festival organisers criticised the chanting by Bob Vylan, which comprises the guitarist-singer with the stage name Bobby Vylan and a drummer known as Bobbie Vylan.
"Their chants very much crossed a line and we are urgently reminding everyone involved in the production of the festival that there is no place at Glastonbury for anti-Semitism, hate speech or incitement to violence," it said on Sunday.
The Israeli Embassy in the UK condemned the "inflammatory and hateful rhetoric expressed" on stage at the festival.
Bob Vylan's band members did not immediately respond to a request for comment.
The BBC said some of the comments made during Bob Vylan's set were deeply offensive.
"During this live stream on iPlayer, which reflected what was happening on stage, a warning was issued on screen about the very strong and discriminatory language," a spokesperson said.
"We have no plans to make the performance available on demand."
Bob Vylan's show on the festival's West Holts stage took place just before controversial Irish rap trio Kneecap played to a huge crowd, leading chants against Starmer and also taking aim at Israel.
During the show, frontman Liam Óg Ó hAnnaidh accused Israel of committing war crimes, saying: "There's no hiding it."
Known by the stage name Mo Chara, he was charged with a terrorism offence last month for allegedly displaying a flag in support of Lebanese militant group Hezbollah at a concert.
He has denied the charge.
Starmer had said it was "not appropriate" for Kneecap to play at the festival.
A senior member of his government, Health Secretary Wes Streeting, earlier on Sunday criticised the chants by Bob Vylan but added that he was also appalled by violence committed by Israeli settlers in the occupied West Bank.
"I'd also say to the Israeli embassy, get your own house in order in terms of the conduct of your own citizens and the settlers in the West Bank," Streeting told Sky News.
"I wish they'd take the violence of their own citizens towards Palestinians more seriously."
with PA
