But the UK government has resisted opposition calls to sanction Musk for the remarks.
Sir Keir Starmer denounced violence on the fringes of Saturday's 100,000 or more-strong 'Unite the Kingdom' demonstration in London organised by far-right campaigner Tommy Robinson.
Police said 26 officers were injured, four seriously, as a minority among the protesters tried to breach lines separating them from a smaller anti-racist counter-demonstration. There were 25 people arrested at the event and the Metropolitan Police said more arrests would follow.
Addressing the demonstration by video link, Musk called for the dissolution of parliament and an early election to remove Sir Keir's centre-left government. He told protesters "violence is coming to you" and "you either fight back or you die".
The prime minister's spokesman Dave Pares said he didn't think "the British public will have any truck with that kind of language".
"The UK is a fair, tolerant and decent country, so the last thing that British people want is dangerous and inflammatory language which threatens violence and intimidation on our streets," he said.
Ed Davey, leader of the Liberal Democrats, the third-largest party in Britain's parliament, urged Sir Keir, Conservative opposition leader Kemi Badenoch and Reform UK leader Nigel Farage to join him in condemning Musk's attempt "to sow discord and incite violence on our streets" and interfere with British democracy.
Davey urged the prime minister to block Tesla from getting government contracts, but Sir Keir's spokesman said the government had no plans to sanction Musk over his comments.
The prime minister wrote on X that peaceful protest "is core to our country's values. But we will not stand for assaults on police officers doing their job or for people feeling intimidated on our streets because of their background or the colour of their skin".
This is not the first time Musk, an erstwhile ally of President Donald Trump, has supported hard-right and far-right figures in Europe, including Robinson, a convicted fraudster and founder of the anti-Islam English Defense League whose real name is Stephen Yaxley-Lennon, and the Alternative for Germany party, or AfD.
Musk also is a critic of attempts by the UK and other European governments to clamp down on harmful online content, something he argues restricts free speech.
Saturday's march was billed as a demonstration in support of free speech, with many of the speeches by influencers and far-right politicians from across Europe aimed largely at the supposed perils of migration.
It followed growing political concern about unauthorised immigration, especially the arrival of migrants across the English Channel in small boats.
More than 30,000 people have made the dangerous crossing from France so far this year despite efforts by authorities in Britain, France and other countries to crack down on the people-smuggling gangs behind the trips.
The use of hotels to accommodate asylum-seekers has become a major political issued in Britain, sparking dozens of small but heated protests over the summer, some of which turned violent.
