An Instagram live stream by a Kazakh blogger showed a fire blazing in the mayor's office in the city of Almaty, with gunshots audible nearby.
Videos posted online also showed the nearby prosecutor's office burning.
Protesters appeared to have broken through security forces' cordons even though the latter deployed stun grenades whose explosions could be heard throughout the city centre.
Kazakhstan is a tightly controlled former Soviet republic which cultivates an image of political stability, helping it attract hundreds of billions of dollars of foreign investment in its oil and metals industries.
President Kassym-Jomart Tokayev accepted the government's resignation on Wednesday, a day after police used tear gas and stun grenades to drive hundreds of protesters out of the main square in Almaty.
On Wednesday a Reuters correspondent saw thousands of protesters pressing ahead towards Almaty city centre, some of them on a large truck, after security forces failed to disperse them with tear gas and flashbang grenades.
Atameken, Kazakhstan's business lobby group, said its members were reporting cases of attacks on banks, stores and restaurants.
The city health department said 190 people had sought medical help, including 137 police. City authorities urged residents to stay home.
The interior ministry said that government buildings were also attacked in the southern cities of Shymkent and Taraz overnight, with 95 police wounded in clashes. Police have detained more than 200 people.
A video posted online showed police using a water cannon and stun grenades against protesters in front of the mayor's office in Aktobe, the capital of another western province
The protests began after the government lifted price controls on liquefied petroleum gas at the start of the year. Many Kazakhs have converted their cars to run on LPG because of its low cost.
The government said the regulated price was causing losses for producers and needed to be liberalised. The president said it had botched the move.
Speaking to acting cabinet members, Tokayev ordered them and provincial governors to reinstate price controls on LPG, and broaden them to petrol, diesel and other "socially important" consumer goods.
He also ordered the government to develop a personal bankruptcy law and consider freezing utility prices and subsidising rent payments for poor families.
He said the situation was improving in protest-hit cities and towns, including Almaty and the surrounding province, where the authorities declared a state of emergency.
Tokayev has said that domestic and foreign provocateurs were behind the violence.
Almaty mayor Bakytzhan Sagintayev said the situation in the city was under control and security forces were detaining "provocateurs and extremists".
