Three members of the security forces and a civilian were killed on Friday and 18 members of the security forces wounded, according to government officials.
Two suicide bombers were killed and at least one blew himself up after entering the police building, officials said.
Pakistani Taliban in a brief statement claimed responsibility.
Government adviser Murtaza Wahab said police and paramilitary forces in a joint operation had cleared the police building within three hours of the attack late Friday.
President Arif Alvi condemned the attack in Karachi, while prime minister Shahbaz Sharif paid tribute to security forces for the successful operation.
Earlier, TV footage showed officers surrounding the city's central police station as residents reported the sound of explosions and gunfire.
Pakistan's interior minister Rana Sanaullah Khan said some of the militants threw hand grenades as they tried to force their way into the police headquarters.
Pakistan has witnessed a surge in militant attacks since November when Pakistani Taliban ended a months-long ceasefire with the government.
Pakistan's outlawed Tehrik-e-Taliban is a separate group but are allies of the Taliban in Afghanistan, who seized power there more than a year ago as US and NATO troops withdrew.
The Taliban takeover in Afghanistan emboldened Pakistani militants, whose top leaders and fighters are hiding across the border.
The assault on Karachi's police headquarters comes two weeks after a suicide bomber disguised as a policeman killed 101 people at a mosque in the northwestern city of Peshawar.
Authorities blamed the TTP for orchestrating last month's mosque bombing and Sarbakaf Mohmand, a TTP commander, claimed responsibility for it.
