He was 73.
The Kremlin said Ivanov died on Friday without providing the cause of death or giving other details.
Putin sent his condolences to Ivanov's family.
Ivanov's death, incongruously, was first announced by a basketball organisation - the VTB United League, where Ivanov was honorary president.
It was later confirmed by the Kremlin.
In 2001, Putin appointed Ivanov, a fellow KGB veteran, to serve as his defence minister, the job he held until 2007, overseeing the second war in Chechnya that crushed the region's separatist bid.
When Putin decided to step down due to term limits and shift into the prime minister's seat in 2008, Ivanov was widely viewed as his most likely successor.
However, Putin picked another longtime associate, Dmitry Medvedev, to serve as his placeholder until reclaiming the presidency in 2012.
Some observers argued that Putin ditched Ivanov's candidacy because he saw Ivanov as overly ambitious and feared that he could try to hold onto the presidential seat.
A fluent English-speaker, Ivanov was a combative figure who regularly jousted with journalists at the Munich Security Conference and cast himself as a pragmatist seeking to move beyond Cold War divisions.
At the same time, he consistently warned that Russia's security interests were being undermined, particularly by US missile defence plans and the erosion of arms control agreements.
Ivanov remained at Putin's side as deputy prime minister from 2007 to 2011, and then served as the Kremlin chief of staff from 2011 to 2016.
In 2016, Ivanov was named a presidential envoy for environment protection and transport, a job that carried no political weight and was widely seen as a honorary retirement.
He stepped down earlier this year.
Along with other Russian top officials, Ivanov has been targeted by United States and European Union sanctions in response to Russia's military action in Ukraine.
Ivanov had framed the NATO military alliance's expansion as a strategic concern for Russia and repeatedly argued that security in Europe should be built on "mutual respect for the concerns and interests of all sides".
Years later, Putin would cast the invasion of Ukraine in 2022 as a forced response to the encroachment of NATO into Russia's sphere of influence, a characterisation rejected by Ukraine and its allies as a false justification for war.
with Reuters
7°C