The prime minister on Friday wrapped up a two-day visit to the region, meeting with Torres Strait Islander women leaders.
"We do need to close the gap in education, in health, in life expectancy, in housing," Mr Albanese said.
"But the way to do that is to make sure that firstly you give people that sense of ownership, that you consult them, it's good manners to do so.
"But also recognising First Nations people in our constitution, which is after all our national birth certificate, will be an important step forward for the entire nation."
Labor pledged at the election to put the Indigenous Voice to a referendum by the end of the term.
Mr Albanese, Indigenous Australians Minister Linda Burney and Queensland senator Nita Green also met with the Torres Strait Regional Authority on Thursday.
Ms Burney said Torres Strait Islanders had "expressed to both the prime minister and myself the desire to be involved, the desire to be consulted".
"They have made it very clear that they'd like to be involved in the discussions and the make-up of the voice," she told ABC Radio.
Employment, housing and governance of the region were also raised, Ms Burney.
She said it was important to understand Torres Strait Islanders held concerns specific to that region.
"It is very distinct, it's absolutely beautiful. They see themselves, as was described to us, as a minority within a minority ... Aboriginal people on the mainland (and) people in the Torres Strait."
The prime minister said he had received unanimous support in his meetings for the Voice to parliament and constitutional recognition for Aboriginal and Torres Strait Islander peoples.
The draft question to be put to the Australian people is: "Do you support an alteration to the Constitution that establishes an Aboriginal and Torres Strait Islander Voice?"
A new provision in the constitution would have three sentences: "There shall be a body, to be called the Aboriginal and Torres Strait Islander Voice.
"The Aboriginal and Torres Strait Islander Voice may make representations to Parliament and the Executive government on matters relating to Aboriginal and Torres Strait Islander Peoples.
"The Parliament shall, subject to this Constitution, have power to make laws with respect to the composition, functions, powers and procedures of the Aboriginal and Torres Strait Islander Voice."
State and territory ministers have agreed to back the Labor government's work towards enshrining the Voice.
A successful referendum needs an overall majority of votes as well as majorities in a majority of states.
There have been 44 proposals for constitutional change with only eight changes passing.
Ms Burney said there was a clear way to address concerns raised by former prime minister John Howard that nothing that gave "special status" to a group of people would work.
"I'm going to stick to the facts," she told ABC Radio.
"John Howard's record in reconciliation is not something he should crow about."
12°C