Scott Morrison is standing firm against growing calls to enshrine an indigenous "voice to parliament" in the constitution.
The prime minister was on Friday asked if he was reconsidering the idea, after a former High Court chief justice came out in favour of the proposal.
"No," Mr Morrison told 2GB Radio.
Former chief justice Murray Gleeson says a new "voice to parliament" for indigenous Australians would be a positive step.
Mr Gleeson on Thursday night delivered a 40-minute speech to a group of high-profile judges and lawyers in Sydney in which he neutralised key arguments made against the federal proposal.
Some politicians, including One Nation leader Pauline Hanson, have argued having such a body amounts to giving Aboriginal and Torres Strait Islanders special treatment.
But Mr Gleeson said section 51 of the constitution already affords special status to indigenous people under the "race power".
He also rejected an argument put forward by former prime minister Malcolm Turnbull that it would stymie the powers of the federal parliament and be an unnecessary "third chamber".
"It is unlikely that parliament will propose a change to the constitution in aid of indigenous recognition if the effect of the change will be to curtail its own legislative power," Mr Gleeson said.
"That appears to have been well understood by the supporters of the voice. What is proposed is a voice to parliament, not a voice in parliament."
He said the "structure, composition, and functions" of the body would be set by parliament and able to be changed by it.
As for the constitutional aspect, which would require a referendum being carried: "What would appear in the constitution would be the minimum requirements necessary to guarantee its continued existence and its essential characteristics."
"A body that has the capacity to speak to the parliament on behalf of indigenous people should be of advantage to parliament and, through it, the nation," Mr Gleeson said.
The Morrison government is working with Labor on a possible referendum by the end of this term to recognise indigenous people in the constitution.
The prime minister does not support a constitutionally-enshrined indigenous advisory body but favours some form of recognition of Aboriginal and Torres Strait Islanders in the constitution.
However, he's open to the idea of a national body - backed by legislation - bringing together key indigenous groups.