"Does it wake me up at night?" Dr Forrest said in response to a question at an Australian Financial Review business summit in Sydney on Wednesday.
"No, it keeps me awake all night.
"I've sat in some of those discussions, and I just thank God that those discussions are taking place."
Holding up a glass bottle of water, Dr Forrest asked whether it was a projectile or a vessel that contained hydrogen and oxygen.
"If the software can't agree, just in the lexicon, you can imagine what can go wrong."
His company, mining and clean energy giant Fortescue, makes extensive use of AI, including for autonomous mining equipment. It has over 200 autonomous haul trucks currently operating across the Pilbara.
It took Fortescue two years, from 2019 to 2021, to get that lexicon right, Dr Forrest said.
"So I can see how incredibly dangerous AI is," he said, pointing to the US-Israel war on Iran and the Pentagon's reported use of Anthropic's Claude AI model.
"That first strike we saw with Iran, that pre-emptive strike mentality to get an advantage, even split seconds, becomes really tempting with AI."
In addition, Dr Forrest said he was really worried by AI's ability to replace the social networks that build humanity.
He recently spoke with a global champion of AI about some of the issues Fortescue had experienced with the technology.
"I'm saying, 'hey, I've seen this in my company, you must have seen it everywhere'," he said.
The unnamed executive admitted he had, but was afraid to speak out.
But Dr Forrest also spoke positively about artificial intelligence, saying he had watched as its ability to help Fortescue balance its "incredibly complex" green energy grid in the Pilbara had dramatically accelerated in the past six months.
When a cloud floats over a solar farm, or when there's a drop in the wind, that creates instant changes across Fortescue's 500km grid, he explained.
"AI has got to the point where we can now manage that," meaning Fortescue should be able to complete the green grid before its 2030 target date, he said.
Fortescue also used AI when it was about to engage in a six-year process to test making green steel.
Executives programmed questions into an artificial intelligence on Friday afternoon and by Monday had received 2650 simulations with five different recommendations.
Dr Forrest wasn't sure if Fortescue was going to implement all of them but seemed incredibly impressed.
"I'm seeing the horsepower of AI basically rise vertically and we should all be very alive to that, because it's like everything in nature, there's a positive and a negative," he said.
"AI is going be incredibly powerful, but also very dangerous."
