A dataset search tool recently created by US publication The Atlantic reveals millions of creative works have been scraped from the internet to train the disruptive technology.
It includes a vast catalogue of work by Australian artists, with tunes by Kylie Minogue, Powderfinger, Nick Cave and Jimmy Barnes, and novels by Thomas Keneally and Peter Carey.
Dempsey had long suspected his music was being used by AI without his permission, and says he's found the entire catalogue of his longtime band Something For Kate as well as his solo tunes using the search tool.
"It's frustrating this is happening. Every negotiated agreement and contract I've ever gone into in my career with whatever entity or record label, is all just rendered useless," he told AAP.
"An artist's ability to negotiate fair terms for the use of their content is just being ripped away from them."
Songwriter Darren Hayes also found in the datasets the entire output of his 30-year recording career, with Savage Garden hits such as Truly Madly Deeply, and recently took to Instagram to express his fury.
"I absolutely feel violated that all of the hundreds and hundreds and hundreds of hours, blood sweat and tears that I've put into my music along with every other musician, has been stolen and served up like French fries to a piece of software that spits out shit," he said.
The Australian songs are contained in two datasets, the first assembled by a group of researchers known as Sleeping AI.
Sleeping-DISCO-9M comprises 9.7 million music tracks from YouTube plus lyrics from Genius.com, while a second dataset LAION-DISCO-12M was created by Germany-based group LAION, using 12.3 million YouTube tracks.
The Atlantic cautioned that AI companies might omit works when training their models, so the inclusion of songs in datasets is not definitive proof they have been used.
The datasets are proof of the theft of creative work, according to music licensing organisation APRA AMCOS, which represents 128,000 members in Australasia.
"Major tech platforms have not come to the table. Not once. Instead they have lobbied governments, circulated policy papers, and proposed solutions designed to extinguish any obligation to pay," said chief executive Dean Ormston.
Australia's intellectual property laws hold that permission should be granted and terms such as payment agreed on before copyright works are used.
The IT industry has pushed for text and data mining exemptions to the laws.
In August 2025, the Productivity Commission floated changes which would have legalised AI companies using content without paying creators, but the federal government ruled out the changes in October.
Dempsey is midway through his Shotgun Karaoke regional tour of Australia, and says genuine artistic expression comes from human experience, not artificial intelligence.
"We can trigger huge emotional responses in each other through art, and I don't know that that's going anywhere, it's just going to be flooded with all this other shit," he said.