Delegates at Labor's national conference in Adelaide, which kicked off on Thursday, agreed to changes to the party's official platform on AI.
The motion, which called for the creation of the task force that would build awareness of the risks and opportunities of AI, passed unanimously.
Labor Senator Tony Sheldon said workers still needed protection from the technology.
"We're the ones to say that those tech bros that say to us that 'they're going to replace everything you do' ... that's not true," he told the conference.
"We need to have the strength, the capability and the purpose to make sure that our human creativity, ambition, integrity and intelligence is at the forefront of every decision that's made in every industry we work in."
The amendments also laid out plans for equitable access to AI education and digital skills training across a wide range of industries and communities.
In his speech opening the conference, Prime Minister Anthony Albanese said handling AI would be one of the defining challenges of modern times.
"Right now, global investors are knocking on Australia's door, and that gives us the chance to determine what AI looks like here," he said.
"This technology is already part of our daily lives, we can't stop it and we wouldn't want to, but we can shape it and we have to.
"Our prosperity, our sovereignty and our national security depend on it."
It comes after Mr Albanese used a speech earlier in July to lay out Australian standards for AI, such as ways for data centres to minimise their water use.
The prime minister said the technology needed to not infringe on the creative works of artists and journalists.
"There are real concerns about how fast AI is moving and the disruption that it might bring," Mr Albanese said.
"We can only manage that and protect ourselves if Australia is active and confident in shaping this change."