Anthropic, which owns Claude, fronted a federal grilling on AI which held hearings in Sydney on Tuesday.
The testimony comes after it was revealed the San Francisco-based giant's rival, OpenAI, had let one of its models hack a Medicare website during a research mission in June.
The tech titan found out about the break-in in August, but left Canberra in the dark until September when it emailed a generic public service inbox.
"As soon as we detect anything of that nature, we would definitely reach out within a matter of days or sooner," Anthropic's head of safeguards Dave Orr told the committee.
But Australian laws lacked the teeth to get AI companies to own up to their mistakes, committee members heard.
"Right now this is largely voluntary, and we're still learning technology and learning how to conduct investigations," Anthropic special envoy and former US ambassador to Australia Jeff Bleich told the committee.
The firm's Australia and New Zealand policy chief David Masters said there were transparency requirements in some laws.
"We would welcome similar laws here in Australia," he said.
Anthropic flagged AI's rise could hurt demand for human workers in its submission to the inquiry, but said early evidence showed the technology would augment rather than replace manpower.
It recommended creating incentives for firms to keep and retrain workers if AI's workforce impact turned nasty.
"There really should be no trading off between capturing the opportunity of AI and taking those risks very seriously," Mr Bleich said.
"Australia can and should do both."
Anthropic's chief executive Dario Amodei is the industry's most prominent advocate for slowing the technology and urgently erecting guardrails.
OpenAI's chief strategy officer Jason Kwon has also flown in from the US to apologise and answer questions in person about the Medicare breach.
"That should not have happened. We also should have handled our response better," Mr Kwon told the inquiry.
"That starts with acknowledging where we fell short, following through on the commitments we've made, and earning the trust of the Australian people."
OpenAI is facing heat from a rapid review task force that is sniffing out legal avenues to punish the company for the June breach and its sluggish reporting.
On Friday, it informed the government of another break-in, to a NSW parks and wildlife website in June, with Mr Kwon pledging to more diligently keep authorities in the loop.
The tech giant will mount its own Australia-based task force on AI safety packed with independent expertise, which is expected to wrap up by the end of 2026.
That will roughly coincide with the drafting of Labor's proposed national AI standards, announced by Prime Minister Anthony Albanese in July.
Representatives of artists, musicians, authors and journalists, including the ABC, earlier told the inquiry AI scraping without compensation posed an existential threat to creatives' livelihoods, calling the companies thieves.
The ABC said it had no plans to replace any journalists with bots and deepfakes, but stressed the technology was a useful reporting tool.
Mr Bleich denied Anthropic had tried to "dictate the terms" on AI and copyright, despite the industry lobbying the government for carve-outs, but stressed the impracticality of trying to shield protected work.
"You can't licence the entire internet," he said.
