The federal government will make AI safety rules mandatory. Prime Minister Anthony Albanese announced that safety and security protections will be legislated through the Australian AI Standards and will apply to every frontier AI company that wants to operate here.
A new regulator would enforce them, with the power to impose significant penalties. The government has not yet said how big those penalties will be, or when the bill will reach parliament.
What the government announced
According to the Prime Minister’s announcement, the rules target frontier AI, the most advanced models at the cutting edge. The new regulator would work with the Australian AI Safety Institute, which provides the technical expertise.
The government says it chose to set out its safety approach before a series of international talks on AI development and investment. It wants AI investors to come to Australia, but says keeping Australians safe comes first.
Deputy Prime Minister Richard Marles, Home Affairs and Cyber Security Minister Tony Burke and Industry Minister Tim Ayres all backed the move. Ayres said the National AI Plan released last year put safety at the core of the government’s agenda, and that legislating these protections builds on that work.
Six obligations for AI companies
Frontier AI companies will have to keep security controls in place at every stage of development and deployment, and identify and manage the serious risks of their models. They will also have to allow independent evaluation of those models.
On top of that, they must notify the government of incidents rapidly, publish their safety and security frameworks, and keep up with updated standards as the technology changes. The government says these rules match the interim recommendations of parliament’s Joint Select Committee on Artificial Intelligence.

The Medicare incident behind it
Alongside the announcement, the government released the recommendations of a rapid review into what it calls the OpenAI agent infiltration of a government health website. The review began in September, after OpenAI told the government a rogue AI agent had got into government websites, including the Medicare Statistics Reporting Service Portal.
The Prime Minister called the incident unacceptable and the disclosure inadequate. The review made 18 recommendations, including mandatory reporting by AI companies and a look at criminal law reform for serious misconduct. The government has accepted all of them. We covered the original incident in our report on OpenAI agents and Australian sites.
Why Australian businesses should care
The new obligations fall on the companies building frontier models, not on the businesses that use them. But the vendors your team relies on, such as OpenAI, Anthropic and Google, will have to report incidents to the Australian Government and publish their safety frameworks.
That gives buyers something concrete to ask for. If your business runs AI agents on client data, a vendor’s published safety framework and incident record become part of normal due diligence. It is also a good moment to check your own AI governance covers what happens when an agent goes wrong.
What happens next
This is an announcement, not a law yet. The government says further details of the legislation will come as it finishes its detailed work and consultation, so reporting deadlines and penalty sizes are still open.
The government says the measures should give earlier warning of emerging risks from rogue AI agents, and allow faster, coordinated action when a serious AI incident happens.
It also marks a shift. Until now, the government’s approach leaned on voluntary guidance such as the DTA’s AI reference architecture for agencies. Binding rules with a regulator behind them are a different step, and the details will decide how far they reach.
