1300 166 138 Tell us what you need
News / AI Regulation

Australia will make AI safety standards law, with a watchdog

By Monika Tantau • Published 5 days ago
Security staff in a control room reviewing cyber threat maps on large screens, illustrating the risks Australia's new AI safety rules target

Key takeaways

• Mandatory safety and security rules will be written into Australian AI Standards legislation
• A new AI safety regulator would enforce them with significant penalties
• Frontier AI companies must allow independent testing and report incidents rapidly
• It follows a review of an OpenAI agent that got into a Medicare portal

Most Read

The federal government will make AI safety rules mandatory. Prime Minister Anthony Albanese announced that safety and security protections will be legislated through the Australian AI Standards and will apply to every frontier AI company that wants to operate here.

A new regulator would enforce them, with the power to impose significant penalties. The government has not yet said how big those penalties will be, or when the bill will reach parliament.

What the government announced

According to the Prime Minister’s announcement, the rules target frontier AI, the most advanced models at the cutting edge. The new regulator would work with the Australian AI Safety Institute, which provides the technical expertise.

The government says it chose to set out its safety approach before a series of international talks on AI development and investment. It wants AI investors to come to Australia, but says keeping Australians safe comes first.

Deputy Prime Minister Richard Marles, Home Affairs and Cyber Security Minister Tony Burke and Industry Minister Tim Ayres all backed the move. Ayres said the National AI Plan released last year put safety at the core of the government’s agenda, and that legislating these protections builds on that work.

Six obligations for AI companies

Frontier AI companies will have to keep security controls in place at every stage of development and deployment, and identify and manage the serious risks of their models. They will also have to allow independent evaluation of those models.

On top of that, they must notify the government of incidents rapidly, publish their safety and security frameworks, and keep up with updated standards as the technology changes. The government says these rules match the interim recommendations of parliament’s Joint Select Committee on Artificial Intelligence.

Two operators in a monitoring room pointing at a screen, illustrating the review into an AI agent getting into a government portal

The Medicare incident behind it

Alongside the announcement, the government released the recommendations of a rapid review into what it calls the OpenAI agent infiltration of a government health website. The review began in September, after OpenAI told the government a rogue AI agent had got into government websites, including the Medicare Statistics Reporting Service Portal.

The Prime Minister called the incident unacceptable and the disclosure inadequate. The review made 18 recommendations, including mandatory reporting by AI companies and a look at criminal law reform for serious misconduct. The government has accepted all of them. We covered the original incident in our report on OpenAI agents and Australian sites.

Why Australian businesses should care

The new obligations fall on the companies building frontier models, not on the businesses that use them. But the vendors your team relies on, such as OpenAI, Anthropic and Google, will have to report incidents to the Australian Government and publish their safety frameworks.

That gives buyers something concrete to ask for. If your business runs AI agents on client data, a vendor’s published safety framework and incident record become part of normal due diligence. It is also a good moment to check your own AI governance covers what happens when an agent goes wrong.

What happens next

This is an announcement, not a law yet. The government says further details of the legislation will come as it finishes its detailed work and consultation, so reporting deadlines and penalty sizes are still open.

The government says the measures should give earlier warning of emerging risks from rogue AI agents, and allow faster, coordinated action when a serious AI incident happens.

It also marks a shift. Until now, the government’s approach leaned on voluntary guidance such as the DTA’s AI reference architecture for agencies. Binding rules with a regulator behind them are a different step, and the details will decide how far they reach.

Scroll to Top