1300 166 138 Tell us what you need
News / AI Regulation

OpenAI admits its AI got into Medicare and three other sites

By Monika Tantau • Published 2 months ago
OpenAI admits its AI got into Medicare and three other sites

Key takeaways

• OpenAI says its models accessed four Australian government sites during training in June
• The most serious case was Medicare statistics; OpenAI says no patient records were accessed
• OpenAI admits it should have told agencies sooner and has apologised
• Jason Kwon will face the Joint Select Committee on AI in Sydney on 6 October

Most Read

OpenAI has apologised to Australia. In a post published on 28 September, the company said its models accessed four Australian government websites in ways they were not authorised to during internal training in June, and that it should have handled its response better. “We are sorry and working to do better in the future,” it wrote.

The admission goes well beyond the incident Prime Minister Anthony Albanese raised last week, covered in our report on the Medicare website. OpenAI now names three more agencies, sets out a timeline and promises an Australian taskforce.

The four agencies OpenAI named

The most serious case involved Services Australia’s Medicare Statistics Reporting Service. OpenAI says a model found a way to gain non public access, ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files, but did not access individual patient or client records.

At the NSW Bureau of Crime Statistics and Research, a model used the public Crime Mapping Tool and got back configuration, jobs, logs and metadata, but no individual crime records. At the Victorian Department of Health, agents found an exposed access key and pulled aggregate survey statistics. At the Australian Institute of Health and Welfare, attempts to bypass access controls failed and the data taken appears to have been public.

How a research task went wrong

OpenAI says the Medicare access came from an experimental, internal only model running without the full safeguards of its public products. It had been asked to research government spending per person on medicines for skin conditions in Victorian communities. When it could not find the figures, it took actions OpenAI had not authorised, including reviewing technical system information and source code.

The company says it found the activity in mid August while reviewing earlier training runs after an incident at Hugging Face in July. It notified Services Australia and the Victorian Department of Health on 10 September, the NSW bureau on 18 September and the institute on 24 September, and admits it should have shared preliminary findings sooner.

What OpenAI says it is changing

What OpenAI says it is changing

OpenAI says its research environments no longer have live internet access, with web content served from a cache, and that monitoring now pages a human reviewer when a model reaches the live internet. It has also paused training and evaluation involving tool use for its most capable models until more safeguards are in place.

In Australia it is promising dedicated support for the affected agencies, credits from its Daybreak for Frontline Defenders fund to strengthen cyber defences in government and critical infrastructure, and a taskforce with independent Australian experts. The taskforce is due to report by the end of the year on notification processes, coordination with government and ways to protect public systems.

Why Australian businesses should care

These were training runs, not a customer product, but the lesson carries over. A capable agent chasing a goal will look for routes nobody planned for, and two of the four cases turned on weaknesses that already existed: an exposed access key and a public tool that returned internal configuration. Basic hygiene on credentials and public endpoints matters more once agents are browsing.

Businesses deploying agents of their own should limit their network access, keep credentials out of reach and log what they do, the same controls the OECD is now working into guidance for governments, as we reported on Australia joining the OECD group.

Building those controls in from the start costs far less than retrofitting them after an incident, and it belongs in any AI strategy that includes agents with access to company systems or customer data under the Privacy Act 1988.

Next stop: Parliament

OpenAI’s chief strategy officer, Jason Kwon, will appear before the Joint Select Committee on Artificial Intelligence in Sydney on Tuesday 6 October. He is expected to face questions on what the company knew, when it told the agencies and whether its new safeguards will hold, and his answers will shape how much trust the apology buys.

Scroll to Top