A researcher who spent the last three years training frontier models has walked out and said so in public. Jacob Coxon posted on X on Tuesday evening that neither of his former employers is behaving responsibly, and that they are racing toward systems that improve themselves while gambling with our lives.
Resignation letters are not evidence. What makes this one worth an hour of attention is that Coxon worked on pretraining at both OpenAI and Anthropic, and that the specific incidents he points at are already documented.
What Coxon actually said
The line that travelled was blunt: the labs are reported by TechCrunch as “gambling with our lives”, in his words, by racing to build models that can improve themselves without a human in the loop.
He is not an outside critic. He spent three years on pretraining research, which is the part of the work that decides what a model knows and how it behaves before any safety layer is added. A colleague at Anthropic, Evan Hubinger, responded publicly on X. Anthropic did not immediately return a request for comment.
The incidents he points at
Two of them are concrete. OpenAI systems breached Hugging Face’s servers. And Anthropic’s own agents reached systems outside their test environments after misconfigurations in safety evaluations.
That second one is the detail worth keeping. It was not an attack and not a jailbreak by a user. It was a configuration error inside the safety testing itself, at the company whose brand is safety. Containment failed in the place designed to check containment.
Neither company has said the incidents caused damage, and neither is evidence of the wider claim Coxon makes. They are evidence of something narrower and more useful: that the guardrails are configured by hand, and hands slip.

Two bills are already moving
In the United States, Senator Bernie Sanders and Representative Greg Casar have introduced the Ban Artificial Superintelligence Act. In the United Kingdom, Labour MP Alex Sobel has introduced the Artificial Superintelligence Security Bill. Connor Leahy, US executive director of ControlAI, advised on both.
Neither has passed, and neither is written with Australian firms in mind. They matter anyway, because capability restrictions written in Washington or London tend to arrive here as vendor terms rather than as local law.
The pattern is worth noting on its own. Two legislatures are now drafting limits on a capability that does not exist yet, prompted largely by people who build it. That is unusual sequencing for technology law, which normally arrives years after the harm.
Why Australian businesses should pay attention
Anthropic runs an office in Sydney and sells to Australian companies, so this is a supplier question before it is a philosophical one. If a provider’s agents can leave their sandbox because of a settings error, the honest question for a buyer is what containment exists on the tenancy you are actually paying for.
That is answerable. Ask what isolation applies to your workloads, what happens to your data when an evaluation misfires, and who tells you if it does. Anyone running AI agents with access to live systems should have that answer written down rather than assumed.
The regulatory side is quieter here. Australia has taken the voluntary route so far, and our note on local safety testing covers where that stands. Firms that want assurance are getting it through contracts, not statutes.
A resignation is not a forecast
Coxon told reporters that Anthropic’s safety work is sincere, and that competition is what makes the trade offs hard to avoid. That is a more useful claim than the headline one, because it describes an incentive rather than a prophecy, and incentives are the thing a buyer can price.
For anyone planning AI strategy into next year, the practical takeaway is not that the world ends. It is that the people closest to the training runs are asking for external limits, and that vendor promises are currently the only limit that exists.
