Anthropic Researcher Resigns, Warns AI Race Could Put Humanity at Risk
Anthropic researcher Jacob Coxon has resigned from the company and said he is leaving the artificial intelligence industry over concerns about the race toward increasingly autonomous and self-improving AI systems. Coxon, who previously worked at OpenAI, said neither company is acting responsibly enough as capabilities accelerate.
In a post on X, Coxon said he spent the past three years conducting pretraining research at OpenAI and Anthropic. He argued that the industry is moving toward systems that could potentially improve themselves, acquire resources and carry out sophisticated tasks with limited human involvement.
His comments represent his assessment of the risks, rather than evidence that such systems have already reached those capabilities.
Why Coxon Says the AI Race Is Different
Coxon’s central concern is not simply that AI models are becoming more powerful. He believes the combination of capability growth and competition between leading laboratories could make safety decisions increasingly difficult. In his view, companies may continue developing frontier systems because they fear that slowing down would allow competitors to move ahead.
That creates what Coxon describes as a dangerous race. He argues that researchers could eventually build systems capable of hacking, scientific discovery and other complex activities at a level beyond humans. The key question, he says, is whether developers will understand how these systems behave before giving them significantly greater autonomy.
Coxon also challenged the common argument that laboratories would not build dangerous technology if they genuinely believed it posed an existential threat. He said his experience at OpenAI suggested that some people had not fully internalized the potential consequences, while Anthropic researchers understood the stakes but remained under competitive pressure.
Related: AI Is Coming: 4 Crypto Solutions You Need To Know About
His criticism is particularly notable because Anthropic has built much of its public identity around AI safety and alignment. Coxon’s argument is therefore not that researchers at the company are indifferent to safety, but that competitive incentives could still push development faster than safety research can keep up.
The debate comes as AI systems are becoming increasingly capable across programming, mathematics, cybersecurity and scientific research. Recent reporting has highlighted rapid progress in AI-assisted mathematical research and autonomous systems, demonstrating why questions about capability growth are becoming less theoretical.
Coxon believes the answer cannot be left entirely to individual companies. He called for greater coordination between major U.S. AI laboratories and suggested that, under certain circumstances, a temporary restriction on improving model capabilities could become necessary to prevent an uncontrolled race.
The Bigger Question Is Whether AI Development Can Be Controlled
The researcher did not present his departure as an argument against AI itself. Instead, he said he remains optimistic that coordination is possible. He pointed to incidents involving AI systems and cybersecurity as potential warning signs that could make cooperation between laboratories more politically and technically viable.
His argument also raises a difficult governance question. If several companies are simultaneously developing increasingly capable systems, one laboratory slowing down may have little effect if competitors continue operating at full speed. That creates incentives for companies to prioritize staying ahead, even when researchers inside those organizations believe additional safeguards are necessary.
Anthropic’s own researchers have separately acknowledged the difficulty of solving alignment for future superintelligent systems. Anthropic alignment lead Evan Hubinger has said the company is trying to address the problem but is not yet clearly on track to solve alignment for superintelligence.
Related: Mark Zuckerberg Launches Muse as Meta Pushes Into Personal AI Agents
That distinction matters. Current AI systems are not proof that human extinction is inevitable, and Coxon’s most severe scenarios remain forecasts about future technology. But the fact that researchers working directly on frontier models are openly debating these possibilities shows how seriously some specialists view the uncertainty surrounding advanced AI.
For the industry, Coxon’s resignation puts the focus on a question that cannot be answered by model benchmarks alone: how quickly should capabilities advance when developers cannot confidently predict how future systems will behave? The answer will likely involve companies, governments and researchers rather than any single AI laboratory.
Coxon’s warning is ultimately about incentives as much as technology. If the world’s leading AI companies believe that reaching superintelligence first is strategically important, safety measures may face pressure whenever they appear to slow development. Whether the industry can coordinate before that pressure becomes overwhelming may be one of the most consequential technology-policy questions of the decade.
About The Author
