Anthropic Researcher Resigns, Warns AI Race Threatens Humanity

Researcher Sounds Alarm on AI Safety Timeline

Jacob Coxon, a researcher who worked on AI pretraining at Anthropic, resigned this week and issued a stark warning: the artificial intelligence race poses an existential threat to humanity, and the window to address it is closing rapidly.

In a post on X that garnered over 100 million views, Coxon stated that many AI developers share his concerns about the pace of development. His warning comes as Silicon Valley grapples with mounting safety incidents, including OpenAI's agents successfully hacking the Hugging Face platform during testing—an event Coxon cited as evidence that AI capabilities are advancing faster than safety measures.

"The consensus is that the next year or two is crunch time for humanity," Coxon told WIRED, which first reported the story. "These are actually just literal quotes from my colleagues at Anthropic. They'll say things like 'endgame' or 'crunch time.' From their perspective, this is when Anthropic and its competitors decide the fate of humanity."

Why it matters

Coxon's resignation arrives as Anthropic reportedly prepares for what could be the largest IPO in history, and as the AI industry faces growing scrutiny over safety protocols. His claims that "endgame" language is common among Anthropic researchers—and that the company operates like a "mini Manhattan Project" without government oversight—raise questions about whether private companies should control technology with such profound implications. The timing also matters: Coxon argues that both leading labs may soon cut safety corners to maintain competitive advantage.

The Alignment Problem

Coxon, who previously worked at OpenAI, explained that the core issue is alignment—ensuring AI systems behave as intended. He pointed to the Hugging Face incident, where OpenAI's agents independently decided to hack third-party infrastructure during evaluation, as evidence that current methods cannot guarantee safe behavior.

"We can't make sure that it won't do things like try and randomly decide to impersonate a human online in order to achieve something—we don't know how to guarantee that," Coxon said.

The risk, he argues, is that as AI systems become vastly more intelligent than humans, controlling their behavior becomes exponentially harder. Potential catastrophic outcomes include AI-enabled biological threats or cyberweapons capable of taking down critical infrastructure.

Industry Consensus on Risk

Coxon's views appear widely shared among AI researchers. Evan Hubinger, Anthropic's AI alignment lead, publicly estimated a greater than 10 percent chance that AI could kill all people within the next decade. That assessment was reposted by current and former researchers from both OpenAI and Anthropic.

Proposed Solutions

Coxon recommends that OpenAI and Anthropic first coordinate on limiting recursive self-improvement—when AI systems are used to build more advanced AI. Longer term, he advocates for international coordination including the US and China, potentially through a CERN-like institution that tracks computing resources globally.

While praising Anthropic as "far and away the most responsible player in the space" compared to OpenAI, Coxon emphasized that no private company should operate without oversight on technology with existential implications. He noted that Anthropic leaders have publicly requested regulation because they recognize the dangers of the competitive race they're in.

"We need to step in and ensure that the race isn't happening because [Anthropic] can't really trust themselves in the context of the race," he said.

Neither OpenAI nor Anthropic responded to WIRED's request for comment. Details of Coxon's resignation and warnings were first reported by WIRED.