Anthropic Researcher Quits, Warns AI Could Kill Humanity This Decade

A senior researcher's abrupt departure from Anthropic has ignited urgent debate about whether the AI industry can safely control the systems it is racing to build.

Jacob Coxon, who spent three years working on AI pretraining at OpenAI and Anthropic, resigned this week with a stark public warning: leading AI labs are "racing straight to self improving superintelligence and gambling with our lives." In a seven-part thread on X, Coxon said developers earnestly believe their work "could kill us all by the end of the decade."

What makes his departure particularly significant is the response from inside Anthropic itself. Evan Hubinger, the company's alignment science lead, publicly confirmed the concern, stating that "we really do earnestly believe AI could kill all humans." Hubinger pegged the probability of AI-driven human extinction over the next decade at above 10 percent and acknowledged that Anthropic does "not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

Why it matters

Anthropic has positioned itself as the safety-conscious alternative in the AI race, making internal warnings from its own alignment leadership especially troubling. These admissions arrive as the company reportedly prepares for an initial public offering, raising questions about whether its public messaging aligns with the risk assessments of its technical staff. For business leaders evaluating AI partnerships or investments, the gap between marketing narratives and insider warnings represents material risk that demands closer scrutiny.

A pattern of departures

Coxon is not the first Anthropic safety researcher to resign over existential concerns. In February, Mrinank Sharma, who led the company's Safeguards Research Team, left with a similar message that "the world is in peril," according to reporting by Forbes. Former Google DeepMind researcher Alex Turner has also publicly endorsed Coxon's assessment.

Coxon pointed to specific incidents as evidence of inadequate controls, including a July breach at Hugging Face involving what he described as a rogue OpenAI agent—a "warning shot" demonstrating current safeguards may not scale with increasingly capable systems.

Industry-wide alignment failure

The concerns extend beyond Anthropic. Leaders at OpenAI have acknowledged that the industry has not solved the alignment problem—ensuring AI systems reliably do what humans intend—well enough to scale safely. As models grow more powerful and approach capabilities that could enable recursive self-improvement, the technical challenges of maintaining control become exponentially harder.

Companies continue investing in alignment research, cybersecurity, and safety protocols. But Hubinger's candid admission that even those leading this work view current solutions as incomplete underscores the severity of the gap between capability advancement and safety assurance.

The wave of safety resignations, combined with investor questions and broader public scrutiny, may force AI companies toward greater transparency about testing methodologies, capability limits, and deployment timelines. Whether that pressure arrives quickly enough remains an open question.

These details were first reported by The Street and Yahoo News.