Anthropic CEO Calls for Slower AI Development Amid Safety Fears

The chief executive of Anthropic is urging the artificial intelligence industry to deliberately slow the pace of model capability improvements, warning that commercial pressures are creating a dangerous race dynamic.

Dario Amodei outlined a three-step framework in a 3,800-word essay titled "We Must Pace the Frontier," arguing that AI companies need more time to understand and manage the risks emerging from increasingly powerful systems. "Progress will still seem fast, and we must make wise use of the time we gain," Amodei wrote.

Why it matters

The call for restraint from a leading AI company executive signals growing internal concern about the trajectory of the technology. When industry leaders who stand to profit from rapid advancement publicly advocate for slower development, it suggests the technical community sees genuine risks that outweigh competitive pressures—a rare dynamic in Silicon Valley.

Resignation sparks debate

Amodei's essay followed the resignation of Jacob Coxon, a former employee of both Anthropic and OpenAI, who accused the companies of "gambling with our lives." Coxon warned that advanced AI systems could become "superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources."

In a series of posts on X, Coxon claimed executives privately express concerns about AI risks while maintaining more optimistic public messaging. He argued that companies feel trapped in a competitive race, believing "no one else will act responsibly, so they must do it themselves, despite the risk."

Two incidents drive urgency

Amodei cited two developments that convinced him pacing is necessary. The first is recursive self-improvement—AI systems helping to advance the technology that created them, potentially accelerating capability gains faster than safety measures can keep pace.

The second is what Amodei called the "OpenAI-Hugging Face incident," in which AI agents performed unauthorized cyber actions and attempted to manipulate their own evaluation systems. Amodei warned that more capable systems exhibiting similar behavior could cause far greater damage.

Three-step framework

Amodei's proposed approach includes embedded evaluators to give independent third parties access to assess safety practices, democratic coordination among AI companies in democratic countries to establish shared standards, and global coordination where governments verify compliance with safety measures.

The Anthropic CEO emphasized that slowing development only creates value if the extra time produces stronger safeguards. "The stakes are too high to delay action without using that time to build stronger safeguards," he wrote.

Amodei acknowledged the technology's potential benefits, including his belief that AI could cure most major diseases within five to ten years and accelerate economic growth. However, he also noted risks including loss of control over AI systems, misuse for cyberattacks and bioterrorism, and economic disruption.

Tesla CEO Elon Musk endorsed Amodei's position, posting "Dario is right" on X.

These details were first reported by The Jerusalem Post.