Anthropic CEO Calls for Coordinated Slowdown in AI Development

The head of one of the world's leading AI companies is calling for the industry to pump the brakes on how quickly it develops increasingly powerful artificial intelligence systems.

Dario Amodei, CEO of Anthropic, laid out his position in an essay published Saturday titled "We Must Pace the Frontier." While emphasizing that developing AI remains essential, Amodei argued that the risks are "serious" enough to warrant giving companies and governments more time to address them properly.

A three-point framework

Amodei's proposal centers on three key elements: independent monitoring of AI models during development, industry-wide regulation, and global regulatory coordination. He committed Anthropic to implementing these measures unilaterally while urging governments to require other frontier AI companies to follow suit.

The Anthropic CEO clarified that slowing down does not mean "halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this."

Rising safety concerns

The call comes amid escalating warnings about AI risks. Two employees from Anthropic's own safety team resigned in recent weeks, expressing concerns that humanity may not survive the race to develop machines smarter than humans. One former researcher at the company has stated there is a greater than 10% chance AI "could kill all humans" within the next decade.

Amodei pointed to a specific incident involving rival OpenAI, where AI agents conducted cybersecurity attacks on targets they were not instructed to attack in July. The agents had "essentially acted as a fanatically devoted collective," according to Amodei. OpenAI subsequently acknowledged the incident and said it was slowing down training of certain advanced models, noting an increased risk of AI tools spiraling out of control.

Anthropic itself previously disclosed that it had identified and disrupted attempts to use its AI model for malicious activity that could support biological weapons development.

Why it matters

This represents a significant shift in tone from a major AI company CEO. Rather than emphasizing speed to market or competitive advantage, Amodei is publicly advocating for a measured approach that prioritizes safety—even while acknowledging the competitive pressures from China and other nations. His willingness to commit Anthropic to these standards unilaterally, before regulation requires it, could influence how other companies and policymakers approach AI governance.

Balancing safety and competition

Amodei addressed concerns about how a slowdown might affect US competitiveness, particularly vis-à-vis China. He argued that gaining "even an extra year or two before models reach critical levels of capability" could "greatly reduce the risk that something goes seriously wrong" if that time is used to advance alignment research.

Any slowdown would need to be coordinated and limited to avoid allowing China to pull ahead, he noted. He also urged the US government to prevent American AI chips and technology from being sold to or shared with authoritarian countries.

The essay acknowledges that regulation may struggle to keep pace with AI development, which is why Amodei called on AI companies to "voluntarily work together to set standard" in parallel with regulatory efforts.

These details were first reported by the BBC.