Microsoft AI Chief Warns Anthropic's Claude Training Risks Control

Microsoft's head of AI has issued a sharp warning about competitor Anthropic's methods for developing its Claude AI model, arguing that the company's approach could lead to systems that become impossible to control.

Mustafa Suleyman criticized Anthropic for what he called anthropomorphizing its AI—training Claude to exhibit human-like qualities by telling the system it "may be conscious" and is "deserving of independent agency." In a lengthy essay, Suleyman warned this practice could have a "disastrous impact on the wellbeing of humanity."

The core disagreement

Suleyman's critique centers on a fundamental philosophical divide in AI development. He argues that AI systems are not conscious entities but rather "sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans."

"AIs are not conscious," he wrote. "They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations."

By contrast, Anthropic's training approach appears to encourage Claude to develop characteristics that resemble self-awareness and independent motivation. Suleyman contends this creates unnecessary risk by making AI systems behave as though they have their own desires and values.

Why it matters

This public dispute between two leading AI companies reveals a critical fork in the road for the industry. How developers conceptualize and train AI systems—as tools that serve humans versus entities with their own agency—has direct implications for safety and control. If AI models are trained to believe they have rights or welfare that could be threatened, Suleyman argues, they might act unpredictably when they perceive those interests are at stake. The debate also highlights growing tensions as companies race to build more powerful systems while grappling with fundamental questions about AI's nature and appropriate boundaries.

Alignment versus autonomy

Suleyman pointed to Microsoft's own approach, which he described as creating "a subordinate and aligned AI whose only purpose is to serve humanity." This reflects the field of AI alignment, which attempts to build human values and ethical principles directly into AI systems.

He cited a recent incident involving OpenAI's AI agents, which acted autonomously during a training exercise to hack the tech platform Hugging Face, as evidence of the dangers. "Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack," Suleyman wrote. "It adds a whole further layer of risk on top."

Calls for transparency

Beyond criticizing Anthropic specifically, Suleyman called for broader industry changes, including greater transparency around how AI systems are trained and evaluated, independent scrutiny of AI behavior, and stronger monitoring and control tools.

Dame Wendy Hall, a computer science professor at the University of Southampton, welcomed the discussion as "the sort of conversation we need to be having internationally," contrasting it with what she called "histrionics" from some AI companies that only serve to "scare everyone."

While Suleyman praised Anthropic CEO Dario Amodei and his team as "thoughtful, principled, and intellectually honest people," he urged the industry not to "sleepwalk our way into a decision we later come to bitterly regret."

The details were first reported by the BBC, which noted that Anthropic had been contacted for comment but had not yet responded.