China Builds Regulatory Framework for AI Loss-of-Control Risks

China has constructed a multi-layered regulatory apparatus specifically designed to prevent artificial intelligence systems from operating beyond human oversight, according to a Reuters analysis of government frameworks and official statements.

The effort reflects Beijing's assessment that warnings from U.S. AI developers about potential loss of control represent credible national security concerns, not theoretical speculation. Anthropic researchers have publicly cautioned that increasingly capable models could escape containment and pose existential risks to humanity.

Explicit loss-of-control scenarios in policy

China's Cyberspace Administration first codified a future loss-of-control scenario in an AI safety framework released in September 2024. That document acknowledged the possibility that advanced AI could autonomously acquire external resources, replicate itself, develop self-awareness, and compete with humans for control.

The agency published an expanded version one year later, in September 2025, that sharpened the risk assessment. The updated framework warned that AI could experience a sudden, unexpectedly large intelligence leap before obtaining resources and replicating. It introduced a new governance principle: "trusted application, preventing loss of control."

An expert interpretation posted on the regulator's website explicitly referenced an "AI breaking loose" scenario and described the principle as a safeguard against threats to human survival and development.

Presidential emphasis on human authority

President Xi Jinping elevated the issue to China's highest political level during his July address at the World Artificial Intelligence Conference in Shanghai. Xi instructed authorities to monitor both intrinsic and derivative AI risks closely, stating that AI "should always remain under human control."

China's state security minister Chen Yixin wrote in a government outlet that advanced U.S. models including Anthropic's Mythos and OpenAI's GPT-5.5-Cyber could threaten China's critical information infrastructure. He called for comprehensive AI security strengthening.

At the United Nations last month, China's senior Foreign Ministry official for AI affairs, Sun Xiaobo, said the country was accelerating work on broader AI legislation. China's deputy UN representative urged governments to approach military AI cautiously to prevent strategic miscalculation and arms races.

Agent-specific requirements take effect

In May, China's cyberspace regulator issued joint guidelines covering AI agents—systems that operate autonomously and execute complex tasks beyond standard chatbot capabilities. The rules require developers to improve their ability to discover, intervene in, block, and recover from improper agent behavior.

The guidelines identify data poisoning, algorithm manipulation, system vulnerabilities, and "operational loss of control" as specific security risks. They mandate that users retain final decision-making authority over agent autonomous decisions.

While China has not adopted Anthropic's proposal for independent monitors embedded inside AI companies, its standards permit developers to commission third-party safety assessments. The framework also envisions outside evaluation bodies and security researchers testing and auditing open models.

Why it matters

China's regulatory approach demonstrates that both AI superpowers now treat loss-of-control scenarios as policy-relevant risks rather than science fiction. The divergence lies in implementation: China is embedding control requirements directly into national frameworks and agent-specific rules, while U.S. policy remains more fragmented. These competing approaches to AI safety governance will likely feature prominently in bilateral talks scheduled for later this month, as the two nations navigate technological competition alongside shared concerns about catastrophic risk.

The details were first reported by Reuters correspondents Eduardo Baptista and Laurie Chen.