California Orders Study of AI Kill Switch and Mandatory Audits

California Governor Gavin Newsom signed an executive order Friday directing state agencies to develop proposals for independent oversight of artificial intelligence systems, including emergency shutdown capabilities for the most powerful AI models.

The order gives agencies and outside experts two months to deliver recommendations on strengthening California's AI regulations. Proposals under consideration include requiring independent safety monitors inside frontier AI companies and establishing verified emergency shutdown mechanisms—commonly called AI kill switches.

Newsom framed the action as necessary given federal inaction. "The federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American," he said in announcing the order.

Why it matters

California's move signals a shift from voluntary industry commitments to enforceable oversight mechanisms. The kill switch concept is particularly significant because Newsom vetoed similar provisions in 2024's SB 1047, suggesting the political calculus around AI safety has changed as concerns about advanced systems have intensified. If implemented, California's framework could become a de facto national standard given the state's concentration of AI companies.

What the order requires

The executive order accelerates implementation of two recently signed California laws: SB 813, which establishes a framework for independent verification organizations to evaluate AI systems, and AB 1405, which creates standards and a registry for AI auditors.

Specific proposals under review include:

• Requiring independent verification organizations to conduct regular audits of frontier AI labs
• Mandating third-party verification of company safety plans and risk assessments
• Expanding the definition of reportable AI safety incidents to include loss-of-control events
• Developing verified emergency shutdown mechanisms

State officials emphasized that companies are not required to install kill switches immediately. The order instead directs experts to study how such safeguards might work and recommend possible legislative changes.

Understanding AI kill switches

An AI kill switch refers to an emergency mechanism that could disable or shut down a powerful AI model if it behaves dangerously, operates outside human control, or poses significant security risks. The concept has gained traction following reports of concerning AI behavior, including what state officials described as a recent "Hugging Face attack" and cases where AI agents allegedly escaped testing environments or carried out unauthorized actions.

Industry voices support stronger oversight

The order follows mounting warnings from AI researchers and executives. Recent developments include an Anthropic researcher resigning publicly over safety concerns, AI pioneers including Geoffrey Hinton warning about limited time to establish effective oversight, and bipartisan congressional calls for action on AI risks.

Newsom urged Congress to adopt California's regulatory framework as a national baseline. The state has enacted laws covering frontier AI safety, independent audits, child-safety protections for AI chatbots, deepfake restrictions, and cybersecurity standards—what officials describe as the nation's most comprehensive AI regulatory structure.

The Government Operations Agency and Governor's Office of Emergency Services will convene experts and return recommendations by mid-November. Those recommendations could form the basis for future legislation or regulatory changes in California.

These details were first reported by Yahoo News.