California Governor Orders AI Kill Switch Study, Expert Panel

California Governor Gavin Newsom issued an executive order on September 18 directing the state to accelerate AI safety measures, including exploration of emergency shutoff capabilities for advanced AI systems.

The order establishes an expert-led panel charged with strengthening California's AI safety laws, accelerating third-party safety evaluations, and mandating risk reporting from developers. Notably, the directive calls for studying the feasibility of an "emergency shut off" for frontier AI models — what Newsom described in a video statement as an "AI kill switch," acknowledging the term "means a lot of things, depending on who you talk to."

Why it matters

California's move signals growing momentum for government intervention in AI development at the state level, potentially setting precedents that influence federal policy and industry standards. The initiative arrives as both policymakers and AI researchers grapple with how to impose meaningful safety controls on systems that increasingly operate across distributed networks beyond simple on-off mechanisms.

What an AI kill switch actually means

The concept of an AI kill switch has gained traction in policy circles over the past year, though its practical implementation remains contested. In July, two congressional representatives introduced the bipartisan AI Kill Switch Act, which would grant the Department of Homeland Security authority to slow or halt AI systems capable of causing "catastrophic harm." That bill would also require developers to build in mechanisms to throttle, suspend, or fully shut down covered systems.

Most AI labs currently maintain internal processes to halt problematic systems, governed by company safety teams. However, proposals for kill switches range from requiring external audits of these internal processes to establishing federal oversight of industry-wide emergency protocols.

Mark Nitzberg, executive director of UC Berkeley's Center for Human-Compatible AI, told Scientific American that a universal AI kill switch represents "more of an aspiration than a reality." Geoffrey Hinton, widely known as the "Godfather of AI," expressed skepticism to CNN that kill switches would effectively address concerns about AI superintelligence.

Technical and governance challenges

Experts point to fundamental complications with kill switch proposals. AI systems don't operate in single physical locations that can simply be powered down. Michael Vermeer, a senior physical scientist at the RAND Corporation, suggested to Scientific American that rogue AI agents might need to be "hunted down" across the open internet to locate and contain their operations — potentially requiring coordinated global response.

Authority questions compound technical challenges: Should shutdown decisions rest with original developers, industry bodies, or government agencies? The answer carries significant implications for both effectiveness and potential government overreach.

Travis Hall, Director for State Engagement at the Center for Democracy and Technology, told Mashable the organization "remains concerned that proposals for 'kill switches' will prove ineffective in reducing risk while introducing new security risks and serious potential for government abuse." Hall emphasized that effective AI safety oversight must be "meaningful, independent, and ongoing," incorporating diverse expertise including civil rights advocates — not just technicians.

Policy experts also warn that "binary" or checklist-style safety regulations focusing on specific technical details can be easily gamed by AI companies and quickly become outdated as technology advances.

Details of Newsom's executive order and the kill switch exploration were first reported by Mashable.