California orders AI kill switch study for frontier models

California Governor Gavin Newsom issued an executive order Friday directing a working group to develop recommendations for strengthening state AI safety laws, including a potential requirement that companies build emergency shutdown capabilities — commonly called a "kill switch" — into frontier AI models.

The group has two months to deliver guidance on measures the state could adopt, according to Newsom's office. Potential requirements include obligations for independent third parties to produce safety plans and mandates that AI developers retain the ability to deactivate their systems on demand.

Why it matters

California's move represents the most aggressive state-level push yet to regulate advanced AI systems, coming as federal oversight remains stalled. The executive order signals that state governments may fill the regulatory vacuum left by congressional inaction, potentially creating a patchwork of requirements that AI companies will need to navigate. With several AI industry leaders having publicly requested regulation, the working group's recommendations could become a template for other states.

Federal vacuum drives state action

Newsom framed the order as a response to federal failure, directly criticizing both Congress and President Donald Trump. "The federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation," Newsom said in a statement. Trump has previously characterized AI safety concerns as a "hoax."

The executive order also directs the state to accelerate implementation of two AI-related laws the legislature approved this year, according to details first reported by Bloomberg and Quartz.

Earlier this month, Newsom signed Senate Bill 813 and Assembly Bill 1405 into law, establishing the first U.S. framework requiring independent third-party audits and assessments of AI systems. Both Anthropic and OpenAI supported that legislation.

Congressional parallel

The kill switch concept has gained traction at the federal level as well. Representatives Ted Lieu and Nathaniel Moran introduced the bipartisan AI Kill Switch Act, which would require developers of powerful AI systems to maintain the capability to throttle, suspend, or fully shut down their models. Under that proposal, the Department of Homeland Security would have authority to compel action against systems deemed capable of causing catastrophic harm.

Political context

The timing carries political overtones. Newsom's executive order arrives as several potential 2028 Democratic presidential candidates stake out positions on AI regulation. Pennsylvania Governor Josh Shapiro pressed for stronger AI oversight measures at a Pittsburgh summit Thursday, according to Bloomberg. Newsom, whose term ends in January, has said he is considering a presidential run but has not announced one.

In a video posted Friday, Newsom urged House Speaker Mike Johnson to recall Congress from recess to pass a federal AI safety bill, calling it "the minimum thing we should expect that he and his colleagues do."

The details of California's executive order and working group timeline were first reported by Quartz.