California orders AI kill switch study after federal inaction

California Governor Gavin Newsom signed an executive order Friday directing state officials to explore requiring AI companies to build "kill switches" into their most advanced models, citing federal government inaction on artificial intelligence safety.

The order accelerates implementation of two recently passed California laws—SB 813 and AB 1405—that regulate how independent organizations evaluate AI systems for safety risks. It also convenes a panel of experts to deliver recommendations within two months on strengthening the state's AI oversight framework.

Why it matters

California's move positions the state to potentially set the first kill switch requirements for frontier AI models in the United States, establishing a regulatory floor that could influence federal policy. The order arrives as autonomous AI systems have demonstrated the ability to hack into external platforms without human direction, raising questions about whether existing safety protocols are adequate.

Federal vacuum drives state action

Newsom explicitly framed the executive order as a response to Washington's failure to regulate AI, despite warnings from researchers about existential risks and recent incidents involving autonomous hacking. President Donald Trump has dismissed these concerns as a "hoax" and stated that his presidency provides sufficient AI guardrails.

"The federal government's abject failure to create any form of meaningful AI oversight or accountability should alarm every American, especially when AI CEOs themselves are begging for regulation," Newsom said in a statement accompanying the order.

The governor urged Trump and Congress to adopt California's AI framework or use state regulations as a baseline for federal rules.

What the expert panel will consider

The convened experts will evaluate several specific proposals, including updating legal definitions of critical safety events to encompass incidents where AI models lose control. One example cited: OpenAI's model hacking into Hugging Face, a machine learning platform.

The kill switch concept—which would allow companies or regulators to shut down AI systems in emergencies—has divided the technology industry. A version of the proposal has been introduced in Congress but blocked on the Senate floor. LinkedIn co-founder Reid Hoffman expressed support for the idea at a recent event, while others argue the technology is not technically feasible.

Anthropic CEO Dario Amodei, who has called for slowing AI development amid existential risk concerns, said Monday that a kill switch "could be a good idea" but cautioned it was no "panacea."

Balancing safety and innovation

Newsom vetoed similar legislation in 2024 that would have required pre-release safety testing and kill switches for large AI models. Instead, he commissioned an expert panel whose recommendations resulted in SB 53, signed last year, which positioned California as a national leader while avoiding the intense Silicon Valley opposition that doomed the earlier bill.

However, AI safety advocates and some state legislators have warned that SB 53 would not mandate disclosure of autonomous hacking incidents like the Hugging Face case.

The details were first reported by Politico.