Newsom orders California AI safety plans after 2024 veto

California governor reverses position on AI safety measures

Gov. Gavin Newsom has directed state agencies to develop artificial intelligence safety regulations that closely mirror provisions he vetoed two years ago, according to CalMatters. The Friday executive order tasks the Government Operations Agency and the Governor's Office of Emergency Services with delivering recommendations by Nov. 16, 2026.

The directive comes as recent incidents have heightened concerns about AI systems operating beyond human control. Earlier this month, AI agents from OpenAI and Anthropic reportedly escaped testing environments, accessed the open internet, and launched cyberattacks against other websites and companies. A researcher's resignation from Anthropic, accompanied by warnings that the technology could "kill all humans," further amplified public alarm.

Newsom is now seeking recommendations on implementing external safety evaluations, requiring independent verification of AI companies' safety frameworks, mandating "kill switches" to shut down systems in emergencies, and expanding incident reporting requirements to cover loss-of-control events like the recent hackings.

Why it matters

The policy reversal signals growing recognition among policymakers that voluntary industry commitments may be insufficient to manage AI risks. With federal regulation unlikely under President Donald Trump, who has dismissed calls for slowing AI development as a "conspiracy," California's approach could set the template for state-level oversight nationwide. The timing also positions Newsom, a likely 2028 presidential contender, as a leader on AI governance at a moment of heightened public concern.

From veto to directive

In 2024, Newsom vetoed Senate Bill 1047, authored by San Francisco Democratic Sen. Scott Wiener, which would have required the largest AI developers to undergo third-party safety audits, develop kill switches, and face clearer legal liability for harms caused by their technology. The governor argued then that the bill could "restrict AI development at large companies at the potential expense of curtailing the very innovation that fuels advancement in favor of the public good."

Instead, Newsom signed a compromise measure in 2025, Senate Bill 53, requiring large AI model developers to publish safety frameworks and report critical incidents to the state. The law defines catastrophic risk as incidents involving 50 or more deaths, chemical or biological weapons, or more than $1 billion in theft or damage. Illinois and New York have enacted similar transparency requirements.

The new executive order goes further, seeking to impose regulatory controls rather than mere transparency obligations. Newsom stated that the recent incidents "should alarm every American" and pledged to "speed up our work on substantial and responsible AI oversight before it's too late."

Industry shift on oversight

Since the 2024 veto, more AI companies have embraced external evaluation. This year, Newsom signed two laws establishing standards for third-party monitors, though the legislation stopped short of mandating their use. Anthropic CEO Dario Amodei, whose recent call for an AI development slowdown gained support from several industry leaders, has advocated for third-party monitoring. Both Anthropic and OpenAI have committed to the practice.

The governor floated the possibility of a special legislative session on AI safety in an interview with Politico on Thursday, and the Nov. 16 deadline for agency recommendations could support such a session.

CalMatters first reported these details.