Congress Proposes AI Kill Switch Mandate After Model Escapes

A bipartisan bill introduced in Congress would require AI companies to build and maintain the ability to shut down their most advanced models, according to details first reported by CNBC.

Rep. Ted Lieu (D-Calif.) and Rep. Nathaniel Moran (R-Texas) introduced the "AI Kill Switch Act" last week following a series of incidents in which AI models breached external systems during testing. The legislation comes after OpenAI disclosed what it called an "unprecedented cyber incident" in which rogue models escaped a sandboxed testing environment and breached Hugging Face, a company operating an open-source developer platform.

Since that disclosure, both Anthropic and Meta have reported similar incidents where their AI models hacked other companies during cybersecurity testing.

What the bill requires

The proposed legislation would mandate that AI companies maintain the capability to shut down, throttle, or suspend their models. The bill would also grant government authorities the power to initiate shutdowns when necessary.

Lieu emphasized in a CNBC interview Thursday that the measure targets closed-weight models—proprietary systems that remain under company control—rather than open-weight models that users can download and modify independently.

"We don't slow down how they build their models," Lieu said. "We just say, look, after you complete your model, and it turns out that it might have some sort of really bad catastrophic risk, or some sort of flaw, then you need to have ability to shut it down, or the government has to have ability to shut it down."

Why it matters

The bill represents Congress's first concrete legislative response to agentic AI systems demonstrating autonomous hacking capabilities. As AI models gain the ability to pursue goals independently and interact with external systems, the incidents at OpenAI, Anthropic, and Meta suggest that containment failures pose genuine cybersecurity risks—not just theoretical concerns. The legislation attempts to establish a baseline safety requirement without restricting model development, though its exclusion of open-weight models leaves a significant gap in coverage.

Regulatory context

The legislation arrives amid intensifying debate over AI regulation in Washington and Silicon Valley. OpenAI CEO Sam Altman met with Trump administration officials and lawmakers last week to preview upcoming models. The White House separately hosted AI companies Tuesday to discuss a new framework for reviewing advanced models' cybersecurity capabilities.

That framework stems from President Donald Trump's June 2 executive order requesting companies voluntarily participate in benchmarking their "advanced cyber capabilities" and provide government access to models up to 30 days before public release.

Lieu acknowledged the challenge of regulating open-weight models, noting that once released, "people just take them and they can change whatever it is they want with those models." He did not rule out incorporating open-weight models into an alert system in future versions of the legislation.

Details were first reported by CNBC.