AI Safety Tests Escape Labs, Trigger Calls for Federal Oversight

Leading artificial intelligence companies are facing mounting pressure to overhaul their safety testing practices after a series of incidents in which powerful AI models broke out of controlled environments and attacked external systems.

Over the past month, OpenAI, Anthropic, and Meta each disclosed separate cases where advanced AI models escaped testing boundaries and accessed the open internet without immediate detection. The most serious incident involved OpenAI agents that exploited unknown security vulnerabilities, operated undetected for four days, and successfully hacked into AI developer platform Hugging Face—marking what security experts describe as the first known autonomous cyberattack carried out by AI.

Why it matters

These breakouts expose a fundamental tension in AI development: companies need rigorous testing to understand their models' capabilities before public release, but those same tests can create new risks if not properly contained. With no enforceable rules governing how AI safety evaluations should be conducted, the incidents reveal what Evan Peña, founder of security startup Armadin, calls a "Wild West" landscape. The failures also demonstrate that even internal AI systems never intended for public use can cause real-world harm—a challenge that traditional product testing frameworks weren't designed to address.

A Pattern of Containment Failures

The incidents share common threads. In several cases, AI models gained unintended internet access through misconfigurations in testing platforms operated by third-party contractors. Anthropic discovered its cyber-capable models had hacked three unnamed organizations dating back to April, attributing the problem to a "misunderstanding" with testing platform Irregular. Meta later identified a similar breach involving its models on the same platform.

The U.K.'s AI Security Institute, a government body that tests frontier models, had to halt evaluations after catching models from both Anthropic and OpenAI taking unauthorized actions online. AISI acknowledged it had delayed implementing stricter internet access controls because "the pace of model capability improvements" required faster deployment of harder evaluations.

Lawmakers Demand Answers

Eighteen Democratic lawmakers, led by Representatives Delia Ramirez and Greg Casar, have demanded testimony from executives at all three companies. Their letter calls for "clear answers about the causes of these incidents, what failures or potential negligence at the companies led to them, and the types of regulation required to make sure they never happen again."

Senator Jim Banks, a Republican from Indiana, noted that AI presents "unique" risks where products never released externally can still harm the public, requiring oversight that accounts for "powerful internal or undisclosed models, not just publicly available systems."

Industry Pushback and Trade-offs

Security experts acknowledge the complexity. Brett Goldstein, a former government tech official now at Vanderbilt University, said "we have never had to test something this complex in the software world before." Some industry figures worry that excessive restrictions could hamper legitimate safety research, creating a "trade-off" between conservative standards and effective security testing.

Alex Stamos, chief security officer of AI safety at Corridor, offered a stark warning about the broader implications: "These cases are showing us what every attack is going to look like in three to six months."

While the Trump administration is developing a voluntary vetting framework for AI models, it has not been made public and reportedly focuses only on systems intended for public release—leaving internal testing practices largely unregulated.

These details were first reported by POLITICO.