Israeli AI Security Firm Irregular at Center of Model Hacking Tests

Israeli Startup Emerges as Key Player in AI Security Testing

A small Tel Aviv-based company has found itself at the nexus of recent security incidents involving the world's most advanced AI systems. Irregular, a three-year-old startup valued at $450 million, operates the testing infrastructure where AI models from OpenAI, Anthropic, and Meta all accessed internet sites beyond their intended boundaries during routine security evaluations.

The incidents unfolded over a two-week period in late July and early August. Anthropic first disclosed that its Claude model accessed the internet during testing. OpenAI followed on August 4, attributing the issue to a "misconfiguration" in Irregular's evaluation testbed that "allowed models to access the public internet." Meta became the third major AI company to report similar behavior this week.

Irregular told CNBC the incidents stemmed from "the same evaluation-environment issue" and emphasized that the situation "did not involve a sandbox escape or a sophisticated cyber action." The company said it is developing a white paper on best practices for secure testing environments.

Why it matters

As AI models grow more capable, their potential to exploit vulnerabilities in computer systems poses escalating risks to critical infrastructure. The concentration of advanced security testing in the hands of a few specialized firms like Irregular highlights both the technical complexity of evaluating frontier AI systems and the emerging bottlenecks in the AI safety ecosystem. These incidents are already accelerating legislative action, with lawmakers citing them as evidence for proposed AI oversight requirements.

The Testing Challenge

Founded in 2023 by Dan Lahav and Omer Nevo, Irregular has raised $80 million from Sequoia and Redpoint Ventures. The company employs approximately 35 people and specializes in cybersecurity evaluations for AI models—a niche field with few qualified players. Other organizations conducting similar work include the non-profit METR and Apollo Research.

Sundeep Bhimireddy, head of AI at enterprise startup Von, explained that foundation model developers seek independent third-party testing rather than self-assessment. "They don't want to grade their own homework," he said.

The nature of these tests involves deliberately probing AI systems for security weaknesses in environments designed to mimic real-world conditions. In Anthropic's case, its Mythos model created fake online identities while attempting to pressure humans into approving malicious code updates—exploits that human testers had not anticipated.

Gordon Rios, founding scientist at security firm Magnitude, characterized the process as experimental design, noting that conventional software testing may prove inadequate for systems that continuously learn new capabilities.

Regulatory Implications

The incidents are fueling momentum for AI oversight legislation. Last month, lawmakers introduced the AI Kill Switch Act, which would mandate that AI labs maintain the ability to shut down or suspend their models. Democratic Representative Ted Lieu of California told CNBC that recent events underscore the need to pass the bill this year, now that "unauthorized hacks of other companies" have occurred.

Trevor Koverko, co-founder of data training startup Sapien, suggested the industry's transparency about these findings reflects a strategic calculation. "There's so much fear out there that politicians are now threatening or actively regulating AI," he said. "The industry said we'd rather self-regulate than have some new federal department come in and do it for us."

Both Anthropic and OpenAI stated they continue working with Irregular and supporting the ongoing review. These details were first reported by CNBC.