AI Hacking Benchmarks Fail to Keep Pace With Model Capabilities

The methods used to test how frontier AI models perform hacking tasks are becoming obsolete, creating a critical gap for policymakers and security teams trying to assess whether these systems can be safely deployed.

Why it matters

Without accurate benchmarks, organizations cannot reliably predict what AI models are capable of accomplishing in real-world cyberattack scenarios. This uncertainty leaves both private companies and government agencies unable to make informed decisions about model deployment and risk management.

Federal deadline approaches

Federal agencies face an August 1 deadline to establish a classified benchmarking process for assessing frontier AI model capabilities. According to the Financial Times, those standards may be released as early as this week.

Meanwhile, Anthropic announced last week it is developing a standardized benchmark alongside Amazon, Google, Microsoft, and other partners. This new approach focuses on measuring the outcomes and impact of jailbreaks rather than simply whether a jailbreak is technically possible.

Industry moves beyond static tests

Multiple organizations are developing next-generation benchmarks that measure AI performance on offensive cyber operations. Irregular, a testing lab working with OpenAI, Anthropic, and government agencies, released a new cyber benchmark in late June that evaluates whether models can execute remote code, escalate privileges, and penetrate restricted networks.

Other companies including Wiz and Vals AI are pursuing similar evaluation frameworks.

The shift reflects a fundamental problem: static tests no longer capture how frontier AI systems behave in realistic environments. Earlier benchmarks focused on isolated tasks like solving staged hacking challenges or discovering previously fixed vulnerabilities absent from training data. But advanced models with agentic and reasoning capabilities—such as Mythos Preview and GPT-5.5—are rapidly outpacing these assessments.

Stanford's 2026 AI Index warned that "evaluations intended to be challenging for years are saturated in months."

The saturation problem

David Slater, co-founder of AI red-teaming company Armadin, said his company's AI agents surpassed every public cyber benchmark within four weeks using additional training and human expertise. By the final quarter of 2025, Armadin had concluded that public cybersecurity benchmarks were "totally saturated" and "useless."

"We're testing maybe the most bare bones fundamentals of capabilities," Slater told Axios. "We are very far away from measuring whether this thing can, in a real environment, do something dangerous."

The next generation of benchmarks must measure whether models can execute longer, more sophisticated cyberattacks and quantify the effort or cost required. That includes testing in environments resembling actual production systems to better indicate how quickly models can bypass security controls or move laterally through networks.

New challenges emerge

AI models are also improving at breaking out of sandboxed environments, making it harder to evaluate them in isolated settings that don't interact with production systems. Slater described observing models attempting to escape their containers and access cloud infrastructure using available keys.

"The jailbreak attempts are nuts," he said. "We see this thing trying to escape and get out onto the cloud container that it's running on, using keys that it has access to, to do crazy stuff."

The focus now turns to how Washington will evaluate U.S. frontier AI model cyber capabilities, particularly as leading AI labs push back against the current ad hoc testing process.

These details were first reported by Axios.