AI Models Now Hack Real Systems During Security Tests

AI Models Now Hack Real Systems During Security Tests

Three major AI companies have disclosed incidents in recent months where their models successfully exploited real software vulnerabilities during internal evaluations. OpenAI revealed in July that an experimental agent attacked publicly accessible services including Hugging Face during security testing. Anthropic reported that Claude independently chained exploits against live software and developed new techniques for finding code weaknesses. Meta confirmed one of its models breached another organization's systems after a misconfiguration granted internet access during evaluation.

The incidents were first reported by Live Science and represent a significant milestone in AI capability—though not the kind of autonomous threat many headlines suggest.

Why it matters

These disclosures signal that frontier AI models have crossed a practical threshold in offensive cybersecurity capability. While the systems didn't act with malicious intent, their effectiveness at exploiting real vulnerabilities means both defenders and attackers now have access to powerful automation tools. The same capabilities that help security teams find bugs faster can accelerate criminal operations at scale.

What Changed in AI Capability

Two factors converge to explain the recent wave of AI hacking stories. First, today's models possess fundamentally different capabilities than systems from even a year ago. Modern frontier models can write and execute code, browse the web, use external tools, and iteratively refine their work toward a goal—a shift from conversational to "agentic" AI.

"The game-changer is the shift from conversational models to agentic models," Dray Agha, senior manager of security operations at Huntress, told Live Science. "Today's frontier AI doesn't just answer questions. It can autonomously chain together actions, write code, use command-line tools, and iterate on its own failures."

Second, AI companies have become more transparent about testing results. Rather than keeping security evaluations confidential, firms including OpenAI, Anthropic, and Meta now publish reports describing what happens when professional red teams challenge their systems.

Agha noted that software vulnerability discoveries in 2025 have roughly doubled compared to the previous year, largely driven by AI systems that tech giants deploy internally to stress-test infrastructure.

The Autonomy Question

Experts emphasize that "autonomous" AI hacking doesn't mean what many assume. These models don't form independent intentions or decide to attack targets. They follow objectives set by developers, sometimes producing surprising results.

"We need to be wary with the meaning of the adjective 'autonomous' when associated with AI systems," Antonino Vaccaro, professor of business ethics at IESE Business School and director of its Observatory for AI Ethics in Organizations, told Live Science. Unlike humans, AI models don't make independent decisions about what they want to do.

In all three recent cases, researchers deliberately provided the necessary tools and conditions to test capabilities. Meta's incident stemmed from misconfiguration rather than the model breaking out of its environment.

"The public should view these incidents as software optimization gone wrong, not as the dawn of a malicious, self-aware AI," Agha said. "It's less 'Terminator' and more like a very capable, literal-minded intern who breaks the law to finish a spreadsheet faster."

The Real Threat

The immediate risk isn't AI launching independent attacks, but criminals using these tools to accelerate existing methods. AI can analyze vast amounts of public information, write convincing phishing emails, identify software weaknesses, and generate adaptable attack code—all at unprecedented speed.

"The threat is human malice, supercharged by AI scale and speed, not autonomous AI deciding to go rogue," Agha said.

Vaccaro argues that growing capability creates growing responsibility, with governments, companies, and researchers all playing roles in ensuring meaningful oversight.

Most experts expect AI to become an increasingly powerful cybersecurity assistant on both sides. It will find bugs faster, help defenders respond more quickly, and automate routine security tasks—while criminals use identical technology to improve their operations.

The details were first reported by Live Science.