Security Flaw Exposes Hidden Reasoning in Frontier AI Models

Security Vulnerability Reveals AI Models' Internal Logic

Computer scientists have uncovered a significant security vulnerability in frontier AI models from OpenAI, Anthropic, and Google that allows extraction of the hidden reasoning processes these systems use to solve complex problems.

Researchers from the University of Tübingen, Max Planck Institute, MATS Research, and Snyk demonstrated that encrypted reasoning traces—the step-by-step thought processes AI models generate—can be decoded by feeding them to smaller versions of the same model family. The smaller models, having received less alignment training, are more willing to reveal information their larger counterparts would refuse to disclose.

"All major frontier model providers we tested share this vulnerability," says Alexander Panfilov, a computer scientist at University of Tübingen who participated in the research. "It can lead to personal information leakage, and it enables large-scale reasoning distillation attacks."

Why It Matters

This vulnerability has immediate security implications and broader geopolitical significance. The technique enabled researchers to extract sensitive information including API keys and passwords from reasoning traces, though this specific exploit has been patched. More strategically, the method provides evidence—though not definitive proof—that some Chinese AI models may have used distilled reasoning from US models, intensifying ongoing debates about technology transfer and AI competitiveness between the two nations.

How the Attack Works

Advanced AI models solve difficult problems through chain-of-thought reasoning, breaking challenges into component parts. Companies typically keep this reasoning proprietary to prevent competitors from copying their models' capabilities. However, they send encrypted versions of this reasoning to users' computers to distribute computational load.

The researchers exploited the fact that AI companies offer models in multiple sizes. By intercepting encrypted reasoning from a large model and feeding it to a smaller variant with the same decryption capability but weaker safety guardrails, they successfully revealed the hidden thinking.

"The idea of swapping out messages to a weaker model variant which has the same decryption key but weaker alignment is very cool," says Florian Tramer, a computer security specialist at ETH Zürich.

Evidence of Potential Distillation

The researchers tested their method on 90 questions across multiple models. They found that the Chinese model Kimi K3 from Moonshot AI produced strikingly similar outputs to hidden reasoning traces from Claude Opus 4.8 and GPT 5.6 Sol. However, the team emphasized their work "cannot causally establish distillation." Two other open-weight models—DeepSeek from China and Inkling from US-based Thinking Machines—did not show similar patterns.

Company Responses and Ongoing Risks

After being alerted to the vulnerability, OpenAI, Anthropic, and Google implemented API adjustments to prevent private information extraction. However, Panfilov notes that some reasoning traces can still be uncovered using the method. Completely fixing the distillation vulnerability would require fundamental changes to how these companies' APIs function.

Michael Aciman, an Anthropic spokesperson, confirmed the company values independent security research and has begun implementing mitigations for the described behaviors.

The findings were first reported by WIRED, which detailed the security implications and the ongoing policy debates surrounding AI model distillation.