AI Hallucination Nearly Triggered US Military Strike on China

Military aircraft abort mission after AI error discovered

US military aircraft were already in the air this spring when officials made a startling discovery: the intelligence justifying an armed operation against a Chinese vessel had been entirely fabricated by an AI chatbot. The operation was called off moments before execution, narrowly avoiding a potential international incident with China, according to CNN's reporting.

The false intelligence emerged during ongoing conflict with Iran. A Special Operations Command analyst had queried an AI chatbot to combine open-source information with classified signals intelligence. The system incorrectly identified the vessel's cargo as components for a nuclear weapons program—a critical misidentification that could have triggered military action based on false premises.

Compounding the error, the analyst used the same AI tool a second time to format the flawed findings into an official-looking intelligence summary. That document then circulated through command channels, gaining credibility as it moved up the chain before anyone questioned its accuracy.

Why it matters

The Pentagon is racing to integrate AI systems to accelerate decision-making and maintain strategic advantages over adversaries. Military leaders have publicly described AI as essential for speeding up the "kill chain"—the process from target identification to engagement. But this incident reveals a dangerous paradox: the same speed that makes AI attractive in military contexts can allow fabricated information to reach decision-makers before human oversight catches errors. When those decisions involve potential use of force, the consequences extend beyond operational failures to international conflict and loss of life.

Speed versus safeguards in military AI

The episode highlights tensions between rapid AI adoption and necessary safeguards. Jake Steckler, a research scholar at GovAI and veteran US Army officer, emphasized that service members must understand the inherent uncertainty in large language models, particularly for decisions involving force, targeting, intelligence analysis, or operational planning.

However, Steckler cautioned against abandoning AI tools entirely. "These tools can be useful in the right contexts and with the right safeguards in place," he told TechCrunch. "But prioritizing adoption speed over all else will likely lead to incidents that only make service members lose trust in these systems, which ultimately is only going to slow adoption."

The incident underscores how AI errors can travel through institutional hierarchies before being questioned—a particular concern in military contexts where time pressure and hierarchical structures may discourage challenges to intelligence assessments.

The details of this near-miss were first reported by CNN on Friday.