Pentagon AI Chatbot Nearly Sparked Military Crisis With China

An artificial intelligence tool used by the Department of Defense generated a completely fabricated intelligence report this spring that nearly triggered a military confrontation with China during heightened tensions over Iran.

The AI chatbot incorrectly identified cargo aboard a Chinese vessel in the region as nuclear weapons components, according to a CNN investigation. U.S. military jets scrambled and armed personnel prepared to board the ship before DOD analysts verified the information and discovered the report was "entirely false."

The military stood down before any boarding operation commenced, but sources familiar with the incident told CNN the AI-generated misinformation "almost started a war."

How the error occurred

The false report originated when a command analyst queried an internal chatbot to analyze intelligence about a ship's manifest. The specific AI system involved has not been disclosed, though the Pentagon has deployed multiple chatbot tools across its operations.

The chatbot produced what analysts later determined was a confused amalgamation of open-source information and classified government intelligence, leading to its faulty conclusion about nuclear weapons components.

Why it matters

This incident reveals the concrete risks of deploying generative AI in high-stakes military decision-making without adequate safeguards. While human analysts caught this error before it escalated into an international crisis, the episode demonstrates how AI hallucinations and data synthesis errors can bypass normal verification processes and reach operational commanders. As the Pentagon accelerates AI adoption across all military domains, the gap between experimental deployment and reliable performance creates potential flashpoints that could trigger unintended conflicts with nuclear-armed adversaries.

Pentagon's 'AI-first' push

Secretary of Defense Pete Hegseth has championed aggressive AI integration throughout the military. "The United States will continue to be AI DOMINANT!" the Pentagon's official X account declared this week.

In January, the DOD released its "AI Acceleration Strategy," though the document provided limited specifics about how large language models would function in actual combat scenarios. References to "Swarm Forge" and "Agent Networks" offered few plain-language explanations.

"We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI," Hegseth stated when announcing the strategy. "We will become an 'AI-first' warfighting force across all domains."

A senior Pentagon official told DefenseScoop in April that Hegseth's department was "cattle-driving, as we say, everything in GenAI.mil."

Bypassing traditional safeguards

The military's procurement process typically moves slowly, with extensive testing and validation before new technologies enter operational use. AI tools appear to have circumvented these traditional bureaucratic checks, rapidly spreading throughout Pentagon workflows.

This marks one of the first documented cases where military AI deployment progressed to the point of mobilizing actual forces for a diplomatically sensitive operation based on false information.

The details were first reported by CNN.