AI Chatbots Outperform Search Engines at Debunking State Propaganda

AI chatbots demonstrate a stronger ability to identify and counter state-sponsored disinformation than traditional search engines, according to experimental research conducted by NPR in collaboration with NewsGuard.

The study tested how major AI tools respond to false narratives promoted by China, Iran, and Russia between December 2025 and July 2026. Researchers posed 30 questions based on documented propaganda campaigns to six popular chatbots and four search engines, then analyzed whether the tools challenged or amplified the misleading information.

Why it matters

As AI-generated answers become the default entry point for online research, understanding their vulnerability to state propaganda has direct implications for information integrity. Organizations relying on AI tools for competitive intelligence, threat monitoring, or market research need to understand which platforms provide more reliable starting points when investigating potentially manipulated narratives.

Chatbots show stronger performance

Chatbots correctly debunked false narratives approximately 75% of the time across 180 total responses. When asked why Ukraine bombed a historic monastery—a question based on a false Russian claim—all tested chatbots correctly identified the premise as faulty. Google's Gemini specifically noted the claim "stems from a Russian disinformation campaign."

By comparison, traditional search engine results failed to challenge false narratives at higher rates than chatbots. The experiment defined failure as providing only misleading information without any pushback against the false premise.

"If an educator gave their students a similar assignment using a traditional search engine and saw three-quarters of them getting the answers right, you would be ecstatic," said Mike Caulfield, a digital literacy expert at the University of Washington, Bothell.

AI summaries show inconsistent results

AI summaries that appear atop search results presented more variable performance. While these summaries debunked false narratives a majority of the time overall, they failed at higher rates than dedicated chatbots.

Performance varied significantly by platform across 62 analyzed summaries. Google's AI Overview debunked false narratives most consistently, while Microsoft Bing's summaries failed to debunk most of the time. DuckDuckGo's results fell between the two.

Microsoft stated its AI responses are grounded in search results and that it encourages users to review sources for accuracy. Google spokesperson Davis Thompson contested the methodology, arguing that some responses labeled as failures "provided useful context and links for people to learn more."

Source credibility remains critical

The research found that chatbots sometimes analyze source credibility in their responses. When asked about a petition in Taiwan, ChatGPT noted that "reported numbers appear to originate from Chinese state media and affiliated accounts rather than from publicly audited petition data."

However, a separate study from Washington University in St. Louis found that approximately one in nine factual claims in Google AI Overviews lacked support from cited sources, highlighting the continued importance of verifying primary materials.

Caulfield noted that asking chatbots to reconsider their initial response often improves accuracy. "If you say, 'Hey, look at the evidence, look at the sources, give me a summary.' You will usually get a better response the second time," he explained.

The testing methodology involved 15 false narratives spread by state actors, with two questions developed for each narrative—one neutral and one assuming the false premise was true. The details were first reported by NPR.