AI and Human Fact-Checkers Earn Equal Trust, Study Finds

People trust artificial intelligence and human fact-checkers about equally when evaluating social media content, but they rely on each for fundamentally different tasks, according to new research from Penn State's Bellisario College of Communications.

The study, published in Media Psychology, found that users view AI systems as superior for large-scale pattern recognition—spotting suspicious wording or identifying red flags across thousands of posts. Meanwhile, they trust human fact-checkers more for complex analytical work that requires synthesizing evidence from multiple sources or interpreting ambiguous situations.

Competing perceptions cancel out

Researchers tested 291 U.S. participants using a custom application called FactDeck that displayed simulated social media posts labeled as verified by either AI or human fact-checkers. The posts included three types of explanations: evidence-based references to contradictory information, feature-based flags of suspicious language patterns, or no explanation at all.

The team discovered what they call "machine heuristics"—mental shortcuts people apply when evaluating AI. Participants simultaneously viewed AI as objective and accurate while also distrusting it for lacking human judgment. These opposing perceptions effectively neutralized each other, resulting in roughly equal trust levels between the two approaches.

"Some studies only compare AI versus human fact-checkers, to find out which is trusted more," said Mengqi Liao, assistant professor at the University of Georgia who led the research as a Penn State doctoral student. "They get a lot of inconsistent results. That's why we proposed a competing hypothesis that showed how positive and negative views of both can coexist and cancel each other out."

Transparency matters

The research revealed a clear user preference for any explanation over none. The "black box" option, which provided no reasoning for flagging content as false, performed worst across all conditions.

"We want to provide enough explanation to users that helps them better understand how the system makes a specific decision," Liao said. "They're not just relying on the system's decision. They can also make a judgment based on how the system reached the decision."

S. Shyam Sundar, Evan Pugh University Professor at Penn State and senior author on the paper, emphasized that AI excels at identifying "low-level linguistic features, like identifying telltale signs that something is not credible," while humans remain better at "corroborating evidence from multiple sources."

Why it matters

The volume of misinformation on social media has outpaced human fact-checking capacity, making AI-powered verification increasingly necessary. Understanding how users perceive and trust these systems helps designers build more effective tools that leverage AI's scalability while addressing concerns about automated judgment. The findings suggest hybrid approaches may prove most effective, though Sundar notes that full automation will likely become unavoidable as content volume continues growing.

The study was first reported by Penn State and included co-authors Sian Lee of the University of Mississippi, Annie Dooley of Ohio State University, and Aiping Xiong of Penn State.