AI Drafts Empathetic Patient Messages But Overdiagnoses, Study Finds

AI Shows Promise and Peril in Clinical Messaging

Artificial intelligence can write more compassionate patient messages than physicians, but it also tends to jump to conclusions and recommend unnecessary treatments, according to new research from Dartmouth Health.

The study examined whether large language models could help physicians manage the flood of patient portal messages that now consume hours of their evenings. Family physician Tim Burdick, who co-authored the research, faces roughly 500 messages weekly at Dartmouth Health in Lebanon, often working until 9:30 p.m. to respond. The workload contributes to widespread clinician burnout.

Burdick and Dr. Sarah Preum, technical associate director at Dartmouth's Center for Precision Health and Artificial Intelligence, trained six LLMs on more than 146,000 anonymized patient-clinician conversations. The models included commercial systems like OpenAI's ChatGPT, Anthropic's Claude 4.5 Sonnet, and Google's Gemini 2.5 Pro, plus three open-source alternatives built on Llama and Qwen AI.

Critical Gaps in Clinical Reasoning

The models struggled with fundamental aspects of medical practice. Where physicians typically ask clarifying questions—when did the pain start, rate it on a scale of one to ten—the LLMs rushed to diagnosis. They also recommended treatments more readily than human clinicians, who exercise greater caution before prescribing medication or suggesting interventions.

"Our study shows that when you look at a very granular level, like looking at what the AI wrote versus what the actual clinician wrote, there is a gap," Preum said.

The errors proved significant enough that some participating physicians told researchers they didn't want to use AI for patient communication. Previous research has shown AI-generated drafts can actually slow doctors down as they spend time correcting mistakes and filling gaps.

Where AI Excels

The technology demonstrated clear strengths in specific areas. All six models produced messages that evaluators rated as more empathetic and compassionate than typical physician responses. Preum noted that empathy represents "a low hanging fruit" for language models, which don't experience the fatigue that leads time-pressed doctors to focus narrowly on problem-solving.

The models also handled administrative tasks well, including appointment scheduling and prescription refills. Preum sees workflow support as the most promising near-term application.

Why it matters

Physician burnout driven by administrative burden is a documented crisis in American healthcare. While AI tools promise relief, this research demonstrates that deploying them prematurely in clinical communication could introduce new risks. The findings suggest that effective AI assistance will require models specifically trained to match the diagnostic caution and iterative questioning that characterize good medical practice—not just generate fluent, empathetic text.

Next Steps

Preum has developed a framework designed to prompt LLMs to ask more personalized questions and diagnose less hastily. The research team is now quantifying how much time physicians spend editing AI-generated responses and testing whether modified models can reduce overall workload.

"Can we teach it in a better way to mimic clinicians' reasoning?" Preum said. "I would say that's actually not an easy problem."

The findings were first reported by New Hampshire Public Radio.