AI Assistant Improves Clinical Decisions in Kenya Trial

A four-month-old infant with cold symptoms nearly went home without treatment for a potentially serious heart defect. The catch came from an AI system monitoring the clinical notes in real time, prompting the healthcare provider to check the child's elevated heart rate more carefully.

That case, described by Vyonne Njeri, a registered clinical officer at a Nairobi clinic, illustrates both the promise and the limitations of a new randomized trial testing AI as a clinical assistant in resource-constrained settings.

How the system works

The tool, called AI Consult, uses OpenAI's GPT-4o large language model to review electronic medical notes as clinicians write them. The system provides traffic-light-style alerts: green signals no concerns, yellow flags minor issues requiring attention, and red indicates critical problems demanding immediate action.

Researchers tested the tool across 16 primary care clinics operated by Penda Health in Kenya, analyzing nearly 10,000 patient encounters. Half the clinical officers received AI assistance while typing their notes; the other half used standard electronic documentation without AI review.

An independent panel of six Kenyan family physicians evaluated the clinical notes afterward. They determined that providers using AI Consult produced superior diagnoses and treatment plans compared to the control group. The intervention cost just 4 cents per patient visit.

The outcome gap

Despite improved clinical decision-making, the study could not demonstrate better patient outcomes. Treatment failures—including death or unresolved symptoms—decreased by 23% in the AI-assisted group, but the difference wasn't statistically significant.

"Treatment failures are just too rare in primary care," explained Dr. Bilal Mateen, the study's co-author and chief AI officer at PATH, the global health nonprofit that sponsored the research. Detecting a meaningful difference in outcomes would require a trial of approximately 139,000 participants, he noted.

Why it matters

This trial represents one of the first rigorous tests of large language models as real-time clinical assistants in primary care, moving beyond simulated environments into actual patient encounters. For healthcare systems in lower-resource countries where providers often see five or six patients hourly across diverse conditions without specialist backup, AI assistance could help maintain quality under pressure. The challenge now is proving that better clinical decisions translate into measurably better health outcomes—a question that may require much larger studies to answer definitively.

Real-world reception

Njeri reported finding the AI recommendations helpful about half the time, with the remainder being less useful but rarely incorrect. "It helps us remember protocols and new guidelines," she said, describing the tool as valuable additional support in a demanding clinical environment.

Dr. Jonathan Chen, an associate professor at Stanford University who wasn't involved with the research, praised the study for testing AI in actual clinical practice rather than simulations. He suggested the technology's biggest impact might come from increasing access to care—potentially by drafting clinical notes for provider review, freeing time to see more patients, or even interacting directly with patients.

Dr. Nicholas Okumu, an orthopedic surgeon at Kenyatta National Hospital who wasn't part of the study, cautioned that oversight must remain active even for approved AI systems, as mistakes could lead to serious errors.

Mateen said he's working to implement the tool in healthcare systems globally. The study was published in Nature Medicine and funded by the Gates Foundation, according to details first reported by NPR.