Doctors warn against using ChatGPT for medical diagnoses

Healthcare providers adopt AI cautiously

Health systems in Minnesota are integrating artificial intelligence into clinical workflows while drawing firm boundaries around its appropriate use. OpenAI recently launched ChatGPT Health, which allows patients to connect medical records and health data to the chatbot for personalized queries. But physicians say the technology's confident tone masks fundamental limitations.

"AI can sound super confident; it can understand results well, but it doesn't understand the patient context," said Dr. David Kunstman, a family physician at Essentia Health, according to Northern News Now.

The distinction matters because generative AI tools like ChatGPT synthesize patterns from training data rather than applying clinical judgment to individual circumstances. Arshia Khan, a University of Minnesota Duluth professor whose research focuses on assistive technology and robotics, put it bluntly: "ChatGPT is not a physician, it has not learned as much as a doctor in a specialized setting has learned. ChatGPT is essentially regurgitating what it has digested."

AI proves valuable for administrative tasks

Despite reservations about patient-facing diagnostic tools, Essentia clinicians have embraced AI for back-office functions. For several years, doctors have used AI-powered draft responses to patient messages submitted through Epic's MyChart app. The system incorporates patient medical history to generate preliminary replies that clinicians then review and refine.

More recently, Essentia deployed ambient documentation tools that listen to patient conversations and automatically extract relevant clinical details for medical notes. Physicians must review every word for accuracy, but the time savings are substantial—eliminating at least an hour of daily paperwork and freeing doctors to focus on patient interaction.

Patient resistance has been minimal. Only three patients have opted out of AI-assisted documentation in two years of use at Essentia.

Data security and diagnostic risks

The bigger concern involves patients using consumer AI tools for self-diagnosis. Clinicians report a surge in patients arriving with ChatGPT-generated assessments, often having shared sensitive health information with systems that lack clear data governance.

"Who controls the data once you provide it to the large language models such as ChatGPT? And what are they going to use it for?" Dr. Kunstman asked.

Khan warned against allowing AI to displace human expertise: "If we go down that slope, and if we start replacing our experts and our skill sets with AI, it is going to really come back and haunt us later on."

Why it matters

The gap between AI's administrative utility and its diagnostic limitations reveals a critical challenge for healthcare technology adoption. While automation can reduce physician burnout by handling routine documentation, the same confident-sounding systems may mislead patients who lack medical training to evaluate AI-generated health advice. The distinction between tool and replacement will shape both patient safety and the future scope of clinical practice.

Medical professionals recommend using AI only to prepare questions before in-person appointments, keeping human clinicians central to diagnosis and care decisions.

These details were first reported by Northern News Now.