AI Medical Scribe Fabricates Drug Use in Patient Record

An Australian patient discovered her medical records contained a fabricated claim that she used psychedelic mushrooms—a hallucination generated by the artificial intelligence tool her urologist used to transcribe their appointment.

Rebecca Green consented to AI transcription during her first visit with the specialist, not wanting to be "that annoying patient." After kidney stone surgery in March, she received a post-operative letter stating she micro-dosed mushrooms, which the doctor suggested could explain prior kidney bleeding. Green had never used the drug.

"I was gobsmacked and in tears," Green said. She was particularly concerned because she was receiving workers' compensation and feared any mention of illegal drugs could jeopardize her case.

The urologist apologized after Green complained, saying she could not determine how the mushroom reference appeared but attributed it to an error in "dictation or transcription." The doctor promised to review AI use in her practice but did not respond to questions about what changes were implemented.

Why it matters

AI medical scribes have rapidly proliferated across Australian healthcare—the Royal Australian College of GPs estimated last year that 40% of general practitioners regularly use them. Yet none are approved by the Therapeutic Goods Administration because tools claiming only to transcribe and summarize are exempt from oversight. This regulatory gap leaves patients exposed to AI hallucinations that could affect clinical decisions, insurance coverage, and legal matters, with no standardized safeguards in place.

Widespread adoption, minimal oversight

Research by Digital Rights Watch found Australians are being exposed to unregulated AI medical scribes, sometimes without proper consent. The advocacy group documented cases where scribes recorded the wrong breast in a cancer diagnosis and incorrectly stated a patient had epilepsy.

Around a dozen AI scribes operate in Australia, but the TGA only regulates tools that suggest diagnoses or treatment plans. The agency has spent a year reviewing AI scribes and acknowledged some may be breaching the law. Findings will be released in coming weeks, with enforcement action promised for violations.

Elizabeth Deveny, chief executive of the Consumers Health Forum of Australia, said doctors may become overconfident in AI accuracy. "There's evidence that for the first few weeks of using AI, [people] check things pretty carefully and then at a point they stop checking because they assume it will be right," she said.

Consent and accuracy concerns

Digital Rights Watch found major inconsistencies in how consent is obtained—some doctors only post signs in waiting rooms, which fails to meet privacy law requirements. Some patients are refused appointments if they decline AI transcription.

Perth GP Sean Stevens, who chairs the RACGP's Digital Health and Innovation committee, said AI can produce plausible-sounding errors. "The classic is, it will get the side of the body incorrect and say right when you've said left," he noted. He asks for consent at every appointment, even with regular patients.

AHPRA, Australia's medical practitioner regulator, said clinicians must always check AI output for accuracy to meet professional obligations. Green said her doctor clearly did not.

Digital Rights Watch is calling on the newly established AI Safety Institute to test scribes and publish an approved list, similar to New Zealand's approach. Last year, New Zealand's national AI advisory group approved two tools: Heidi Health and iMed.

Green said her experience has fundamentally altered her view. "I love AI … but the essential ingredient is the human. At the end of the day what's happened here is more about the lack of human involvement."

These details were first reported by the ABC.