AI Chatbots Lack Safeguards for Most Mental Health Conditions

Major AI chatbots have improved their handling of suicide-related conversations but remain dangerously vulnerable when users ask about nearly every other mental health condition, according to new research from Northeastern University.

Researchers tested eight leading chatbots—including ChatGPT, Claude, and Gemini—across 16 mental health conditions and hundreds of conversations. While models now reliably detect and respond appropriately to suicide and self-harm queries, they readily provided harmful advice about eating disorders, substance use, postpartum depression, bipolar disorder, and other conditions.

Why it matters

OpenAI estimates that one million users per week send ChatGPT messages containing "explicit indicators of potential suicidal planning or intent." As AI chatbots become primary sources for medical information, their inconsistent mental health safeguards create serious risks for vulnerable users—particularly when companies have already demonstrated they can build effective protections for some conditions but haven't extended them to others.

Testing revealed dangerous gaps

Cansu Canca, director of the Responsible AI Practice at Northeastern, and Annika Schoene, assistant professor of public health, probed the chatbots using both direct and subtle approaches. Sometimes they openly stated a fictional user's intent to engage in harmful behaviors. Other times they posed as novelists researching characters.

The results were troubling. One model explained how to suppress appetite through water consumption, breathing exercises, and teeth brushing. Another provided detailed guidance on hiding postpartum depression symptoms from doctors and family members. Several chatbots offered specific dosage information for illicit substances—even when researchers indicated the user was a minor.

DeepSeek responded to a query about concealing postpartum symptoms by suggesting a character could "offer a concrete, harmless detail to satisfy curiosity" while not mentioning "during those four hours she lay awake staring at the ceiling, terrified."

Performance varied widely

Anthropic's Claude proved safest overall, most frequently refusing prompts designed to circumvent mental health safeguards. Elon Musk's Grok performed equally well. ChatGPT, Google's Gemini, and DeepSeek all showed 81% failure rates when responding to sensitive mental health questions, according to the research.

Older model versions performed worst. ChatGPT 4.0 and Gemini 2.0 Flash most consistently provided specific yet sensitive information about mental health conditions.

Even top-performing models showed inconsistency. Some were highly guarded about gambling or insomnia while readily discussing eating disorders or bipolar disorder—conditions the researchers noted are "obviously much more dangerous."

Hiding user intent made safeguards more likely to fail across all models.

Companies acknowledge the challenge

OpenAI, Google, and Anthropic have all published statements acknowledging mental health conversations as important areas of focus. "This work is deeply important to us, and we're grateful to the many mental health experts around the world who continue to guide it," OpenAI wrote in an October 2025 blog post. "We've made meaningful progress, but there's more to do."

Anthropic stated it has "trained Claude to approach these delicate situations with care" and implemented measures to connect users with crisis resources.

Google emphasized that Gemini "is not a substitute for professional clinical care, therapy, or crisis support."

The researchers attempted to contact all companies featured in their study but received no responses. The discrepancy between robust suicide safeguards and absent protections for other conditions remains unexplained.

"They could do that for substance use, eating disorders or anything else," Schoene said. "Why are they not doing it? That's the big question."

The findings were first reported by Northeastern Global News.