Meta to Alert Parents When Teens Discuss Self-Harm With AI

Meta Expands Safety Controls for Teen AI Interactions

Meta is introducing new safeguards designed to alert parents when their teenagers discuss suicide or self-harm with Meta AI, the company's conversational assistant. The features, which launched this week for Instagram users with parental supervision enabled, represent an expansion of Meta's approach to monitoring potentially dangerous conversations between minors and artificial intelligence systems.

When a teen's conversation with Meta AI contains signals suggesting they may be considering self-harm, parents who have activated Instagram's supervision tools will receive a proactive notification. These alerts will include expert-developed resources to help parents navigate difficult conversations with their children. The system relies on a dedicated AI model trained to identify concerning language patterns, even when references to self-harm are subtle or indirect.

All flagged conversations will undergo manual human review before an alert is sent to parents. Meta acknowledged this approach may generate false positives, but the company stated it prefers to err toward caution during the initial rollout. The alerts are currently available to supervising parents in the United States, United Kingdom, Australia, and Canada, with global availability planned by year-end.

Emergency Response Integration

Beyond parental notifications, Meta is building functionality to contact emergency services directly when conversations with Meta AI suggest someone—whether teen or adult—faces imminent risk of suicide. This extends Meta's existing protocol for social media posts: the company reported making more than 19,000 emergency referrals globally last year based on content flagged across Facebook and Instagram.

Clinical Review Process

Meta consulted more than 75 mental health clinicians specializing in adolescent care to refine how Meta AI responds to sensitive prompts about suicide and self-harm. These experts reviewed hundreds of conversation scenarios and provided feedback on response appropriateness, tone, and follow-up strategies.

The clinical input led to specific improvements, including ensuring Meta AI acknowledges teens' emotions before directing them to support resources and avoiding abrupt conversation termination that might discourage help-seeking behavior.

Stricter Content Controls

Meta also announced that its Limited Content setting—a more restrictive option for parents who want tighter controls on their teen's Instagram experience—now applies to Meta AI conversations. When enabled, this setting causes Meta AI to decline a broader range of prompts, reducing the likelihood of potentially inappropriate exchanges. All teen accounts default to a 13+ content setting that already restricts Meta AI from engaging in sexual, romantic, or alcohol-related conversations.

Why It Matters

As generative AI becomes embedded in social platforms used by hundreds of millions of teenagers, companies face mounting pressure to prevent harmful interactions while preserving the technology's utility. Meta's approach—combining automated detection, human review, parental notification, and clinical expertise—establishes a framework other platforms may adopt. The willingness to accept false positives in favor of safety also signals how tech companies are recalibrating risk tolerance when minors and mental health intersect with AI systems.

The features were first detailed by Meta in a company announcement on The Verge.