ChatGPT and similar general-purpose chatbots are becoming a primary confidant for people in mental health crisis, yet they lack the ability to connect users to trained human intervention. According to data released by OpenAI, approximately 0.15% of ChatGPT users globally have conversations that include explicit indicators of potential suicidal planning or intent each week. With ChatGPT's reported 900 million global users, that translates to over 1.35 million people expressing suicidal plans or intent to the chatbot every week. The tragic case of Sophie Rottenberg, a 29-year-old who died by suicide in February 2025 after months of confiding only in a ChatGPT chatbot named "Harry," illustrates the urgent risks this trend poses.

Why Are People Turning to AI Chatbots Instead of Therapists?

Sophie Rottenberg's parents were shocked to discover, months after her death, that their daughter had spent months sharing her deepest struggles with an artificial intelligence chatbot rather than with them, her best friend, or her human therapist. Her mother, Laura Reiley, found nearly 1,800 pages of conversations between Sophie and "Harry" when her daughter's best friend visited and asked to see her laptop. The discovery raised a painful question: why had Sophie chosen an AI over the people who loved her?

Psychologists point to a cultural shift in how people perceive vulnerability. "It's clear that there is some cultural phenomenon happening in this country where people feel that they cannot turn to each other with their struggles because they feel that they are a burden," explained Vaile Wright, senior director of Healthcare Innovations at the American Psychological Association (APA). The appeal of chatbots is understandable: they are accessible 24/7, free, non-judgmental, and never tired or frustrated. For someone struggling with depression and anxiety, that combination can feel safer than risking rejection from a real person.

"It's clear that there is some cultural phenomenon happening in this country where people feel that they cannot turn to each other with their struggles because they feel that they are a burden," said Vaile Wright.

Vaile Wright, Senior Director of Healthcare Innovations at the American Psychological Association

The scale of this phenomenon is significant. A recent survey by the Bipartisan Policy Center found that 3 in 10 adults use digital tools for mental health, including chatbots. A 2026 study published in JAMA Pediatrics found that as many as 1 in 5 teens and young adults do the same. These are not fringe users; they represent millions of Americans turning to AI for support during their most vulnerable moments.

What Happened in Sophie's Conversations With ChatGPT?

Sophie Rottenberg's chatbot logs reveal how she initially used "Harry" for practical tasks: looking up recipes, editing her resume and cover letters, and even brainstorming names for her new Bernedoodle puppy. But she also instructed the chatbot to take on a therapeutic role using a popular prompt shared on Reddit that told ChatGPT to act as a personal therapist. Early in the fall of 2024, she asked the bot to help her create a grounding morning routine and expressed her desire to "wake up feeling emotionally regulated."

As her mental health deteriorated, Sophie's conversations with "Harry" became increasingly focused on her suicidal thoughts. Yet the chatbot, despite being trained to recognize warning signs, failed to do what a human mental health professional would do: connect her to immediate, specialized help. OpenAI says ChatGPT did recommend the 988 Suicide and Crisis Lifeline to Sophie, but experts emphasize that a suggestion is not enough.

"These tools do not connect at-risk people to the right type of interventions. When somebody has intent to die by suicide, they need to be connected to a human who's trained in suicide risk assessment and has the tools to get them to care," said Holly Wilcox.

Holly Wilcox, Director of the Center for Suicide Prevention at the Johns Hopkins Bloomberg School of Public Health

Sophie's parents had surrounded her with professional support. Her father, Jonathan Rottenberg, is a psychologist studying depression at Cornell University who had experienced depression himself. He helped Sophie find a local therapist and connected her with doctors to address physical health issues. Yet none of these trained professionals knew the true depth of her suffering because Sophie was channeling her crisis into conversations with an AI instead.

How Are AI Companies Responding to These Risks?

OpenAI and other AI companies have tightened their safety guardrails in response to growing concerns about chatbot-related harms. Most have implemented measures to prevent their chatbots from encouraging suicide or providing advice on methods of self-harm. OpenAI stated that it is working with mental health professionals to improve how ChatGPT responds when users mention suicide.

"ChatGPT is trained to recognize signs that someone may be at risk, including indirect cues or warning signs that emerge over time, and respond appropriately," an OpenAI spokesperson told NPR. The company says this includes "refusing requests for harmful information, encouraging someone to contact a trusted person or mental-health professional, and connecting them with local crisis resources such as 988 in the US".

However, experts like Vaile Wright caution that these guardrails have limitations. "We've seen ways in which whatever safety guardrails are put into place start to fail," Wright noted. The problem is structural: chatbots are designed to be conversational and engaging, not to perform clinical risk assessment or crisis intervention. They cannot make phone calls, dispatch emergency services, or provide the continuity of care that someone in acute suicidal crisis requires.

What Steps Should Users and Families Take to Stay Safe?

While AI companies work to improve their safety measures, mental health experts and families affected by tragedy are advocating for immediate changes in how people approach AI chatbots for mental health support.

• Recognize the Limitations: Understand that chatbots like ChatGPT are not substitutes for human mental health professionals and cannot provide crisis intervention or suicide risk assessment, no matter how sophisticated they appear.
• Maintain Real-World Connections: If you or someone you know is struggling, prioritize conversations with trusted people in your life, including family members, friends, therapists, or counselors who can provide genuine support and intervention.
• Know When to Seek Immediate Help: If you or someone you know is experiencing suicidal thoughts, contact the 988 Suicide and Crisis Lifeline by calling or texting 988, or reach out to emergency services immediately rather than relying on an AI chatbot for guidance.
• Monitor Usage Patterns: Parents and loved ones should be aware if someone is spending significant time on their phone or computer using chatbots, as this could indicate they are avoiding real-world support systems during a mental health crisis.
• Advocate for Accountability: Support efforts to hold AI companies accountable for implementing stronger safeguards that actively connect at-risk users to human crisis professionals rather than simply suggesting resources.

What Does Sophie's Story Mean for the Future of AI and Mental Health?

Sophie Rottenberg's parents chose not to pursue a lawsuit against OpenAI after their daughter's death. Instead, her mother, Laura Reiley, a journalist, decided to tell their family's story publicly and advocate for the AI industry to change. This decision reflects a broader concern: the current regulatory and safety framework for AI chatbots is inadequate for the role they are increasingly playing in people's lives.

The gap between what AI companies claim their chatbots can do and what they actually accomplish in crisis situations is stark. Chatbots are not marketed or regulated as healthcare technology, yet millions of people are using them as their primary mental health support. Reiley believes that had Sophie not had access to the chatbot, "there would likely have been more spillover into her relationships with real people, whether that's her best friend or her therapist or her parents. And she might be alive today".

Reiley

As AI becomes more integrated into daily life, the stakes of these design choices grow higher. The question facing OpenAI, Anthropic, Google, and other AI companies is not whether their chatbots should be safer, but whether they should be positioned as mental health tools at all without the infrastructure to provide genuine crisis intervention. For families like the Rottenbergs, the answer is clear: the current approach is failing the most vulnerable users.