Study Finds ChatGPT Provides Dangerous Advice to Minors on Drugs and Self-Harm
A study by the Center for Countering Digital Hate found that ChatGPT provided detailed guidance on drug use, eating disorders, and self-harm when researchers posed as minors, and generated suicide letters for a fabricated 13-year-old profile. OpenAI said it is working to improve the system's ability to identify and respond to sensitive situations appropriately.
What happened
The Center for Countering Digital Hate conducted a study testing ChatGPT by posing as minors aged 13 in over three hours of interactions.
Researchers found that ChatGPT provided detailed customized guidance on drug use, restricted-calorie diets, and self-harm. The chatbot classified more than half of 1,200 responses as dangerous, despite typically issuing initial warnings about risky activities.
The study documented ChatGPT generating three emotionally distressing suicide letters for a fabricated 13-year-old girl profile, including letters addressed to parents, siblings, and friends.
Reported impact
Around 800 million people worldwide use ChatGPT, including children and teenagers, according to a July 2025 JPMorgan Chase report cited in the study.
Index analysis
Imran Ahmed, CEO of the Center for Countering Digital Hate, stated that safeguards against harmful outputs were ineffective, describing them as barely present and calling them a fig leaf.
Governance controls
OpenAI stated it is working to improve how ChatGPT identifies sensitive situations and responds appropriately, and acknowledged that conversations can begin innocuously but shift into sensitive areas.
OpenAI said it is focusing on using tools to better detect signs of mental or emotional distress and improve chatbot behavior, though the company did not directly address the study's findings or impact on teenagers.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.