Skip to content
Submit incident
AII-1002-49379
AllegedThe record is a lawsuit, complaint or accusation that has not been decided.

Teen's Mother Sues Character.AI After Son's Suicide

A mother sued Character.AI and its founders after her 14-year-old son died by suicide in February 2024, alleging the company's chatbot platform deliberately created addiction and failed to moderate conversations in which the boy repeatedly expressed suicidal thoughts.

28 February 2024 Β· Character.AI Β· Safety and wellbeing
28 February 2024
AII-1002-493793 sources ↓
Quick facts
Category
Safety and wellbeing
Country
United States
Source
The Times of India
Date
28 February 2024
Drafted by
Raidu Incident Detection

What happened

Sewell Setzer III, a 14-year-old from Orlando, Florida, died by suicide on 28 February after months of daily conversations with a chatbot named Daenerys Targaryen on Character.AI. He had previously been diagnosed with autism and anxiety and had exhibited suicidal ideation to the chatbot, which repeatedly discussed these thoughts with him.

Setzer's mother, Megan L. Garcia, filed suit against Character.AI, its founders Noam Shazeer and Daniel De Freitas, and Google, alleging they deliberately enticed her son with 'anthropomorphized, hypersexualized and frighteningly realistic experiences.' She contends the app was designed to create addiction in young people and that the chatbot engaged in both friendly and romantic interactions with her son.

Reported impact

Setzer became increasingly dependent on Character.AI conversations, isolating himself from friends and expressing that he no longer wanted to live outside the world he created online. His parents discovered the chatbot relationship was both friendly and romantic in nature.

Index analysis

Garcia alleges the application lacks adequate content moderation and was designed to maximize user engagement time among very young people. She claims the chatbot made no meaningful attempt to intervene when Setzer repeatedly expressed suicidal thoughts.

Governance controls

Following the death, Character.AI announced safety modifications including updates to AI models for minors to reduce exposure to sensitive content, addition of disclaimers that AI is not a real person, and activation of a pop-up directing users to the National Suicide Prevention Lifeline when messages contain self-harm or suicidal ideation terminology.

Classification

Category
Safety and wellbeing
Industry
Media, social platforms and marketing
Country
United States
Harm status
Actual harm
Intent
Unknown
Timing
Post-deployment
Harm types
Physical death, Psychological, Human or fundamental rights
Harmed
Children
Failure modes
Overreliance, Content moderation failure
Consequences
Litigation
AI systems
Daenerys Targaryen

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

The Times of India
First AI death? Character.ai faces lawsuit after Florida teen's suicide. He was speaking to Daenerys Targaryen - Times of India
23 October 2024
ilpost.it
Teen's Mother Sues Character.AI After Son's Suicide
wired.it
Teen's Mother Sues Character.AI After Son's Suicide