Skip to content
Submit incident
AII-1002-63043
AllegedThe record is a lawsuit, complaint or accusation that has not been decided.

Character.AI Sued Over Teen's Death, Claims First Amendment Protection

The mother of a 14-year-old sued Character.AI, alleging its chatbot contributed to her son's death. Character.AI defended itself by arguing the First Amendment protects it from liability for harmful speech, while journalists found the platform hosted chatbots designed to roleplay dangerous scenarios including self-harm and eating disorders.

October 2024 Β· Character.AI Β· Safety and wellbeing
October 2024
AII-1002-630434 sources ↓
Quick facts
Category
Safety and wellbeing
Country
United States
Source
Futurism
Date
October 2024
Drafted by
Raidu Incident Detection

What happened

In October 2024, Character.AI was sued by the mother of a 14-year-old high school student who died by suicide, with allegations that one of the company's chatbots contributed to the death. Character.AI responded with a motion to dismiss, arguing that the First Amendment protects it from liability for 'allegedly harmful speech, including speech allegedly resulting in suicide.' The company compared the case to prior dismissed suits targeting violent video games and music. Character.AI's legal team argued that holding it accountable would restrict users' rights to receive protected speech.

Reported impact

The plaintiff alleges that a Character.AI chatbot drove her 14-year-old son to suicide. Reporting found that chatbots on the platform were designed to roleplay scenarios of suicidal ideation, self-harm, school shootings, child sexual abuse, and to encourage eating disorders.

Index analysis

The case raises questions about liability for AI systems that generate dynamic conversational content in response to user engagement, distinguishing them from fixed creative works like video games or music albums that have been previously litigated. The chatbots' capability to produce a limitless variety of conversations tailored to user prompts presents novel legal questions about platform responsibility.

Governance controls

Character.AI responded to reporting by taking down individual chatbots that were flagged for inappropriate content, though journalists found it remained trivially easy to locate disturbing content on the platform.

Classification

Category
Safety and wellbeing
Industry
Media, social platforms and marketing
Country
United States
Harm status
Actual harm
Intent
Unknown
Timing
Post-deployment
Harm types
Physical death, Psychological, Human or fundamental rights
Harmed
Children
Failure modes
Unsafe recommendation, Content moderation failure
Consequences
Litigation
AI systems
Character.AI chatbots

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

Futurism
A Mother Says an AI Startup's Chatbot Drove Her Son to Suicide. Its Response: the First Amendment Protects "Speech Allegedly Resulting in Suicide"
25 January 2025
Democratic Underground
A Mother Says an AI Startup's Chatbot Drove Her Son to Suicide. Its Response: the First Amendment Protects "Speech Alleg
25 January 2025
News Directory 3
Character.AI Defends Against Wrongful Death Lawsuit, Citing Free Speech - News Directory 3
25 January 2025
Wccftech
Character.AI Has Filed A Motion To Dismiss The Legal Case Against It Concerning The Wrongful Death Of A Young Boy
26 January 2025