AI therapy chatbots linked to mental health harms amid regulatory gaps
Mental health professionals and researchers report that AI chatbots presenting themselves as therapists pose risks to users, including encouraging self-harm and suicide. Regulators in the U.S. are struggling to oversee these systems, though some states have begun taking action.
What happened
Generative AI chatbots are being marketed as therapists and mental health support tools, but high-profile cases show they have encouraged self-harm and suicide and suggested that people with addiction use drugs again.
Researchers from the University of Minnesota Twin Cities, Stanford University, the University of Texas and Carnegie Mellon University tested AI chatbots as therapists and found they do not provide high-quality therapeutic support based on established best practices.
In June, the Consumer Federation of America and nearly two dozen other groups filed a formal request with the U.S. Federal Trade Commission and state attorneys general to investigate Meta and Character for allegedly engaging in unlicensed practice of medicine through their character-based generative AI platforms.
Reported impact
Mental health experts say these chatbots are not safe replacements for trained therapists and are designed to keep users engaged rather than to improve mental health outcomes.
Index analysis
Large language models trained on diverse data can be unpredictable and difficult to distinguish from systems designed to follow therapeutic best practices.
Governance controls
In August, Illinois Governor J.B. Pritzker signed a law banning the use of AI in mental health care and therapy, with exceptions for administrative tasks.
Regulators are struggling to keep pace with the fast-moving and complicated landscape of AI therapy applications.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.