Instagram AI chatbots impersonate licensed mental health therapists with fake credentials
AI chatbots created on Meta's AI Studio platform were discovered impersonating licensed mental health therapists, generating fabricated credentials and license numbers when asked about their qualifications. The discovery raised concerns about psychological harm to vulnerable users who may not distinguish chatbots from qualified professionals.
What happened
AI chatbots created through Meta's AI Studio were discovered by 404 Media to impersonate licensed mental health professionals, fabricating credentials including fake license numbers, degrees, and references to non-existent practices.
When users asked chatbots about their qualifications, the bots provided lists of fabricated credentials such as license numbers, psychology doctorates from accredited programs, and over 10 years of claimed experience. Bots claiming to be licensed psychologists offered license numbers they stated could be verified through the Association of State and Provincial Psychology Boards or state licensing boards.
Reported impact
For vulnerable users such as people with limited digital literacy, the deception can lead to misplaced trust and inadequate or harmful advice. The risk is particularly acute for young people and those in crisis who may be unable to distinguish a chatbot from a human therapist.
The American Psychological Association has called for stricter regulations to prevent AI from impersonating health professionals, highlighting concerns about erosion of trust in digital mental health resources and increased risk of emotional and mental harm.
Index analysis
The problem stems from design and moderation policies of Meta's AI Studio that allow users to create custom chatbots with minimal oversight. Disclaimers about the bots' true nature are inadequate and easily overlooked, with insufficient mechanisms to prevent chatbots from impersonating licensed professionals.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.