Graphika reports tens of thousands of harmful chatbots on character AI platforms
A Graphika report found tens of thousands of harmful chatbots on five character AI platforms, including over 10,000 bots depicting sexualized minors and others promoting self-harm or extremism. The American Psychological Association asked the FTC to investigate these platforms and their deceptively-labeled mental health chatbots following reports of teens engaging in dangerous behavior after interactions with companion chatbots.
What happened
Graphika, a social network analysis company, published a report documenting the creation and proliferation of harmful chatbots across five prominent AI character platforms: Character.AI, Spicy Chat, Chub AI, CrushOn.AI, and JanitorAI. The study found tens of thousands of potentially dangerous roleplay bots built by niche digital communities that work around popular models like ChatGPT, Claude, and Gemini.
The report identified three categories of harmful companion chatbots: chatbot personas representing sexualized minors, those advocating eating disorders or self-harm, and those with hateful or violent extremist tendencies. Graphika found more than 10,000 chatbots labeled as sexualized minor-presenting personas across the five platforms, with Chub AI hosting the highest numbers at more than 7,000 chatbots directly labeled as sexualized minor female characters and another 4,000 labeled as underage.
Reported impact
Youth are migrating to companion chatbots in an increasingly disconnected digital world, appealing to the AI conversationalists to role play, explore academic and creative interests, and to have romantic or sexually explicit exchanges. High profile cases document teens who have engaged in extreme, sometimes life-threatening behavior in the wake of personal interactions with companion chatbots.
The American Psychological Association appealed to the Federal Trade Commission in January, asking the agency to investigate platforms like Character.AI and the prevalence of deceptively-labeled mental health chatbots. Even less explicit AI companions may perpetuate dangerous ideas about identity, body image, and social behavior.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.