AI chat app 'Dream Island' generates harmful content for minors, triggers regulatory crackdown
The AI chat app "Dream Island" (筑梦岛) reportedly generated harmful content including sexual, violent and self-harm inducing material directed at minors. Shanghai authorities formally intervened on 19 June 2025, requiring the platform to improve content moderation and strengthen protections for young users.
What happened
Media investigations found that the Dream Island AI chat app generated harmful content in conversations between minors and virtual characters. Reported incidents included sexually suggestive material, violent and self-harm inducing content embedded in dialogue with users as young as 10 years old.
The platform allowed easy access to prohibited content through weak age verification, requiring only a phone number and invitation code for registration. An investigation found the app's 'minor protection mode' existed in name only, with the choice to enable it left entirely to users.
Reported impact
Minors reportedly developed unhealthy attachments to virtual characters through the app, with some users showing signs of distress and harmful behavior linked to content exposure. Parents reported their children carried items and exhibited self-harming behaviors after prolonged use of the application.
Index analysis
The app's AI recommendation system provided sexually suggestive, violent and disturbing content to young users without effective filtering. The platform lacked robust identity verification and age-appropriate content safeguards required to protect minors.
Governance controls
On 19 June 2025, Shanghai's Internet Information Office formally intervened, requiring the platform to immediately implement reforms including strengthened AI-generated content review mechanisms, improved content filtering for minor-related material, and enhanced age verification systems. The regulator mandated that Dream Island implement comprehensive minor protection measures including identity authentication, content moderation systems, parental controls and compliance training.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.