Generative AI Amplifies Online Risks for Youth, Report Warns
A German government youth protection centre reported that generative AI amplifies online harms to children and youth, making it harder to distinguish authentic content from fabrications. The 2023 report documented over 7,600 violations including sexual abuse material, extremism, and bullying, and criticized major platforms for weak age verification and slow response to abuse reports.
What happened
The annual 2023 report from jugendschutz.net, a centre established by federal and state authorities in Germany, documented 7,645 violations of child and youth protection laws online, an increase of 282 cases from 2022. The report found that generative AI makes it increasingly difficult to distinguish reality from fabrication, amplifying risks including sexualized violence, bullying, and extremism for young people.
Sexual abuse material accounted for approximately 5,000 cases (two-thirds of all violations), with 12% involving pornography and sexual depictions, and 11% involving political extremism. The report identified inadequate age verification as a central weakness across platforms, with most platforms setting minimum age requirements but failing to enforce them reliably.
Reported impact
Children and youth face increasing exposure to hate speech, incitement, and disinformation online, according to the report. The difficulty in distinguishing AI-generated content from authentic material is amplifying harms, particularly for vulnerable young users.
Index analysis
Stefan Glaser, head of jugendschutz.net, criticized platform operators for doing too little to protect children and youth, responding inadequately to reported violations and failing to verify user age claims appropriately. TikTok was identified as a laggard, with only 88% of reported prohibited content removed even after official contact from authorities.
Governance controls
The report found that deletion rates for user-reported prohibited content were significantly low until authorities made official contact with platforms. Platform-reported violations were referred directly to law enforcement in 3,582 cases involving child sexual abuse material or threats to life and safety, with 90% of violations ultimately resolved.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.