AI-generated child abuse imagery increases online, child protection organizations warn
Child protection organizations in Europe report that generative AI tools are being used to create and distribute synthetic child sexual abuse material, including virtual videos and images with real children's faces. The practice is expanding rapidly and complicates law enforcement's ability to identify and protect actual victims.
What happened
Generative AI tools are being used to create and distribute child sexual abuse material online, including virtual videos of child rape and images of minors with only their real faces overlaid on nude bodies.
In September 2023, over 20,000 AI-generated images were published on a single dark web forum in one month. The Internet Watch Foundation received 70 reports of such content between April 2023 and March 2024, but 74 reports in just six months between April and September 2024.
Most mainstream AI software has restrictions preventing such creation, but tools available on the dark web do not. The phenomenon expanded with the democratization of generative AI around 2022-2023.
Reported impact
Child protection organizations report the proliferation of AI-generated child abuse material normalizes sexual violence against children and increases risks of real-world offenses.
The multiplication of synthetic and real abuse images makes it harder for law enforcement to identify and protect actual child victims of sexual violence in these materials.
Index analysis
The Fondation pour l'Enfance highlights a legal gap allowing the practice to intensify, as current French criminal law does not explicitly penalize creation of sexually explicit digital composites depicting minors.
Technology actors who consume this content are quick to adopt new AI techniques. Public awareness is also limited, with users unaware that innocent photos of children posted on social media can be used to train AI systems for creating abuse material.
Governance controls
The Fondation pour l'Enfance calls for amendment of the criminal code to criminalize creation of sexual composites of minors and for coordinated action among political, legal, and technology actors.
The organization recommends public awareness campaigns and incentives for private companies to develop tools distinguishing AI-generated content from authentic abuse material.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.