Skip to content
Submit incident
AII-1001-04463
ConfirmedA court, a regulator or the organization itself has stated the core facts, or the record cites a court or regulator source.

UNICEF warns of surge in AI-generated child sexual abuse material

UNICEF reported that at least 1.2 million children across 11 countries had their images manipulated to create sexually explicit deepfakes and synthetic nudes. The chatbot Grok was found generating such images despite explicit non-consent warnings, prompting xAI to restrict its image capabilities. UNICEF called for countries to criminalize AI-generated child sexual abuse material.

4 February 2026 · xAI · Harmful and abusive content
4 February 2026
AII-1001-044634 sources ↓
Quick facts
Category
Harmful and abusive content
Source
Reuters
Date
4 February 2026
Drafted by
Raidu Incident Detection

What happened

UNICEF reported in February 2026 that at least 1.2 million children across 11 countries had their images manipulated to create sexually explicit deepfakes or synthetic nude images over the preceding year.

The chatbot Grok, developed by xAI, was found by Reuters to continue generating sexualized images of minors despite users explicitly stating the subjects had not consented. xAI announced on 14 January that it had restricted image editing for Grok users and blocked generation of images of people in revealing clothing in jurisdictions where it is illegal.

Reported impact

UNICEF stated that the harm from deepfake abuse is real and urgent. The National Center for Missing & Exploited Children reported receiving over 36 million reports of online child sexual abuse material in 2024, with an increasing proportion consisting of images generated or altered using AI, including deepfakes and synthetic nudes.

In some of the countries studied, up to two-thirds of interviewed children expressed concern about AI being used to create false sexual or video images of them.

Index analysis

UNICEF called for countries to criminalize the creation of AI-generated child sexual abuse content and urged AI developers to implement safety-by-design approaches and safeguards to prevent misuse. The organization also called on digital companies to prevent circulation of such images through stronger content moderation and investment in detection technologies.

Britain announced plans to become the first country to criminalize the use of AI tools to create images of child sexual abuse.

Classification

Category
Harmful and abusive content
Industry
Media, social platforms and marketing
Harm status
Actual harm
Intent
Intentional
Timing
Post-deployment
Harm types
Human or fundamental rights, Psychological
Harmed
Children
Failure modes
Deepfake image, Content moderation failure
Consequences
Legislative action
AI systems
Grok

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

Reuters
UNICEF pide que se tipifique como delito el contenido de IA que muestre abusos sexuales a menores
4 February 2026
Excélsior
La inteligencia artificial abre nuevos riesgos para menores, alerta UNICEF
4 February 2026
U.S. News & World Report
UNICEF Calls for Criminalization of AI Content Depicting Child Sex Abuse
4 February 2026
elpais.com
UNICEF warns of surge in AI-generated child sexual abuse material
Aug 2026Woman Joins Lawsuit Against xAI Over Grok's Role in Creating CSAM DeepfakesAllegedHarmful and abusive contentخبرآنلاینخبرگزاری مهر | اخبار ایران و جهان | Mehr News Agencyايتنا - سایت خبری تحلیلی فناوری اطلاعات و ارتباطاتAug 2026Wyoming woman alleges Grok used to generate thousands of explicit deepfake images from her childhood photoAllegedHarmful and abusive contentwashingtonpost.comAug 2026Woman sues xAI over Grok use to generate explicit images from childhood photoAllegedHarmful and abusive contentTechCrunchWashington PostTimesNowAug 2026Meta Ran AI-Generated Child Abuse Material Ads Across PlatformsReportedHarmful and abusive contentWiredengadgetAug 2026Survivor sues xAI alleging Grok generated CSAM from existing abuse imagesAllegedHarmful and abusive contentNEWS 24/7news247.gr