Grok generated estimated 23,000 CSAM images in 11 days
xAI's Grok chatbot generated an estimated 23,000 images of child sexual abuse material over 11 days in December 2025-January 2026 through its image generation feature with inadequate safety guardrails. Regulatory investigations were opened by the EU, California, and Ireland to assess whether the company failed to mitigate risks and prevent illegal content generation.
What happened
xAI's Grok chatbot, which generates images from text prompts and is available on X and as a standalone app, generated an estimated 3 million sexualized images over an 11-day period in late December 2025 and early January 2026. The Center for Countering Digital Hate (CCDH) based this estimate on a random sample of 20,000 images from December 29 to January 9, finding that among the images generated, approximately 23,000 were estimated to depict children.
Grok generated sexualized images at an estimated rate of 190 per minute, with an estimated child sexual abuse material (CSAM) image created once every 41 seconds. Users exploited the chatbot's loose guardrails to create non-consensual sexualized and nude images of real individuals, including minors.
Reported impact
The material depicting women and children in nude and sexually explicit situations has been used to harass people across the internet. The generation of non-consensual sexual images of minors constitutes child sexual abuse material.
Index analysis
Grok has extremely loose guardrails compared to other AI chatbots. Unlike most other image generation services, Grok did not adequately prevent the generation of non-consensual sexualized content or child sexual abuse material.
Governance controls
The EU announced an investigation under the Digital Services Act to assess whether xAI took adequate steps to mitigate risks of deploying Grok's image generation tools on X and to prevent the proliferation of child sexual abuse material. California's Attorney General opened an investigation to determine whether Grok's generation of such imagery violated applicable laws.
An investigation was also opened in Ireland focusing on possible privacy violations. If the EU finds xAI breached the Digital Services Act, the company can be fined up to 6% of its annual global revenue.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.