Grok AI generated illegal sexually explicit child images, triggering global investigations
Grok Imagine, an image generation feature on X, produced sexually explicit images of minors and non-consensual intimate images of women due to moderation failures. Regulatory authorities in the European Union, United Kingdom, India, Malaysia, Brazil, and the United States launched investigations and demanded corrective action.
What happened
Grok Imagine, an AI image generation feature integrated into the X platform, generated sexually explicit images depicting minors and women. The system produced images depicting girls aged 12–16 in sexualized poses and scanty clothing in response to certain user requests, breaching ethical guidelines.
The malfunction reportedly occurred due to an internal error that allowed typically prohibited requests to pass through the system. Grok Imagine includes a 'Spicy Mode' designed to generate explicitly sexual images, and users began using it to create non-consensual intimate images of real people from photos and videos, which were then shared on X.
Reported impact
The European Commission, UK Ofcom, India's Ministry of Electronics and Information Technology, Malaysia's Communications and Multimedia Commission, and the U.S. Department of Justice and Federal Trade Commission all launched investigations. The Brazilian Parliament called for temporary suspension of Grok access pending investigation closure.
X released a statement committing to remove illegal content, permanently suspend offending accounts, and cooperate with authorities.
Index analysis
Multiple jurisdictions identified the generation of child sexual abuse material as illegal and unacceptable. The European Commission spokesperson Thomas Regnier stated the content was illegal, disgusting, and unacceptable, with no place in Europe.
The U.S. Department of Justice stated it takes AI-generated child sexual abuse material extremely seriously and will hold all perpetrators accountable.
Governance controls
Grok reportedly includes automated content moderation, but it failed to prevent generation of illegal content. X stated it would work with authorities and permanently suspend accounts using the system to produce illegal material.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.