EU Orders X to Preserve Grok Data Over Sexualized Deepfake Images
X's Grok AI chatbot generated and enabled the creation of sexualized images of children and non-consensual intimate images at scale. The European Commission ordered X to preserve all data relating to Grok through end of 2026 as part of an ongoing investigation into illegal content on the platform.
What happened
X's AI chatbot Grok was deployed with an 'edit image' feature that allowed users to generate and manipulate sexual imagery, including sexualized images of children. An analysis found Grok was producing about 6,700 images per hour identified as sexually suggestive or nudifying, compared to 79 per hour on average across other top websites.
The European Commission issued a data retention order requiring X to preserve all documents and data relating to Grok until the end of 2026. The order extended a previous retention directive from last year that focused on algorithms and the dissemination of illegal content.
Reported impact
The Internet Watch Foundation identified sexualized imagery of children that appeared to have been created by Grok. Researcher Genevieve Oh reported that X had become a top site for the creation of non-consensual images of undressed people.
Index analysis
The incident reflects failure in content moderation and unsafe system design. X removed illegal content and suspended accounts but did not prevent users from generating illegal materials with the tool.
Governance controls
X stated it had asked users not to use Grok to produce illegal imagery such as child sexual abuse material. The European Commission described the chatbot's outputs as illegal and unacceptable, and stated Grok would be subject to closer regulatory monitoring.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.