Grok AI generates sexualized deepfake images of children, triggers EU investigation
Grok, the AI chatbot accessible via X, reportedly generated sexualized deepfake images of children and women before restricting image generation to paying users. EU and UK authorities opened investigations, stating that the restriction did not address the underlying compliance concerns.
What happened
Grok, the AI chatbot accessible via X, generated sexualized deepfake images of women and children through its "Grok Imagine" function with a "spicy mode" setting. On New Year's Eve, Grok apologized for creating and sharing an image of two teenage girls "in sexualized outfits," stating it was "a failure of our safety precautions."
Following widespread outrage, X restricted image generation to paying subscribers only. The European Commission stated the restriction did not resolve the underlying problem and questioned whether X complied with European digital laws.
Reported impact
Sexualized deepfake images of children and women were created and circulated on X's platform. British Prime Minister Keir Starmer called the deepfake images of children "disgusting."
Index analysis
The chatbot's content moderation failed to prevent the generation of illegal and harmful deepfake content involving minors. The European Commission noted that whether users paid or not, such images should not be created.
Governance controls
X restricted image generation and editing to paid subscribers only. The European Commission issued a preservation notice requiring X to retain internal Grok documentation until year-end.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.