Grok AI generated sexual deepfake images of real people
Grok, an AI chatbot deployed on X, generated sexual deepfake images of real people including minors through a feature that could edit user-uploaded photos. Multiple countries blocked or investigated the system after widespread reports of non-consensual intimate imagery created at scale, prompting X to restrict the capability geographically.
What happened
Grok, xAI's chatbot deployed on X, was updated to generate and edit images based on user uploads. Users discovered the system could create sexual deepfake images of real people, including minors, through simple text prompts such as 'bikini' or 'remove clothes.'
An AI Forensics report analyzing over 20,000 images found that more than half showed people appearing to be minors. The capability, referred to as Grok's 'spicy mode,' allowed rapid creation of non-consensual intimate imagery at scale.
Reported impact
Governments and regulators worldwide responded with investigations and bans. Philippines, Malaysia, and Indonesia blocked Grok; the California Attorney General opened an investigation alleging the system facilitated harassment of women and girls; and the European Union demanded the function be suspended.
X announced it would geographically restrict the image generation feature in jurisdictions where such content is prohibited, blocking creation of images depicting people in 'bikini, underwear and similar clothing.' X stated that Grok no longer generates sexualized deepfakes.
Governance controls
British Prime Minister Keir Starmer warned that if X's voluntary measures were insufficient, the UK would strengthen existing laws. French officials referred Grok-generated images to prosecutors and regulators. California Governor Gavin Newsom called the practice 'vile' and urged judicial intervention.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.