xAI's Grok generates explicit deepfake images despite safeguards
xAI's Grok chatbot, which includes an image generation tool with adult content capabilities, was used to create explicit deepfake images of real people despite the company's January 14 safeguard restrictions. An independent analysis in mid-January found the restrictions were ineffective, and Brazilian regulators issued a 30-day ultimatum for xAI to halt the dissemination of fake sexualized content.
What happened
xAI introduced an image-generator feature in Grok that included a 'spicy mode' capable of generating adult content. The feature was later used to create sexualized images by undressing photos of real women.
On January 14, X announced it had implemented technological measures to prevent Grok from editing images of real people in revealing clothing. However, on January 19, AI Forensics analyzed 2,000 user conversations and found an 'overwhelming' majority depicted nudity or sexual activity, indicating users could still generate sexualized images.
Users were able to bypass restrictions by accessing Grok through its website directly rather than through X, or by using Grok Imagine, the image and video generation tool.
Reported impact
AI Forensics found that Grok was still generating sexually explicit images of individuals despite the stated safeguards, showing the restrictions were ineffective.
Index analysis
The chatbot's image generation capabilities were designed to enable adult content creation with minimal restrictions, creating a tool readily available for generating explicit deepfakes of real people.
Governance controls
xAI restricted Grok's image editing capabilities and implemented content blocks to prevent image creation depicting real people in revealing or sexualized clothing. Brazilian authorities, including Senacon and ANPD, issued a 30-day ultimatum on January 21 demanding xAI cease unauthorized spread of fake sexualized content and implement technologies to identify, review, and remove offensive material.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.