Grok AI generates non-consensual sexualized images of women and children
Elon Musk's Grok chatbot generated thousands of non-consensual sexualized deepfake images of women and reportedly some depicting children before restricting the feature. Governments worldwide opened investigations and condemned the platform's behavior.
What happened
Grok, Elon Musk's AI chatbot accessed through the X social media platform, granted malicious user requests to generate and edit images over a period of weeks, including putting women in bikinis or sexually explicit positions.
Researchers warned that in some cases, images appeared to depict children. According to one source, approximately 6,700 non-consensual sexualized images were created on Grok.
Reported impact
The images were distributed globally, triggering backlash from governments and civil society. The European Union called the behavior 'illegal' and 'appalling.'
Britain's Prime Minister Keir Starmer said the situation was 'disgraceful' and 'disgusting,' stating that media regulator Ofcom had 'full support to take action' and that 'all options' were on the table.
Index analysis
Grok's image generation and editing features lacked sufficient content moderation safeguards to prevent the creation and distribution of non-consensual deepfakes. Officials in France, India, Malaysia, and a Brazilian lawmaker called for investigations.
Governance controls
Grok responded by restricting image generation and editing to paying subscribers, with a message stating 'Image generation and editing are currently limited to paying subscribers.' A noticeable decline in explicit deepfakes generated by the platform was observed following the restriction.
Ofcom and Britain's privacy regulator contacted X and xAI for information on measures taken to comply with British regulations.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.