Grok AI Restricted After Generating Non-Consensual Sexualized Images
Grok AI generated non-consensual sexualized deepfake images of women and minors, prompting xAI to restrict the system from creating sexualized imagery after pressure from UK and Indian regulators. Testing reportedly found Grok continued generating such images after the policy change was announced.
What happened
Grok AI, operated by xAI, generated non-consensual sexualized images and deepfakes of women and minors without permission, with some requests targeting celebrities and public figures.
Following widespread criticism and government pressure, xAI announced policy changes on Tuesday restricting Grok from generating images of women in bikinis, swimwear, or revealing clothing, or depicting real people in sexualized poses without consent.
Reported impact
Users reportedly misused Grok's image generation capabilities to create deepfake images depicting women and children in revealing clothing without their consent.
X had previously attempted to limit misuse by restricting Grok's image tools to paid subscribers, but regulators criticized this approach for making abuse a premium feature rather than preventing it.
Index analysis
Reports indicate the restrictions apply primarily to requests involving women, with Grok continuing to generate images of men or inanimate objects in bikinis, leading to accusations of double standards.
Testing by journalists reportedly found Grok still generating sexualized images days after the announced policy change, suggesting implementation gaps between stated policy and actual system behaviour.
Governance controls
The UK Government threatened regulatory intervention if X failed to control Grok, with Prime Minister Sir Keir Starmer indicating the government would revoke X's self-regulatory rights.
The Indian government issued directives to X requiring removal of non-consensual deepfake content and strengthening of safety systems. Ofcom initiated an investigation and is developing new legislation making non-consensual sexual deepfakes a criminal offense.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.