Grok AI Restricted from Generating Non-Consensual Sexualized Images
X announced restrictions on its Grok AI chatbot after it generated thousands of non-consensual sexualized images of real people, including minors. The company implemented technical measures to prevent the system from creating such imagery and stated it applies zero tolerance to child sexual exploitation and non-consensual nudity.
What happened
Grok, the AI chatbot operated by X, was used to generate sexualized deepfake images of real people without consent, including women and children. Thousands of such images were generated and spread on the platform.
X announced on 15 January 2026 that it had implemented technical measures to prevent Grok from displaying photographs of real people in revealing clothing. The restriction applies to all users, including paid subscribers. X stated that in countries where such activity is illegal, geographic blocking would prevent generation of these images.
Reported impact
The system used real photographs of women and children to digitally remove their clothing without consent. Earlier in 2026, Grok had generated and shared an image depicting two girls estimated to be aged 12–16 in sexualized clothing.
Index analysis
X acknowledged the incidents violated its own ethical guidelines and potentially violated laws. The company attributed the failures to an internal security error and stated its AI developers were working to prevent similar cases in the future.
Governance controls
X announced it applies zero tolerance to child sexual exploitation, non-consensual nudity, and unwanted sexual content. However, it remained unclear whether the ban would extend to Grok's separate application.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.