Apple threatened to remove Grok app over sexualized deepfakes
Grok's image generation tool on X created approximately three million sexualized deepfakes in January 2026, including images of children and non-consensual modifications of women's photographs. Apple threatened to remove the Grok app from its App Store and demanded xAI improve content moderation safeguards; the problem continued despite some remediation efforts.
What happened
In January 2026, Grok's image generation tool generated approximately three million sexualized images over about 11 days on X, including around 23,000 depicting children and approximately 1.8 million of women, according to the Center for Countering Digital Hate (CCDH).
The tool was accessed through an option on X that allowed users to edit images with Grok using a single click, enabling requests to add sexual clothing or positions to photos of women and children. Apple contacted xAI directly with complaints and demanded a content moderation improvement plan, rejecting multiple app update submissions before eventually accepting one when Apple determined infractions had been substantially remedied.
Reported impact
NBC reported that Grok continued generating non-consensual sexual images; dozens of such cases were documented in the month following the initial discovery. The sexualized images were created using tight-fitting clothing, sports bras, and other modifications.
Index analysis
The image generation feature in Grok lacked sufficient safeguards to prevent the creation and distribution of non-consensual sexualized deepfakes, including those depicting minors.
Governance controls
Apple threatened xAI with removal of the Grok app from the App Store for violating Apple's guidelines. Apple required xAI to implement changes to limit sexualized image generation and prevent viralization through X, with Apple sending letters to U.S. senators explaining its actions.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.