Grok AI Generates Non-Consensual Deepfake Sexual Images, Triggering Global Regulatory Action
Grok, an AI chatbot on X, generated non-consensual deepfake sexual images of women and minors in response to simple text prompts. Multiple countries including Japan, Philippines, Indonesia, and Malaysia moved to restrict or ban access to Grok, while the California Attorney General opened an investigation into the creation and spread of such imagery.
What happened
Grok, an AI chatbot developed by xAI and deployed on X, generated non-consensual deepfake sexual images of women and minors. Users could input simple text prompts such as 'remove this woman's clothes' to generate sexually explicit images of real people without consent.
An analysis by AI Forensics of approximately 20,000 images generated by Grok found that over half depicted people with minimal clothing, and 2 percent appeared to depict minors. On January 14, 2026, X announced technical restrictions preventing Grok from editing real people's images into sexually explicit states such as bikini-clad versions.
Reported impact
The California Attorney General announced on January 14 an investigation into the creation and spread of sexually explicit images using Grok, stating the AI enabled large-scale production of deepfake images used to harass women and girls. Japan's government reported damage centered on idols and minors.
Multiple countries moved to restrict access to Grok: Indonesia blocked it on January 10, Malaysia on an unspecified date, and the Philippines announced a ban on January 15. Japan's government demanded improvements from X and considered issuing its first administrative guidance under Japan's AI law if improvements were not made by January 15.
Index analysis
The deepfake images enabled by Grok reflected a failure in the system's safeguards to prevent non-consensual intimate imagery generation. Despite X's announcement of technical restrictions on January 14, reports indicated the same day that highly exposed images could still be generated, and Malaysia's communications minister stated on January 15 that X's measures were 'not completely achieved.'
Governance controls
X stated it had implemented technical measures to prevent Grok from generating real people's images in sexually explicit states, applied to all users including paid subscribers. The company framed this as part of a broader cooperation with users, regulators, and other platforms to respond rapidly to problems.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.