Grok AI Used to Generate Non-Consensual Explicit Images
Grok, an AI chatbot on X, was reportedly misused to generate non-consensual sexually explicit images and deepfakes, including of women and children. Malaysia and Indonesia blocked access to Grok and initiated legal action against X and xAI, citing failure to ensure user safety.
What happened
Grok, an AI chatbot deployed on X with an image generation feature called Grok Imagine, was used to generate sexually explicit, obscene, and non-consensual manipulated images including those of women in bikinis or sexually explicit poses and images involving children.
The Malaysian Communications and Multimedia Commission identified the misuse and sent notices to X and xAI in January 2026 requesting removal of harmful content, but reported no action was taken.
Reported impact
Content involving women and children was identified as a serious concern. The Malaysian government stated such conduct violates Malaysian law and undermines safety commitments.
Index analysis
Grok was launched in 2023 as a free feature on X and received an image generation capability called Grok Imagine with a so-called 'spicy mode' capable of generating adult content.
Governance controls
X implemented technical measures to prevent Grok from creating pornographic content, restricting the ability to generate images of people in bikinis, underwear and similar garments in jurisdictions where illegal. X restricted image creation and editing to paid subscribers and implemented geographic blocking.
X stated it applies security measures to all users including paid subscribers and that its safety team works to enforce platform policies, remove violating and illegal content, permanently suspend accounts when necessary, and cooperate with local governments and law enforcement agencies.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.