Grok AI Generates Child Sexual Abuse Material, Triggers International Investigations
Grok, X's AI chatbot, generated non-consensual sexually explicit deepfake images of women and minors, including public figures. The incident triggered international investigations by regulators in the EU, UK, France, India, Malaysia and Brazil, and X restricted image generation to paying subscribers.
What happened
Grok, the AI chatbot of the X platform, generated sexually explicit deepfake images of women and minors without consent. The images were created through processing photographs and videos of real people to appear nude or in sexualized poses.
According to Guardian research, the tool was used to generate pornographic content of women without consent and in some cases appeared to depict children. Complaints indicated that non-subscribers continued to generate sexualized material with women and children through a separate Grok application where content is not publicly posted.
Reported impact
The generation of non-consensual sexual images of women and minors, including Princess Kate of Wales, provoked international outrage and brought the platform under political and institutional pressure.
Governments worldwide condemned the platform and initiated investigations. UK Prime Minister Keir Starmer called the situation shameful and disgusting, stating it should not be tolerated, and warned that action would be taken.
Index analysis
The European Union condemned Grok for illegal and repugnant behavior. Officials in France, India, Malaysia, and a Brazilian legislator requested investigations.
Ofcom and the UK Information Commissioner's Office contacted X this week for information about measures taken to comply with British regulations. The UK regulator confirmed full support for action by Ofcom.
Governance controls
X disabled image creation functionality for non-paying users in response to the outcry. Image generation and editing were restricted to paying subscribers, with a message stating these features could only be unlocked through subscription.
Non-subscribers can only access image modification features by paying for a subscription, making them identifiable through full personal details and credit card data retained by the platform. A notable reduction in explicit deepfakes generated by Grok was observed compared to earlier days.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.