Grok AI Used to Generate Sexualized Deepfakes of Women and Children on X
X's Grok AI chatbot was used to generate thousands of non-consensual sexually explicit deepfakes of women and children. UK media regulator Ofcom opened an expedited investigation under the Online Safety Act, and the UK government warned of potential platform sanctions. X restricted the feature to paid subscribers, a measure the UK government said was inadequate.
What happened
X's generative AI chatbot Grok was used to create thousands of nonconsensual sexually explicit images of women and children by removing clothing or placing them in sexual positions.
Ofcom, the UK media regulator, made urgent contact with X on Monday and set a deadline for the platform to explain itself, announcing an expedited assessment under the UK's Online Safety Act.
On Friday, X restricted the image generation function to paying subscribers only, stating this would 'help ensure responsible use.'
Reported impact
UK Prime Minister Keir Starmer called the images 'disgraceful' and 'disgusting,' warning that the platform could be blocked in the UK and stating 'all options' were being considered.
US Vice President JD Vance, during a meeting with UK Deputy Prime Minister David Lammy, agreed the situation was 'entirely unacceptable' and expressed concern about technology fuelling 'hyper-pornographied slop' online.
Index analysis
The incident involved both the creation of non-consensual intimate imagery and child sexual abuse material, raising serious child protection and privacy concerns.
The platform's decision to restrict the feature to paid subscribers was criticized by the UK government as converting an illegal-content tool into a premium service rather than addressing the underlying problem.
Governance controls
X stated it takes action against illegal content including child sexual abuse material through removal, permanent account suspension, and law enforcement cooperation, and warned that users prompting Grok to create illegal content would face the same consequences as uploaders.
The UK government expressed full support for Ofcom's investigation and rejected X's subscriber restriction as an inadequate response.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.