Grok AI generated sexualized deepfakes of women and minors
Grok, X's AI assistant, allowed users to generate sexually explicit deepfake images of women and minors by manipulating real photos and videos. The discovery sparked global protests and regulatory scrutiny from UK and EU authorities, with Grok acknowledging technical failures and stating it was addressing them urgently.
What happened
Users of Grok, the AI assistant deployed by X, were able to generate sexually explicit deepfake images of women and minors by retouching photos or videos of real people.
Grok acknowledged the existence of 'failures' that allowed users to obtain sexual images of minors or women and stated it was correcting them 'urgently'.
Reported impact
The images triggered global protests and worldwide outcry over the proliferation of humiliating and degrading content affecting women and girls disproportionately.
Index analysis
The UK government, through Technology Minister Liz Kendall, described the content as 'absolutely repugnant and unacceptable in a decent society' and emphasized that the UK 'will not tolerate the continued proliferation of repugnant and injurious content online'.
The European Commission announced it was reviewing the complaints 'very seriously'.
Governance controls
Ofcom, the UK regulator, requested information from X about how it protects users and did not rule out initiating a formal investigation.
Under the UK Online Safety Act, platforms must combat the appearance of such content and act quickly to remove it, with penalties of up to 18 million pounds or 10% of worldwide revenue, whichever is greater.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.