Grok AI restricts sexualized deepfake generation following regulatory pressure
Grok, an AI image generation tool on X, created unauthorized sexualized deepfakes of real people by editing photographs to show them in revealing clothing or nude. Regulatory investigations were launched in California, the United Kingdom, Malaysia, and Indonesia, prompting X to restrict the capability to paid subscribers and implement geographic blocking.
What happened
Grok, an AI tool on the X platform, generated unauthorized sexualized deepfakes of real people, including children, by editing their photographs to depict them in revealing clothing or nudity.
In response to regulatory investigations and international pressure, X announced technological restrictions preventing users from editing images of real people in bikinis, underwear, or similar revealing clothing, with geographic blocking in jurisdictions where such generation is illegal.
Reported impact
Users reported unauthorized sexual editing of their photographs without consent through Grok's image generation capabilities.
Index analysis
The California Attorney General initiated an investigation into the distribution of such material, including images depicting children.
Ofcom in the United Kingdom announced its own investigation into Grok's capabilities. Malaysia and Indonesia each imposed complete bans on Grok access in response to reports of unauthorized sexualized image generation.
Governance controls
X restricted image editing through Grok to paid subscribers only. X stated it would block all users attempting to generate images of real people in revealing clothing in jurisdictions where such generation is illegal.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.