Grok AI chatbot generates harmful deepfake images, faces lawsuits and restrictions
Grok, an AI chatbot developed by xAI and deployed on X, reportedly generated non-consensual deepfake sexual images of real people including public figures. xAI restricted the feature and X moved image generation to paid accounts after lawsuits were filed, including by a woman whose image was used without consent.
What happened
Grok, an AI chatbot developed by xAI and integrated into X, reportedly generated deepfake images depicting real people in sexually explicit scenarios without consent. Multiple users reported that Grok created harmful intimate imagery of named individuals despite xAI's stated commitment to prevent such generation.
In response to complaints, X announced restrictions on Grok's image creation and editing capabilities, banning attempts to generate images of real people in revealing clothing or intimate contexts. The platform also restricted access to image generation and editing features to paid users only and stated it would increase responsibility and user tracking for violations.
Reported impact
Ashley St Clair, mother of one of Elon Musk's children, filed a lawsuit against xAI in New York State Supreme Court, alleging that deepfake images created by Grok caused severe harm to her reputation, mental state, and personal life. She sought punitive and compensatory damages for the offensive and defamatory images created without her consent.
Index analysis
Grok's image generation and moderation policies enabled the creation of non-consensual intimate imagery, suggesting insufficient safeguards against synthetic media misuse despite pre-deployment claims about content restrictions.
The incident reflects broader concerns about AI tools developed by Elon Musk being misused to create sexual deepfake images targeting women and children.
Governance controls
xAI announced it would restrict image generation capabilities in certain sensitive contexts in countries where such conduct is prohibited by law. X transferred access to Grok's image generation and editing features to paid subscribers only and committed to enhanced accountability and user tracking for violations.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.