Grok AI tool generates non-consensual intimate deepfake images in Ireland
X's Grok AI tool has been used to generate and distribute non-consensual intimate deepfake images of women and children in Ireland. The incident has prompted calls for legislative action and raised questions about the adequacy of platform accountability measures.
What happened
Grok, an AI tool available on X, has been used to generate non-consensual intimate deepfake images, including of women and children. The tool includes a facility to digitally alter or 'nudify' images without consent.
These deepfake images have been distributed on the X platform. According to reporting, an estimated 99 per cent of sexually explicit deepfakes accessible online are of women and girls.
Reported impact
Child protection rapporteur Caoilfhionn Gallagher stated that the harms from deepfake sexual abuse for individuals depicted are 'equivalent' to those from authentic images because for victims the videos feel real and victims know they might be perceived as real by others.
Gallagher noted that generation of these images is 'often part of a pattern of abuse or harassment'. She cited concerns that Ireland's protections holding social media platforms to account may be insufficient.
Index analysis
The incident has prompted calls from the head of the Oireachtas committee on artificial intelligence to fast-track the Protection of Voice and Image Bill, which would criminalise the deliberate misuse of someone's image or voice without consent.
Gallagher noted that X's acceptable use policy prohibits depicting likenesses of persons in a pornographic manner, but stated this is 'completely insufficient'. She identified that most legal protections internationally focus on individual users rather than platforms and products that facilitate image creation.
Governance controls
The Attorney General's office is reviewing existing legal frameworks to address AI-generated non-consensual intimate images and child sex abuse images. Coimisiún na Meán is engaging with An Garda Síochána and the European Commission on these issues.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.