Dutch Court Bans Grok from Generating Non-Consensual Sexual Deepfakes; EU Parliament Votes Ban
An Amsterdam court banned Grok and X from generating non-consensual sexual deepfakes after finding the platforms' safety measures ineffective. On the same day, the European Parliament voted to prohibit AI services in the EU that can generate such deepfakes without consent.
What happened
Offlimits, a Dutch online violence expertise centre, sued Grok and X after the platforms introduced a feature in Grok allowing users to generate non-consensual sexual deepfakes of women and children from real photographs.
The Amsterdam District Court found that Grok and X's stated safety measures were ineffective. The court noted that Offlimits produced a deepfake video of a naked person using Grok shortly before the hearing.
The Dutch court banned Grok and X from generating nude images of people residing in the Netherlands and from producing or distributing such images in the country, with violations punishable by a daily fine of €100,000 capped at €10 million.
Reported impact
On the same day, the European Parliament approved an EU-wide prohibition on AI services that can remove clothing from people without consent, passing the measure with 569 votes in favour and 45 against.
Governance controls
Grok and X included prohibitions of non-consensual sexual image generation in their terms of service and stated they had taken measures to prevent generation and distribution of such content.
The court determined that these measures were insufficient to prevent the harm, finding reasonable doubt about their effectiveness.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.