Google reports 258 complaints of Gemini-generated terrorist deepfakes to Australian eSafety Commission
Google reported to Australian authorities that it received over 250 user complaints globally between April 2023 and February 2024 that its Gemini AI was used to generate deepfake terrorism material, plus dozens of reports of AI-generated child exploitation content. The eSafety Commission called the disclosure "world-first insight" and noted Google did not use the same detection system for terrorist content as it did for child abuse material.
What happened
Google reported to the Australian eSafety Commission that it received 258 user reports about suspected AI-generated deepfake terrorist or violent extremist content made using Gemini, and 86 user reports alleging AI-generated child exploitation or abuse material, during the reporting period of April 2023 to February 2024.
Reported impact
Google said the reports represented the total global volume of user reports, not confirmed policy violations. Google did not say how many of the complaints it verified.
Index analysis
Google used hash-matching to identify and remove child abuse material made with Gemini, but did not use the same system to detect terrorist or violent extremist material generated with the AI, the regulator noted.
Governance controls
Google said it does not allow the generation or distribution of content related to facilitating violent extremism or terror, child exploitation or abuse, or other illegal activities. Under Australian law, tech firms must supply the eSafety Commission periodically with information about harm minimisation efforts or risk fines.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.