AI-generated deepfakes spread as misinformation on social media and in news
AI-generated deepfakes depicting false military events and forged Holocaust imagery spread on social media and infiltrated news organizations, eroding public trust and historical memory. Researchers documented coordinated AI agent networks designed to fabricate false social consensus, and institutions developed detection tools to identify synthetic media and expose coordinated manipulation.
What happened
A video claiming to show Israeli soldiers distressed during Iranian rocket and drone attacks spread widely on social media platform X, reaching 1.6 million views. German broadcaster ZDF's investigation found the video was AI-generated, with inconsistencies including flickering military insignia with gibberish text, unrealistic uniform styles, and artificial audio details.
In February, ZDF suspended and dismissed a New York correspondent after discovering they had submitted a report on U.S. immigration enforcement that included a false video generated by the Sora model, which contained a visible Sora watermark. ZDF subsequently announced mandatory training for staff on proper use of AI tools in journalism.
Around 30 German concentration camp memorials and museums issued an open letter urging social media platforms to combat AI-generated content related to the Holocaust. Historians noted that AI-generated images mimicking historical photos appeared superficially authentic but contained anomalies, including unrealistic uniformity in facial features, height, and attractiveness of depicted individuals.
Reported impact
Researchers warned that AI can fabricate 'social consensus' by deploying coordinated clusters of AI agents that interact with real users, adapt to emotional changes, and reinforce false narratives to create an illusion of widespread agreement where none exists.
The spread of synthetic media threatens public trust in news institutions, historical memory, and social discourse. Historians stated that beautified AI-generated Holocaust imagery distorts historical record and erodes the authority and credibility of memorials, archives, museums and research institutions.
Index analysis
Researchers from the University of Konstanz noted that AI agent clusters differ significantly from earlier botnets by operating across platforms, maintaining stable identities and memories, and amplifying impact through mutual reinforcement rather than displaying mechanical behavior.
The German AI Research Center (DFKI) and Rhineland-Palatinate Political Education Center developed a deepfake detection tool combining forensic analysis of images and audio with cross-verification against verified fact databases, including data from news agency Deutsche Presse-Agentur's fact-checking team.
Governance controls
ZDF established mandatory training programs for employees on proper use of AI tools in news reporting, stating that disregard for journalistic standards causes severe damage to reporting credibility.
Researchers advocated for independent monitoring mechanisms to continuously observe how AI agents influence public discussion across platforms and promptly expose false content networks. DFKI's detection tool allows users to submit suspicious social media content for analysis to determine whether material was AI-generated or altered.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.