AI-generated images used to spread misinformation about disasters
Police in China verified that images of a trapped boy shared after an earthquake were AI-generated, originally posted in November 2024 and later misused. Similarly, a video of the Hollywood sign engulfed in wildfire during California fires was also determined to be false, with the sign containing spelling errors and not actually affected by the fire.
What happened
Photos of a boy trapped under rubble were shared widely following the Jiangze earthquake, but Chinese police verified that the images were created using AI tools. The original images were published in November 2024 and were repurposed by misinformation spreaders after the earthquake occurred.
A video showing the Hollywood sign engulfed by wildfires circulated on social platforms during California wildfires, but the fire did not reach the sign's location and the sign in the video contained spelling errors.
Reported impact
The incidents exploited public panic and caused serious harm. AI-generated content made misinformation appear more credible, expanded the gap between the speed of creating false content and debunking it, and increased the difficulty of identifying and regulating synthetic media.
AI models' opacity made it difficult to understand how they generated misleading content or to establish legal frameworks for governance.
Index analysis
Deep learning models contain thousands of nodes and connections with decision logic hidden within multiple neural network layers. The nonlinear activation functions make the relationship between input and output extremely complex and difficult to track.
Governance controls
Technical measures should include integrating anomaly detection mechanisms into models to flag unusual behavior, and improving AI identification tools and digital watermarks. Current watermarks can be easily removed, so exploring metadata embedding and hard-to-remove invisible watermarks is needed.
Platforms must strengthen content review and rapid response mechanisms, and bear responsibility for using high-computing-power AI identification tools to verify content. Users should improve media literacy and information evaluation skills.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.