AI-generated violent fruit and vegetable videos on TikTok linked to youth radicalization
Researchers from GNET warned that AI-generated videos of violent fruits and vegetables spreading on TikTok use cute visual disguises to bypass content moderation and present radicalization risks to young people, with some videos replicating extremist messaging from groups including IS and neo-Nazi organizations.
What happened
AI-generated videos depicting fruits and vegetables committing acts of violence are spreading on TikTok. The videos use a 'cute' and 'Pixar-like' visual style that bypasses the platform's moderation policies designed to ban extreme violent content.
Reported impact
Researchers from GNET warned that the videos present a 'radicalising potential' for young people. Experts stated that violence-based AI content reinforces fascination with violence and normalises cruelty and emotional numbness through algorithmically amplified shock culture.
Index analysis
Some videos convey messages from extremist organisations including extreme right 'accelerationism', 'nihilistic violence extremism', and the Islamic State group. One example features a carrot making a potato 'explode' replicating official IS videos, and another features a 'Carrot Khalifah' telling his army 'the Carrot State will rise again', evoking an IS slogan. An account shares violent fruit and vegetable AI content while referencing neo-Nazi terrorist organisation Atomwaffen Division and the nihilistic violence subculture No Lives Matter.
Governance controls
TikTok's moderation policies are designed to strictly ban extreme violent content, but experts found that the cute visual disguise of these videos allows them to bypass these policies.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.