Skip to content
Submit incident
AII-1002-43771
Under investigationA regulator, court or police has opened a formal inquiry and reached no finding yet.

South Korea investigates deepfake abuse of K-pop idols on Telegram

South Korea's police opened a formal investigation into Telegram for allegedly facilitating deepfake sexual exploitation of over 200 female K-pop idols, whose non-consensual synthetic sexual imagery circulated on pornographic sites. The platform reportedly declined to provide investigative data to law enforcement, prompting government action to enforce removal of the material.

August 2024 Β· Telegram Β· Harmful and abusive content
August 2024
AII-1002-437712 sources ↓
Quick facts
Category
Harmful and abusive content
Country
South Korea
Source
tuoitre.vn
Date
August 2024
Drafted by
Raidu Incident Detection

What happened

South Korea's police opened an investigation into Telegram, alleging the platform facilitated deepfake sexual crimes. The Seoul police began an internal investigation after complaints that Telegram aided in the distribution of deepfake pornographic content.

Over 200 female K-pop idols had their faces manipulated into sexual imagery and videos posted on pornographic websites, circulating online. Affected idols included members of groups such as NewJeans, IVE, TWICE, and LE SSERAFIM.

Telegram reportedly refused to provide investigative data such as account information to South Korean police or other state investigation agencies, including U.S. authorities.

Reported impact

The deepfake sexual exploitation caused significant harm to female K-pop idols, whose images were non-consensually manipulated into pornographic content and distributed without their consent.

Index analysis

The incident represents the convergence of generative AI deepfake technology with platform governance failures, where a messaging application lacked adequate content moderation and cooperation mechanisms for illegal sexual exploitation material.

Governance controls

President Yoon Suk Yeol publicly declared deepfake sexual content a crime and ordered officials to respond with strict enforcement. The government requested Telegram and other social media platforms remove deepfake images from their systems.

Classification

Category
Harmful and abusive content
Industry
Arts, entertainment and recreation
Country
South Korea
Harm status
Actual harm
Intent
Intentional
Timing
Unknown
Harm types
Human or fundamental rights, Psychological, Reputational
Harmed
Women
Failure modes
Deepfake image, Deepfake video
Consequences
Regulatory investigation, Police investigation
AI systems
Deepfake technology

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

tuoitre.vn
South Korea investigates deepfake abuse of K-pop idols on Telegram
vietnamnet.vn
South Korea investigates deepfake abuse of K-pop idols on Telegram