Skip to content
Submit incident
AII-1002-44267
Under investigationA regulator, court or police has opened a formal inquiry and reached no finding yet.

South Korea Investigates Telegram Over Deepfake Sexual Crimes

Deepfake sexual exploitation material proliferated on Telegram in South Korea, with police opening an investigation in September 2024 after the platform became a distribution hub for non-consensual sexual imagery, predominantly involving minors as both perpetrators and victims.

5 September 2024 · Telegram · Harmful and abusive content
5 September 2024
AII-1002-442672 sources ↓
Quick facts
Category
Harmful and abusive content
Country
South Korea
Source
韩联社(韩国联合通讯社)
Date
5 September 2024
Drafted by
Raidu Incident Detection

What happened

South Korean police opened a pre-investigation into Telegram in September 2024 after the platform became a hub for distributing deepfake sexual exploitation material. Telegram's monthly active users in South Korea surged by 31.1% in August 2024, with users aged 19 or younger increasing by 50%, attributed to illegal deepfake videos being shared on the platform.

As of August 25, 2024, a support center for digital sexual crime victims had received requests for assistance in 781 deepfake-related cases, with 36.9% involving minors. The police reported 118 deepfake-related complaints in one five-day period, with 73.6% of arrested perpetrators in 2024 being teenagers.

Reported impact

The majority of both perpetrators and victims in deepfake sexual crime cases involving Telegram in South Korea are minors. A US cybersecurity firm reported that 53% of victims in over 90,000 deepfake sexual exploitation items distributed globally were South Korean.

Victims faced obstacles in obtaining relief, with some paying millions of won to private companies for video removal after government systems could not quickly delete exploitation material. An estimated 260,000 illegal image deletion requests from 2020 onward remained unprocessed.

Index analysis

The deepfake sexual exploitation material appeared designed to exploit minors, with Telegram's architecture enabling mass distribution. The platform's teenage user surge and the demographics of cases indicate the technology was creating new vectors for harm against children.

Governance controls

South Korea's government faced criticism for delayed policy responses, with the Women and Family Ministry lacking a confirmed minister for six months. Proposed legislation included 33 bills addressing penalties for sexual violence crimes, but lawmakers were accused of drafting them hastily with overlapping provisions.

Telegram had begun working with South Korea to delete some deepfake sexual exploitation content. The platform updated its user terms to add reporting functions for private chats and group messages.

Classification

Category
Harmful and abusive content
Industry
Media, social platforms and marketing
Country
South Korea
Harm status
Actual harm
Intent
Intentional
Timing
Unknown
Harm types
Psychological, Human or fundamental rights
Harmed
Children, Consumers
Failure modes
Deepfake video, Deepfake image
Consequences
Regulatory investigation, Police investigation
AI systems
Deepfake technology

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

韩联社(韩国联合通讯社)
通讯软件"电报"8月在韩用户环比增幅创新高 | 韩联社
5 September 2024
韩国最大的传媒机构《中央日报》中文网
【社论】民众陷入深伪技术的痛苦深渊,政策却毫无作为
6 September 2024