Skip to content
Submit incident
AII-1002-49829
ReportedReported by fewer than three independent publishers, with no official finding.

AI-generated deepfakes and robocalls fuel fraud surge across Asia

AI-generated robocalls and deepfake technology are fueling fraud across Asia, particularly in Japan where robocalls target personal information and romance fraud schemes use fake images and videos on dating apps to extract money from victims. Survey data shows most victims suffer psychological distress and eroded trust in communications systems.

26 October 2024 ยท Eureka (Pairs) ยท Fraud, scams and manipulation
26 October 2024
AII-1002-498292 sources โ†“
Quick facts
Category
Fraud, scams and manipulation
Country
Japan
Source
ใ‚ฑใƒผใ‚ฟใ‚ค Watch
Date
26 October 2024
Drafted by
Raidu Incident Detection

What happened

AI-generated robocalls and deepfake technology are being misused for fraud schemes across Asia, including Japan. A survey by Ribbon Communications found that approximately 90% of Japanese respondents were unaware of robocalls, yet 1 in 5 reported receiving suspicious automated voice calls for personal information collection or fraud purposes.

Romance fraud has increased on dating platforms, with criminals using deepfake images and AI-generated videos in conversations to build false relationships and extract money. Victims predominantly include men aged 50-60 and women aged 40-50, though younger users are also affected.

Reported impact

Over 90% of those who received suspicious robocalls reported negative feelings about answering phones afterward, indicating erosion of trust in telecommunications. About 71% of respondents believe personal measures against fraud calls are insufficient.

Romance fraud victims experience significant psychological harm, as trust relationships are fabricated before financial exploitation occurs. Victims often remain in contact with perpetrators before recognizing the deception.

Index analysis

Generative AI technology capable of voice synthesis was used to create fraudulent robocalls, demonstrating real-world deployment of voice-cloning capabilities for criminal purposes. The technology's increasing sophistication poses growing risks as AI generation capabilities improve.

Governance controls

Pairs (Eureka) deployed Amazon Rekognition machine learning for identity verification and adopted My Number Card IC chip verification before other platforms. The platform analyzes suspicious user behavior patterns and removes malicious accounts via AI detection and human review.

Pairs established a Safety Center with guidance materials, YouTube educational videos, and safety handbooks distributed to users. The platform coordinates with local government and victim support organizations like CHARMS to conduct educational seminars on safe dating app use.

Classification

Category
Fraud, scams and manipulation
Industry
Financial and insurance services
Country
Japan
Harm status
Actual harm
Intent
Intentional
Timing
Unknown
Harm types
Economic or property, Psychological
Harmed
Consumers, General public, Women
Failure modes
Voice clone, Deepfake image, Deepfake video
AI systems
AI-generated robocalls, Generative AI voice synthesis, Amazon Rekognition

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

ใ‚ฑใƒผใ‚ฟใ‚ค Watch
ใƒญใƒžใƒณใ‚น่ฉๆฌบ่ขซๅฎณใ‚’AIใชใฉๆŠ€่ก“ใง้˜ฒๆญขใ€ใƒšใ‚ขใƒผใ‚บใŒๅฎ‰ๅ…จใซๅ‘ใ‘ใŸๅ–ใ‚Š็ต„ใฟใ‚’่ชฌๆ˜Ž
28 October 2024
CNET
ใ€Žใƒญใƒœใ‚ณใƒผใƒซใ€ใ‚’็Ÿฅใ‚‰ใชใ„ไบบ ็ด„9ๅ‰ฒ
30 October 2024