Skip to content
Submit incident
AII-1002-43567
Well documentedThree or more independent publishers report the core facts.

Deepfake nude sites exploited Sign in with Apple

Deepfake websites designed to generate non-consensual nude images were discovered using sign-in systems from Apple, Google, Discord, and other major tech platforms to gain credibility. After investigation revealed the abuse, Apple and other companies blocked access to their sign-in infrastructure.

29 August 2024 Β· Apple Β· Harmful and abusive content
29 August 2024
AII-1002-435677 sources ↓
Quick facts
Category
Harmful and abusive content
Country
United States
Source
Wired
Date
29 August 2024
Drafted by
Raidu Incident Detection

What happened

A Wired investigation revealed that 16 major deepfake websites designed to generate non-consensual nude images were using sign-in infrastructure from major tech companies, including Google, Apple, Discord, Twitter, Patreon, and Line. Six of these sites specifically offered Sign in with Apple, giving the unethical sites an appearance of legitimacy.

After learning about the abuse, Apple quickly blocked API access for deepfake website developers, removing their access to the Sign in with Apple feature. Discord, Google, and Patreon also took steps to address the issue by blocking or restricting accounts that violated their terms.

Reported impact

The deepfake sites enable users to upload photos and digitally alter them to remove clothing without consent. Cases have been documented where such technology was used to create explicit content involving high school students.

Index analysis

The abuse demonstrates how authentication services designed for user convenience and privacy can be misused by bad actors to lend credibility to harmful services. The availability of trusted sign-in options made it easier for users to create accounts on these deepfake sites without recognizing their dubious nature.

Governance controls

Apple's swift action to block API access reduced the sites' legitimacy and operational ease. Mastercard stated that its network does not allow purchases of non-consensual deepfake content and indicated readiness for further action.

Classification

Category
Harmful and abusive content
Industry
Media, social platforms and marketing
Country
United States
Harm status
Actual harm
Intent
Intentional
Timing
Post-deployment
Harm types
Human or fundamental rights, Psychological, Reputational
Harmed
General public, Women
Failure modes
Deepfake image, Manipulation or dark pattern
Consequences
System suspension or termination
AI systems
Sign in with Apple

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

Wired
Google, Apple, and Discord Let Harmful AI 'Undress' Websites Use Their Sign-On Systems
29 August 2024
Ars Technica
Harmful "nudify" websites used Google, Apple, and Discord sign-on systems
30 August 2024
9to5Mac
Sign in with Apple was available on multiple deepfake nudes sites
29 August 2024
AppleInsider
Deepfake website developers blocked form Sign in with Apple
29 August 2024
Tech Times
Deepfake Sites Exploiting Apple's Sign-in Options: Here's the Worse Part
29 August 2024
The Unofficial Apple Weblog (TUAW)
Apple Blocks Deepfake Websites from Using "Sign in with Apple"
1 September 2024
The Tech Report
Nudify" Websites Are Using Sign-on Systems from Google, X, Discord, Patreon & Line
2 September 2024