AI-generated deepfakes and fake reviews harm consumers, women and businesses in e-commerce
Deepfake technology and AI-generated content are being intentionally misused to create non-consensual synthetic sexual material targeting women and to perpetrate e-commerce fraud through fake product images and fabricated reviews. Current legal protections and platform enforcement remain inadequate, leaving victims with limited recourse and vulnerable populations exposed to significant financial and psychological harm.
What happened
Deepfake technology is being used to create non-consensual synthetic sexual content, with 98% of deepfake videos containing pornographic material and nearly all involving women. Victims' images and videos are altered and distributed without consent across multiple platforms, reaching thousands to millions of people online.
In e-commerce, AI-generated content is being misused to create fake product images, fabricated customer reviews, and false effect demonstrations to deceive consumers. Sellers use low-cost, high-yield synthetic media to pose fraudulent products as genuine, bypassing verification systems. Deepfake video numbers increased 550% from 2019 to 2023, with free and readily available tools making creation increasingly accessible.
Reported impact
Women and girls suffer severe harm including financial losses, psychological trauma, and digital exposure to millions. Vulnerable populations including elderly consumers are disproportionately affected by fake product claims and fraudulent reviews, experiencing both property losses and erosion of trust in e-commerce platforms.
Legitimate businesses face reputational damage and reduced consumer confidence as honest merchants compete unfairly with those using fraudulent AI-generated marketing. The abuse of AI-generated content undermines market competition and damages broader trust in online commerce ecosystems.
Index analysis
Current legal protections are inadequate, with most countries lacking specific legislation addressing deepfake misuse. Existing 'revenge porn' laws predate AI technology and create legal gaps. Although some jurisdictions including the EU, United States, and Brazil have attempted legislation, enforcement remains limited and perpetrators often evade punishment.
Platform response times are slow, with content remaining visible for days after reporting. Evidence disappears as content spreads across platforms and is saved to private devices, making complete removal nearly impossible. Cross-border cases present investigation difficulties and resource constraints in enforcement.
Governance controls
China's regulations including the 'Live E-commerce Supervision and Management Measures' explicitly prohibit using AI to fabricate or spread false commercial information. The 'AI-Generated Synthetic Content Labeling Measures' requires all AI-generated content to include active disclosure and prominent marking.
UN Women calls for governments to enact laws covering deepfake abuse with clear provisions for consent rights, perpetrator liability, platform deletion obligations, and cross-border enforcement. Platforms must be legally mandated to monitor content, promptly remove harmful material, and cooperate with law enforcement. Systems should include survivor support through free legal aid, trauma-informed training, and improved reporting processes to prevent re-victimization.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.