Deepfake videos and non-consensual pornography proliferate on social media and search engines
Deepfake videos of political figures and non-consensual explicit content are spreading via social media and search engines. Studies show people struggle to identify deepfakes, and search engines including Google and Microsoft continue displaying tutorials for creating such content despite new regulations.
What happened
AI-generated deepfake videos depicting political figures saying or doing things that did not happen were shared during the 2024 US election campaign. Microsoft reported in late October that Russian actors created deepfake videos of Vice President Kamala Harris making derogatory comments about Donald Trump, another showing her accused of illegal poaching in Zambia, and another spreading disinformation about Democratic vice president nominee Tim Walz that gained over 5 million views on X in 24 hours.
Search engines including Google, Microsoft's Bing and Yahoo are displaying tutorials and guides for creating deepfake pornography, hosting non-consensual explicit content, and promoting NSFW AI applications, according to a Glamour Magazine investigation. Non-consensual deepfake videos have been created of political figures in Australia, including Prime Minister Anthony Albanese and Opposition Leader Peter Dutton, and scammers impersonated Sunshine Coast Mayor Rosanna Natoli in a fake video call.
Reported impact
Studies suggest people can accurately identify deepfake facial images only 50% of the time and deepfake faces in videos 24.5% of the time. AI-based detection methods are marginally better than humans but become less effective when videos are compressed for social media.
The prevalence of deepfakes poses a threat to democratic processes. Without action, countries including Australia could become vulnerable to the same AI-driven political disinformation seen in the US. An ecosystem of actors online continues to create and distribute non-consensual sexually explicit deepfakes faster than platforms can address them.
Index analysis
The UK Online Safety Act criminalises the sharing and distribution of deepfake pornography with a potential two-year sentence and addition to the Sex Offenders register if intent to cause harm or distress is proven. The previous UK government announced an amendment to the Criminal Justice Bill in April making deepfake creation punishable with a criminal record and unlimited fines, prompting two popular deepfake pornography sites to block UK access, though they remain accessible via VPN.
Google, Microsoft and Yahoo have policies prohibiting the sharing or creation of sexually intimate images without permission and provide reporting and takedown procedures for victims. However, hundreds of listings for free deepfake-making software, NSFW AI apps, guides and forums remain easily discoverable through simple searches on these platforms.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.