Australia blocks AI deepfake 'nudify' services after child abuse incidents
Australia's eSafety Commission blocked access to AI 'nudify' deepfake services that generated fake nude images of school children after determining they breached mandatory online safety codes. The UK-based provider received about 100,000 monthly visits from Australians and was actively marketing features designed to target minors, prompting the regulator to withdraw access and Hugging Face to revise its terms of service.
What happened
Three widely used AI 'nudify' services linked to generation of fake nude images of school children were blocked in Australia. A UK-based company withdrew access to Australians after the eSafety Commission issued an official warning in September over the services creating artificially generated child sexual exploitation material.
The eSafety Commission found the provider failed to prevent safeguards and actively marketed features including 'undressing any girl,' 'schoolgirl' image generation, and 'sex mode.' About 100,000 Australians visited the services monthly, and the tools featured in high-profile cases of students creating fake nude images of classmates.
Separately, AI model-hosting platform Hugging Face changed its terms of service after warnings that Australians misused some of its generative tools to create child sexual exploitation material. Hugging Face's new terms require users to minimise risks associated with uploaded models to prevent generating child exploitation material.
Reported impact
The eSafety Commission reported doubling of reports about digitally altered images and deepfakes from people under 18 in the past 18 months, with four out of five reports targeting women and girls.
Commissioner Julie Inman Grant stated the tools have been 'used to devastating effect in Australian schools' and that blocking access would have 'a tangible impact on the number of Australian school children falling victim to AI-generated child sexual exploitation.'
Index analysis
The services used AI-generated deepfake technology to create non-consensual sexual images of minors without their knowledge or permission, exploiting images taken from social media, streets, schools and yearbooks.
Governance controls
The eSafety Commission issued an official warning in September under Australia's mandatory Codes and Standards, which require online industry members to take meaningful steps to tackle child sexual exploitation material. The UK-based provider breached this requirement by failing to implement safeguards.
Hugging Face implemented revised terms of service requiring users to minimise risks of child sexual exploitation material, with enforcement requirements if breaches become known.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.