TikTok and Instagram ban accounts for unlabeled AI-generated Black female avatars
TikTok and Instagram hosted dozens of accounts with AI-generated sexualized black female avatars that lacked required content labels and were used to drive traffic to paid explicit content sites. One account also stole videos from a real content creator, Riya Ulan, overlaying a synthetic face onto her body without consent.
What happened
TikTok banned 20 accounts after the BBC highlighted the use of AI-generated black female influencers to drive users to sites promoting sexually explicit content. The BBC and researchers from Riddance identified 60 accounts on Instagram and TikTok featuring highly sexualized black female digital characters or avatars that were generated by AI but not labeled as such, in apparent breach of the platforms' guidelines.
The avatars were often shown dressed in skimpy swimwear or revealing clothing with exaggerated body shapes and digitally manipulated, exceptionally dark skin tones. Account names included terms such as 'black', 'noir', 'dark' and 'ebony', with posts containing comments about white men such as 'loves white men' and 'why I need a white guy in my life'.
One account shut down by TikTok had stolen videos from real content creator Riya Ulan, a model based in Malaysia, overlaying an AI-generated face with exceptionally dark artificially created skin tone onto Ulan's body, movements, clothing and backdrop.
Reported impact
The accounts were used to drive users to third-party sites promoting sexually explicit content. The BBC and Riddance identified 60 accounts that carried links or chains of links to paid-for sexually explicit content, though the third-party sites labeled the imagery as AI-generated while the Instagram accounts did not.
Riya Ulan expressed anger at having her videos manipulated and posted without consent as part of the AI-generated avatar account.
Index analysis
The incident reflects a trend on Instagram and TikTok that has been criticized as racist, exploitative and misleading because of racial tropes and language used. Many of the accounts followed or liked each other, suggesting coordinated activity.
Governance controls
Meta told the BBC it was investigating but did not say it had taken any action on Instagram. TikTok removed 20 accounts in response to the BBC investigation, though the broader network of similar accounts continued to operate.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.