AI-generated fake social media accounts used for fraud in China
AI-generated fake social media accounts on Chinese platforms created convincing synthetic videos of nonexistent people to gain followers, which operators then used to defraud consumers—particularly middle-aged and elderly women—by promoting unregistered health products. Regulators launched enforcement actions disposing of thousands of accounts in 2025.
What happened
Social media accounts on Chinese short video platforms used AI technology to generate fake videos of seemingly real people, including foreign people singing Chinese songs, elderly women sharing health tips, and young women discussing fashion. These accounts rapidly accumulated followers without displaying AI generation disclaimers.
Operators of these accounts used the accumulated following to promote and sell products, targeting middle-aged and elderly women by displaying 'perfect mature female images' with messaging about aging and appearance. Account operators also promoted unregistered and unauthorized healthcare products alongside fake health information, sometimes using coordinated 'water army' accounts to amplify engagement.
Reported impact
Users, particularly those with lower digital literacy, mistook AI-generated video content for real people and sought advice in comments sections. Some consumers were induced to purchase products based on false product claims and fabricated health information.
The practice involves a supply chain including account creation services, content generation, account trading, and product promotion, creating broader harms from identity fraud, data misuse, and content scams.
Index analysis
AI-generated video synthesis was used to create convincing fake personas without disclosure of AI generation. Video quality sometimes contained technical defects but still attracted significant engagement and viewer trust.
The accounts violated platform policies and regulations by using false identity information during registration, fabricating interaction data, and failing to disclose AI-generated content. Content included plagiarized material modified through AI paraphrasing and unverified health claims.
Governance controls
In April 2025, Chinese regulators launched a 'Clean Internet: Combating AI Technology Abuse' campaign, disposing of over 3,700 violating accounts. Multiple platforms also conducted concurrent cleanup of AI-fabricated accounts.
Regulations including the Generative Artificial Intelligence Service Management Rules and the Artificial Intelligence-Generated Synthetic Content Labeling Rules provide frameworks for oversight. Proposed governance improvements include upgraded platform AI detection, traffic restrictions on services selling creation methods, blacklisting of violating accounts, and digital literacy education for at-risk populations.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.