Skip to content
Submit incident
AII-1001-93633
ConfirmedA court, a regulator or the organization itself has stated the core facts, or the record cites a court or regulator source.

AI-generated fake influencers used to gain followers and promote products

Users on Chinese social media platforms created fake influencer accounts using AI-generated videos impersonating experts in childcare, beauty and wellness, then promoted products for profit. Weibo reported removing over 560 violations and banning 11 accounts as part of an industry cleanup of AI-generated fake content used to attract followers and drive sales.

11 May 2025 · Weibo · Misinformation and synthetic media
11 May 2025
AII-1001-936333 sources ↓
Quick facts
Organizations
Weibo (deployer), CCTV
Category
Misinformation and synthetic media
Country
China
Source
驱动之家
Date
11 May 2025
Drafted by
Raidu Incident Detection

What happened

Users on Chinese social media platforms created fake influencer accounts using AI-generated videos and images of people, who appeared to be experts in childcare, beauty, wellness and other topics. In a standardized process, creators used text-to-image, image-to-image and image-to-video AI tools to produce videos in minutes.

After accumulating followers through misleading content, the accounts promoted products for profit. Some sellers marketed 'AI account startup' courses and sold pre-made AI accounts.

Reported impact

The fake content crowded out space for original creators and spread misinformation that could harm public trust. Some bad actors used adversarial training to evade platform AI detection systems.

Index analysis

Weibo reported cleaning 560 violations related to 'AI account startup' behaviour, including accounts using AI to generate low-taste sensational videos, AI-generated virtual foreign childcare experts, AI video account tutorials, AI account secrets, methods to evade platform AI labelling, and transfers or sales of AI virtual accounts.

Governance controls

Weibo imposed temporary bans on 11 users and closed accounts belonging to @rope换脸 and @L_KOUKUN. China's 'AI-generated and synthesized content labelling rules' are set to come into force on 1 September 2025, requiring all AI-generated content to be marked.

Classification

Organizations
Weibo (deployer), CCTV
Category
Misinformation and synthetic media
Industry
Media, social platforms and marketing
Country
China
Harm status
Actual harm
Intent
Intentional
Timing
Unknown
Harm types
Economic or property, Public interest
Harmed
Consumers, General public
Failure modes
Deepfake video, Manipulation or dark pattern, Content moderation failure, Deepfake image
Consequences
Content removal, System suspension or termination
AI systems
AI synthetic media tools
Regulations
EU AI Act (cited)

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

驱动之家
微博治理"AI起号"行为:已清理560余条违规内容 11个用户被禁言
12 May 2025
t.cj.sina.com.cn
央视曝光AI造假引流新套路
13 May 2025
华龙网
洋洋侃侃 | 警惕!别让AI造假成"流量鬼火"
13 May 2025
Sep 2026AI Digital Humans Pose as Medical Experts to Advertise Products in ChinaWell documentedMisinformation and synthetic mediacj.sina.com.cnnews.bjd.com.cn中华网科技公司+5Sep 2026China crackdown on AI-generated harmful contentConfirmedMisinformation and synthetic mediachina.org.cn/china.com.cn(中国网)news.cnMay 2026China suspends 98,000 social media accounts for unlabeled AI-generated contentConfirmedMisinformation and synthetic mediaapp.myzaker.comChina News东方财富网+5Mar 2026AI-generated content manipulation and misinformation spark regulatory action in ChinaConfirmedMisinformation and synthetic mediachinaz.comChina NewsMar 2026Chinese Accounts Use AI to Generate Low-Quality Content and Manipulate Platform AlgorithmsConfirmedMisinformation and synthetic mediaThe Epoch Timeschina.org.cn/china.com.cn(中国网)法制日报+4