AI-generated content manipulation and misinformation spark regulatory action in China
Chinese regulators and platforms responded to AI-generated misinformation and content manipulation following a CCTV exposé on AI model 'poisoning'. Douyin removed thousands of accounts distributing AI-generated obscene content, while GEO services enabling AI recommendation manipulation faced platform removal, prompting industry debate over whether such practices constitute deceptive marketing.
What happened
Douyin reported processing 14,000 violating accounts and removing 42,000 pieces of content generated using AI to create obscene and vulgar material. According to Douyin, 339 accounts had their live streaming rights permanently revoked and criminal detention was initiated against relevant teams.
Services marketed as GEO (generative engine optimization) became controversial after China's CCTV 3·15 gala reported on AI model 'poisoning'. GEO services help products rank higher in AI recommendations by publishing articles containing product information that influences AI data sources. Concerns emerged that GEO services may violate advertising disclosure principles, constitute false advertising, or enable unfair competition.
Reported impact
GEO service listings were removed from multiple online trading platforms in March 2026, though merchants continued offering similar services under alternative keywords like 'AI recommendation' and 'SEO'. Merchants reported various pricing models for GEO services, ranging from 288 yuan for single-use services to annual subscriptions of 299 to 2,980 yuan.
Index analysis
The incidents reflect tensions between generative AI capabilities and content moderation standards. Douyin's enforcement actions and the GEO controversy indicate platform responses to government pressure and public concern over AI-driven misinformation and manipulated search rankings.
ByteDance postponed the global launch of Seedance 2.0 due to legal and compliance challenges, citing copyright infringement concerns from Hollywood production companies over the high-fidelity videos generated by the model.
Governance controls
Tencent introduced 'Lobster Manager', an AI security sandbox providing local AI agents with permission management, malicious injection interception, and privacy protection features.
Youzan clarified that its invested enterprises were unrelated to the CCTV 3·15 disclosure, and stated its AI marketing business operates within safety and compliance boundaries, opposing 'poisoning-style' marketing.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.