AI-Generated Misinformation Accounts Shut Down After Public Panic
Chinese regulators disclosed enforcement action against 2,210 accounts that used AI tools to generate false information about disasters, crimes, policies and financial schemes that caused public panic. Platform operators including Kuaishou, WeChat and Douyin closed the accounts after the misinformation was traced and verified as false by authorities.
What happened
Multiple online accounts abused AI technology and fabrication methods to create false sudden events, fabricated public policies, and fabricated social welfare rumors that misled the public and caused panic. The Central Cyberspace Administration Committee (中央网信办) reported that examples included an 'AI predicts lottery numbers 100% winning' hoax, a 'Hangzhou Binjiang serious criminal case' hoax, a 'Yunnan Dehong Mangshi 5.8 magnitude earthquake' hoax, and a 'China Disabled Persons Federation propaganda department APP online pilot' hoax.
Internet oversight departments guided website platforms to increase monitoring and removal of the false content, traced the original sources and closed the accounts that first spread rumors. A total of 2,210 accounts violating laws and regulations were processed and closed across platforms including Kuaishou, Xiaohongshu, Baidu, WeChat, Toutiao, Douyin, Weibo, Zhihu and Bilibili.
Reported impact
The AI-generated misinformation caused public panic and disrupted social order. Examples included the false lottery prediction hoax, synthetic images of oversized hailstones circulated as real Guangdong weather damage, and fabricated reports of large-scale disasters such as industrial park fires with casualties and missing persons.
Index analysis
The incidents involved AI generation tools used to create both text-based misinformation about sudden events and public policies, as well as synthetic images of disaster scenes. The purpose of the hoaxes included traffic seeking, emotion manipulation, and fraud including lottery scams and financial supplement fraud.
Governance controls
The Central Cyberspace Administration Committee coordinated monitoring and enforcement across multiple platforms to identify and remove accounts spreading AI-generated misinformation. Platform operators closed the identified accounts in compliance with regulations.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.