China: AI deepfakes and data leaks pose national security risks
China's Ministry of State Security warned that AI systems pose national security risks through deepfake videos spread by foreign hostile forces, data leaks exposing personal information, and manipulated social media content aimed at social destabilization. The ministry identified legitimate uses of AI in entertainment but cautioned against misuse and urged Chinese citizens to strengthen security awareness and information verification practices.
What happened
According to the Ministry of State Security, unspecified foreign anti-China hostile forces have used deepfake technology to generate false videos for circulation within China to mislead public opinion and create panic.
The Ministry reported that due to an open-source library malfunction, users of an unspecified foreign AI model could see chat records of other active users including titles, names, payment addresses and other private information, causing personal information leaks that could expose personnel to espionage and subversion activities.
The Ministry stated that foreign countries and external forces have manipulated social media accounts in bulk to spread specific viewpoints or emotional content, attempting to exploit gender conflicts and labor disputes to infiltrate and damage China, and damage social stability.
Reported impact
The Ministry identified risks to China's critical data, basic infrastructure and social order from misuse of AI tools by unauthorized persons.
AI-generated false information was described as potentially misleading public opinion and causing severe impacts during early stages of social events when circumstances are unclear and public emotions accumulate.
Index analysis
The Ministry warned that deepfake capabilities combining deep learning with forgery could be misused to threaten national security, though the technology has legitimate uses in film, gaming and advertising industries.
Generative AI's strong learning capabilities create risks because the systems collect and store user input information needed for their learning and decision-making processes.
Governance controls
China has issued the Generative AI Service Management Interim Measures and released an AI Safety Governance Framework to strengthen AI governance within the legal framework.
The Ministry advised citizens to cultivate critical thinking regarding AI-generated content, verify information before sharing, avoid uploading personal privacy information or sensitive files to AI platforms, and limit AI tool permissions to prevent non-essential access to contact lists, photo albums and location data.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.