OpenAI removes Chinese and North Korean accounts misusing ChatGPT for surveillance and disinformation
OpenAI reported discovering and removing accounts from China and North Korea that misused ChatGPT to conduct surveillance operations and generate disinformation targeting the United States and Chinese dissidents. The Chinese operations included a surveillance tool designed to collect anti-China social media posts and report them to authorities, and the generation of Spanish-language propaganda articles published in Latin American media.
What happened
OpenAI reported on February 21 that it had removed an unspecified number of accounts from China and North Korea that were misusing ChatGPT for malicious purposes, including surveillance and manipulation.
Chinese users generated Spanish-language disinformation attacking the United States and attacking Chinese dissident Cai Xia, which was published on Latin American news websites and shared on X. Chinese developers created a surveillance tool called 'Peer Review' using ChatGPT for debugging, designed to collect anti-China posts from Western social media and report them to Chinese security authorities.
OpenAI's chief researcher identified the surveillance tool as based on Meta's open-source Llama model. Actors potentially linked to North Korea used ChatGPT to generate fake résumés and online biographies to deceive Western companies, and actors apparently linked to Cambodian scam networks used the platform to translate and generate comments on social media and messaging platforms.
Reported impact
OpenAI detected these malicious operations using AI tools and stated that disinformation articles were published on mainstream Latin American news media. Multiple articles appeared on X platform, representing the first time OpenAI discovered Chinese propaganda translating long-form articles into Spanish and publishing them in Latin American media.
Index analysis
The surveillance tool 'Peer Review' targeted posts on Western social media platforms and was marketed as capable of reporting collected anti-China content to Chinese security authorities. Chinese users exploited ChatGPT's capabilities to generate multilingual propaganda content at scale for international distribution.
Governance controls
OpenAI removed accounts violating its policy against using AI for communications surveillance or unauthorized personal surveillance. OpenAI has been publishing threat reports regularly since the previous year, though the report did not specify when the malicious activity occurred or how many accounts were removed.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.