OpenAI uncovers Chinese surveillance tool 'Peer Review' targeting Western social media
OpenAI discovered that a Chinese security operation built an AI surveillance tool called Peer Review to monitor anti-Chinese sentiment on Western social media platforms. The operators used ChatGPT to debug code for the tool and generate content, and OpenAI subsequently banned their accounts.
What happened
OpenAI uncovered evidence that a Chinese security operation built an AI-powered surveillance tool called Peer Review to gather real-time reports about anti-Chinese posts on social media services in Western countries. The campaign's code was partially based on an open-source version of Meta's Llama model.
The operators used ChatGPT to debug and edit code for the surveillance tool, which was designed to monitor anti-Chinese sentiment on X, Facebook, YouTube, Instagram and other platforms. They prompted ChatGPT to generate sales pitches for the program and proofread claims that insights had been sent to Chinese embassies abroad and to intelligence agents monitoring protests in the United States, Germany and the United Kingdom.
The same group used ChatGPT to generate social media posts and articles critical of Cai Xia, a Chinese political scientist and dissident living in exile in the US, and to generate articles in Spanish critical of the US that were published by mainstream news organizations in Latin America.
Reported impact
The operation was particularly interested in spotting calls for protests against human rights violations in China, with the intent of sharing those insights with China's authorities.
Index analysis
OpenAI identified the campaign after discovering that someone working on the tool used OpenAI's technologies to debug computer code. The network consisted of ChatGPT accounts that operated in a time pattern consistent with mainland Chinese business hours, prompted models in Chinese, and used tools with a volume and variety consistent with manual prompting rather than automation.
Governance controls
OpenAI banned the accounts of the Chinese users who used ChatGPT to debug and edit code for the surveillance tool. OpenAI stated that assessing the impact of the activity would require inputs from multiple stakeholders, including operators of open-source models.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.