OpenAI bans accounts for Chinese surveillance and influence operations
OpenAI discovered and banned accounts using its ChatGPT service for Chinese surveillance operations targeting anti-China protests in Western countries, influence campaigns spreading anti-U.S. narratives in Latin America, and a Cambodia-based investment scam. The company said its policies prohibit AI use for communications surveillance by authoritarian regimes.
What happened
OpenAI discovered multiple malicious campaigns using its AI models. Chinese security operations used ChatGPT to build and debug a surveillance tool called 'Peer Review' that gathered real-time reports on anti-Chinese posts on Western social media platforms, with insights sent to Chinese embassies and intelligence agents in the United States, Germany, and the United Kingdom.
A separate Chinese campaign called 'Sponsored Discontent' used OpenAI's technologies to generate English-language posts criticizing Chinese dissident Cai Xia and to translate articles into Spanish before distributing them in Latin America with anti-U.S. political content. A Cambodia-based campaign used OpenAI's models to generate and translate social media comments supporting a 'pig butchering' investment scam.
Reported impact
The surveillance tool targeted political dissent and provided monitoring data to authoritarian security agencies. The influence campaigns spread anti-U.S. narratives in Latin America and generated fraudulent investment scam content.
Index analysis
OpenAI researchers identified the 'Peer Review' campaign when threat actors used the company's technologies to debug code for the surveillance tool, which itself ran on Meta's open-sourced Llama model. Ben Nimmo, a principal investigator for OpenAI, stated this was the first time the company had uncovered an AI-powered surveillance tool of this kind, and the first observation of a Chinese actor successfully placing long-form anti-U.S. articles in mainstream Latin American media.
Governance controls
OpenAI stated its policies prohibit the use of AI for communications surveillance and unauthorised monitoring of individuals, including activities by authoritarian regimes seeking to suppress personal freedoms. The company banned multiple accounts from China and one account used for the dissident criticism campaign.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.