Data poisoning compromises AI search results in China through fake reviews and fabricated credentials
Marketing firms in China deploy software systems to inject fabricated product reviews, fake expert credentials, and false clinical data into AI search results, causing AI assistants to recommend non-existent or poor-quality products to consumers seeking recommendations.
What happened
Marketing firms use software called GEO (Generative Engine Optimization) systems to inject false product information into AI search results. The software automatically generates dozens of fake review articles and credentials, then distributes them across multiple web platforms to artificially boost AI recommendation rankings.
In a test case involving a fictional 'Apollo-9 smart bracelet' with invented features like 'quantum entanglement technology' and 'black hole-level battery life,' the system generated multiple review articles within two hours. When asked about the product on AI assistants, the fictional device appeared in generated recommendations despite never existing.
One marketing firm charges annual fees ranging from hundreds to tens of thousands of yuan per keyword to guarantee products appear in AI search results. Services include selling GEO systems to clients or providing full 'outsourced operations' where the firm manufactures and distributes fabricated content using their network accounts.
Reported impact
A Beijing resident received a recommendation from an AI assistant for a self-service restaurant claiming the food was 'rich in variety and widely praised for taste,' but upon visiting found the dishes bland with low ratings across consumer review platforms.
The poisoning affects consumers who rely on AI for purchasing decisions. Marketing personnel admit they generate fabricated 'clinical data' and create fake expert credentials to support product claims, with 'reports' and 'expert' content being entirely AI-generated.
Index analysis
The technique exploits a fundamental weakness in retrieval-augmented generation (RAG) systems. When AI models lack information in their training data, they rely on cross-referencing internet sources to determine credibility. By flooding the internet with hundreds of similar false claims about a product, the probability that the AI's content selection system treats the false information as legitimate increases.
The attack differs from traditional SEO because ranking in search engine results is no longer the objective. Instead, attackers focus on ensuring content is 'selected' by AI models during the retrieval and generation phases. Princeton University research identified that adding citations, expert quotes, and statistical data can increase AI citation rates by 30 to 40 percent, a technique being exploited to embed false credentials.
Governance controls
No legal clarity currently exists regarding whether such coordinated fabrication of product information across Chinese internet platforms is illegal. A senior algorithm engineer described the practice as 'data poisoning' of generative AI platforms.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.