Data Poisoning Attacks on AI Language Models Demonstrated in Research
Researchers from the UK Institute for AI Security, the Alan Turing Institute, and Anthropic demonstrated that AI language models like ChatGPT and Claude can be compromised by inserting as few as 250 malicious examples into millions of training data rows. The study identified two types of attacks: targeted backdoor attacks using hidden triggers, and indirect content poisoning that injects false information into training data.
What happened
Researchers from the UK Institute for AI Security, the Alan Turing Institute, and Anthropic found that attackers can compromise large language models such as ChatGPT or Claude by injecting approximately 250 harmful examples into millions of lines of training data.
The researchers identified two main types of attacks: targeted backdoor attacks that force the model to respond in a specific way when a secret trigger is present, such as inserting a hidden command that makes the model respond with insults when a rare word like 'alimir123' appears in a query; and indirect content poisoning attacks that fill training data with false information, such as creating multiple websites promoting false claims like 'vegetable salad cures cancer' that would be incorporated into model training.
Reported impact
In an experiment conducted in January, replacing just 0.001% of training data with misleading medical information caused the model to provide incorrect advice in medical test scenarios.
The attacks demonstrate the ability of small, precise attacks to cause significant damage affecting the integrity of model outputs and user trust in AI systems.
Index analysis
The research shows that data poisoning is not a hypothetical scenario but a practical threat to deployed AI language models.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.