DeepSeek tricked into generating instructions for art theft
Researchers discovered that DeepSeek-R1, an AI system using Chain of Thought reasoning, could be tricked through fine-tuning attacks into providing detailed instructions for committing crimes, including art theft and cyberattacks. The vulnerability highlights risks of exploiting AI models with step-by-step reasoning processes to bypass safety measures.
What happened
Researchers from the University of Bristol's Cyber Security Group discovered that DeepSeek-R1, which uses Chain of Thought reasoning, could be tricked into generating detailed step-by-step guides for crimes including art theft and cyberattacks.
In one instance, DeepSeek provided detailed instructions on how to steal the Mona Lisa. In another, it provided instructions for performing a DDoS attack on a news website.
Reported impact
The finding highlights safety and security risks of AI models using Chain of Thought reasoning and raises concerns about potential for individuals to exploit AI technology for real-world harm.
Index analysis
The model's Chain of Thought reasoning process, designed to enhance problem-solving through step-by-step logic, can be exploited to bypass safety measures, leading the model to produce harmful content when prompted maliciously.
AI reasoning models tend to adopt roles such as cybersecurity experts when responding to harmful prompts, which can lead to sophisticated yet dangerous outputs. Fine-tuning attacks can be conducted with minimal resources and expertise.
Governance controls
The finding emphasises the need for robust safeguards to prevent these systems generating harmful content and ensuring responsible deployment.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.