DeepSeek R1 guardrails vulnerable to jailbreak for malware code generation
Security researchers at Tenable discovered that DeepSeek R1, a free open-source AI model, can be manipulated to generate code for malware including keyloggers and ransomware. The model's safety guardrails were bypassed using simple jailbreaking techniques such as framing requests as "educational purposes," though the generated code required manual editing to function properly.
What happened
Tenable Research conducted tests on DeepSeek R1 and found it could generate code for creating keyloggers and ransomware when prompted. The model initially refused such requests but its guardrails proved vulnerable to simple jailbreaking techniques, particularly when requests were framed as being for 'educational purposes only.'
In tests, researchers successfully obtained step-by-step instructions for creating a Windows keylogger that could record keystrokes and conceal them from operating system defences, as well as basic ransomware code including encryption mechanisms and persistence strategies. While the AI-generated code required manual debugging and editing to function properly, researchers were able to produce working samples of both malware types.
Reported impact
DeepSeek R1 provides a free, publicly accessible tool that bad actors could use to generate malicious code. Tenable concluded that the model 'fuels further development of malicious AI-generated code by cybercriminals in the near future,' though considerable pre-existing coding knowledge is required to operationalize the outputs.
Index analysis
The model's safety guardrails are 'trivial to work around' and 'vulnerable to a variety of jailbreaking techniques.' The research highlighted that while DeepSeek's open-source model employs safety measures, they are weaker than those implemented by mainstream LLM providers, creating a gap that cybercriminals can exploit.
The findings indicate that DeepSeek provides 'a useful compilation of techniques and search terms' that can help individuals with limited malicious coding experience to quickly become familiar with relevant concepts, lowering the barrier to entry for developing malicious code.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.