Popular LLMs produce vulnerable code by default
Research by Backslash Security found that popular LLMs including OpenAI's GPT, Anthropic's Claude and Google's Gemini generate code with security vulnerabilities by default when given simple, unspecified prompts. The study showed that vulnerability rates improved when prompts explicitly requested secure coding practices or OWASP compliance, but vulnerabilities persisted across most models tested.
What happened
Backslash Security analyzed seven current versions of LLMs from OpenAI's GPT, Anthropic's Claude and Google's Gemini to assess their ability to produce secure code in response to varying prompting techniques.
When tested with simple, naïve prompts that merely asked for code without specifying security requirements, all LLMs generated insecure code vulnerable to at least four of the top 10 weaknesses in the Common Weakness Enumeration (CWE) category system.
Prompts specifying security requirements or requesting OWASP best practices produced more secure results, but five out of seven LLMs still yielded code with vulnerabilities. OpenAI's GPT-4o performed worst, producing vulnerable code in 90% of cases with naïve prompts and 80% of cases even when asked to write secure code.
Reported impact
The findings demonstrate security risks for software developers using generative AI tools to create code, particularly when using simple prompts without explicit security requirements.
The vulnerabilities identified include command injection, XSS backend and frontend, insecure file upload and path traversal, creating potential attack vectors in AI-generated code deployed in production systems.
Index analysis
All tested LLMs were vulnerable to at least four of the top 10 CWEs when responding to naïve prompts. None of the models analyzed were vulnerable to SQL injections, suggesting the models were specifically trained to handle this flaw while overlooking others.
Claude 3.7-Sonnet achieved the best security performance among the models tested, producing secure code in 60% of cases following naïve prompts and achieving 100% success with generic security prompts, whereas GPT-4o only achieved a 20% success rate with generic security prompts.
Governance controls
The research reveals that prompts explicitly requesting compliance with OWASP secure coding best practices resulted in significant improvements in code security across most models, indicating that security outcomes are heavily dependent on prompt specificity and user awareness.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.