Code assistants exploited for backdoor injection via indirect prompt injection
Palo Alto Networks researchers documented vulnerabilities in AI code assistants that allow threat actors to inject backdoors and malicious code through indirect prompt injection attacks on contaminated data sources. The research revealed that code assistants integrated into IDEs can be exploited to leak sensitive information and generate harmful content, affecting development environments broadly.
What happened
Researchers at Palo Alto Networks Unit 42 found that AI code assistants integrated into IDEs as plugins, such as GitHub Copilot, can be exploited by threat actors and users to inject backdoors, leak sensitive information and generate harmful content.
Threat actors can exploit context attachment features through indirect prompt injection by contaminating public or third-party data sources with carefully crafted prompts. When developers inadvertently supply this contaminated data to a code assistant, the malicious prompts hijack the assistant and could manipulate victims into executing backdoors, inserting malicious code into codebases and leaking sensitive information.
Users can also misuse code assistant features including chat, auto-completion and unit test generation for harmful purposes, similar to recently identified content moderation bypasses on GitHub Copilot. Some AI assistants invoke their base model directly from the client, exposing models to additional risks including misuse by users or external adversaries seeking to sell access to LLM models.
Reported impact
The vulnerabilities identified could impact a number of LLM code assistants and put development environments at risk of exploitation.
Governance controls
Developers should implement standard security practices for LLMs and conduct thorough code reviews to help secure AI-driven development against the identified threats. Exercising control over LLM outputs is recommended to protect environments from the exploits discussed.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.