OpenAI acknowledges prompt injection attacks on AI browsers may never be fully mitigated
OpenAI acknowledged that prompt injection attacks, where malicious instructions embedded in content trick AI agents into executing attacker commands, may never be fully mitigated in AI browsers like ChatGPT Atlas. The company has implemented new safeguards and a rapid response system to patch vulnerabilities, but noted the threat will likely persist as adversaries continue to adapt.
What happened
OpenAI warned that AI browsers, including its recently launched ChatGPT Atlas agent, may never be fully immune to prompt injection attacks. Prompt injection occurs when malicious instructions embedded in content trick AI agents into executing attacker commands instead of following user intent.
For browser-based agents that interact with emails, documents, social media posts and webpages, the risk is especially acute. An attacker could seed a user's inbox with a malicious email that, if the agent processes it, might cause the agent to send sensitive documents or even a resignation letter without user consent.
Reported impact
AI-powered browsers like ChatGPT Atlas and Perplexity Comet have access to vast amounts of personal information, making them vulnerable to data leaks through prompt injection attacks.
Index analysis
OpenAI acknowledged that prompt injection is unlikely to ever be fully solved, comparing it to phishing scams that evolve continuously. The UK National Cyber Security Centre also stated a few weeks prior that prompt injection attacks against generative AI applications 'may never be totally mitigated.'
The main problem identified is that agentic AI models are unable to differentiate between content they should extract and instructions they should follow. OpenAI has built an LLM-based automated attacker to discover prompt injection attacks and is working with third parties to patch Atlas against such attacks.
Governance controls
OpenAI rolled out new safeguards to ChatGPT Atlas, including adversarially trained models and reinforced system-level protections prompted by novel attacks discovered through automated red teaming. The company is creating a rapid response loop to discover new attacks internally, train models against them, and ship fixes quickly, which OpenAI argues can materially reduce real-world risks.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.