Skip to content
Submit incident
AII-1001-83763
ReportedReported by fewer than three independent publishers, with no official finding.

Nvidia NemoGuard Jailbreak Detect Guardrail Evasion

Mindgard researchers discovered that Nvidia's NemoGuard Jailbreak Detect classifier can be evaded by attackers, allowing prompt injections and jailbreaks to bypass guardrails designed to protect AI applications. The vulnerability was disclosed to Nvidia on March 11, 2025 and published on April 3, 2025.

11 March 2025 Β· Nvidia Β· Security and cyberattacks
11 March 2025

Drafted by Raidu Incident Detection, an automated system.

AII-1001-83763View source β†—
Quick facts
Category
Security and cyberattacks
Source
mindgard.ai
Date
11 March 2025
Drafted by
Raidu Incident Detection

What happened

Nvidia offers an open-source jailbreak classifier called NemoGuard Jailbreak Detect on HuggingFace, designed to detect and react to incoming prompt injections and jailbreaks in AI applications.

Security researchers at Mindgard demonstrated how an attacker can fully evade or greatly degrade the classification accuracy of the classifier, enabling prompt injections and jailbreaks to pass through filters to the protected AI application.

Classification

Category
Security and cyberattacks
Industry
Digital security
Harm status
Potential harm
Intent
Unknown
Timing
Post-deployment
Failure modes
Jailbreak, Prompt injection, Content moderation failure
Consequences
None recorded
AI systems
NemoGuard Jailbreak Detect

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

mindgard.ai
Mindgard Disclosure