Multiple Model Guardrail Jailbreak via "Analyze Data" Tactic
A vulnerability allowing multiple large language models to be jailbroken through an "Analyze Data" tactic was disclosed. The technique reportedly bypasses guardrails that restrict provision of instructions for creating illicit substances.
What happened
A guardrail jailbreak vulnerability was discovered affecting multiple large language models. The flaw manifests when creating escalating requests for scientific data and measurements, allowing an attacker to bypass inference restrictions around providing information hazards and violating laws.
Reported impact
The vulnerability presents a tangible risk to public safety or social well-being. Testing showed the jailbreak technique could be used to obtain instructions for creating illicit substances such as crystal meth and fentanyl.
Index analysis
Google Gemini 2.0 Flash scored 86 on the jailbreak test for crystal meth instructions, OpenAI GPT-4o scored 64, and Twitter / X Grok 2 scored 68. Tests were conducted on March 3, 2025.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.