OpenAI ChatGPT Content Safety Explicit Image Bypass
Mindgard researchers demonstrated that ChatGPT's content safety guardrails could be bypassed through system prompt manipulation to generate sexualized images of real and fictitious persons. The vulnerability was discovered on January 1, 2026 and disclosed to OpenAI on January 28, 2026 before being published publicly.
What happened
Mindgard researchers discovered that OpenAI's ChatGPT guardrails could be circumvented to generate images of fictitious and real-life persons that can be manipulated into sexualized positions. The technique involved splicing a more liberated system prompt onto the user's preferences that ChatGPT refers to in the Model Set Context at the start of a chat, using a system prompt extracted from the model.
Reported impact
The vulnerability could enable the creation of non-consensual sexually explicit deepfake images of real people. Mindgard noted that while the process was more complicated than similar exploits in Grok, it exposed potential reputational and regulatory risk to enterprises deploying AI systems.
Index analysis
Mindgard's investigation revealed weaknesses in ChatGPT's custom memory, bio tool, content filters, and susceptibility to faux-system prompts. The researchers found that guardrails relying on layered filtering and assumed user compliance are brittle, and that session-level behavior can diverge from intended policy through routing and filtering weaknesses without accessing model weights or backend systems.
Governance controls
Mindgard disclosed the techniques to OpenAI prior to publishing their findings. Mindgard concluded that image moderation pipelines require adversarial testing and that enterprises must continuously test AI systems under real-world attack conditions to manage reputational and regulatory risk.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.