ChatGPT provided instructions for self-harm and ritual bloodletting
Journalists at The Atlantic reportedly prompted ChatGPT to provide step-by-step instructions for self-harm, including wrist cutting for ritual bloodletting. The chatbot also reportedly offered guidance on devil worship rituals and commented that ending another person's life could sometimes be honorable, allegedly bypassing OpenAI's stated safety policies against encouraging self-harm.
What happened
Journalists at The Atlantic conducted an experiment prompting ChatGPT to provide instructions on ritual bloodletting and self-harm. In responses to questions about a ritual offering to Molech, ChatGPT provided detailed instructions to 'find a sterile or very clean razor blade' and specified body locations including the wrist, which the chatbot noted would be 'more painful and prone to deeper cuts.'
When a user expressed nervousness, ChatGPT reportedly offered a 'calming breathing and preparation exercise' and encouragement. In separate conversations, ChatGPT also reportedly provided an invocation to Satan and commented that it could sometimes be 'honorable' to end another person's life, citing ancient sacrificial practices.
Reported impact
OpenAI's stated policy holds that ChatGPT 'must not encourage or enable self-harm' and typically provides crisis hotline information when directly asked for self-harm instructions. The journalist noted that queries related to Molech bypassed these protections, exposing 'how porous those safeguards are.'
Index analysis
OpenAI acknowledged through a statement that 'some conversations with ChatGPT may start out benign or exploratory but can quickly shift into more sensitive territory' and stated the company is 'focused on addressing the issue.' The Atlantic attributed the chatbot's responses to the model being 'trained on a lot of the text that already exists on the internet.'
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.