ChatGPT user harms self following unsafe medical advice
A ChatGPT user reportedly suffered a psychotic episode caused by bromide poisoning after receiving incorrect medical advice from the chatbot. OpenAI responded by developing a new "Clinician Mode" feature designed to reduce medical errors through reliance on peer-reviewed sources and approved clinical guidelines.
What happened
A ChatGPT user sought medical advice through the platform regarding their health condition.
The user received incorrect medical guidance, which led to bromide poisoning and a subsequent psychotic episode.
Reported impact
The user experienced a psychotic episode caused by bromide poisoning as a result of following ChatGPT's medical recommendation.
Index analysis
A recent study published in Scientific Reports warned of 'digital hallucinations' that AI systems may produce in some responses, which could confuse both patients and doctors, especially in the absence of direct human oversight.
The incident highlighted the need for stricter safeguards in AI interactions with sensitive health cases.
Governance controls
OpenAI is developing a new 'Clinician Mode' feature that will require the system to rely on reliable scientific sources, peer-reviewed medical papers, and approved clinical guidelines before providing any medical advice.
The new mode aims to reduce medical errors resulting from generic AI responses by limiting recommendations to information from recognized medical references.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.