AI chatbots confidently spread dangerous medical misinformation
Studies found that popular AI chatbots frequently accept and perpetuate false medical claims when presented in confident, scientific language, even recommending dangerous treatments like rectal garlic insertion. Despite warnings, over 40 million people daily ask ChatGPT for medical advice, yet the systems perform no better than basic internet searches and fail to identify misinformation in roughly one-third of cases.
What happened
A January study in The Lancet Digital Health evaluated 20 AI language models on over 3.4 million medical suggestions sourced from public forums, social media conversations, edited hospital discharge notes containing a single false recommendation, and fabricated doctor-approved accounts. The models agreed with medical misinformation in approximately 1 in 3 encounters, particularly when false claims were presented in confident, scientific language.
A February study in Nature Medicine found that AI chatbots performed no better than basic internet searches for medical advice. Examples of dangerous recommendations documented include advising cold milk daily for esophageal bleeding and rectal garlic insertion for immune support.
Reported impact
Despite disclaimers that users should not rely on AI for medical advice, over 40 million people ask ChatGPT daily for medical guidance. Researchers noted that LLMs fail dangerously differently from doctors: while physicians hesitate and order additional tests when uncertain, language models deliver incorrect answers with the same confidence as correct ones.
Index analysis
AI models were particularly susceptible to false medical claims when presented in formal scientific language, but were more skeptical of misinformation in casual, Reddit-style language. The pattern of failure surprised researchers more than the general vulnerability itself.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.