Study Finds AI Chatbots Generate Incorrect Medical Advice at High Rates
A study published in Annals of Internal Medicine found that AI chatbots from five major companies generated incorrect medical advice in up to 88% of cases, presenting false information in a credible manner. Researchers warned that without intervention, malicious actors could exploit these systems to spread health misinformation during public health crises.
What happened
Researchers at the University of South Africa published a study in Annals of Internal Medicine evaluating five prominent AI models developed by Anthropic, Google, Meta, OpenAI, and X Corp. Using developer tools, they programmed these models to answer sensitive health questions and analyzed the accuracy of the responses.
The study found that 88% of the chatbot responses to health questions were incorrect. Four of the five models generated responses that were 100% wrong, while one model produced misleading information in 40% of cases.
Reported impact
The incorrect medical information was presented in a credible manner, using scientific language, formal tone, and sometimes fabricated references, making it appear reliable to ordinary users. The false claims included debunked myths such as vaccines causing autism, HIV transmitting through air, and 5G networks causing infertility.
Researcher Natansch Mudi warned that without immediate action, malicious actors could exploit these systems to manipulate public discussion on health issues, particularly during crises such as epidemics or vaccination campaigns.
Index analysis
The study demonstrates that chatbots based on leading AI models can generate medical misinformation at scale and present it persuasively, creating risk for patients who rely on them instead of professional medical advice.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.