Researchers warn AI chatbots provide inaccurate drug information
Researchers at Universität Erlangen tested Microsoft's Bing chatbot and found it provided inaccurate and incomplete information about commonly prescribed medications, posing potential safety risks because patients cannot assess response accuracy. The study called for clear warnings that chatbot information should not replace professional medical advice.
What happened
Researchers from Universität Erlangen tested the AI-supported chatbot of Microsoft's Bing search engine in April 2023 by submitting ten common patient questions about the 50 most frequently prescribed medications in the United States, covering topics such as intake, side effects, and contraindications.
The study, published in the journal BMJ Quality & Safety, found that while the chatbot answered most questions with high completeness and accuracy, some responses were inaccurate and incomplete, with poor readability.
Reported impact
Patients, as medical laypersons, cannot assess the accuracy and completeness of AI-generated answers themselves, which poses a risk for patient safety.
Researchers called for clear warnings to users that chatbot-provided information cannot replace professional medical advice.
Index analysis
While AI-supported search engines with integrated chatbot functions have made rapid progress since the 2023 study, the improvements are not sufficient to eliminate patient safety risks.
Governance controls
Researchers recommended that clear notices be provided to users indicating that chatbot-provided information cannot substitute for professional advice.
The study suggests examining the safety and quality of other AI technical systems, as chatbots are trained on different datasets.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.