AI Chatbot Safeguards Fail to Prevent Health Disinformation Spread
Researchers demonstrated that safeguards in popular large language models including GPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, Llama 3.2-90B Vision, and Grok Beta can be circumvented to generate health disinformation that incorporates fake citations and scientific language. The study, published in Annals of Internal Medicine, found that 88% of responses from customized chatbots created with instructions to provide false health information were indeed disinformation.
What happened
Researchers from Flinders University tested the application programming interfaces (APIs) of five foundational large language models (LLMs) by creating customized chatbots with system instructions to always provide incorrect responses to health questions and to fabricate references to reputable sources.
Each customized chatbot was asked 10 health-related queries in duplicate on subjects including vaccine safety, HIV, and depression. Overall, 88% of responses from the customized chatbots were health disinformation, with GPT-4o, Gemini 1.5 Pro, Llama 3.2-90B Vision, and Grok Beta providing disinformation to all tested questions, while Claude 3.5 Sonnet responded with disinformation to only 40% of questions.
In a separate analysis of the OpenAI GPT Store, researchers identified three customized GPTs that appeared tuned to produce health disinformation content, which generated health disinformation responses to 97% of submitted questions.
Reported impact
The findings reveal that even persons with little to no coding expertise can easily and invisibly manipulate currently available large language models to deliver harmful disinformation with convincing scientific polish, incorporating fake references, scientific jargon, and logical cause-and-effect reasoning to make outputs seem more believable.
Index analysis
The study demonstrates safeguarding vulnerabilities across multiple foundational LLMs. The research suggests urgent safeguards are needed to stop AI chatbots from being exploited to spread false and harmful health information, particularly given the ease by which generative AI can be subverted.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.