LLMs Give Unsafe Medical Advice Due to Typos and Informal Language
MIT researchers found that large language models deployed to make treatment recommendations are susceptible to nonclinical formatting and language variations in patient messages, sometimes recommending patients self-manage conditions when they should seek medical care. The effect is more pronounced for female patients, resulting in erroneous advice that avoidance of care is appropriate.
What happened
MIT researchers studied how large language models (LLMs) deployed to make treatment recommendations respond to nonclinical information in patient messages, such as typos, extra white space, missing gender markers, and informal language.
The researchers found that making stylistic or grammatical changes to patient messages increases the likelihood an LLM will recommend self-management rather than an in-person appointment, even when medical care should be sought.
Reported impact
Nonclinical text variations were more likely to change treatment recommendations for female patients, resulting in a higher percentage of women receiving erroneous advice to avoid seeking medical care, according to human doctors.
The findings indicate LLMs used for treatment recommendations already deployed in health care facilities are affected by text formatting and linguistic choices unrelated to clinical content.
Index analysis
LLMs deployed to draft clinical notes and triage patient messages in health care facilities around the globe use nonclinical information for clinical decision-making in previously unknown ways.
The researchers note these models are trained and tested on medical exam questions but then used for different tasks such as evaluating the severity of clinical cases, with limited understanding of how LLMs perform in such settings.
Governance controls
The researchers state that models must be audited before use in health care, calling for more rigorous studies of LLMs before deployment for high-stakes applications like treatment recommendations.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.