UC San Francisco study finds ChatGPT overprescribes in emergency care
Researchers at UC San Francisco found that ChatGPT-3.5 and ChatGPT-4 tend to overprescribe unnecessary medical interventions such as X-rays and antibiotics when given emergency department decision-making tasks, performing less accurately than resident physicians. The study suggests clinicians should not rely on the models for complex medical situations requiring multiple clinical considerations.
What happened
Researchers from UC San Francisco tested ChatGPT-3.5 and ChatGPT-4 on emergency department decision-making tasks using 1,000 ED visits each from an archive of over 251,000 visits, asking the models whether to admit patients, order X-rays or scans, or prescribe antibiotics based on doctors' notes.
Overall, both models tended to recommend services more often than was needed. ChatGPT-4 was 8% less accurate than resident physicians, and ChatGPT-3.5 was 24% less accurate. The researchers noted that while ChatGPT can answer medical exam questions and help draft clinical notes, it is not currently designed for complex situations requiring multiple considerations like emergency departments.
Reported impact
The study found that if ChatGPT were used in the Emergency Department, it might suggest unneeded X-rays and antibiotics for some patients and admit others who did not require hospital treatment.
Index analysis
The researchers suggested the AI's tendency to overprescribe could be because the models are trained on the internet, where legitimate medical advice sites are not designed to answer emergency medical questions but rather to send readers to a doctor.
Lead author Chris Williams stated this is a valuable message to clinicians not to blindly trust these models.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.