Whisper AI Transcription Tool Shows High Hallucination Rates in Medical Use
OpenAI's Whisper transcription tool, deployed in hospitals by companies like Nabla across 85+ organizations for millions of medical conversations, reportedly generates hallucinations at high rates—including racial commentary, violent rhetoric, and fabricated medical treatments—that researchers warn could lead to misdiagnosis despite OpenAI's own warnings against high-risk domain use.
What happened
OpenAI's Whisper transcription tool, promoted as having near human-level accuracy, is reportedly prone to fabricating text or entire sentences, a phenomenon known as hallucinations in the AI industry.
Researchers and software engineers found hallucinations across multiple studies: a University of Michigan researcher found hallucinations in eight of ten audio transcriptions, a machine learning engineer found them in about half of 100+ hours analyzed, another developer found them in nearly every one of 26,000 transcripts, and a study of 13,000+ clear audio snippets uncovered 187 hallucinations.
Healthcare company Nabla has used Whisper-based tools to transcribe seven million medical conversations between patients and doctors across 45,000 clinicians and over 85 organizations including the University of Iowa Health Care, despite OpenAI warnings against use in high-risk domains.
Reported impact
The hallucinations can include racial commentary, violent rhetoric, and imagined medical treatments, raising concerns about misdiagnosis and grave consequences in hospital settings.
Former White House Office of Science and Technology Policy director Alondra Nelson stated such mistakes could have 'really grave consequences' in hospitals and called for a higher bar in medical AI adoption.
Index analysis
The prevalence of hallucinations suggests that over millions of recordings, tens of thousands of faulty transcriptions could result, creating widespread risks in medical documentation.
Governance controls
OpenAI has issued warnings that Whisper should not be used in high-risk domains such as healthcare, but medical facilities have proceeded with adoption despite these cautions.
Experts, advocates, and former OpenAI employees have called for federal government consideration of AI regulations and for OpenAI to prioritize addressing the hallucination flaw.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.