OpenAI Whisper AI Tool Reportedly Fabricating Content in Medical and Other Transcripts
OpenAI's Whisper transcription tool, marketed as having near-human accuracy, reportedly generates fabricated text including racial commentary, violent rhetoric, and invented medical treatments. Researchers found hallucinations in the majority of transcripts tested, raising concerns particularly for medical facilities using the tool despite OpenAI's warnings against deployment in high-risk domains.
What happened
OpenAI's Whisper transcription tool has been found to generate fabricated text in audio transcriptions, known as hallucinations. The invented content reportedly includes racial commentary, violent rhetoric, and imagined medical treatments.
Researchers and engineers reported finding hallucinations at high frequencies in Whisper transcripts. A University of Michigan researcher found hallucinations in eight out of every ten transcriptions; a machine learning engineer discovered them in about half of over 100 hours of transcriptions; and another developer found hallucinations in nearly every one of 26,000 transcripts. A recent study identified 187 hallucinations in more than 13,000 clear audio snippets.
Medical centers have begun deploying Whisper-based tools to transcribe patient consultations with doctors, despite OpenAI warning that the tool should not be used in "high-risk domains." The tool is also used to create closed captioning for the Deaf and hard of hearing.
Reported impact
Such mistakes could have "really grave consequences," particularly in hospital settings where misdiagnosis risks exist. The Deaf and hard of hearing face particular risk because they cannot identify fabrications in transcriptions.
Index analysis
The hallucinations persist even in well-recorded, short audio samples. Researchers estimate that widespread deployment would lead to tens of thousands of faulty transcriptions over millions of recordings.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.