AI-Generated Fabricated Citations Found in Biomedical Papers
Researchers auditing millions of biomedical papers discovered over 4,000 citations to non-existent studies, with the rate of fabricated references growing 12-fold in three years. AI tools used to draft or polish research papers are inadvertently inserting fake citations that slip through peer review, potentially influencing clinical guidelines and patient care decisions.
What happened
An audit of millions of biomedical papers found more than 4,000 citations to non-existent research, according to findings published in The Lancet by researchers including Maxim Topaz, an associate professor at the Columbia School of Nursing.
The fabricated references spanned nearly 3,000 academic papers, with the number of such erroneous citations having grown 12-fold over the last three years.
AI tools used by researchers to polish scientific papers are inadvertently inserting fake citations. In some cases, the AI tool cites a real author while inventing research and attributing it to that person; in other cases, citations are completely fabricated.
Reported impact
None of the mistakes identified have been corrected or retracted and could still be influencing patient care.
Fabricated citations undermine clinical guidelines that health care professionals rely on to provide care, potentially influencing treatment decisions for patients. As Topaz stated, 'Your doctor could be making decisions around treatment based on studies that never existed.'
Index analysis
Topaz explained that such mishaps arise when an author asserts a statement of fact and asks AI for a citation, which in some cases the AI inserts inadvertently.
Fabricated citations can 'look perfectly real,' making it difficult for researchers to identify them without rigorous fact-checking.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.