AI-generated fake citations discovered in over 146,000 scientific papers
Researchers from Cornell University and UCLA found over 146,900 AI-generated fake citations across four major scientific databases. Large language models like ChatGPT and Gemini can create plausible-sounding false references, a problem known as hallucination, which researchers may unknowingly include in published papers.
What happened
Cornell University and UCLA researchers analyzed 11.1 million references in 2.5 million scientific papers across four major research databases (arXiv, bioRxiv, SSRN, and PubMed Central) and found 146,900 fake citations that do not match any published source.
The research team identified that after large language models became widely available, nonexistent references increased sharply compared to before 2023. The false citations are distributed across many papers rather than concentrated in a few, indicating widespread reliance on AI-generated references without sufficient verification by researchers.
Reported impact
Usha Haley, a business professor at Wichita State University, stated that fake citations undermine trust in the academic record that underpins peer review and knowledge accumulation, and expressed concern that such doubts now exist within academia, including among early-career researchers.
arXiv announced on May 19 that it would ban authors who submit manuscripts containing hallucinated citations or traces of insufficiently checked AI-generated content. The scientific director of arXiv stated that much AI-generated content is clearly wrong, meaningless, or just noise that makes it harder to find what is actually happening and may mislead people.
Index analysis
One large language model limitation is its tendency to create plausible-sounding misinformation, known as hallucination. If researchers allow chatbots to create citations without verification, completely fabricated references may be generated and included in published papers.
Governance controls
arXiv implemented a policy to suspend authors who post manuscripts with hallucinated citations or traces of AI content that has not been sufficiently reviewed.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.