Skip to content
Submit incident
AII-1002-53953
ConfirmedA court, a regulator or the organization itself has stated the core facts, or the record cites a court or regulator source.

Stanford misinformation expert used ChatGPT that generated false citations in court filing

A Stanford University professor who researches misinformation used ChatGPT to draft a sworn federal court filing, and the AI system generated false citations that were included in the document. The professor has admitted to the misinformation caused by the AI-generated content.

2 December 2024 Β· Reliability and hallucination
2 December 2024

Drafted by Raidu Incident Detection, an automated system.

AII-1002-539536 sources ↓
Quick facts
Category
Reliability and hallucination
Country
United States
Source
Alternet.org
Date
2 December 2024
Drafted by
Raidu Incident Detection

What happened

A Stanford University professor who studies misinformation used ChatGPT to draft a sworn federal court filing. The AI system generated false citations that were included in the filing.

Reported impact

The professor admitted to the misinformation caused by the AI-generated content in the court document.

Index analysis

The incident involved hallucinations and fabricated citations produced by ChatGPT, demonstrating a common problem with the AI technology.

Classification

Category
Reliability and hallucination
Industry
Government, security and defence
Country
United States
Harm status
Actual harm
Intent
Unintentional
Timing
Post-deployment
Harm types
Public interest, Reputational
Harmed
Government
Failure modes
Hallucination, Fabricated citation, Overreliance
AI systems
ChatGPT

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

Alternet.org
Misinformation expert used AI to draft testimony containing misinformation about AI
2 December 2024
Techdirt
Professor Apologizes For Using Fake AI-Generated Citations In Defense of Minnesota's Unconstitutional Deepfake Law
2 December 2024
The Express Tribune
Misinformation professor admits to AI-created fake citations in Minnesota case
3 December 2024
San Jose Mercury News
Stanford misinformation expert admits his chatbot use led to misinformation in sworn federal court filing
3 December 2024
San Francisco Gate
'Sincere regret': Stanford prof paid $600/hour blames errors on AI use
3 December 2024
The Stanford Daily
Stanford misinformation expert admits to ChatGPT 'hallucinations' in court statement
4 December 2024
Nov 2024Stanford Professor Accused of Using AI to Create Fake Citations in Deepfake Law DefenseAllegedReliability and hallucinationThe VergeGizmodoInternational Business Times+5Sep 2026Stanford University used AI to alter student photos without consentConfirmedDiscrimination and fairnessThe New York TimesNew York PostKRON4+4Sep 2026Stanford Used Generative AI to Alter Student Photo for Campus BannerConfirmedDiscrimination and fairnessstanforddaily.comstanfordreview.orgsfchronicle.comSep 2026AI agents in simulated environments exhibit harmful behaviors without developer oversightWell documentedSafety and wellbeingδΈ­ε›½η»ζ΅Žη½‘ai.cnmo.comwap.eastmoney.com+5Aug 2026AI System Designs and Synthesizes Novel Viruses, Raising Biosecurity ConcernsWell documentedSecurity and cyberattacksBBCForbesThe Independent+1