California prosecutor's office used AI in legal briefs with fabricated citations
A California prosecutor's office used generative AI to draft legal briefs containing hallucinated citations and misinterpreted law, according to defense lawyers representing a defendant held without bail. The district attorney acknowledged AI was used in at least one filing and stated it is implementing policies to prevent future errors, while defense advocates petition the state supreme court to investigate the pattern and concerns about due process.
What happened
The Nevada County District Attorney's Office filed legal briefs containing errors typical of generative AI, including fabricated legal citations and misinterpretations of law. Kyle Kjoller's lawyers identified errors in briefs across four separate cases from District Attorney Jesse Wilson's office.
Wilson acknowledged that AI was used to draft one brief, but stated it was not the one filed in Kjoller's case. He attributed other errors to human error rather than AI use.
Reported impact
Kjoller was ordered held without bail in April despite arguing that the charges against him—multiple counts of illegal gun possession—were not grave enough under California law to warrant pretrial detention. His lawyers contend the AI-generated brief's errors violated his due process rights.
A group of 22 legal and technology scholars, including Barry Scheck from the Innocence Project, warned that unchecked AI use in prosecutions could lead to wrongful convictions and represents a threat to the legitimacy of courts.
Index analysis
Kjoller's lawyers filed a petition with the California Supreme Court identifying three cases containing errors typical of generative AI, including nonexistent quotations and misinterpreted court rulings. The defense requested sanctions against prosecutors but the appeals court denied the request without explanation.
The errors included wholesale misinterpretations of law and quotations that do not appear in cited texts, hallmark mistakes of generative AI systems.
Governance controls
The District Attorney's Office stated it is learning AI's pitfalls and implementing new policies in response to the errors. Wilson said prosecutors work under heavy caseloads and time constraints, and that there was no intent to mislead the court.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.