California Court Orders Probe of DA After AI Errors in Cases

California Court Orders Probe of DA After AI Errors in Cases

A California appeals court has appointed a judge to investigate potential sanctions against Nevada County District Attorney Jesse Wilson after prosecutors repeatedly filed criminal court documents containing AI-generated errors last fall. The decision follows revelations that mistakes appeared in four separate felony cases over a two-month period when prosecutors used generative AI to write or proofread legal filings.

The investigation stems from a sworn declaration filed this month by Madison Maxwell, a former Nevada County prosecutor who handled three of the four affected cases. Maxwell alleges that her supervisor, Assistant District Attorney Lydia Stuart, prevented her from immediately reporting and apologizing for the AI-generated mistakes to the court, defense attorneys, and defendants.

Why it matters

This case represents one of the only known instances nationwide where public prosecutors—who face heightened ethical standards—have presented arguments containing falsehoods to courts because of AI use. The outcome could establish precedent for how California handles prosecutorial misconduct involving generative AI and whether district attorneys can be held accountable when their offices adopt the technology without adequate safeguards. With more than 130 documented cases of AI-generated falsehoods in California court filings, the investigation may shape statewide standards for AI use in criminal justice.

Claims of inadequate oversight

In her declaration, Maxwell argues that Stuart encouraged prosecutors to use AI despite having no policy or training in place for responsible use of the technology. She describes an office environment where "junior lawyers often operated with minimal-if any-oversight" and claims she was unaware that large language models could fabricate case citations.

Maxwell, who started at the Nevada County DA's office in July 2024 as her first job after law school, also cited medical issues including brain fog from unmanaged diabetes and increased workload following senior staff departures. She resigned in May and now works in the neighboring Sutter County district attorney's office.

One of the cases Maxwell handled involved a habeas corpus petition concerning whether to release a man from jail—a situation where fabricated legal precedents could directly impact someone's liberty.

Calls for deeper investigation

The California Supreme Court directed the appeals court in January to consider a deeper investigation into the extent of misconduct and its impact on cases. More than 20 AI experts have urged the court to order a comprehensive probe beyond the four acknowledged cases.

Wilson and Stuart said an audit covering 18 months found no pattern of AI-attributable errors outside the four cases identified in February. However, it remains unclear whether that audit has been made public.

Civil Rights Corps, which assisted the Nevada County Public Defender's Office with habeas corpus and sanction petitions, argues that Maxwell's declaration strengthens the case for investigation. "An independent investigation is needed to understand the scope of what really happened here and how many people and cases have been impacted," attorney Peter Santina said.

Broader implications

Independent researcher Damien Charlotin has documented more than 130 instances of generative AI leading to court filings with falsehoods in California alone. Judges have issued monetary sanctions to 19 lawyers in the state for AI-related errors.

Kate Chatfield, executive director of the California Public Defender's Association, emphasized the heightened stakes when prosecutors make such errors. When a prosecutor cites a nonexistent case to justify holding someone in custody, "that has serious implications for that defendant and their family," she said.

The Nevada County district attorney's office did not respond to requests for comment on Maxwell's allegations or the appeals court ruling, according to CalMatters, which first reported these details.