California now reports six high-risk AI systems after zero last year

California state agencies are now reporting the use of six high-risk automated decision systems that affect residents' lives, a sharp reversal from last year when officials claimed they used none.

The disclosure, released June 12 by the state's technology department, marks the second year of reporting under a 2023 law requiring agencies to inventory "high-risk automated decision systems" — tools that assist or replace human judgment in consequential areas like criminal justice, employment, education, and healthcare.

According to CalMatters, which first reported the details, the newly disclosed systems include:

• COMPAS software used by the California Department of Corrections and Rehabilitation to predict whether incarcerated individuals will reoffend
• Fraud detection tools employed by the state's employment department to evaluate unemployment claims
• Remote proctoring systems for California State University exams
• Detection software that flags when college students use generative AI to complete assignments

Why it matters

The gap between last year's empty inventory and this year's disclosure raises questions about state compliance and oversight of AI systems already in use. The COMPAS recidivism tool, for example, has been deployed by California corrections officials for at least a decade. The employment department's fraud detection system previously paused benefits for 600,000 Californians during the 2020 holiday period, according to a Legislative Analyst's Office report. These systems were operating but went unreported under the transparency law's first year.

The state technology department attributed the improved reporting to more thorough evaluation of agency responses, including direct meetings and follow-up questions about their systems.

What remains unreported

The inventory still excludes several categories of AI use across California government. Notably absent are generative AI pilot projects supported by the governor's office, including an AI assistant named Poppy that uses Anthropic's Claude model to draft documents and research policy. The state website indicates 67 departments participated in Poppy's pilot phase, with statewide rollout scheduled for July 2026.

Also missing: a California State University contract with OpenAI to provide ChatGPT access, and AI systems used by the judicial branch and University of California system, which are exempt from the reporting requirement.

The report identified two additional high-risk systems not currently in use: the Department of Cannabis Control is developing AI to analyze marijuana packaging for child-appeal violations, and CSU discontinued a language model previously used for job application reviews.

Six other systems were initially flagged as high-risk but later determined not to meet the threshold, including AI used by the California Department of Finance for legislative bill analysis.

Broader context

The disclosures arrive as California cities including San Jose and San Francisco release their own AI inventories. Recent surveys show most Californians prioritize AI safety over innovation, while Americans remain divided on trusting AI systems generally.

Legislation that would have prohibited state employees from using automated systems as the sole basis for decisions was killed in May 2026 during the appropriations process.

CalMatters is building a comprehensive inventory of automated decision systems used by California state and local agencies to provide transparency into government AI deployment.

These details were first reported by Khari Johnson at CalMatters.