Michigan's $20M AI Fraud Settlement Shows Why Caseworkers Matter

When Automation Runs Unchecked

Michigan's unemployment fraud system operated largely on autopilot between 2013 and 2015, issuing more than 53,000 fraud determinations with minimal human oversight. The Michigan Integrated Data Automated System flagged income discrepancies, sent questionnaires to inactive online accounts, then terminated benefits and seized wages before recipients understood what had happened.

When the state later reviewed approximately 22,000 cases, it overturned 93 percent of them. The class-action settlement finalized in 2024 cost Michigan $20 million.

Now similar technology is returning to benefits offices, backed by federal encouragement. In April 2025, the Office of Management and Budget issued guidance urging agencies to accelerate AI adoption while maintaining risk controls for high-impact systems—a category that includes benefits eligibility and fraud detection.

Why it matters

As agencies adopt AI tools to manage overwhelming caseloads, the Michigan experience reveals a fundamental tension: algorithms can process data faster than humans, but they cannot replicate the contextual judgment that prevents catastrophic errors. Without clear safeguards, the next generation of automated systems risks repeating the same mistakes at greater scale.

The Constitutional Problem With Algorithmic Decisions

Frontline caseworkers operate as what researcher Michael Lipsky termed "street-level bureaucrats"—public servants who translate statutes into decisions about individual circumstances. That discretion represents the core of their work, not an inefficiency to eliminate.

Constitutional due process requirements remain intact regardless of automation. The Supreme Court's Goldberg v. Kelly decision still mandates timely notice and fair hearings before benefit termination. No algorithm satisfies that standard independently, and vendor contracts cannot override those obligations.

The documented failures extend beyond Michigan. Arkansas replaced nurse assessments with an algorithm that determined Medicaid home-care hours, resulting in significant care reductions for disabled residents whose conditions had not changed. In the Netherlands, a child-benefit algorithm that used nationality as a fraud indicator wrongly accused thousands of families, contributing to a government crisis.

Research by Saar Alon-Barkat and Madalina Busuioc identified a pattern they call "selective adherence": officials most readily accept algorithmic recommendations that confirm existing beliefs, particularly about individuals with less power to challenge decisions. The technology does not eliminate human judgment—it shapes how that judgment gets applied.

Five Principles for Accountable AI

Effective human oversight requires more than keeping someone "in the loop." Five operational principles can preserve meaningful review:

First, algorithms should inform decisions, not issue them. Every adverse action requires review from a designated human decision-maker.

Second, override authority must be genuine. Agencies should track override rates—a caseworker who never disagrees with the system is not meaningfully reviewing it. Workers should face no penalties for questioning recommendations.

Third, every recommendation needs an explanation in clear language before action. Agencies should embed this requirement in vendor contracts.

Fourth, agencies must audit for disparate impact before deployment and continuously after. The National Institute of Standards and Technology's AI Risk Management Framework offers a useful structure.

Fifth, residents retain rights to notice, explanation, and appeal. Individuals affected by automated decisions should know when AI was used, understand the reasoning, and access meaningful human review.

Support, Not Substitution

The strongest case for AI in benefits offices acknowledges reality: workers face crushing caseloads, and tools that handle routine lookups can return time for complex cases requiring human attention. But the Roosevelt Institute found that when workers lack input over these tools, errors and workloads often increase rather than decrease.

Code for America and Anthropic recently built a SNAP policy navigator that answers caseworkers' policy questions while deliberately leaving eligibility determinations with the worker. That distinction—support rather than substitution—defines the boundary between useful automation and the failures Michigan experienced.

Government agencies will continue adopting these tools. Whether they establish safeguards before the next crisis or explain failures afterward remains within their control.

These details were first reported by Albert Nii Noi Okwei, a doctoral researcher at Virginia Commonwealth University's Research Institute for Social Equity, writing in PA Times.