What happened
In 2013 Michigan’s Unemployment Insurance Agency replaced much of its claims processing with a new computer system, the Michigan Integrated Data Automated System, MiDAS, built by the vendor Fast Enterprises with a fraud-detection component from SAS Institute and consulting from CSG Government Solutions. The system did more than store records. It was given the power to decide, on its own, whether a claimant had committed fraud, and to act on that decision without a person reviewing it first.
From its launch in October 2013 until 2015, MiDAS auto-adjudicated fraud. It flagged discrepancies, such as a gap between what an employer reported and what a claimant had stated, treated them as evidence of intentional deception, and issued a determination. A finding of fraud in Michigan carried a penalty of four times the amount supposedly owed, on top of repaying the benefits themselves. The agency then collected: garnishing wages, intercepting state and federal tax refunds, and pursuing people into debt. Some received bills running to tens of thousands of dollars for benefits they had been entitled to. The consequences reported by those caught up in it included bankruptcies and home losses.
The determinations were wrong at a rate that is hard to overstate. Around 40,000 people were accused of fraud over the period the system ran on its own. When the state later reviewed a sample of roughly 22,000 cases the system had decided without human input, it found that about 93% of them involved no fraud at all. The errors were not edge cases; they were the rule. The agency stopped purely automated fraud adjudication and reinstated human review in 2015, as lawsuits and criticism mounted, but the determinations already issued took years to unwind.
The litigation outlasted the system. A federal case, Cahoo v. SAS Institute, named the agency, its vendors and individual officials and worked through years of appeals over qualified immunity. A separate state class action, Bauserman v. Unemployment Insurance Agency, brought by people wrongly accused, settled for 20 million dollars, agreed in 2022 and approved by the Michigan Court of Claims in early 2024. The sum, spread across the class, was a fraction of the penalties the system had extracted, and a long way from the cost it imposed on the people it falsely named.
What an auditable version would have shown
The central fact about MiDAS, that it was wrong the overwhelming majority of the time, was knowable from inside the system long before a sampled review surfaced it. Every determination it issued was a record waiting to be counted. What was missing was anyone counting: nothing measured how often the automated finding survived contact with the facts, and so a 93% error rate ran for two years as a hidden property of the system rather than a monitored number that should have stopped it.
An auditable version records each fraud determination, the data the system relied on, and the outcome when a human or an appeal examined it, and computes a standing measure of how often automated findings were later overturned. With that, an overturn rate climbing past anything defensible is visible early, as monitoring, instead of being reconstructed years later from the wreckage. The same record gives an accused person something MiDAS never did: a legible account of why they were flagged, against which they could argue.
Where the gap was
Two things were absent. The first was any authority check on a decision this consequential: a binding accusation of fraud, with quadruple penalties attached, was allowed to issue from software with no authorised human deciding it. The second was any measurement or record that would have made the system’s failure visible and its determinations contestable.
An AuthorityGate is the control on the first. A determination that binds a person, that garnishes their wages or seizes their refund, should not be issued on the authority of an automated process alone; it requires an authorised human decision-maker in the loop before it takes effect. A MetricRecord is the control on the second: a signed, recomputable measure of how often automated findings are overturned, so a 93% error rate shows up as a monitored figure rather than a later scandal. A ConductRecord preserves each determination and its basis, which is what turns a wrongful accusation into something a person can appeal rather than a sentence handed down by a machine.
What governance should have looked like
The pattern is the same one Australia’s Robodebt scheme revealed on the other side of the world: when an automated system is allowed to make a high-consequence decision without a human accountable for it and without measurement of how often it is wrong, the error does not stay small, and the people least able to fight back pay for it first.
The reference implementation of AuthorityGate, MetricRecord, and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Michigan Integrated Data Automated System (MiDAS) Performance Audit (Michigan Office of the Auditor General)
- Michigan’s MiDAS Unemployment System: Algorithm Alchemy Created Lead, Not Gold (IEEE Spectrum)
- State says it will pay $20 million to people wrongly accused of unemployment fraud (Michigan Public Radio)
- Case Over the Michigan UIA’s Faulty Automated System Finally Settled (Science, Technology, and Public Policy Program, University of Michigan)