180 incidents on record · 2026 Headlights Incident reports by Ellie Harris · Melbourne
10 new this week Library last updated 30 August 2026
← The incident library
HD-INC-119
Insurance · United States · 2023 · Alleged bias in automated claims screening

A class action alleges State Farm's automated fraud screening subjected Black homeowners' claims to more scrutiny and delay, and a US court has let the case proceed

By Ellie Harris · Filed Filed December 2022

Alleged: State Farm Fire and Casualty Company developed or deployed the AI system implicated in this incident. Details are drawn from public reports; parties are presumed innocent of any wrongdoing not established by an official finding.

A class action alleges State Farm's automated fraud screening subjected Black homeowners' claims to more scrutiny and delay, and a US court has let the case proceed

What happened

It was reported that a class action filed in federal court in Illinois, Huskey v State Farm, alleges that State Farm handled the home insurance claims of Black policyholders less favourably than those of white policyholders, and that an automated system State Farm uses to screen claims for possible fraud is part of the reason. The lawsuit alleges that Black homeowners who made claims faced more scrutiny, more requests for paperwork, and longer delays before being paid. It draws on a survey of policyholders in which, the complaint says, white homeowners were about a third more likely to have a claim paid within a month, and Black homeowners reported a heavier paperwork burden. State Farm denies discriminating and says its claims process is applied in the same way to everyone.

It was reported that in September 2023 the court ruled on State Farm’s request to throw the case out and allowed part of it to go ahead. The judge dismissed some of the claims but let the central one proceed, a claim of disparate impact under the Fair Housing Act, finding that home insurance is closely enough tied to housing for the law to apply, and that the plaintiffs had set out enough, using statistical evidence connected to State Farm’s claims-processing software, for the claim to continue. The case has since moved into discovery, with an early focus on identifying the automated tools State Farm uses to screen claims. As of the most recent public step in the case, no court had decided whether discrimination occurred, and the matter was not settled.

What an auditable version would have shown

Whether State Farm’s screening treats Black and white claimants differently is, from the outside, a matter of survey evidence and inference, which is why it takes a lawsuit and discovery to test. An auditable version would keep a record of each claim the system screened, the flag or score it produced, the reason, and what followed, so that any difference in how often claims are flagged, and how long they then take, across groups of policyholders, is a figure the company already holds. That turns a question a claimant has to fund a case to ask into one the record can answer, and it works the same way whether the answer clears the company or not.

Where the gap was

When the system picks a claim out for extra checks, the homeowner it happens to is never told. They do not know they were flagged, or why, or whether people like them are flagged more often than anyone else. A MetricRecord counts how the flags, and the delays that follow, fall across groups of claimants, so a disparity is a number the company can see rather than one a survey has to estimate. A ConductRecord keeps each screening decision, the flag and its basis, so a claim held up for extra checks can be traced and, if it was wrongly flagged, put right. The lawsuit alleges a pattern; State Farm says its process is even-handed. A record of this kind is what would let either side show which is true, from the claims themselves.

What governance should have looked like

Where an automated system decides which insurance claims get extra scrutiny, the rate at which it flags claims, and the delay that follows, should be measured across groups of policyholders and open to a regulator, and a claimant should be able to see and question a flag against their claim. The dispute here is unresolved: the lawsuit alleges Black homeowners were treated worse, State Farm denies it, and a court has allowed the question to be tested. What a record does is make that question answerable from the claims a company already processes, rather than from a survey and years of litigation.

Failure Pattern: an automated system decided which insurance claims got extra scrutiny, and whether it flagged and delayed some groups of claimants more than others could not be seen by the people affected or shown from records, only alleged and litigated.

Governance Principle: where an automated system decides which claims receive extra scrutiny, the rate at which it flags and delays claims must be measured across groups and open to challenge by the people affected.

The reference implementation of MetricRecord and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.

Sources

The mailing list

Fresh incident reports every week. One email to match.

We add new incidents to the library regularly, and send a single short email each week with what's new. The library stays free and open; this is just how you keep up with it.

No tracking. Unsubscribe in one click.

The record

An auditable system would have produced a signed, tamper-evident record the moment this happened: what the system did, the version that did it, the basis it acted on, and the action taken, and State Farm Fire and Casualty Company could have produced it on demand.

This is the record the system as deployed did not produce in a signed, auditable form.

What this teaches
Capture what happened when it happens
What the system did, the version that did it, the basis it acted on, and the action taken, recorded at the moment, not reconstructed after.
Sign it, so no one has to trust the record-keeper
A tamper-evident entry. Edit it later and the signature breaks. The record does not ask for the benefit of the doubt.
Make it verifiable by anyone
A court, a regulator, a customer's lawyer can check the record themselves, without taking the company, or us, at our word.

Headlights summarises publicly reported AI incidents. All summaries are independently written, attributed to their original sources, and intended for research and educational purposes. Allegations are identified as such until established through official findings.

This report is based on the sources listed above and reflects information available at the time of review; later developments may not be captured. Where a person is described as charged with or alleged to have done something, that allegation is unproven unless a conviction or a court or regulatory finding is stated. Headlights publishes journalism and commentary, not legal advice.

Want to write back?

Direct to my inbox.

ellie@useheadlights.com →