234 incidents on record · 2026HeadlightsIncident reports by Ellie Harris · Melbourne
5 new this weekLibrary last updated 14 October 2026
← The incident library
HD-INC-227
Healthcare · Kenya · 2025 · An AI-driven system used to process and approve health insurance claims from hospitals was relied on without sufficient human oversight, and facilities said it flagged legitimate claims as fraudulent and sent rejections too late to resubmit

Kenyan MPs Found an AI Claims System Led to Unjust Denials of Hospital Claims

By Ellie Harris · Filed Claims processed after the Social Health Authority replaced the National Health Insurance Fund in 2024

Alleged: Social Health Authority (SHA), Kenya developed or deployed the AI system implicated in this incident. Details are drawn from public reports; parties are presumed innocent of any wrongdoing not established by an official finding.

Kenyan MPs Found an AI Claims System Led to Unjust Denials of Hospital Claims

What happened

Kenya’s Social Health Authority (SHA) replaced the National Health Insurance Fund in 2024. It pays hospitals and clinics for treating its members.

It was reported that the National Assembly’s Departmental Committee on Health visited hospitals and clinics to examine how the authority was spending its funds and what problems facilities were facing. Its report was adopted on 27 November 2025.

The committee heard from facilities struggling with the claims system. One nursing home said: “The AI-driven system frequently flags legitimate claims as fraudulent, while some rejections are communicated months later, far beyond the legally stipulated 90-day period, leaving facilities unable to resubmit claims in time.”

What an auditable version would have shown

The committee raised concerns about hospitals being denied payments without enough human oversight.

Its report stated: “The increasing reliance on AI-driven claim processing and approvals without sufficient human oversight has led to unjust denials, causing significant financial strain on facilities.”

One hospital asked for human oversight to be brought back alongside the AI system. It also wanted the 90-day payment rule enforced for approved claims.

The report does not say how many claims were wrongly rejected or how much money was involved.

An auditable record would show why each claim was rejected, whether a person reviewed the decision and when the hospital was told.

Where the gap was

The government gave a different account. It was reported that on 27 February 2026, the Health Cabinet Secretary said claims worth about 12.7 billion shillings “were flagged, rejected, and unpaid by our new digital gatekeeper.”

The committee, however, found that relying on AI without enough human oversight had led to unjust denials and left hospitals under financial strain.

A ConstraintGate is designed to check an action against declared rules before it goes ahead, such as a rule that a rejection must reach the facility within the legal time limit and go to a person for review when the facility disputes it.

A ConductRecord keeps each claim decision, the reason given, who reviewed it and when the facility was told.

ConstraintGate and ConductRecord are Headlights designs.

What governance should have looked like

The Star reported that the committee called for “human verification for all AI-flagged claims within 45 days”. It also wanted hospitals to be told why a claim was rejected and given a chance to resubmit it through the digital portal.

That wording could not be found in the copy of the committee’s report reviewed, so it remains attributed to The Star.

The findings came from Parliament, rather than a court or regulator. As of 4 October 2026, the sources reviewed showed no published audit of the system’s rejected claims.

For hospitals relying on these payments, best practice would be to have a person review disputed claims, explain why they were rejected and give hospitals enough time to resubmit them.

Failure Pattern: according to a Kenyan parliamentary committee, a national health insurer relied on AI to process and approve hospital claims without sufficient human oversight, leading to unjust denials.

Governance Principle: an insurer that uses AI to reject claims should keep a person able to review each rejection and should tell the claimant in time to respond.

The reference implementation of ConstraintGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.

Sources

The mailing list

Fresh incident reports every week. One email to match.

We add new incidents to the library regularly, and send a single short email each week with what's new. The library stays free and open; this is just how you keep up with it.

Sent through Substack. Unsubscribe in one click.

The record

An auditable system would have produced a signed, tamper-evident record the moment this happened: what the system did, the version that did it, the basis it acted on, and the action taken, and Social Health Authority (SHA), Kenya could have produced it on demand.

This is the record the system as deployed did not produce in a signed, auditable form.

What this teaches
Capture what happened when it happens
What the system did, the version that did it, the basis it acted on, and the action taken, recorded at the moment, not reconstructed after.
Sign it, so no one has to trust the record-keeper
A tamper-evident entry. Edit it later and the signature breaks. The record does not ask for the benefit of the doubt.
Make it verifiable by anyone
A court, a regulator, a customer's lawyer can check the record themselves, without taking the company, or us, at our word.

Headlights summarises publicly reported AI incidents. All summaries are independently written, attributed to their original sources, and intended for research and educational purposes. Allegations are identified as such until established through official findings.

Last reviewed October 2026. This report is based on the sources listed above and reflects information available at the time of review; later developments may not be captured. Where a person is described as charged with or alleged to have done something, that allegation is unproven unless a conviction or a court or regulatory finding is stated. Headlights publishes journalism and commentary, not legal advice.

Want to write back?

Direct to my inbox.

ellie@useheadlights.com →