How Headlights works

An incident is eligible when an AI system, or an automated model, score or decision rule, contributed to harm, an operational failure or an adverse finding against the organisation that used it.

This deliberately includes rule-based and statistical decision systems as well as machine learning and generative AI. Scoring and eligibility systems that decide about people, such as welfare debt calculations, credit scores or fraud flags, fail in the same ways and raise the same governance questions, whatever the underlying technique.

Inclusion requires a public record strong enough to write from:

• Court or tribunal decisions
• Regulator, ombudsman or audit findings
• Parliamentary records
• Official statements by the organisation
• Reporting by reputable news organisations

The library is curated, not a census. New entries are chosen for spread across industries, countries and kinds of failure. Areas already heavily covered, such as lawyers citing fabricated cases, need a new jurisdiction or a new kind of outcome to be added.

• Primary sources firstWhere secondary reporting conflicts with a court decision, a finding or an official record, the primary source is followed.
• Everything attributedEvery factual statement is attributed: "the regulator found", "the court held", "the company said", "it was reported that".
• Findings are not allegationsFindings, allegations, organisational statements and press reports are kept distinct. An allegation is never presented as a finding.
• Gaps are namedWhere the record does not establish something, the entry says "the sources reviewed do not show", rather than assuming it did not happen.
• Claims are datedNegative claims are dated. If a finding was appealed, overturned or annulled, that travels with every mention of it.
• Roles, not namesIndividuals are referred to by role, not name. Organisations, regulators and courts are named.

1. Drafted from the sources
2. Fact-check pass
3. Second fact-check pass
4. Corrections checked against the primary source
5. Published, with the month last reviewed

Each entry goes through at least two editorial fact-check passes before publication, covering dates and figures, quotations and attribution, the scope of findings and allegations, and the accuracy of the account. Every proposed correction is checked against the primary source before it is applied. A summary of a source is never treated as the source.

These are editorial checks, not external peer review. Each entry shows the month it was last reviewed. Corrections are welcome at ellie@useheadlights.com and are made as soon as they are confirmed.

Each incident is read against a fixed list of thirteen behaviours. Each has a test the record must meet. For what each one means in everyday language, see the 13 behaviours in plain language.

• Normalisation of deviance9 incidents

  The record must showA departure from the rule or design was repeated over time and treated as normal, without anyone re-deciding it was acceptable.

• Overconfidence58 incidents

  The record must showThe organisation asserted a capability, accuracy or safety level it had not established, or relied on one without evidence.

• Automation bias28 incidents

  The record must showA person or process accepted a system's output as correct when the information needed to question it was available.

• Judgement surrendered12 incidents

  The record must showA decision that required human discretion was made by, or effectively handed to, a system.

• Diffusion of responsibility5 incidents

  The record must showNo identified role owned the decision or the check, or each party relied on another to have done it.

• Authority ambiguity5 incidents

  The record must showIt was unclear who was entitled to decide, instruct or approve, and the system or staff acted anyway.

• Rules-behaviour gap38 incidents

  The record must showA written policy, procedure or legal requirement existed and practice did not follow it.

• Delayed intervention8 incidents

  The record must showThe problem was known, or reported, and the organisation did not act for a material period.

• Incentive pressure4 incidents

  The record must showSpeed, cost, targets or commercial advantage are recorded as shaping the decision.

• Group conformity4 incidents

  The record must showA risk was known inside the organisation, or advice against the course was given to it, and the organisation carried on regardless.

• Confirmation bias0 incidents

  The record must showEvidence that contradicted the preferred conclusion was available and was not weighed.

• Defensive denial5 incidents

  The record must showAfter the failure, the organisation denied, minimised or refused to reconsider, on the record.

• Concealment10 incidents

  The record must showThe organisation kept the system's operation, its workings or the record of what it did from people entitled to see it, or removed the means of checking afterwards. Someone must have asked and been refused, or the withholding must itself have been found or admitted.

The rules

1. A tag is Headlights' reading of documented conduct. It is not a psychological diagnosis, and not a finding by any court, regulator or company.
2. A tag describes conduct in the record, not a state of mind. If the only basis would be a guess about what someone thought, the answer is "Not established".
3. Every entry gets one main behaviour, at most two contributing ones, and one attributed sentence from the record supporting the main tag. That sentence is shown on the entry.
4. Where proceedings are unresolved, a tag is used only if the conduct itself is recorded, not merely alleged.
5. When two tags fit, the more specific one wins, then the one with the stronger source, then the one closest to the failure the entry is about.
6. "Not established" is recorded where the record cannot support any behaviour. It applies to about a fifth of the library.

Changes to the list

The list has changed as the library grew. Each change is dated, and affected entries were re-tagged.

1. 24 September 2026

   Behaviour tags introduced: twelve behaviours plus "Not established".

2. 24 September 2026

   Concealment added as a thirteenth behaviour. Nine entries re-tagged.

3. 24 September 2026

   Concealment tightened: someone must have asked and been refused, or the withholding must be found or admitted. A system that simply cannot explain itself no longer counts. Six uses removed.

4. 24 September 2026

   Group conformity widened to cover official findings that a risk was known inside the organisation, or advice against the course was given. Three entries re-tagged.

Each entry ends with analysis: where the gap was, and what governance and record-keeping would have helped. This is Headlights' view and is kept separate from the factual record. Where an entry describes a control Headlights has designed, such as ConductRecord or VerificationGate, it says what the control is designed to do, never that it would have prevented the incident. The designs are published as open-source code.

IDs

Permanent and never reused. If two entries turn out to describe the same incident, they are merged and the old address redirects.

Industry

One of thirteen parent industries.

Governance domain

One to three of fourteen, from a fixed list.

Year

The year of the defining event or principal finding, not the start of the problem or a later appeal.

The open index has one row per incident, linking to the full entry where the sources, classification evidence and analysis can be examined.

AI tools are used in the work. They help find candidate incidents and their sources, draft entries, run fact-check passes and plain-language edits, and propose behaviour tags.

• One editorEntries and classifications are written and approved by one person. The library has not been externally peer reviewed.
• Tags not yet independently testedA blind second assessment of a sample of entries, with the agreement rate published, is planned.
• Curated, not a censusCounts describe patterns within the documented collection. They are not estimates of how often AI fails.
• Uneven public recordsCoverage leans towards countries with open courts, active regulators and English-language reporting.
• The record movesAppeals, settlements and new findings can change an entry after publication.

Cite an entry by its ID and link, for example:

Cite the dataset as:

Questions about the method, or a classification you think is wrong: ellie@useheadlights.com