← All topics
Human behaviour
AI incidents involving automation bias
Headlights has recorded automation bias as the main behaviour in 28 of the 234 incidents in the library (12%), and as a contributing behaviour in 5 more. They span 2017 to 2026. Most come from Professional & legal services (10), Policing, justice & courts (8), Government & public administration (4).
Headlights' test for automation bias
A person or process accepted a system's output as correct when the information needed to question it was available.
A behaviour tag is Headlights' reading of the public record, not a finding by any court, regulator or company. Each entry gives the one sentence in the record that supports it.
Main behaviour in the record · 28
HD-INC-207India's Supreme Court Voided a Tribunal Order Built on Fake AI-Generated PrecedentsIndia · 2026HD-INC-206Brazil's Superior Court Threw Out a Police Report Made With Generative AIBrazil · 2026HD-INC-204AI-Generated Intelligence Report Nearly Led to a US Military Boarding, CNN ReportedUnited States · 2026HD-INC-202Israel's Supreme Court Found Uncontrolled AI Use in a City's Refusal LetterIsrael · 2025HD-INC-194A Safety Board Found Driver Overreliance Contributed to Two Fatal BlueCruise CrashesUnited States · 2026HD-INC-185It was reported that an Argentine court annulled a criminal sentence after a chatbot instruction was found in the judgment. Months later, that decision was overturned.Argentina · 2025HD-INC-180South Africa withdrew its draft national AI policy after fictitious references were found in its evidence baseSouth Africa · 2026HD-INC-179Police intelligence used to support an away fan ban contained eight reported errors, including a Copilot result about a match that never happenedUnited Kingdom · 2025HD-INC-176A self-represented worker relied on AI generated material that argued the wrong date, and was ordered to pay costsAustralia · 2026HD-INC-158A US drug manufacturer told inspectors it had used AI agents to write the documents that define how its products are made, and said it had not known about a validation requirement because the AI had not raised itUnited States · 2026HD-INC-144America's biggest newspaper chain published AI sports write-ups with the team names left as blank tags, and paused the toolUnited States · 2023HD-INC-142Facebook translated a Palestinian man's 'good morning' as 'attack them', and police arrested himWest Bank · 2017HD-INC-128São Paulo built one of Latin America's largest facial recognition networks, with tens of thousands of cameras planned to match passers-by against watchlists, and reporting found it identified wanted people but also stopped some who were innocentBrazil · 2024HD-INC-087The Chicago Sun-Times ran a summer reading list with ten books that did not exist, from a syndicated section a freelancer had built with AI and not checkedUnited States · 2025HD-INC-070US immigration authorities leaned on machine translation for asylum cases, and errors as small as a pronoun reportedly led to rejectionsUnited States · 2023HD-INC-068Google's Gemini CLI botched a file move, declared the data destroyed, and days passed before anyone knew the truthUnited States · 2025HD-INC-062An Australian university accused thousands of students of AI misconduct using a detector it knew was unreliableAustralia · 2025HD-INC-054Detroit police arrested the man in his driveway on a facial-recognition match, the first wrongful arrest of its kind publicly known in the USUnited States · 2020HD-INC-037CNET Published 77 AI-Written Articles. More Than Half Needed Corrections.United States · 2023HD-INC-036A Misinformation Expert's Court Declaration on Deepfakes Cited Studies That Did Not ExistUnited States · 2024HD-INC-034Google's Bard Ad Got a Telescope Fact Wrong, and Alphabet Lost 100 Billion USD in a DayUnited States · 2023HD-INC-025An Australian Lawyer Relied on ChatGPT and Filed 17 Cases That Did Not ExistAustralia · 2025HD-INC-013Melbourne Law Firm Ordered to Pay Costs Over Fake Citations in a Native Title ClaimAustralia · 2025HD-INC-012A Senior Barrister Filed AI-Invented Cases and Quotes in a Victorian Murder TrialAustralia · 2025HD-INC-011The First Australian Lawyer Disciplined for AI Misuse Handed a Judge Fake CasesAustralia · 2024HD-INC-010Deloitte's $440K AUD report for the Australian government cited a federal court quote that did not existAustralia · 2025HD-INC-003Google Bard Invented Three Court Cases, and a Lawyer Filed Them Without CheckingUnited States · 2023HD-INC-002Mata v. Avianca: A Lawyer Cited Six Fake Cases and Asked ChatGPT to Confirm ThemUnited States · 2023
Also present · 5
HD-INC-231A Court Struck Part of Anthropic's Expert Evidence After Claude Changed a CitationUnited States · 2025HD-INC-151Two US federal judges withdrew orders that contained misstated case outcomes and a citation to a case that did not exist, and both told a Senate committee that staff had used AI to draft themUnited States · 2025HD-INC-111Spain's VioGén algorithm rated a woman who reported her former partner as 'medium' risk, and she was killed three weeks later, in a case that fits a pattern an independent audit had already foundSpain · 2025HD-INC-066A child protection worker used ChatGPT on a Children's Court report, and the regulator found it disclosed a child's details and understated riskAustralia · 2024HD-INC-045An Uber self-driving car detected a woman crossing the road, worked out it needed to brake, and had been built to neither brake nor warn its driverUnited States · 2018
Related topics
- Professional & legal services 10
- Policing, justice & courts 8
- Government & public administration 4
- Rules-behaviour gap 3
- Concealment 1
- Overconfidence 1
- United States 16
- Australia 8
- Brazil 2
Cite this page: Harris, E. (2026). AI incidents involving automation bias. Headlights. https://useheadlights.com/topics/behaviour/automation-bias/