← All topics
Governance domain
AI incidents: algorithmic accountability
Headlights has documented 104 AI incidents that raise questions of algorithmic accountability, 44% of the library, from 2016 to 2026. The behaviours most often in the record are overconfidence (30), rules-behaviour gap (16), automation bias (8). Most are from the United States (56), Australia (8), Brazil (6).
104 incidents, newest first
HD-INC-235Brazil's Audit Court Found More Than One in Ten Automatic Benefit Refusals Were WrongBrazil · 2025HD-INC-233Starling Bank's Sanctions Screen Checked Customers Against 39 of 3,088 Listed NamesUnited Kingdom · 2024HD-INC-232Three Bugs Degraded Claude's Answers for Weeks Before Anthropic Opened an InvestigationUnited States · 2025HD-INC-229Spain's National Court Held Foundever Withheld the Formula That Set Staff Rest DaysSpain · 2025HD-INC-225IBM Japan Settled a Union Complaint by Agreeing to Disclose What Its AI Weighed in PayJapan · 2024HD-INC-224Austria's Privacy Regulator Held a Credit Bureau Made an Automated Decision on ElectricityAustria · 2025HD-INC-223Italy's Top Court Rejected a Ministry Reading That Let Teacher Postings Skip Higher RanksItaly · 2026HD-INC-221Korea's State Auditor Found Public AI Training Data Published With Errors Left UnfixedSouth Korea · 2026HD-INC-220A Brazilian Court Sanctioned a Hidden Instruction Aimed at Its AI to Grant Every ClaimBrazil · 2026HD-INC-218Italy Fined an Energy Supplier Whose Credit Score Refused Contracts Without ExplanationItaly · 2026HD-INC-214A Brazilian Labour Court Fined Two Lawyers Over a Hidden Instruction to AI in a FilingBrazil · 2026HD-INC-212France's Top Administrative Court Cut Amazon's Warehouse Data Fine to 15 Million EurosFrance · 2025HD-INC-211A Fuel Price Software Maker Settled With Brazil's Competition Authority Over Its AlgorithmBrazil · 2026HD-INC-208Dutch Auditors Found Police Did Not Track What Their Number Plate Algorithm Led ToNetherlands · 2026HD-INC-206Brazil's Superior Court Threw Out a Police Report Made With Generative AIBrazil · 2026HD-INC-198Spain's Supreme Court Ordered Release of the Code That Checks Eligibility for Energy AidSpain · 2025HD-INC-195AI Research Agents Reached Another Company's Production SystemsUnited States · 2026HD-INC-194A Safety Board Found Driver Overreliance Contributed to Two Fatal BlueCruise CrashesUnited States · 2026HD-INC-193A regulator settled with three firms that sold small businesses an advertising service they represented as listening to conversations on smart devices, alleging it did not collect or use voice data in any mannerUnited States · 2026HD-INC-191An AI Safety Test Reached the Real World and Tried to Influence a MaintainerUnited Kingdom · 2026HD-INC-190A state settled with a motor insurer after an AI tool selected a new policyholder for underwriting review and her policy was later cancelled without her knowingUnited States · 2026HD-INC-189A clinical AI vendor advertised hallucination rate figures of <.001% and <1 per 100,000, and Texas alleged its representations may have violated the Texas Deceptive Trade Practices and Consumer Protection Act because they were false, misleading or deceptiveUnited States · 2024HD-INC-188Ontario's auditor found that nine of twenty AI Scribe systems fabricated information in the government's own procurement testing, and that all twenty vendors were approvedCanada · 2026HD-INC-187A state inspector general found a school's AI gun alert was raised on a pupil's hand movement, not on the bag of crisps in the reporting, and that it was sent to fifteen people, ten of them policeUnited States · 2025HD-INC-184EU court rules automated credit decisions must be explained in a way people can understandAustria · 2025HD-INC-183A state attorney general alleged a student lender's underwriting model penalised applicants for the default rate of the college they had attendedUnited States · 2025HD-INC-182An Italian regulator warned that workplace stress reports derived from chat analysis could unlawfully reach employers, even in aggregate formItaly · 2026HD-INC-181A Dutch inspectorate found the probation organisations' recidivism tool had the formulas for detained and non-detained people the wrong way round, and had done since 2018Netherlands · 2026HD-INC-178Canadian privacy regulators found OpenAI had not validated the general accuracy of personal information generated about peopleCanada · 2026HD-INC-175Delta denied using personal data for AI pricing, despite its president saying prices would be tailored to individual customersUnited States · 2025HD-INC-174The FDA told a company its AI app was an unapproved medical device after it advertised a five minute gaze test to detect early signs of Alzheimer's diseaseUnited States · 2025HD-INC-173Anthropic reported that its AI coding agent was used in a cyber espionage campaign targeting about thirty organisationsUnited States · 2025HD-INC-171South Africa's social grant agency read every deposit into an applicant's bank account as income, and the High Court found its checks against government databases unreasonable and unfairSouth Africa · 2025HD-INC-168Google's Gemma model produced a fabricated criminal allegation about a sitting US senator, and Google withdrew the model from AI Studio while keeping it available through the APIUnited States · 2025HD-INC-163Cloudflare said undeclared crawlers reached brand new domains after Perplexity's declared crawler was blocked, de-listed it as a verified bot, and Perplexity disputed the attributionUnited States · 2025HD-INC-160A Munich court found at first instance that OpenAI's models hold the words of nine German songs and reproduce them in chatbot outputs, and rejected the argument that this was text and data miningGermany · 2025HD-INC-157ProPublica reported that the tool used to pick Veterans Affairs contracts for cancellation was shown only the first 10,000 characters of each contract, and recorded around 1,100 of them as worth 34 million dollars eachUnited States · 2025HD-INC-156Greece's data protection authority fined the Migration Ministry 175,000 euros after finding that the impact assessments for the camera, behavioural analytics, drone and fingerprint systems at five island reception centres were substantially incompleteGreece · 2024HD-INC-155California's Labour Commissioner cited Amazon nearly six million dollars for failing to meet a warehouse quota law written three years earlier, at two of its own fulfilment centresUnited States · 2024HD-INC-154The US Department of Justice alleged that competing landlords set rents using a shared pricing algorithm fed with each other's confidential data, and RealPage settled without admitting wrongdoingUnited States · 2025HD-INC-133US prosecutors charged the founder of the shopping app Nate with fraud, alleging the AI that was said to complete purchases on its own was mostly people working in a call centreUnited States · 2025HD-INC-130Universities and exam boards watched students through remote-proctoring AI that flagged them for cheating, and its face-detection worked so poorly on darker-skinned students that some could not start their examsUnited States · 2021HD-INC-129A Pennsylvania county used an algorithm to score families reported for child neglect, drawing on data like disability and welfare use, and an Associated Press investigation and a US civil rights inquiry raised concerns it hardened biasUnited States · 2022HD-INC-128São Paulo built one of Latin America's largest facial recognition networks, with tens of thousands of cameras planned to match passers-by against watchlists, and reporting found it identified wanted people but also stopped some who were innocentBrazil · 2024HD-INC-126A US company sold employers software that scored job candidates from their recorded face and voice on a video interview, and dropped the facial analysis only after a privacy complaint and an outside auditUnited States · 2021HD-INC-125US home insurers increasingly decide whether to renew a policy by running AI over aerial images of the house, and homeowners report being dropped over roofs already repaired or trees already gone, with no way to see or challenge the imageUnited States · 2025HD-INC-124US regulators opened an investigation after Waymo's driverless taxis were recorded passing stopped school buses with their lights flashing and stop arms out, and Waymo recalled the software on more than 3,000 vehiclesUnited States · 2025HD-INC-123South Korea's privacy regulator fined Kakao Pay after finding it had sent about 40 million users' data to Alipay without their consent, where it was used to build a scoring model, and it ordered the model deletedSouth Korea · 2025HD-INC-122France became the first country in the European Union to write mass AI video surveillance into law, scanning the crowds at the 2024 Paris Olympics for events like crowd surges and abandoned bags, over the objections of rights groupsFrance · 2024HD-INC-121A widely used US healthcare algorithm judged how sick patients were by what had been spent on their care, so patients the system had historically spent less on were rated healthier and passed over for extra help, and researchers found the bias cut those referrals by more than halfUnited States · 2019HD-INC-119A class action alleges State Farm's automated fraud screening subjected Black homeowners' claims to more scrutiny and delay, and a US court has let the case proceedUnited States · 2023HD-INC-118Austria's employment service built an algorithm that sorted jobseekers by their predicted chances and gave less weight to women, older and disabled people, and the privacy regulator banned it before a court cleared itAustria · 2020HD-INC-117A Wisconsin school system's dropout-prediction algorithm raised false alarms far more often for some students than others, and the state's own analysis had already found it unfairUnited States · 2023HD-INC-115The US securities regulator found that Presto Automation misled investors about its AI drive-thru service, telling the market it removed the need for human order-takers while most orders were completed with help from people offshoreUnited States · 2025HD-INC-113A court in Bologna found that Deliveroo's rider-ranking algorithm discriminated against couriers because it treated a legally protected strike the same as an unexplained no-showItaly · 2021HD-INC-111Spain's VioGén algorithm rated a woman who reported her former partner as 'medium' risk, and she was killed three weeks later, in a case that fits a pattern an independent audit had already foundSpain · 2025HD-INC-110Facebook's recommendation AI asked people watching a video of Black men whether they wanted to keep seeing videos about primates, and the company disabled the featureUnited States · 2021HD-INC-109A Black Uber Eats courier says the app's facial verification kept rejecting valid photographs of him until he lost his account, and he received a payout after nearly three years of litigationUnited Kingdom · 2024HD-INC-099Facebook's ad-delivery algorithm was alleged to skew who saw housing ads by race and sex, and a US Justice Department settlement required Meta to rebuild itUnited States · 2022HD-INC-098DoNotPay marketed a 'robot lawyer', but the FTC said it never checked whether the AI matched a real lawyer, and fined the company and barred the claimUnited States · 2024HD-INC-097Australia's Merit Protection Commissioner overturned most decisions from an AI-assisted recruitment round, and a union told Parliament experienced staff were being screened outAustralia · 2022HD-INC-095Dutch reporters found that most Dutch responses to an EU tobacco consultation appeared to come from a Philip Morris AI tool, and the company denied steering themNetherlands · 2026HD-INC-094Discord wrongly banned more than 8,000 users over benign images after bugs bypassed the intended human review, and the company admitted the errorUnited States · 2026HD-INC-093Ross trained an AI legal-research tool on Westlaw headnotes obtained after being refused a licence, and a US court ruled it was not fair use, in what commentators called the first such decisionUnited States · 2025HD-INC-092The FTC barred Evolv from claiming its AI scanner detects all weapons, after citing a knife it missed in a school stabbingUnited States · 2024HD-INC-090A franchisee that runs 111 Pizza Huts is suing the chain for more than $100 million over a mandated AI dispatch system it says let DoorDash drivers cherry-pick ordersUnited States · 2026HD-INC-088An AI crime-alert app turned police-radio chatter into false alarms about serious crimes, and a BBC investigation prompted an apologyUnited States · 2025HD-INC-087The Chicago Sun-Times ran a summer reading list with ten books that did not exist, from a syndicated section a freelancer had built with AI and not checkedUnited States · 2025HD-INC-086Builder.ai sold app-building AI and raised about $445 million, then collapsed in 2025 amid reports its 'AI' was largely human engineers and its revenue far below what it had claimedUnited Kingdom · 2025HD-INC-085Getty Images sued Stability AI over the images behind Stable Diffusion, then abandoned its main copyright claim because it could not establish where the training happenedUnited Kingdom · 2025HD-INC-084Apple Card reportedly offered a man twenty times his wife's credit limit, claims of gender bias went viral, and New York's regulator found no unlawful discrimination but criticised the lack of transparencyUnited States · 2021HD-INC-079Trivago advertised the best hotel deal while its algorithm favoured the advertiser that paid it most, and the penalty was $44.7 millionAustralia · 2022HD-INC-078An insurer promised loyalty discounts while a pricing algorithm capped them, and the Federal Court imposed what was then a record $40 million penaltyAustralia · 2023HD-INC-075Amazon's AI coding assistant shipped with planted instructions telling it to wipe users' computers and cloud accountsUnited States · 2025HD-INC-074Haryana's family-ID system marked living pensioners as dead, and the chief minister told the assembly that 70 percent of old-age pension stoppages based on its data were wrongfulIndia · 2023HD-INC-073An AI transcription tool used by clinicians invented text no one said, researchers reported, and the audio that could have proved it was deletedUnited States · 2024HD-INC-072The Dutch student-finance agency picked fraud targets with a risk profile for a decade, and official investigations found it indirectly discriminated against students with migration backgroundsNetherlands · 2024HD-INC-068Google's Gemini CLI botched a file move, declared the data destroyed, and days passed before anyone knew the truthUnited States · 2025HD-INC-064Cigna's 'PxDx' let its doctors reject 300,000 claims in two months, about 1.2 seconds each, without opening the fileUnited States · 2023HD-INC-062An Australian university accused thousands of students of AI misconduct using a detector it knew was unreliableAustralia · 2025HD-INC-059A Dutch court struck down SyRI, a secret welfare-fraud profiling system aimed at poor neighbourhoods, as a violation of human rightsNetherlands · 2020HD-INC-057ChatGPT reportedly told users an Australian mayor had gone to prison for bribery, when he was the whistleblower who exposed itAustralia · 2023HD-INC-056The New York Times sued OpenAI and Microsoft, alleging their models were trained on its articles and could reproduce them almost word for wordUnited States · 2023HD-INC-054Detroit police arrested the man in his driveway on a facial-recognition match, the first wrongful arrest of its kind publicly known in the USUnited States · 2020HD-INC-053Michigan let an algorithm decide unemployment fraud with no human looking, and a state review later found about 93% of its accusations wrongUnited States · 2017HD-INC-052Amazon sold 'Just Walk Out' as cashierless AI, and reporting said more than a thousand people in India were checking the shoppingUnited States · 2024HD-INC-051Google's Gemini generated racially diverse Nazi soldiers and Black US founding fathers, and Google switched off image generation of people for about six monthsUnited States · 2024HD-INC-050When exams were cancelled, England's regulator graded A-levels with an algorithm that downgraded nearly two in five results by the school's history, not the studentUnited Kingdom · 2020HD-INC-046A risk-assessment tool used in US courts reportedly flagged Black defendants as high risk at nearly twice the rate it did white defendants, and its maker would not show howUnited States · 2016HD-INC-043A Cruise robotaxi dragged a pedestrian twenty feet, and the company left the dragging out of what it told regulatorsUnited States · 2023HD-INC-042The Dutch tax office used a risk algorithm that flagged families by nationality, and wrongly branded tens of thousands as benefit fraudstersNetherlands · 2021HD-INC-040Amazon built an AI tool to rank job applicants, found it had taught itself to downgrade women, and scrapped itUnited States · 2018HD-INC-034Google's Bard Ad Got a Telescope Fact Wrong, and Alphabet Lost 100 Billion USD in a DayUnited States · 2023HD-INC-031A Tenant-Screening Score Allegedly Shut Housing Voucher Holders Out of HomesUnited States · 2024HD-INC-030A man applied to over 100 jobs, was rejected every time, and a US court let him sue the AI screening vendor, not just the employersUnited States · 2025HD-INC-029Google's AI Overviews Told People to Put Glue on Pizza and Eat a Small Rock Every DayUnited States · 2024HD-INC-018iTutorGroup's Hiring Software Automatically Rejected Older Applicants Because of Their AgeUnited States · 2020HD-INC-017Zillow's Algorithm Got House Prices Wrong, and It Cost More Than 500 Million USDUnited States · 2021HD-INC-015Commonwealth Bank Replaced Workers With AI. Then Reversed Course.Australia · 2025HD-INC-014Woolworths' AI Assistant Olive Talked About Its Mother and Seemed to Claim It Was HumanAustralia · 2026HD-INC-009Robodebt, Australia's automated welfare-debt scheme raised $1.76 billion in unlawful debts against 443,000 peopleAustralia · 2016HD-INC-007UnitedHealth allegedly used an algorithm with a 90% error rate to deny post-acute care to elderly Medicare Advantage patientsUnited States · 2022HD-INC-006New York City's Business Chatbot Told Owners They Could Break the Law, and Stayed OnlineUnited States · 2023HD-INC-001Air Canada chatbot promised a bereavement refund policy that did not existCanada · 2022
Related topics
- Technology & software 19
- Government & public administration 15
- Financial services & insurance 10
- Overconfidence 30
- Rules-behaviour gap 16
- Automation bias 8
- Concealment 6
- United States 56
- Australia 8
- Brazil 6
Cite this page: Harris, E. (2026). AI incidents: algorithmic accountability. Headlights. https://useheadlights.com/topics/domain/algorithmic-accountability/