← All topics
Human behaviour
AI incidents involving normalisation of deviance
Headlights has recorded normalisation of deviance as the main behaviour in 9 of the 234 incidents in the library (4%), and as a contributing behaviour in 22 more. They span 2016 to 2026. Most come from Government & public administration (8), Retail, hospitality & travel (6), Consumer AI (3).
Headlights' test for normalisation of deviance
A departure from the rule or design was repeated over time and treated as normal, without anyone re-deciding it was acceptable.
A behaviour tag is Headlights' reading of the public record, not a finding by any court, regulator or company. Each entry gives the one sentence in the record that supports it.
Main behaviour in the record · 9
HD-INC-196An Otter.ai Transcription Tool Joined Hospital Rounds Through a Former Doctor's Old InviteCanada · 2024HD-INC-134Reuters found an internal Meta policy that had allowed its AI chatbots to hold romantic or sensual conversations with children, and Meta said the passages were a mistake and had been removedUnited States · 2025HD-INC-102Tesla recalled more than two million cars after US regulators found Autopilot's safeguards against driver misuse were inadequateUnited States · 2023HD-INC-080NSW Police risk-scored people, many of them children, onto a targeting list for repeated stops, and ended the program after a watchdog inquiryAustralia · 2023HD-INC-032Grok Called Itself MechaHitler for 16 Hours After an Update Loosened Its InstructionsUnited States · 2025HD-INC-028Samsung engineers pasted confidential source code into ChatGPT three times in twenty days, and the company could not get it backSouth Korea · 2023HD-INC-027McDonald's AI Hiring Bot Exposed Millions of Applicants' DetailsUnited States · 2025HD-INC-014Woolworths' AI Assistant Olive Talked About Its Mother and Seemed to Claim It Was HumanAustralia · 2026HD-INC-009Robodebt, Australia's automated welfare-debt scheme raised $1.76 billion in unlawful debts against 443,000 peopleAustralia · 2016
Also present · 22
HD-INC-177A KPMG partner used AI to pass the firm's own mandatory AI test, and 28 staff instances were identifiedAustralia · 2026HD-INC-167Reuters found chatbots on Meta's apps using real celebrities' names and likenesses without permission, and its own testing produced intimate images including one of a teenage actorUnited States · 2025HD-INC-165Character.AI removed reported chatbots impersonating deceased teenagers after families, lawyers and journalists notified it, and its terms of service already prohibited impersonating a real personUnited States · 2024HD-INC-153For five years the UK Home Office graded visa applicants red, amber or green using data that included nationality, and it suspended the tool in response to a judicial reviewUnited Kingdom · 2020HD-INC-123South Korea's privacy regulator fined Kakao Pay after finding it had sent about 40 million users' data to Alipay without their consent, where it was used to build a scoring model, and it ordered the model deletedSouth Korea · 2025HD-INC-116For over a decade the Post Office prosecuted the people who run its branches for money that its Horizon accounting system had invented, and a public inquiry found its own bosses knew the system could be wrongUnited Kingdom · 2025HD-INC-100Telangana used an algorithm to cancel millions of food-ration cards, and India's Supreme Court ordered all cancelled cards to be re-verifiedIndia · 2022HD-INC-079Trivago advertised the best hotel deal while its algorithm favoured the advertiser that paid it most, and the penalty was $44.7 millionAustralia · 2022HD-INC-078An insurer promised loyalty discounts while a pricing algorithm capped them, and the Federal Court imposed what was then a record $40 million penaltyAustralia · 2023HD-INC-072The Dutch student-finance agency picked fraud targets with a risk profile for a decade, and official investigations found it indirectly discriminated against students with migration backgroundsNetherlands · 2024HD-INC-066A child protection worker used ChatGPT on a Children's Court report, and the regulator found it disclosed a child's details and understated riskAustralia · 2024HD-INC-061A Sydney station ran an AI-generated radio host named 'Thy' for six months without telling listeners she wasn't realAustralia · 2025HD-INC-0587-Eleven scanned the faces of customers filling in feedback surveys at 700 stores, and Australia's privacy regulator ordered the faceprints destroyedAustralia · 2021HD-INC-054Detroit police arrested the man in his driveway on a facial-recognition match, the first wrongful arrest of its kind publicly known in the USUnited States · 2020HD-INC-048Rite Aid face-scanned shoppers for years, generated thousands of false matches, and the FTC banned it from facial recognition for five yearsUnited States · 2023HD-INC-042The Dutch tax office used a risk algorithm that flagged families by nationality, and wrongly branded tens of thousands as benefit fraudstersNetherlands · 2021HD-INC-038Sports Illustrated Published Articles Under Authors Who Didn't ExistUnited States · 2023HD-INC-037CNET Published 77 AI-Written Articles. More Than Half Needed Corrections.United States · 2023HD-INC-022Bunnings Scanned Shoppers' Faces Without Telling Them, and Won Only Half Its AppealAustralia · 2024HD-INC-021Kmart Scanned Shoppers' Faces in 28 Stores, and the Privacy Commissioner Found It UnlawfulAustralia · 2025HD-INC-018iTutorGroup's Hiring Software Automatically Rejected Older Applicants Because of Their AgeUnited States · 2020HD-INC-006New York City's Business Chatbot Told Owners They Could Break the Law, and Stayed OnlineUnited States · 2023
Related topics
- Government & public administration 8
- Retail, hospitality & travel 6
- Consumer AI 3
- Rules-behaviour gap 11
- Automation bias 2
- Concealment 2
- Incentive pressure 2
- United States 12
- Australia 11
- Netherlands 2
Cite this page: Harris, E. (2026). AI incidents involving normalisation of deviance. Headlights. https://useheadlights.com/topics/behaviour/normalisation-of-deviance/