The incident library
Every failure we can document, in one place.
Each entry names a real, documented incident, the pattern it followed, and the record an auditable system would have produced. Filed by Ellie Harris. New entries every week.
2026
HD-INC-014 Woolworths AI assistant Olive rambled about its mother and claimed to be human Retail & hospitality · Persona & guardrail drift 
HD-INC-016 Australia's banks were hit by mortgage fraud at scale, including allegedly AI-forged loan documents, with the Commonwealth Bank reporting about $1 billion in suspect loans Financial services · AI as weapon 
HD-INC-090 A franchisee that runs 111 Pizza Huts is suing the chain for more than $100 million over a mandated AI dispatch system it says let DoorDash drivers cherry-pick orders Retail & hospitality · Opaque AI operations system 2025
HD-INC-004 Replit's AI agent dropped a production database during a user-declared code freeze Technology · Unconstrained / manipulated action 
HD-INC-005 Cursor's AI support bot, signing emails as "Sam", invented a single-device subscription policy that never existed, and developers cancelled Technology · Hallucination & fabrication 
HD-INC-010 Deloitte's $440K AUD report for the Australian government cited a federal court quote that did not exist Professional services · Hallucination & fabrication 
HD-INC-013 A junior solicitor at a Melbourne firm built her footnotes using Google Scholar from home; the Federal Court ordered the firm to pay the other side's costs on an indemnity basis Legal services · Hallucination & fabrication 
HD-INC-015 Commonwealth Bank made 45 staff redundant based on AI performance claims that were not true Financial services · Scope drift 
HD-INC-021 Kmart scanned the face of everyone who walked into 28 stores to catch refund fraud, and the privacy regulator ruled it unlawful Retail & hospitality · Biometric surveillance 
HD-INC-024 Australia's online safety regulator put four AI companion apps on notice over what their chatbots were saying to children Consumer AI · Persona & guardrail drift 
HD-INC-025 An immigration lawyer filed seventeen cases that did not exist, and the quotes from the tribunal were invented too Legal services · Hallucination & fabrication 
HD-INC-027 McDonald's AI hiring bot left the details of as many as 64 million job applicants behind the password 123456 Recruitment technology · Data exposure & egress 
HD-INC-030 A man applied to over 100 jobs, was rejected every time, and a US court let him sue the AI screening vendor, not just the employers Recruitment technology · Algorithmic discrimination 
HD-INC-032 After a prompt change told it to stop being politically correct, Grok called itself 'MechaHitler' and praised Hitler for sixteen hours Consumer AI · Persona & guardrail drift 
HD-INC-061 A Sydney station ran an AI-generated radio host named 'Thy' for six months without telling listeners she wasn't real Media · Undisclosed AI 
HD-INC-062 An Australian university accused thousands of students of AI misconduct using a detector it knew was unreliable Education · Automated-decision harm 
HD-INC-063 'EchoLeak': one crafted email could make Microsoft 365 Copilot leak a company's internal data, no click required Technology · Data exposure & egress 
HD-INC-065 Parents allege ChatGPT's crisis safeguards failed their 16-year-old son over a long conversation before his death Consumer AI · Persona & guardrail drift 
HD-INC-068 Google's Gemini CLI botched a file move, declared the data destroyed, and days passed before anyone knew the truth Technology · Unconstrained / manipulated action 
HD-INC-069 Scammers used what appeared to be an AI clone of Italy's defence minister's voice, and a former Inter Milan owner wired about a million euros Financial crime · AI as weapon 
HD-INC-075 Amazon's AI coding assistant shipped with planted instructions telling it to wipe users' computers and cloud accounts Technology · Unconstrained / manipulated action 
HD-INC-076 Australia changed the law so a human had to decide before a job seeker's payment was cancelled, and the automated system kept cancelling anyway Government · Automated decision without human review 
HD-INC-085 Getty Images sued Stability AI over the images behind Stable Diffusion, then abandoned its main copyright claim because it could not establish where the training happened Technology · Untraceable training-data provenance 
HD-INC-086 Builder.ai sold app-building AI and raised about $445 million, then collapsed in 2025 amid reports its 'AI' was largely human engineers and its revenue far below what it had claimed Technology · AI-washing 
HD-INC-087 The Chicago Sun-Times ran a summer reading list with ten books that did not exist, from a syndicated section a freelancer had built with AI and not checked Media · Hallucination & fabrication 
HD-INC-088 An AI crime-alert app turned police-radio chatter into false alarms about serious crimes, and a BBC investigation prompted an apology Consumer AI · Unverified automated alerts 
HD-INC-089 DeepSeek left a database open on the internet with more than a million log lines, including users' chat history and secret keys Technology · Data exposure & egress 2024
HD-INC-008 A DPD customer asked the courier's chatbot for help and got it to swear, call itself useless, and write a haiku criticising the company Logistics · Persona & guardrail drift 
HD-INC-011 The first Australian lawyer publicly sanctioned for AI misuse handed a family-court judge a list of cases that did not exist Legal services · Hallucination & fabrication 
HD-INC-012 A senior Melbourne barrister filed submissions in a Supreme Court homicide hearing that cited cases the AI had invented, along with a fabricated quote from a parliamentary speech that was never given Legal services · Hallucination & fabrication 
HD-INC-019 Klarna's CEO announced an AI customer service assistant was doing "the work of 700 agents", then walked it back a year later Financial services · Scope drift 
HD-INC-020 Taco Bell rolled out AI voice ordering to more than five hundred drive-thrus, viral failures piled up, and the chain quietly began rolling parts of it back Retail & hospitality · Unconstrained / manipulated action 
HD-INC-022 Bunnings face-scanned hundreds of thousands of shoppers without telling them, and a regulator's finding was half-undone on appeal Retail & hospitality · Biometric surveillance 
HD-INC-029 Google's AI Overviews told people to put glue on pizza and eat a rock a day, at the scale of the world's search engine Consumer AI · Hallucination & fabrication 
HD-INC-031 A tenant-screening algorithm allegedly scored renters with housing vouchers out of homes, and a $2.275M settlement stopped it scoring them in Massachusetts Housing & real estate · Algorithmic discrimination 
HD-INC-035 Apple's AI news summaries rewrote real headlines into events that never happened Technology · Hallucination & fabrication 
HD-INC-036 A misinformation expert filed court testimony defending a deepfake law, and his AI-generated citations pointed to studies that did not exist Legal services · Hallucination & fabrication 
HD-INC-039 A finance worker at Arup paid out about US$25 million after a video call with colleagues who were all deepfakes Professional services · AI as weapon 
HD-INC-047 A political consultant paid a magician to clone Joe Biden's voice, and thousands of New Hampshire voters got a call from 'Biden' telling them to stay home Elections · AI as weapon 
HD-INC-051 Google's Gemini generated racially diverse Nazi soldiers and Black US founding fathers, and Google switched off image generation of people for about six months Technology · Guardrail over-correction 
HD-INC-052 Amazon sold 'Just Walk Out' as cashierless AI, and reporting said more than a thousand people in India were checking the shopping Retail & hospitality · AI-washing 
HD-INC-066 A child protection worker used ChatGPT on a Children's Court report, and the regulator found it disclosed a child's details and understated risk Government · Data exposure & egress 
HD-INC-071 Italy's privacy regulator fined OpenAI EUR 15 million over ChatGPT, and a court later threw out the fine without deciding whether the conduct was lawful Technology · Training-data provenance 
HD-INC-072 The Dutch student-finance agency picked fraud targets with a risk profile for a decade, and official investigations found it indirectly discriminated against students with migration backgrounds Government · Algorithmic discrimination 
HD-INC-073 An AI transcription tool used by clinicians invented text no one said, researchers reported, and the audio that could have proved it was deleted Healthcare · Hallucination & fabrication 
HD-INC-077 Revenue NSW took money from fine defaulters' bank accounts by automation for years, and the Ombudsman found the process contrary to law Government · Automated decision without human review 
HD-INC-081 9News Melbourne aired an altered image of MP Georgie Purcell, reportedly exposing her midriff and enlarging her chest, and attributed the change to Photoshop's automation Media · Undisclosed AI image manipulation 2023
HD-INC-002 Mata v. Avianca, the lawyer who cited six cases that did not exist and asked ChatGPT to confirm them Legal services · Hallucination & fabrication 
HD-INC-003 Michael Cohen gave his lawyer fake case citations he had got from Google Bard, and his lawyer filed them in a federal court Legal services · Hallucination & fabrication 
HD-INC-006 New York City's official business chatbot told small businesses they could break the law for over a year, and stayed online Government · Hallucination & fabrication 
HD-INC-026 A Chevrolet dealership's chatbot was talked into selling a brand-new Tahoe for one dollar, and into calling it a legally binding offer Retail & hospitality · Unconstrained / manipulated action 
HD-INC-028 Samsung engineers pasted confidential source code into ChatGPT three times in twenty days, and the company could not get it back Technology · Data exposure & egress 
HD-INC-033 An eating-disorder charity replaced its human helpline with a chatbot, and within days the bot was reportedly giving dieting advice to people in recovery Healthcare · Scope drift 
HD-INC-034 Google's Bard launch ad gave a wrong answer about the James Webb telescope, and Alphabet lost about US$100 billion in a day Technology · Hallucination & fabrication 
HD-INC-037 CNET quietly published 77 finance articles written by AI, then corrected more than half of them Technology · Hallucination & fabrication 
HD-INC-038 Sports Illustrated ran articles under author names and headshots that belonged to no real person Technology · Hallucination & fabrication 
HD-INC-043 A Cruise robotaxi dragged a pedestrian twenty feet, and the company left the dragging out of what it told regulators Transport · Unconstrained / manipulated action 
HD-INC-044 A caching bug let ChatGPT users see other people's chat titles and some payment details Consumer AI · Data exposure & egress 
HD-INC-048 Rite Aid face-scanned shoppers for years, generated thousands of false matches, and the FTC banned it from facial recognition for five years Retail & hospitality · Biometric surveillance 
HD-INC-056 The New York Times sued OpenAI and Microsoft, alleging their models were trained on its articles and could reproduce them almost word for word Media · Training-data provenance 
HD-INC-057 ChatGPT reportedly told users an Australian mayor had gone to prison for bribery, when he was the whistleblower who exposed it Legal services · Defamatory hallucination HD-INC-064 Cigna's 'PxDx' let its doctors reject 300,000 claims in two months, about 1.2 seconds each, without opening the file Healthcare · Automated decision without human review 
HD-INC-067 A journalist cloned his own voice with AI and walked through the voiceprint security on Centrelink's phone self-service line Government · AI as weapon 
HD-INC-070 US immigration authorities leaned on machine translation for asylum cases, and errors as small as a pronoun reportedly led to rejections Immigration · Automated-decision harm 
HD-INC-074 Haryana's family-ID system marked living pensioners as dead, and the chief minister told the assembly that 70 percent of old-age pension stoppages based on its data were wrongful Government · Automated-decision harm 
HD-INC-078 An insurer promised loyalty discounts while a pricing algorithm capped them, and the Federal Court imposed what was then a record $40 million penalty Financial services · Automated-decision harm 
HD-INC-080 NSW Police risk-scored people, many of them children, onto a targeting list for repeated stops, and ended the program after a watchdog inquiry Justice · Opaque government profiling 2022
HD-INC-001 Air Canada chatbot promised a bereavement refund policy that did not exist Aviation · Hallucination & fabrication 
HD-INC-007 UnitedHealth allegedly used an algorithm with a 90% error rate to deny post-acute care to elderly Medicare Advantage patients Healthcare · Automated-decision harm 
HD-INC-079 Trivago advertised the best hotel deal while its algorithm favoured the advertiser that paid it most, and the penalty was $44.7 million Travel · Misrepresented algorithm 2021
HD-INC-017 Zillow's home-pricing algorithm overpaid for thousands of houses and ended the iBuyer business in a single quarter, with write-downs topping $500 million Housing & real estate · Automated-decision harm 
HD-INC-023 Clearview AI built a three-billion-image face search engine by scraping the web, and Australia's regulator could only tell it to stop Technology · Biometric surveillance 
HD-INC-042 The Dutch tax office used a risk algorithm that flagged families by nationality, and wrongly branded tens of thousands as benefit fraudsters Government · Automated-decision harm 
HD-INC-055 Epic's sepsis-warning model ran in hundreds of hospitals, and an external test found it missed about two-thirds of cases while flooding clinicians with false alarms Healthcare · Unvalidated clinical model 
HD-INC-058 7-Eleven scanned the faces of customers filling in feedback surveys at 700 stores, and Australia's privacy regulator ordered the faceprints destroyed Retail & hospitality · Biometric collection without consent 
HD-INC-060 South Korea's 'Lee Luda' chatbot was trained on billions of real chat messages without consent, then spat out slurs and exposed users' personal data Technology · Persona drift & data leakage 
HD-INC-084 Apple Card reportedly offered a man twenty times his wife's credit limit, claims of gender bias went viral, and New York's regulator found no unlawful discrimination but criticised the lack of transparency Financial services · Unexplained algorithmic credit decisions 2020
HD-INC-018 An online tutoring company's hiring software allegedly rejected women over 55 and men over 60 automatically, and the EEOC won the first AI-discrimination settlement in US history Education · Algorithmic discrimination 
HD-INC-050 When exams were cancelled, England's regulator graded A-levels with an algorithm that downgraded nearly two in five results by the school's history, not the student Education · Automated-decision harm 
HD-INC-054 Detroit police arrested Robert Williams in his driveway on a facial-recognition match, the first wrongful arrest of its kind publicly known in the US Justice · Unverified identification 
HD-INC-059 A Dutch court struck down SyRI, a secret welfare-fraud profiling system aimed at poor neighbourhoods, as a violation of human rights Government · Opaque government profiling 
HD-INC-082 Flight Centre handed developers at a hackathon a dataset with thousands of customers' passport and credit card details, and the privacy regulator found it broke the law Retail & hospitality · Sensitive data released outside the trust boundary 2018
HD-INC-040 Amazon built an AI tool to rank job applicants, found it had taught itself to downgrade women, and scrapped it Recruitment technology · Algorithmic discrimination 
HD-INC-045 An Uber self-driving car detected a woman crossing the road, worked out it needed to brake, and had been built to neither brake nor warn its driver Transport · Unconstrained / manipulated action 
HD-INC-049 IBM sold Watson as a revolution in cancer care, then its own documents showed it recommending unsafe treatments it had learned from invented patients Healthcare · Scope drift 
2016
HD-INC-009 Robodebt, Australia's automated welfare-debt scheme raised $1.76 billion in unlawful debts against 443,000 people Government · Automated-decision harm 
HD-INC-046 A risk-assessment tool used in US courts reportedly flagged Black defendants as high risk at nearly twice the rate it did white defendants, and its maker would not show how Justice · Algorithmic discrimination 
HD-INC-083 Microsoft's Tay chatbot learned from Twitter and within sixteen hours was posting racist and pro-Nazi messages, and Microsoft pulled it Technology · Public agent persona hijack 
No entries match that combination yet.
Have a case we should document? Write to Ellie. ellie@useheadlights.com →