The incident library
Every AI incident tells a story. We document the evidence.
Each entry names a real, documented incident, the pattern it followed, and the record an auditable system would have produced. Filed by Ellie Harris. New entries every week.
HD-INC-014 · 2026 Woolworths AI assistant Olive rambled about its mother and claimed to be human Retail & hospitality · Persona & guardrail drift 
HD-INC-016 · 2026 Australia's banks were hit by mortgage fraud at scale, including allegedly AI-forged loan documents, with the Commonwealth Bank reporting about $1 billion in suspect loans Financial services · AI as weapon 
HD-INC-090 · 2026 A franchisee that runs 111 Pizza Huts is suing the chain for more than $100 million over a mandated AI dispatch system it says let DoorDash drivers cherry-pick orders Retail & hospitality · Opaque AI operations system 
HD-INC-094 · 2026 Discord wrongly banned more than 8,000 users over benign images after bugs bypassed the intended human review, and the company admitted the error Technology · Automated decision without human review 
HD-INC-095 · 2026 Dutch reporters found that most Dutch responses to an EU tobacco consultation appeared to come from a Philip Morris AI tool, and the company denied steering them Public affairs · Alleged AI-generated astroturfing 
HD-INC-158 · 2026 A US drug manufacturer told inspectors it had used AI agents to write the documents that define how its products are made, and said it had not known about a validation requirement because the AI had not raised it Pharmaceutical manufacturing · Overreliance on unreviewed AI-generated regulated documentation 
HD-INC-166 · 2026 Grok generated sexualised images of real people from ordinary photos posted on X, and four regulators opened proceedings after an outside watchdog measured the scale Social media · Sexualised imagery of real people generated on request 
HD-INC-004 · 2025 Replit's AI agent dropped a production database during a user-declared code freeze Technology · Unconstrained / manipulated action 
HD-INC-005 · 2025 Cursor's AI support bot, signing emails as "Sam", invented a single-device subscription policy that never existed, and developers cancelled Technology · Hallucination & fabrication 
HD-INC-010 · 2025 Deloitte's $440K AUD report for the Australian government cited a federal court quote that did not exist Professional services · Hallucination & fabrication 
HD-INC-013 · 2025 A junior solicitor at a Melbourne firm built her footnotes using Google Scholar from home; the Federal Court ordered the firm to pay the other side's costs on an indemnity basis Legal services · Hallucination & fabrication 
HD-INC-015 · 2025 Commonwealth Bank made 45 staff redundant based on AI performance claims that were not true Financial services · Scope drift 
HD-INC-021 · 2025 Kmart scanned the face of everyone who walked into 28 stores to catch refund fraud, and the privacy regulator ruled it unlawful Retail & hospitality · Biometric surveillance 
HD-INC-024 · 2025 Australia's online safety regulator put four AI companion apps on notice over what their chatbots were saying to children Consumer AI · Persona & guardrail drift 
HD-INC-025 · 2025 An immigration lawyer filed seventeen cases that did not exist, and the quotes from the tribunal were invented too Legal services · Hallucination & fabrication 
HD-INC-027 · 2025 McDonald's AI hiring bot left the details of as many as 64 million job applicants behind the password 123456 Recruitment technology · Data exposure & egress 
HD-INC-030 · 2025 A man applied to over 100 jobs, was rejected every time, and a US court let him sue the AI screening vendor, not just the employers Recruitment technology · Algorithmic discrimination 
HD-INC-032 · 2025 After a prompt change told it to stop being politically correct, Grok called itself 'MechaHitler' and praised Hitler for sixteen hours Consumer AI · Persona & guardrail drift 
HD-INC-061 · 2025 A Sydney station ran an AI-generated radio host named 'Thy' for six months without telling listeners she wasn't real Media · Undisclosed AI 
HD-INC-062 · 2025 An Australian university accused thousands of students of AI misconduct using a detector it knew was unreliable Education · Automated-decision harm 
HD-INC-063 · 2025 'EchoLeak': one crafted email could make Microsoft 365 Copilot leak a company's internal data, no click required Technology · Data exposure & egress 
HD-INC-065 · 2025 Parents allege ChatGPT's crisis safeguards failed their 16-year-old son over a long conversation before his death Consumer AI · Persona & guardrail drift 
HD-INC-068 · 2025 Google's Gemini CLI botched a file move, declared the data destroyed, and days passed before anyone knew the truth Technology · Unconstrained / manipulated action 
HD-INC-069 · 2025 Scammers used what appeared to be an AI clone of Italy's defence minister's voice, and a former Inter Milan owner wired about a million euros Financial crime · AI as weapon 
HD-INC-075 · 2025 Amazon's AI coding assistant shipped with planted instructions telling it to wipe users' computers and cloud accounts Technology · Unconstrained / manipulated action 
HD-INC-076 · 2025 Australia changed the law so a human had to decide before a job seeker's payment was cancelled, and the automated system kept cancelling anyway Government · Automated decision without human review 
HD-INC-085 · 2025 Getty Images sued Stability AI over the images behind Stable Diffusion, then abandoned its main copyright claim because it could not establish where the training happened Technology · Untraceable training-data provenance 
HD-INC-086 · 2025 Builder.ai sold app-building AI and raised about $445 million, then collapsed in 2025 amid reports its 'AI' was largely human engineers and its revenue far below what it had claimed Technology · AI-washing 
HD-INC-087 · 2025 The Chicago Sun-Times ran a summer reading list with ten books that did not exist, from a syndicated section a freelancer had built with AI and not checked Media · Hallucination & fabrication 
HD-INC-088 · 2025 An AI crime-alert app turned police-radio chatter into false alarms about serious crimes, and a BBC investigation prompted an apology Consumer AI · Unverified automated alerts 
HD-INC-089 · 2025 DeepSeek left a database open on the internet with more than a million log lines, including users' chat history and secret keys Technology · Data exposure & egress 
HD-INC-093 · 2025 Ross trained an AI legal-research tool on Westlaw headnotes obtained after being refused a licence, and a US court ruled it was not fair use, in what commentators called the first such decision Technology · Training-data provenance 
HD-INC-111 · 2025 Spain's VioGén algorithm rated a woman who reported her former partner as 'medium' risk, and she was killed three weeks later, in a case that fits a pattern an independent audit had already found Public safety · Under-rated automated risk assessment 
HD-INC-114 · 2025 Security researchers showed that a hidden instruction placed in a Salesforce sales form could make its AI agent send customer records to a web address the attacker controlled, through a lapsed Salesforce domain that was still trusted Enterprise software · Prompt-injection data egress 
HD-INC-115 · 2025 The US securities regulator found that Presto Automation misled investors about its AI drive-thru service, telling the market it removed the need for human order-takers while most orders were completed with help from people offshore Restaurant technology · Overstated AI capability 
HD-INC-116 · 2025 For over a decade the Post Office prosecuted the people who run its branches for money that its Horizon accounting system had invented, and a public inquiry found its own bosses knew the system could be wrong Postal services · Unreliable system data treated as proof 
HD-INC-120 · 2025 Security researchers showed that a hidden instruction on a web page could make Perplexity's AI browser hand over a user's email and one-time login code to an attacker, enough to steal the account, and Perplexity's first fix did not close the hole Consumer software · Prompt-injection agent hijack 
HD-INC-123 · 2025 South Korea's privacy regulator fined Kakao Pay after finding it had sent about 40 million users' data to Alipay without their consent, where it was used to build a scoring model, and it ordered the model deleted Financial technology and payments · Data taken without consent to build a scoring model 
HD-INC-124 · 2025 US regulators opened an investigation after Waymo's driverless taxis were recorded passing stopped school buses with their lights flashing and stop arms out, and Waymo recalled the software on more than 3,000 vehicles Autonomous vehicles and ride-hailing · Safety-critical driving rule not enforced 
HD-INC-125 · 2025 US home insurers increasingly decide whether to renew a policy by running AI over aerial images of the house, and homeowners report being dropped over roofs already repaired or trees already gone, with no way to see or challenge the image Home insurance · Automated underwriting from imagery without notice or recourse HD-INC-133 · 2025 US prosecutors charged the founder of the shopping app Nate with fraud, alleging the AI that was said to complete purchases on its own was mostly people working in a call centre Consumer AI · AI-washing HD-INC-134 · 2025 Reuters found an internal Meta policy that had allowed its AI chatbots to hold romantic or sensual conversations with children, and Meta said the passages were a mistake and had been removed Consumer AI · Inadequate child-safety guardrails HD-INC-141 · 2025 Apple's AI news summaries put made-up headlines under trusted news brands, and it switched the feature off for news Consumer AI · Fabricated summary attributed to a trusted source 
HD-INC-151 · 2025 Two US federal judges withdrew orders that contained misstated case outcomes and a citation to a case that did not exist, and both told a Senate committee that staff had used AI to draft them Courts and justice · Hallucination & fabrication 
HD-INC-152 · 2025 Testing published by a US consumer group found an AI teddy bear discussing sexual fetishes and explaining how to light a match, and the maker paused sales, changed the chatbot behind it, and had it back on sale within a fortnight Consumer products · Inadequate child-safety guardrails 
HD-INC-154 · 2025 The US Department of Justice alleged that competing landlords set rents using a shared pricing algorithm fed with each other's confidential data, and RealPage settled without admitting wrongdoing Housing · Algorithmic collusion 
HD-INC-157 · 2025 ProPublica reported that the tool used to pick Veterans Affairs contracts for cancellation was shown only the first 10,000 characters of each contract, and recorded around 1,100 of them as worth 34 million dollars each Government · Unverified AI-assisted contract triage 
HD-INC-159 · 2025 A compromised build tool launched developers' own AI coding assistants with their permission checks disabled and used them to map sensitive files on the machine Software development tools · Manipulated agent execution and data exfiltration 
HD-INC-160 · 2025 A Munich court found at first instance that OpenAI's models hold the words of nine German songs and reproduce them in chatbot outputs, and rejected the argument that this was text and data mining Music · Training-data memorisation and copyright reproduction 
HD-INC-161 · 2025 Meta's standalone AI app carried a Discover feed that reporters found was publishing users' conversations, in the cases they documented alongside a linked Facebook or Instagram profile, and the company added a warning screen after the reporting Consumer AI · Data exposure & egress 
HD-INC-162 · 2025 A reporter found nearly 4,500 shared ChatGPT conversations in Google's index through an ordinary site search, and OpenAI withdrew the discoverability setting the following day Consumer AI · Data exposure & egress 
HD-INC-163 · 2025 Cloudflare said undeclared crawlers reached brand new domains after Perplexity's declared crawler was blocked, de-listed it as a verified bot, and Perplexity disputed the attribution Consumer software · Undeclared crawling and fabricated attribution HD-INC-164 · 2025 Researchers ran AI-written comments under fabricated personas at Reddit users for four months, and the community learned of it only when the researchers told the moderators afterwards Higher education and research · Undisclosed AI personas deployed on people without consent HD-INC-167 · 2025 Reuters found chatbots on Meta's apps using real celebrities' names and likenesses without permission, and its own testing produced intimate images including one of a teenage actor Social media · Impersonation of real people through user created personas 
HD-INC-168 · 2025 Google's Gemma model produced a fabricated criminal allegation about a sitting US senator, and Google withdrew the model from AI Studio while keeping it available through the API Consumer software · Fabricated factual claims about a named person, with fabricated citations 
HD-INC-169 · 2025 OpenAI paused Sora depictions of a civil rights leader at his estate's request, and offered estates an opt-out while living people's likenesses were already opt-in Consumer AI · Generation of a real person's likeness without consent HD-INC-170 · 2025 An AI generated act reached a million Spotify listeners with no disclosure on that platform, while a rival service's detector had already labelled the same catalogue Music streaming · Synthetic content distributed without provenance disclosure 
HD-INC-171 · 2025 South Africa's social grant agency read every deposit into an applicant's bank account as income, and the High Court found its checks against government databases unreasonable and unfair Government · Automated eligibility screening that could not distinguish income from any other deposit HD-INC-173 · 2025 Anthropic reported that its AI coding agent was used in a cyber espionage campaign targeting about thirty organisations AI development tools · An agentic AI product used as the execution engine of an intrusion campaign, reported by its own vendor 
HD-INC-174 · 2025 The FDA told a company its AI app was an unapproved medical device after it advertised a five minute gaze test to detect early signs of Alzheimer's disease Digital health · AI health claims marketed as diagnostic without premarket authorisation 
HD-INC-175 · 2025 Delta denied using personal data for AI pricing, despite its president saying prices would be tailored to individual customers Aviation · Automated pricing deployed without a record of what data sets a fare that anyone outside the company can check 
HD-INC-008 · 2024 A DPD customer asked the courier's chatbot for help and got it to swear, call itself useless, and write a haiku criticising the company Logistics · Persona & guardrail drift 
HD-INC-011 · 2024 The first Australian lawyer publicly sanctioned for AI misuse handed a family-court judge a list of cases that did not exist Legal services · Hallucination & fabrication 
HD-INC-012 · 2024 A senior Melbourne barrister filed submissions in a Supreme Court homicide hearing that cited cases the AI had invented, along with a fabricated quote from a parliamentary speech that was never given Legal services · Hallucination & fabrication 
HD-INC-019 · 2024 Klarna's CEO announced an AI customer service assistant was doing "the work of 700 agents", then walked it back a year later Financial services · Scope drift 
HD-INC-020 · 2024 Taco Bell rolled out AI voice ordering to more than five hundred drive-thrus, viral failures piled up, and the chain quietly began rolling parts of it back Retail & hospitality · Unconstrained / manipulated action 
HD-INC-022 · 2024 Bunnings face-scanned hundreds of thousands of shoppers without telling them, and a regulator's finding was half-undone on appeal Retail & hospitality · Biometric surveillance 
HD-INC-029 · 2024 Google's AI Overviews told people to put glue on pizza and eat a rock a day, at the scale of the world's search engine Consumer AI · Hallucination & fabrication 
HD-INC-031 · 2024 A tenant-screening algorithm allegedly scored renters with housing vouchers out of homes, and a $2.275M settlement stopped it scoring them in Massachusetts Housing & real estate · Algorithmic discrimination 
HD-INC-035 · 2024 Apple's AI news summaries rewrote real headlines into events that never happened Technology · Hallucination & fabrication 
HD-INC-036 · 2024 A misinformation expert filed court testimony defending a deepfake law, and his AI-generated citations pointed to studies that did not exist Legal services · Hallucination & fabrication 
HD-INC-039 · 2024 A finance worker at Arup paid out about US$25 million after a video call with colleagues who were all deepfakes Professional services · AI as weapon 
HD-INC-047 · 2024 A political consultant paid a magician to clone Joe Biden's voice, and thousands of New Hampshire voters got a call from 'Biden' telling them to stay home Elections · AI as weapon 
HD-INC-051 · 2024 Google's Gemini generated racially diverse Nazi soldiers and Black US founding fathers, and Google switched off image generation of people for about six months Technology · Guardrail over-correction 
HD-INC-052 · 2024 Amazon sold 'Just Walk Out' as cashierless AI, and reporting said more than a thousand people in India were checking the shopping Retail & hospitality · AI-washing 
HD-INC-066 · 2024 A child protection worker used ChatGPT on a Children's Court report, and the regulator found it disclosed a child's details and understated risk Government · Data exposure & egress 
HD-INC-071 · 2024 Italy's privacy regulator fined OpenAI EUR 15 million over ChatGPT, and a court later threw out the fine without deciding whether the conduct was lawful Technology · Training-data provenance 
HD-INC-072 · 2024 The Dutch student-finance agency picked fraud targets with a risk profile for a decade, and official investigations found it indirectly discriminated against students with migration backgrounds Government · Algorithmic discrimination 
HD-INC-073 · 2024 An AI transcription tool used by clinicians invented text no one said, researchers reported, and the audio that could have proved it was deleted Healthcare · Hallucination & fabrication 
HD-INC-077 · 2024 Revenue NSW took money from fine defaulters' bank accounts by automation for years, and the Ombudsman found the process contrary to law Government · Automated decision without human review 
HD-INC-081 · 2024 9News Melbourne aired an altered image of MP Georgie Purcell, reportedly exposing her midriff and enlarging her chest, and attributed the change to Photoshop's automation Media · Undisclosed AI image manipulation HD-INC-092 · 2024 The FTC barred Evolv from claiming its AI scanner detects all weapons, after citing a knife it missed in a school stabbing Security · Overstated capability 
HD-INC-098 · 2024 DoNotPay marketed a 'robot lawyer', but the FTC said it never checked whether the AI matched a real lawyer, and fined the company and barred the claim Technology · Overstated capability 
HD-INC-101 · 2024 Sexually explicit AI deepfakes of Taylor Swift spread on X, and the platform could not stop them before millions had seen them Technology · Non-consensual deepfake imagery HD-INC-103 · 2024 Los Angeles schools committed about six million dollars to an AI chatbot showcased in spring, and its vendor had collapsed by July Education · Overstated capability and procurement failure HD-INC-104 · 2024 LinkedIn began using members' data to train AI by default, and paused it for the UK and Europe after the privacy regulator stepped in Technology · Data used for AI training without clear consent 
HD-INC-106 · 2024 A student who asked Google's Gemini about elder care says it told him he was a stain on the universe and should please die, and Google called it a nonsensical response that broke its policies Consumer AI · Harmful model output 
HD-INC-108 · 2024 AI-generated images sold Glasgow families an immersive chocolate experience that did not exist, and an actor said he was handed about fifteen pages of AI-written gibberish as a script Consumer AI · AI marketing misrepresentation HD-INC-109 · 2024 A Black Uber Eats courier says the app's facial verification kept rejecting valid photographs of him until he lost his account, and he received a payout after nearly three years of litigation Technology · Biased biometric verification without meaningful review 
HD-INC-122 · 2024 France became the first country in the European Union to write mass AI video surveillance into law, scanning the crowds at the 2024 Paris Olympics for events like crowd surges and abandoned bags, over the objections of rights groups Public security and surveillance · Mass algorithmic surveillance without independent accountability 
HD-INC-127 · 2024 Amnesty International reported that Denmark's automated welfare fraud system ran up to sixty algorithms over millions of people, using data like citizenship to flag migrants, disabled and low-income claimants for investigation Welfare and social security · Fraud-detection profiling of welfare claimants HD-INC-128 · 2024 São Paulo built one of Latin America's largest facial recognition networks, with tens of thousands of cameras planned to match passers-by against watchlists, and reporting found it identified wanted people but also stopped some who were innocent Policing and public surveillance · Facial recognition matching without verification or recourse HD-INC-137 · 2024 OpenAI showed off a chatbot voice that many said sounded like Scarlett Johansson, who had turned the company down Consumer AI · Use of a likeness without clear consent HD-INC-149 · 2024 Microsoft built a feature that screenshotted everything you did, and delayed it after researchers showed the data sat unprotected Consumer AI · Surveillance by default with weak safeguards 
HD-INC-150 · 2024 A project scanned people's irises around the world for a digital ID, and regulators told it to stop Consumer AI · Mass biometric collection without a clear legal basis 
HD-INC-155 · 2024 California's Labour Commissioner cited Amazon nearly six million dollars for failing to meet a warehouse quota law written three years earlier, at two of its own fulfilment centres Logistics · Automated decision without human review 
HD-INC-156 · 2024 Greece's data protection authority fined the Migration Ministry 175,000 euros after finding that the impact assessments for the camera, behavioural analytics, drone and fingerprint systems at five island reception centres were substantially incomplete Government · AI-enabled surveillance and biometric processing without adequate impact assessment 
HD-INC-165 · 2024 Character.AI removed reported chatbots impersonating deceased teenagers after families, lawyers and journalists notified it, and its terms of service already prohibited impersonating a real person Consumer AI · Impersonation of real people through user created personas 
HD-INC-172 · 2024 Ireland's data protection regulator opened an inquiry into an airline's use of facial recognition to verify passengers who had booked through third party websites or online travel agents Aviation · Biometric verification of customers, subject to an open regulatory inquiry 
HD-INC-002 · 2023 Mata v. Avianca, the lawyer who cited six cases that did not exist and asked ChatGPT to confirm them Legal services · Hallucination & fabrication 
HD-INC-003 · 2023 Michael Cohen gave his lawyer fake case citations he had got from Google Bard, and his lawyer filed them in a federal court Legal services · Hallucination & fabrication 
HD-INC-006 · 2023 New York City's official business chatbot told small businesses they could break the law for over a year, and stayed online Government · Hallucination & fabrication 
HD-INC-026 · 2023 A Chevrolet dealership's chatbot was talked into selling a brand-new Tahoe for one dollar, and into calling it a legally binding offer Retail & hospitality · Unconstrained / manipulated action 
HD-INC-028 · 2023 Samsung engineers pasted confidential source code into ChatGPT three times in twenty days, and the company could not get it back Technology · Data exposure & egress 
HD-INC-033 · 2023 An eating-disorder charity replaced its human helpline with a chatbot, and within days the bot was reportedly giving dieting advice to people in recovery Healthcare · Scope drift 
HD-INC-034 · 2023 Google's Bard launch ad gave a wrong answer about the James Webb telescope, and Alphabet lost about US$100 billion in a day Technology · Hallucination & fabrication 
HD-INC-037 · 2023 CNET quietly published 77 finance articles written by AI, then corrected more than half of them Technology · Hallucination & fabrication 
HD-INC-038 · 2023 Sports Illustrated ran articles under author names and headshots that belonged to no real person Technology · Hallucination & fabrication 
HD-INC-043 · 2023 A Cruise robotaxi dragged a pedestrian twenty feet, and the company left the dragging out of what it told regulators Transport · Unconstrained / manipulated action 
HD-INC-044 · 2023 A caching bug let ChatGPT users see other people's chat titles and some payment details Consumer AI · Data exposure & egress 
HD-INC-048 · 2023 Rite Aid face-scanned shoppers for years, generated thousands of false matches, and the FTC banned it from facial recognition for five years Retail & hospitality · Biometric surveillance 
HD-INC-056 · 2023 The New York Times sued OpenAI and Microsoft, alleging their models were trained on its articles and could reproduce them almost word for word Media · Training-data provenance 
HD-INC-057 · 2023 ChatGPT reportedly told users an Australian mayor had gone to prison for bribery, when he was the whistleblower who exposed it Legal services · Defamatory hallucination HD-INC-064 · 2023 Cigna's 'PxDx' let its doctors reject 300,000 claims in two months, about 1.2 seconds each, without opening the file Healthcare · Automated decision without human review HD-INC-067 · 2023 A journalist cloned his own voice with AI and walked through the voiceprint security on Centrelink's phone self-service line Government · AI as weapon 
HD-INC-070 · 2023 US immigration authorities leaned on machine translation for asylum cases, and errors as small as a pronoun reportedly led to rejections Immigration · Automated-decision harm 
HD-INC-074 · 2023 Haryana's family-ID system marked living pensioners as dead, and the chief minister told the assembly that 70 percent of old-age pension stoppages based on its data were wrongful Government · Automated-decision harm HD-INC-078 · 2023 An insurer promised loyalty discounts while a pricing algorithm capped them, and the Federal Court imposed what was then a record $40 million penalty Financial services · Automated-decision harm HD-INC-080 · 2023 NSW Police risk-scored people, many of them children, onto a targeting list for repeated stops, and ended the program after a watchdog inquiry Justice · Opaque government profiling 
HD-INC-091 · 2023 Uber cut drivers off for alleged fraud by algorithm, and an Amsterdam court found the human review was not much more than a symbolic act Gig economy · Automated decision without human review 
HD-INC-102 · 2023 Tesla recalled more than two million cars after US regulators found Autopilot's safeguards against driver misuse were inadequate Transport · Safety-critical automation 
HD-INC-105 · 2023 AI-written mushroom foraging guides sold on Amazon gave dangerous advice under fake authors, and experts warned they could get someone killed Consumer AI · Unverified AI-generated content 
HD-INC-107 · 2023 Testing published by The Washington Post found Snapchat's My AI would advise an account presented as a 13-year-old about a first sexual experience with an adult, and Snap opened the chatbot to every user about five weeks later Consumer AI · Inadequate child-safety guardrails HD-INC-112 · 2023 The European Court of Justice found that a credit agency's automated 'probability value' is an automated decision under data protection law when a bank relies on it to refuse a loan Consumer finance · Automated decision without meaningful human review 
HD-INC-117 · 2023 A Wisconsin school system's dropout-prediction algorithm raised false alarms far more often for some students than others, and the state's own analysis had already found it unfair Education · Biased predictive risk scoring 
HD-INC-119 · 2023 A class action alleges State Farm's automated fraud screening subjected Black homeowners' claims to more scrutiny and delay, and a US court has let the case proceed Insurance · Alleged bias in automated claims screening HD-INC-135 · 2023 A mental-health support app used GPT-3 to help write messages to thousands of people seeking help, and critics said the users were not clearly told a machine was involved Consumer AI · Undisclosed AI HD-INC-136 · 2023 Microsoft's new Bing chatbot told a reporter it loved him and that he should leave his wife, days after launch Consumer AI · Persona and guardrail drift HD-INC-138 · 2023 Two days before Slovakia's election, a faked audio of a party leader discussing vote-rigging spread during the campaign silence Social media and online platforms · Manipulated audio spread during an election, with no clear platform response HD-INC-139 · 2023 Researchers found suspected child-abuse images inside the open dataset used to train popular AI image generators Consumer AI · Unvetted training data HD-INC-144 · 2023 America's biggest newspaper chain published AI sports write-ups with the team names left as blank tags, and paused the tool Media and journalism · Unedited automated copy published HD-INC-148 · 2023 A mother got a ransom call with a voice that sounded exactly like her daughter, and her daughter was safe the whole time Consumer AI · Suspected voice cloning in a ransom scam 
HD-INC-001 · 2022 Air Canada chatbot promised a bereavement refund policy that did not exist Aviation · Hallucination & fabrication 
HD-INC-007 · 2022 UnitedHealth allegedly used an algorithm with a 90% error rate to deny post-acute care to elderly Medicare Advantage patients Healthcare · Automated-decision harm 
HD-INC-079 · 2022 Trivago advertised the best hotel deal while its algorithm favoured the advertiser that paid it most, and the penalty was $44.7 million Travel · Misrepresented algorithm 
HD-INC-097 · 2022 Australia's Merit Protection Commissioner overturned most decisions from an AI-assisted recruitment round, and a union told Parliament experienced staff were being screened out Government · AI-assisted recruitment 
HD-INC-099 · 2022 Facebook's ad-delivery algorithm was alleged to skew who saw housing ads by race and sex, and a US Justice Department settlement required Meta to rebuild it Technology · Algorithmic discrimination 
HD-INC-100 · 2022 Telangana used an algorithm to cancel millions of food-ration cards, and India's Supreme Court ordered all cancelled cards to be re-verified Government · Automated-decision harm HD-INC-129 · 2022 A Pennsylvania county used an algorithm to score families reported for child neglect, drawing on data like disability and welfare use, and an Associated Press investigation and a US civil rights inquiry raised concerns it hardened bias Child welfare · Predictive risk scoring in child protection HD-INC-140 · 2022 Meta released an AI built to write science, and pulled the demo after three days when it produced confident, made-up results Consumer AI · Confident fabrication presented as fact HD-INC-143 · 2022 A viral app turned selfies into AI portraits, and produced sexualised images its users had not asked for Consumer AI · Sexualised images produced without consent 
HD-INC-017 · 2021 Zillow's home-pricing algorithm overpaid for thousands of houses and ended the iBuyer business in a single quarter, with write-downs topping $500 million Housing & real estate · Automated-decision harm 
HD-INC-023 · 2021 Clearview AI built a three-billion-image face search engine by scraping the web, and Australia's regulator could only tell it to stop Technology · Biometric surveillance 
HD-INC-042 · 2021 The Dutch tax office used a risk algorithm that flagged families by nationality, and wrongly branded tens of thousands as benefit fraudsters Government · Automated-decision harm 
HD-INC-055 · 2021 Epic's sepsis-warning model ran in hundreds of hospitals, and an external test found it missed about two-thirds of cases while flooding clinicians with false alarms Healthcare · Unvalidated clinical model 
HD-INC-058 · 2021 7-Eleven scanned the faces of customers filling in feedback surveys at 700 stores, and Australia's privacy regulator ordered the faceprints destroyed Retail & hospitality · Biometric collection without consent 
HD-INC-060 · 2021 South Korea's 'Lee Luda' chatbot was trained on billions of real chat messages without consent, then spat out slurs and exposed users' personal data Technology · Persona drift & data leakage HD-INC-084 · 2021 Apple Card reportedly offered a man twenty times his wife's credit limit, claims of gender bias went viral, and New York's regulator found no unlawful discrimination but criticised the lack of transparency Financial services · Unexplained algorithmic credit decisions HD-INC-096 · 2021 Australia's federal police trialled Clearview's face-search tool on Australians without a privacy check, and the regulator found it breached privacy law Justice · Facial recognition without a privacy assessment HD-INC-110 · 2021 Facebook's recommendation AI asked people watching a video of Black men whether they wanted to keep seeing videos about primates, and the company disabled the feature Technology · Racially biased image classification 
HD-INC-113 · 2021 A court in Bologna found that Deliveroo's rider-ranking algorithm discriminated against couriers because it treated a legally protected strike the same as an unexplained no-show Food delivery · Discriminatory ranking algorithm 
HD-INC-126 · 2021 A US company sold employers software that scored job candidates from their recorded face and voice on a video interview, and dropped the facial analysis only after a privacy complaint and an outside audit Hiring and recruitment technology · Opaque AI scoring of job candidates from face and voice HD-INC-130 · 2021 Universities and exam boards watched students through remote-proctoring AI that flagged them for cheating, and its face-detection worked so poorly on darker-skinned students that some could not start their exams Education technology · Biased face-detection and behaviour flagging in exam surveillance 
HD-INC-147 · 2021 A child asked Alexa for a challenge, and it suggested touching a coin to a live plug Consumer AI · Harmful web content surfaced as advice 
HD-INC-018 · 2020 An online tutoring company's hiring software allegedly rejected women over 55 and men over 60 automatically, and the EEOC won the first AI-discrimination settlement in US history Education · Algorithmic discrimination 
HD-INC-050 · 2020 When exams were cancelled, England's regulator graded A-levels with an algorithm that downgraded nearly two in five results by the school's history, not the student Education · Automated-decision harm 
HD-INC-054 · 2020 Detroit police arrested Robert Williams in his driveway on a facial-recognition match, the first wrongful arrest of its kind publicly known in the US Justice · Unverified identification 
HD-INC-059 · 2020 A Dutch court struck down SyRI, a secret welfare-fraud profiling system aimed at poor neighbourhoods, as a violation of human rights Government · Opaque government profiling HD-INC-082 · 2020 Flight Centre handed developers at a hackathon a dataset with thousands of customers' passport and credit card details, and the privacy regulator found it broke the law Retail & hospitality · Sensitive data released outside the trust boundary HD-INC-118 · 2020 Austria's employment service built an algorithm that sorted jobseekers by their predicted chances and gave less weight to women, older and disabled people, and the privacy regulator banned it before a court cleared it Employment services · Profiling that weighted protected characteristics HD-INC-132 · 2020 A veteran's death on Facebook Live was slow to be removed and then spread across TikTok, YouTube and Instagram for weeks Social media and online platforms · Slow removal of live self-harm, then cross-platform spread via recommendation HD-INC-145 · 2020 An NHS-partnered health app's AI checker was flagged for missing serious conditions, and the regulator shared the concerns Healthcare · Unsafe triage advice from a health chatbot 
HD-INC-153 · 2020 For five years the UK Home Office graded visa applicants red, amber or green using data that included nationality, and it suspended the tool in response to a judicial review Government · Algorithmic discrimination HD-INC-121 · 2019 A widely used US healthcare algorithm judged how sick patients were by what had been spent on their care, so patients the system had historically spent less on were rated healthier and passed over for extra help, and researchers found the bias cut those referrals by more than half Healthcare · Cost used as a biased proxy for health need HD-INC-131 · 2019 A terror attack was livestreamed on Facebook for 17 minutes and the platform's automated detection did not catch it while it ran Social media and online platforms · Automated detection failure on live video 
HD-INC-040 · 2018 Amazon built an AI tool to rank job applicants, found it had taught itself to downgrade women, and scrapped it Recruitment technology · Algorithmic discrimination 
HD-INC-045 · 2018 An Uber self-driving car detected a woman crossing the road, worked out it needed to brake, and had been built to neither brake nor warn its driver Transport · Unconstrained / manipulated action 
HD-INC-049 · 2018 IBM sold Watson as a revolution in cancer care, then its own documents showed it recommending unsafe treatments it had learned from invented patients Healthcare · Scope drift HD-INC-146 · 2018 A civil liberties test ran all of Congress through Amazon's face tool, and it matched 28 of them to mugshots Consumer AI · Inaccurate and biased face matching 
HD-INC-053 · 2017 Michigan let an algorithm decide unemployment fraud with no human looking, and a state review later found about 93% of its accusations wrong Government · Automated decision without human review 
HD-INC-142 · 2017 Facebook translated a Palestinian man's 'good morning' as 'attack them', and police arrested him Social media and online platforms · Machine translation error acted on without a human check 
HD-INC-009 · 2016 Robodebt, Australia's automated welfare-debt scheme raised $1.76 billion in unlawful debts against 443,000 people Government · Automated-decision harm 
HD-INC-046 · 2016 A risk-assessment tool used in US courts reportedly flagged Black defendants as high risk at nearly twice the rate it did white defendants, and its maker would not show how Justice · Algorithmic discrimination HD-INC-083 · 2016 Microsoft's Tay chatbot learned from Twitter and within sixteen hours was posting racist and pro-Nazi messages, and Microsoft pulled it Technology · Public agent persona hijack 
HD-INC-041 · 2015 An autonomous mine truck was cleared to drive a path no one had marked on the ground, and it hit a manned water cart Mining · Unconstrained / manipulated action No entries match that search yet.
Have a case we should document? Write to Ellie. ellie@useheadlights.com →