# HD-INC-146: A civil liberties test ran all of Congress through Amazon's face tool, and it matched 28 of them to mugshots

> In July 2018 the American Civil Liberties Union ran photos of all members of the US Congress through Amazon's Rekognition face-matching tool, using the default settings, and it falsely matched 28 of them to police mugshots. Nearly 40 percent of the false matches were people of colour, who made up about a fifth of Congress. The test cost 12.33 dollars, and it showed how a tool sold to police could be wrong before anyone was arrested.

- Source: https://useheadlights.com/library/HD-INC-146/
- Publisher: Headlights (useheadlights.com), author Ellie Harris
- Published: 2026-08-11
- Company: Amazon (Rekognition); test run by the ACLU
- Sector: Consumer AI
- Country: United States
- Year: 2018
- Behaviour in the record (Headlights reading): Overconfidence
- Governance domains: Civil liberties, Consumer protection
- Licence: CC BY-NC 4.0. Cite as: Harris, E. (2026). A civil liberties test ran all of Congress through Amazon's face tool, and it matched 28 of them to mugshots. Headlights AI Incident Library, HD-INC-146. https://useheadlights.com/library/HD-INC-146/

## What happened

It was reported that in July 2018 the American Civil Liberties Union tested Amazon's face-matching tool, Rekognition. The ACLU reported that it ran photos of all members of the US Congress against 25,000 publicly available arrest photos, using the tool's default settings and an 80 percent confidence threshold, and that the tool falsely matched 28 members of Congress to police mugshots. The ACLU said nearly 40 percent of those false matches were people of colour, even though they made up about 20 percent of Congress, and that six of those wrongly matched were members of the Congressional Black Caucus. The whole test, it noted, cost 12.33 dollars.

Amazon said the ACLU had used the default 80 percent setting, and that for law enforcement it recommended a higher confidence threshold and human review, while critics pointed out that the default was the setting many customers would use out of the box. In June 2020, amid wider protests over policing, Amazon announced a one-year moratorium on police use of Rekognition, and in 2021 it said the moratorium would continue indefinitely. It continued offering Rekognition for other, non-police image-analysis uses.

## What an auditable version would have shown

If you are going to use a tool to identify people, you need to know how often it gets that identification wrong. That means testing the settings customers actually use and breaking the results down by skin tone, age and sex, so a buyer can see the false-match rate rather than just a confidence score attached to one result, and so a match is presented as something to investigate rather than as a fact. Those records would have shown buyers what happened at the default setting, including who was more likely to be falsely matched, before the technology was ever pointed at real people.

## Where the gap was

The problem was not simply that Rekognition made mistakes. It was what the person using it could see when it did. A confident-looking match appeared without a clear explanation of how often the system could be wrong, the default setting produced false matches, and nothing in front of the buyer showed that those errors fell more heavily on people of colour. When the consequence of a wrong identification could involve the police, that information matters.

## What governance should have looked like

Before a face-matching tool is used to identify people, it should be tested for accuracy and bias using the same settings customers will actually run. A VerificationGate is designed to hold a tool like this against accuracy and fairness tests before it reaches a setting where a wrong match matters, and if it does not pass, it does not move forward. A MetricRecord would keep the evidence behind that decision, including match and error rates broken down by group, so buyers and regulators could see the numbers rather than relying on a general claim of accuracy. The question should be answered before deployment: how often is this system wrong, and who is it getting wrong?

The reference implementation of VerificationGate and MetricRecord is open source. It lives at [github.com/saffronandindia/headlights-oss](https://github.com/saffronandindia/headlights-oss), Apache 2.0 licensed, free for any company to install. The repository is public now.

## Sources

- [Amazon's Face Recognition Falsely Matched 28 Members of Congress With Mugshots (ACLU)](https://www.aclu.org/news/privacy-technology/amazons-face-recognition-falsely-matched-28)
- [Amazon facial AI matched politicians with criminals in ACLU test (Bloomberg)](https://www.bloomberg.com/news/articles/2018-07-26/amazon-facial-ai-matched-politicians-with-criminals-in-aclu-test)
- [ACLU says Amazon's facial recognition matched Congress members to mugshots (TechCrunch)](https://techcrunch.com/2018/07/26/aclu-says-amazon-facial-recognition-associated-congress-members-with-mugshots/)

## Related

- [HD-INC-054](/library/HD-INC-054)
- [HD-INC-048](/library/HD-INC-048)
- [HD-INC-023](/library/HD-INC-023)