What happened
It was reported that in November 2024 the United States Federal Trade Commission took action against Evolv Technologies, alleging that the company had made false or unsupported claims about its AI-powered weapons scanner, Evolv Express, which had been sold to schools, stadiums and hospitals. According to the FTC, Evolv represented that the system would detect all weapons while ignoring harmless personal items, that it was more accurate and faster than metal detectors, that it reduced false alarms, and that it cut labour costs by about 70 percent, and the FTC said these claims were not backed by adequate evidence. In its complaint the FTC noted that, for all the language about artificial intelligence, the only things the scanners detect are metallic, which it described as a marketing distinction rather than a real one.
The agency also pointed to the system’s performance in the field. The FTC said the scanners had failed to detect a seven-inch knife that was brought into a school in October 2022 and used to stab a student; the school was in Utica, New York, according to reporting on the case. Separately, NBC reported that a school district in Rockford, Illinois recorded more than 85,000 false alerts on laptops between August 2023 and April 2024, while recovering five knives. Under the settlement, which the FTC’s commissioners approved by a vote of five to nothing, Evolv was barred from making unsupported claims about detection, accuracy, false-alarm rates, speed and cost, and certain schools that had signed contracts between April 2022 and June 2023 were allowed to cancel them. The order carried no monetary penalty and no admission of wrongdoing. Evolv said it disagreed with the allegations, had not admitted any wrongdoing, and considered the matter to be about past marketing language rather than the value of its technology.
What an auditable version would have shown
The claim at the centre of the case was measurable: does the system detect the weapons it says it detects, and how often does it alarm on something harmless? An auditable version is designed to answer that continuously, from the system’s own operation. A signed record of each scan, what triggered an alert, whether a weapon was actually found, and what was missed, aggregated across the sites where it is deployed, can produce the detection rate and the false-alarm rate as figures a school board, a regulator or a parent can read. If those numbers had existed and been shared, any gap between what was claimed and what the scanners did could have shown up in the first months, in the schools relying on them, rather than being inferred later from a stabbing and a district’s spreadsheet of false alerts.
Where the gap was
The FTC’s case came down to a simple gap: Evolv made safety claims it had not backed with evidence. A MetricRecord turns the system’s real performance into a standing, signed number, detection and false-alarm rates by site and over time, so a claim about catching weapons can be measured against what the machine actually does instead of taken on the vendor’s word. A VerificationGate holds a claim like “detects all weapons” to independent proof before it is made, so it has to be shown, not just said. The technology was not doing nothing; what was missing was any measured record standing between the marketing and the reality.
What governance should have looked like
A system that people are asked to trust with physical safety, especially the safety of children, should carry a higher burden of proof than a marketing page. Before such a claim is made, and continuously after deployment, the capability should be measured against real outcomes, and the results kept in a form an independent party can audit. As the FTC put it, claims about technology, including artificial intelligence, need to be backed up, and that is especially important when the claims involve the safety of children. Where a safety capability cannot be shown from the record to perform as promised, the claim, not the parent’s confidence, is what should give way.
Failure Pattern: a safety-critical AI capability was marketed as proven without independently established detection and false-alarm rates.
Governance Principle: a claim that an AI system detects or prevents a real-world harm must be backed by measured, independently verifiable performance before it is sold or relied on, and that performance must be tracked in production.
The reference implementation of MetricRecord and VerificationGate is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- FTC takes action against Evolv Technologies for deceiving users about its AI-powered security screening (FTC media release)
- Settlement approved in FTC case against weapons-detection company Evolv (NBC Chicago)
- Evolv announces resolution of FTC inquiry (Evolv Technologies)