What happened
It was reported that on 20 October 2025 an AI weapons detection system raised an alert on a pupil outside Kenwood High School in Baltimore County, Maryland. Police responded. The pupil was carrying a bag of crisps. It was widely reported that officers handcuffed him.
The Maryland Office of the Inspector General for Education investigated. Two complaints: that the school system ran an ineffective and inefficient AI gun detection project, and that the software “violated students’ civil rights because its detection algorithm targets disadvantaged students and students of color more than other students”.
It upheld neither. On bias, the office found the system “does not consider an individual’s gender or race in its probability algorithm; it focuses solely on the moving object”. On the crisps, it found that “the Omnilert system did not detect a single blue potato chip bag as a threat”. The system reacted to the pupil’s hand, its movement, position and angle, and an unidentified item that “extended slightly beyond the fingers and was held at a slight angle near the hip or pocket level”.
Between November 2023 and November 2025 the system “filtered out 96% of the objects it alerted on and did not require further action”, and “fewer than 1% of alerts forwarded to BCPS by Omnilert required notifying law enforcement”.
Fifteen people received the 20 October alert, five from the school system and ten from the county police. Staff decided the threat was invalid and sent a cancellation by text. The principal saw it only after opening the video, ringing two colleagues and speaking to police, who told him they had cleared the grounds.
What an auditable version would have shown
The office established the alert volume from the school system’s own data. It reviewed that data from November 2023 to November 2025, and found that 96 per cent of objects initially alerted on were filtered out without further action, and that fewer than 1 per cent of alerts forwarded to the school system required notifying law enforcement. Those figures put the alert in proportion: it reached ten county police officers and five school staff.
In the Headlights design, a standing record would carry those rates without anyone assembling them afterwards. For each alert it would hold the site, the time, what the system flagged, who was notified and in what order, whether a cancellation was issued and when, and what each recipient did after it arrived. Added up by site and by month, the same records give the alert rate, the escalation rate, and the time between an alert and its cancellation.
The office also found it could not verify that training had taken place, because there were no attendance logs. It had the presentation used before the system was switched on, and no way to show who attended.
Where the gap was
The alert was not unreviewed. The office found the monitoring system filtered out 96 per cent of the objects it alerted on, and on 20 October staff reviewed this one and cancelled it within minutes.
The gap was in who had already been told. The notification process sent the alert to school staff and to county police, so assessment and response ran alongside each other. Once law enforcement was involved, the office found, the school system could no longer control external notifications and response methods. It also found the delegation of authority did not address “inadvertent communications of previously canceled threats being forwarded as an active threat”, and that recipients could go round the protocol by phoning or texting police directly.
A MetricRecord is designed to keep alert and escalation rates as standing signed figures, by site and over time, so those patterns do not have to be assembled after an incident. A ConductRecord is designed to keep each alert, the cancellation, who received both and what followed, so the sequence can be reconstructed rather than assembled from memory afterwards.
The office’s recommendations do not include either design. MetricRecord and ConductRecord are Headlights designs.
What governance should have looked like
The office made three recommendations. The first was that the school system review how many people receive alert notifications and use the supplier’s two tier notification, so school staff assess an alert first and police are told only of verified threats, with a default protocol if the primary recipients do not respond promptly. The second was that designated staff move from text messages to the supplier’s app, so everyone sees the notification in the same format. The third was that the school system run twice yearly training for school executive staff and county police on the notification protocols.
The school system said it would adopt the recommendations, said implementation was under way, said all administrators had been retrained in November 2025, and said reminders about the alert process would be added to its annual scenario based safety training. It said county police partners had taken part in the retraining and had reminded school resource officers about after hours procedures.
Where an automated alert can summon armed officers, best practice would be to treat the notification list as part of the safety case rather than as a distribution setting. That means a recorded assessment step between the detection and the police. It means a cancellation that reaches every recipient on the channel the alert used. And it means a standing measure of how often alerts are cancelled, so recipients know what proportion come to nothing.
Failure Pattern: it was reported that a detection system’s alert was sent to armed police and school staff under the same notification process, that staff cancelled it within minutes, and that external recipients continued the notification protocol after the cancellation.
Governance Principle: where an automated alert can summon a response that cannot be undone, an organisation should be able to show who was told, in what order, what each of them did, and whether the cancellation reached them before they acted.
The reference implementation of MetricRecord and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Investigative Synopsis, Case 25-0014-I, Baltimore County Public Schools, amended (Maryland Office of the Inspector General for Education, 25 November 2025)
- Reporting on the state investigation and its findings (The Baltimore Banner, 26 November 2025)
- Reporting that the same company’s system did not detect the gun used in a shooting at a school in Nashville (WSMV, 23 January 2025)
- Reporting carrying the school system’s response to the report (WMAR, November 2025)