What happened
On the night of 18 March 2018, a modified Volvo XC90 operated by Uber’s Advanced Technologies Group was driving itself north along Mill Avenue in Tempe, Arizona. Elaine Herzberg, aged 49, was walking a bicycle across the road, away from a crosswalk. A safety operator, Rafaela Vasquez, sat in the driver’s seat to take over if the automated system failed. The car was travelling at about 39 miles per hour, roughly 63 kilometres per hour, when it struck and killed Herzberg. It was the first recorded death of a pedestrian caused by an autonomous test vehicle.
The car’s sensors found Herzberg about 5.6 seconds before impact. What the software could not do was understand what it was seeing. It classified her as a vehicle, then an unknown object, then a bicycle, revising where it expected her to go each time, and it never classified her as a pedestrian or correctly anticipated her path. At about 1.2 seconds before impact the system concluded that emergency braking was needed to avoid a collision.
It did not brake. Uber had disabled the automated emergency braking that ran under computer control, on the reasoning that sudden braking would make the vehicle behave erratically, and relied instead on the human operator to intervene. The system was not designed to alert that operator when it identified a hazard. It also held back any planned braking for one second while it handed control to the driver. The Volvo’s own factory collision-avoidance braking, which might have caught the error, was switched off whenever the car was in autonomous mode. The operator, who investigators found had been looking down at a personal phone, looked up about a second before impact, too late to do anything.
The National Transportation Safety Board adopted its final report on 19 November 2019. It gave the probable cause as the operator’s failure to monitor the road because she was visually distracted by her phone. It then set out what lay behind that: Uber’s inadequate safety risk assessment, its ineffective oversight of vehicle operators, and its lack of any mechanism for managing the automation complacency it should have expected, all of which the Board attributed to an inadequate safety culture. Herzberg’s crossing away from a crosswalk and the state’s thin oversight of testing were also cited.
Vasquez was charged with negligent homicide in 2020 and pleaded guilty to endangerment in 2023, receiving three years of supervised probation. Uber was not criminally charged, settled with Herzberg’s family on undisclosed terms, and suspended its testing.
What an auditable version would have shown
The car generated a precise record of what it perceived and decided: the moment it detected Herzberg, each time it reclassified her, the instant it determined that braking was needed, and the design choices that stopped the braking from happening. The failure was not a gap in the data. It was that the chain from “the system knows a collision is imminent” to “the system does nothing and says nothing” had been built deliberately and was surfaced to no one who could act. An auditable version records each of these as signed events, the detection, the shifting classifications, the braking decision, and the suppression of that braking, so that a vehicle which has decided it needs to stop and has been configured not to is a visible, reviewable state rather than a fact recovered from a crash investigation.
Where the gap was
Two safety controls had been removed and nothing recorded their removal as the risk it was. The first was emergency braking, switched off under automation. The second was any warning to the operator, never built at all. A ConstraintGate is the control on that: removing a safety constraint as consequential as emergency braking is not a quiet configuration default but an action that requires explicit, recorded authorisation, so that someone has to own the decision to rely entirely on a distracted human. A ConductRecord preserves what the system saw and decided, including the braking call it was not allowed to act on, which is the difference between an investigation reconstructing the last seconds and an operator being able to see, in advance, that the car would not stop itself. An autonomous machine that has correctly identified an emergency and been designed not to respond to it is the clearest possible case for binding a high-consequence action to a recorded, enforced rule.
What governance should have looked like
Testing an automated driving system on a public road puts its failures in front of people who never agreed to be part of the experiment. The lesson of this crash is that the dangerous decision was made long before the night itself, in an office, when emergency braking was disabled and no alert was built, and nothing forced those choices to be justified or recorded. The controls are ordinary: a safety-critical function is not removed without explicit, recorded sign-off, the system’s perception and decisions are logged as signed events, and a human expected to be the last line of defence is actually told when the machine needs them. Uber’s car did almost everything except the one thing that mattered, and it had been built that way on purpose.
The reference implementation of ConstraintGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona (NTSB investigation page, HWY18MH010)
- Highway Accident Report NTSB/HAR-19/03 (full final report, PDF)
- Backup driver in 2018 Uber self-driving death pleads guilty (NPR)