What happened
It was reported that in April 2023 the Amsterdam Court of Appeal ruled that Uber’s decisions to deactivate drivers on its platform over alleged fraud were, for the purposes of European data protection law, solely automated, because the human involvement in them was, in the court’s words, not much more than a purely symbolic act. The case was brought by the App Drivers and Couriers Union and Worker Info Exchange on behalf of drivers in Britain and a driver in Portugal, who said they had been accused of fraudulent activity, dismissed without a hearing, and told the decision had been taken remotely at an Uber office in Krakow. In a linked judgment the same day, the court considered a similar data-access claim against the ride-hailing company Ola Cabs, whose system the applicants said produced fraud probability scores.
The court found that ending a driver’s access in this way, along with functions such as assigning trips, allocating fares and rating drivers, amounted to automated decision-making that evaluated drivers’ behaviour and reliability, and that it affected them to a considerable extent because it could remove their income. Under Article 22 of the General Data Protection Regulation, which restricts certain decisions made solely by automated means, the drivers were entitled to meaningful information about the logic of those decisions so they could challenge them. The court held there was insufficient evidence of actual human intervention, and rejected the companies’ argument that disclosing the logic would expose trade secrets, finding that objection disproportionate in light of the effect of unexplained automated dismissal and discipline on workers. Uber said the flagged accounts had been reviewed by its trust and safety teams, which it described as standard practice when its systems detect potentially fraudulent behaviour. Separately from the April ruling, TechCrunch reported that in October 2023 the Amsterdam District Court found Uber had still not complied and confirmed penalty payments of 4,000 euros a day, which by then had reached about 584,000 euros.
What an auditable version would have shown
Whether a real person decided to end a driver’s access is a question that should be answerable from the record, and in this case the court found insufficient evidence of actual human intervention. An auditable version binds each deactivation to the decision behind it: the automated signal or fraud score that raised the case, the identity and role of the person who reviewed it, the material they considered, and the reason they gave. With that record, a driver could see why access had been removed, and it would be easier for a court to assess whether a human had been meaningfully involved. What the court found missing, evidence of actual human intervention, is the kind of information such a record is meant to capture at the moment the decision is made.
Where the gap was
A driver’s income could be stopped, and the court found insufficient evidence that a human had actually made that decision. A VerificationGate is built to insist on a genuine human decision, and to record it, before an action this serious goes ahead, so that a rubber stamp cannot be logged as a real review. A ConductRecord keeps the whole story of a deactivation: the signal that flagged it, any score attached, who reviewed it, and the reason they gave. With both in place, the answer to “was a person really behind this, and why” comes from the file rather than from years in court.
What governance should have looked like
When a system can take away someone’s livelihood, the human check that is supposed to stand in the way has to be real, and it has to leave a mark. A review that keeps no record of who did it, what they looked at, or why they agreed is, in practice, hard to tell apart from no review at all. The lesson of these rulings, as reported, is that “a human was involved” is something a system should be able to prove from its own records, and that a person whose living is on the line should be able to see why the decision went the way it did, not be told it is a trade secret.
Failure Pattern: automation made a consequential decision while the human review meant to authorise it was little more than symbolic, and the record did not show how the decision was reached.
Governance Principle: when a human must authorise a consequential automated decision, that review has to be a genuine, recorded act that can be shown to have happened, and its basis must be explainable to the person affected.
The reference implementation of VerificationGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Amsterdam Court of Appeal, ECLI:NL:GHAMS:2023:793 (4 April 2023)
- Historic digital rights win for WIE and the ADCU over Uber and Ola at Amsterdam Court of Appeal (Worker Info Exchange)
- Uber ordered to pay drivers over algorithmic transparency failures (TechCrunch)