What happened
It was reported that the Los Angeles Unified School District, the second-largest school district in the United States, signed a contract worth up to about six million dollars with a company called AllHere to build “Ed”, an AI chatbot meant to help students and families with academic and wellbeing support, and was later reported to have paid roughly three million of that before the project unravelled. In the spring of 2024 the district’s superintendent, Alberto Carvalho, presented Ed at the ASU+GSV Summit, a major education-technology conference, describing it in ambitious terms. Within months the picture changed, because in June 2024 AllHere furloughed most of its staff, citing financial trouble, and the company filed for Chapter 7 bankruptcy.
According to reporting by The 74, a whistleblower who had worked at AllHere alleged that the company had mishandled students’ personal information while building the tool. When the vendor collapsed, questions followed about where the student data it held had gone. The company’s founder, Joanna Smith-Griffin, left the company, and in November 2024 federal prosecutors in New York charged her with securities fraud, wire fraud and aggravated identity theft, alleging she had misled investors about the company’s revenue and customers, including false claims that it held contracts with other school districts; she is entitled to the presumption of innocence unless convicted. The district said student data had been protected, but the episode left a publicly funded AI project defunct within months of its launch.
What an auditable version would have shown
When a public body hands a vendor both a large budget and children’s personal data, two things should be answerable from the start: whether the tool could actually do what was claimed, and where the data went. An auditable version records the evidence behind the capability claims before the money is committed, and keeps a standing record of what student data was collected, where it was stored, who could reach it, and what happens to it if the vendor fails. With those records, the collapse of a supplier becomes a controlled handover rather than a scramble to work out what data existed and who now holds it.
Where the gap was
A strong claim about an AI product, and a large amount of sensitive student data, appear to have been largely taken on trust, and when the vendor ran into trouble there was little clear record standing behind either. A VerificationGate holds a capability claim to evidence before a contract is signed, so “transformative AI platform” has to be shown rather than announced. An EgressGate governs student data leaving the district for a third-party service, recording what was sent and under what terms and requiring that it can be retrieved or destroyed on demand, so a supplier’s bankruptcy does not become a data black hole. The failure was not that a school district tried AI, but that neither the claim nor the data had a record standing behind it.
What governance should have looked like
Public money and children’s data raise the bar. Before a district buys an AI tool, it should require evidence that the tool works rather than a demonstration, and it should know exactly what data the vendor will hold and be able to get it back or have it destroyed if the vendor disappears. The lesson of Ed is that an AI procurement is only as safe as the records behind it, and that “the vendor collapsed” should never be the first time anyone asks where the data went.
Failure Pattern: a public body committed significant funds and sensitive student data to an AI product on the strength of unproven claims, with no durable record of the tool’s capability or of where the data went when the vendor collapsed.
Governance Principle: before a public body deploys an AI tool, the capability claim should be backed by evidence and the handling of any personal data should be recorded and recoverable, so a vendor’s failure does not put money or data beyond account.
The reference implementation of VerificationGate and EgressGate is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- L.A. schools probe charges its hyped, now-defunct AI chatbot misused student data (The 74)
- An education chatbot company collapsed. Where did the student data go? (EdSurge)
- Los Angeles Unified’s AI meltdown: 5 ways districts can avoid the same mistakes (Education Week)
- CEO of artificial intelligence startup company charged with defrauding investors (US Attorney, SDNY)