What happened
It was reported that on 27 or 28 August 2019, a passenger was travelling overnight from El Salvador to New York on an Avianca flight when a metal serving cart struck his knee. In 2022, he sued the airline in the United States District Court for the Southern District of New York, but Avianca argued that he had brought the claim too late under the Montreal Convention, an international treaty governing airline liability.
To challenge the airline’s argument, his lawyer, an experienced personal injury attorney with more than thirty years of practice, submitted a filing in March 2023 citing six federal court decisions that appeared to support his position. The cases included Varghese v. China Southern Airlines, Martinez v. Delta Airlines, Shaboon v. EgyptAir, Petersen v. Iran Air, Estate of Durden v. KLM Royal Dutch Airlines and Miller v. United Airlines.
When Avianca’s lawyers tried to locate the decisions, they could not find them. Court staff encountered the same problem, and it eventually emerged that the passenger’s lawyer had used ChatGPT to conduct his research. Rather than checking the citations against a legal database, he had asked ChatGPT whether the cases were genuine, and the system assured him that they were. None of the six cases existed.
On 22 June 2023, the judge imposed sanctions on the lawyer, another lawyer who had signed the filing, and their firm, Levidow, Levidow & Oberman. The second lawyer had not conducted the research and later said he had “no reason to doubt the sincerity” of his colleague’s work, but he had signed the document submitted to the court. The total financial sanction was USD 5,000, and the lawyers were also ordered to send copies of the sanctions decision and the offending filing to their client and to the federal judges whose names had been falsely associated with the fabricated cases.
The lawyer explained that he had not understood ChatGPT could invent legal authorities, believing instead that it operated like a search engine connected to a database of genuine court decisions. The judge rejected that explanation as a defence and found subjective bad faith based on conscious avoidance, concluding that the lawyers had failed to meet their professional obligation to verify the authorities they submitted to court.
What an auditable version would have shown
According to the sanctions decision, the lawyer used ChatGPT to find cases supporting his argument and then returned to the same system when questions arose about whether those cases existed. The problem was that ChatGPT could generate plausible legal citations without retrieving genuine court decisions, and asking it to confirm its own answers provided no independent verification.
The lawyer had mistaken information generated by a language model for information retrieved from a legal source. ChatGPT could produce convincing case names, legal reasoning and quotations, but the appearance of a genuine court decision did not mean that any court had issued it.
An auditable record would have made the distinction between generated and verified information visible throughout the research process. Each citation could have been checked against an independent legal database such as Westlaw, LexisNexis or PACER, with the result recorded alongside the material the lawyer intended to use. A genuine decision would have a verifiable source, while a fabricated citation would fail that check.
The record would also have shown that the lawyer had asked ChatGPT itself to verify the citations rather than consulting an independent source. Had the six cases been checked against an authoritative database before the filing was submitted, their absence could have been identified before they reached the court.
The issue was not simply that ChatGPT produced false information. It was that the information moved from a chatbot into a federal court filing without the verification that legal research requires.
Where the gap was
The court’s decision showed how an established professional responsibility could be overlooked when a new technology was introduced into a familiar task. Lawyers had long used electronic legal databases to locate decisions, read judgments and verify citations, but the passenger’s lawyer treated ChatGPT as though it performed the same function.
Unlike a conventional legal research database, a general-purpose language model can generate text that resembles a legal decision without retrieving an actual case. The lawyer did not recognise that distinction, and when Avianca’s lawyers questioned the citations, he returned to the same system for reassurance rather than checking the authorities independently.
Headlights identifies automation bias in the lawyer’s decision to trust ChatGPT even after questions had been raised about the citations. Rather than checking the cases against an independent legal database, he returned to the same system for reassurance and accepted its answers.
The incident also reveals a rules-behaviour gap. Lawyers were already required to verify the authorities they submitted to court, but that responsibility was overlooked when ChatGPT became part of the research process. The professional obligation had not changed, yet the lawyer’s behaviour had.
There was also diffusion of responsibility. The lawyer explained that he had misunderstood ChatGPT’s capabilities, while the judge made clear that responsibility for the filing remained with the lawyers who submitted it.
These are Headlights’ interpretations of the behaviour described in the court record, not findings made by the judge.
Other courts have since encountered legal submissions containing fabricated AI-generated citations, showing that the underlying problem extends beyond this case. The risk arises whenever a professional treats a convincing AI-generated answer as verified information without checking the source.
What governance should have looked like
The sanctions decision made clear that lawyers remain responsible for the accuracy of the authorities they submit to court, regardless of whether artificial intelligence helped prepare the material. A workable governance process would allow AI to assist with identifying potentially relevant cases, while requiring every citation to be verified against an independent, authoritative legal database before it could appear in a final filing.
The important distinction is between suggesting a citation and confirming that it exists. A language model may help a lawyer identify possible authorities, but verification needs to take place outside the model, using a source capable of establishing whether the decision is genuine.
A CitationVerifier could perform that independent check, while a VerificationGate could prevent unverified citations from moving into a final document. A ConductRecord could preserve the citations generated by the system, the sources consulted during verification and the outcome of each check, creating a record of what happened rather than relying on the lawyer’s recollection.
One useful field would be model_consulted_for_verification, recording whether the lawyer had asked the language model itself to confirm a citation. In this case, the field would have shown that ChatGPT was used both to produce the authorities and to provide reassurance that they were genuine, without an independent database check.
These controls would sit alongside ordinary professional review, including mandatory citation verification, clear identification of AI-generated material, compliance with applicable court disclosure requirements and a separate review of legal authorities before filing. The purpose would not be to prevent lawyers from using AI, but to ensure that introducing a new tool did not remove the checks already required of them.
Failure Pattern: a lawyer relied on fabricated legal citations generated by ChatGPT, then asked the same system to confirm they were genuine. The citations were submitted to a federal court without independent verification.
Governance Principle: AI-generated legal authorities must be verified against independent, authoritative sources before they are relied on in court. Professional responsibility remains with the lawyers submitting the material.
The reference implementation of CitationVerifier, VerificationGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Mata v. Avianca, Inc. Sanctions Opinion (Justia)
- Mata v. Avianca, Inc. (Wikipedia)
- Practical Lessons from the Attorney AI Missteps in Mata v. Avianca (ACC)
- Fake Cases, Real Consequences: Misuse of ChatGPT (Goldberg Segalla)