180 incidents on record · 2026 Headlights Incident reports by Ellie Harris · Melbourne
10 new this week Library last updated 30 August 2026
← The incident library
HD-INC-151
Courts and justice · United States · 2025 · Hallucination & fabrication

Two US federal judges withdrew orders that contained misstated case outcomes and a citation to a case that did not exist, and both told a Senate committee that staff had used AI to draft them

By Ellie Harris · Filed Orders docketed 30 June and 20 July 2025

Alleged: United States District Court for the District of New Jersey; United States District Court for the Southern District of Mississippi developed or deployed the AI system implicated in this incident. Details are drawn from public reports; parties are presumed innocent of any wrongdoing not established by an official finding.

Two US federal judges withdrew orders that contained misstated case outcomes and a citation to a case that did not exist, and both told a Senate committee that staff had used AI to draft them

What happened

It was reported that on 30 June 2025 Judge Julien Xavier Neals of the United States District Court for the District of New Jersey signed and docketed an opinion on a motion to dismiss in In re CorMedix Inc. Securities Litigation, No. 2:21-cv-14020. On 22 July the defendants checked the authorities and wrote to the court. They identified three cases whose outcomes had been stated backwards, with motions to dismiss described as denied when they had been granted, and passages attributed to decisions that did not contain them. The court withdrew the opinion after the defendants raised the errors.

It was reported that around the same period Judge Henry Wingate of the United States District Court for the Southern District of Mississippi withdrew a temporary restraining order docketed on 20 July 2025 in Jackson Federation of Teachers v. Lynn Fitch, No. 3:25-cv-00417. Reporting identified parties who were not involved in the case, misstatements of state law and a reference to a case that did not exist. In October both judges answered questions from the Senate Committee on the Judiciary. Neals said a law school intern had used ChatGPT for legal research without authorisation or disclosure, contrary to a chambers policy that had previously been communicated only verbally. He has since put that policy in writing for all law clerks and interns. Wingate said a law clerk had used Perplexity to assist with an early draft based on public docket information. The draft reached the docket before the standard review process. His chambers now requires a second law clerk to review each draft independently, with cited cases printed and attached to final drafts.

What an auditable version would have shown

In the New Jersey case the defendants found the problems by reading the opinion and checking its authorities. The public record therefore shows an external check working only after the document had been signed and docketed. An auditable version would preserve, during drafting and review, how AI was used, which authorities were checked independently, what source was used for verification and who completed the final review. That record would neither establish that an AI-assisted document was correct nor replace judicial review. It would show whether the required checks happened before the document acquired the authority of the court. The Senate correspondence eventually supplied that explanation, months after the orders were issued.

Where the gap was

It was reported that the errors in both orders could be checked against primary legal sources. A case exists or it does not, a motion was granted or denied, and attributed words appear in the judgment or they do not. In New Jersey lawyers for one of the parties performed those checks and identified the discrepancies. In Mississippi Wingate said an early draft bypassed the standard review process and reached the docket. A CitationVerifier compares each authority and attributed passage against a recognised legal database before release, while a ConductRecord preserves where AI assistance was used and who reviewed the result. Neither control replaces the person responsible for the document. They make verification and accountability visible within the process instead of leaving another party to establish them after the court has acted.

What governance should have looked like

It was reported that the Administrative Office of the United States Courts established an advisory Artificial Intelligence Task Force in early 2025 and distributed interim guidance to federal courts on 31 July 2025. The guidance warned against delegating core judicial functions to AI, recommended independent verification of AI-generated material and placed responsibility for AI-assisted work on the user. Both orders were already on the docket when the guidance arrived, the first by about a month and the second by eleven days. Neals subsequently converted a verbal restriction into a written policy. Wingate added a second reviewer to the drafting process. A document carrying judicial authority requires both, a rule understood by the people doing the work and a workflow step capable of catching errors before issue.

Failure Pattern: machine-drafted text entered a signed judicial order without anyone verifying its citations, and the error was found by a party reading the document rather than by any check inside the court.

Governance Principle: before a document that binds people is issued, every citation in it should be checked against the real source, and there should be a record of what was drafted with what assistance and who reviewed it.

The reference implementation of CitationVerifier and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.

Sources

The mailing list

Fresh incident reports every week. One email to match.

We add new incidents to the library regularly, and send a single short email each week with what's new. The library stays free and open; this is just how you keep up with it.

No tracking. Unsubscribe in one click.

The record

An auditable system would have produced a signed, tamper-evident record the moment this happened: what the system did, the version that did it, the basis it acted on, and the action taken, and United States District Court for the District of New Jersey; United States District Court for the Southern District of Mississippi could have produced it on demand.

This is the record the system as deployed did not produce in a signed, auditable form.

What this teaches
Capture what happened when it happens
What the system did, the version that did it, the basis it acted on, and the action taken, recorded at the moment, not reconstructed after.
Sign it, so no one has to trust the record-keeper
A tamper-evident entry. Edit it later and the signature breaks. The record does not ask for the benefit of the doubt.
Make it verifiable by anyone
A court, a regulator, a customer's lawyer can check the record themselves, without taking the company, or us, at our word.

Headlights summarises publicly reported AI incidents. All summaries are independently written, attributed to their original sources, and intended for research and educational purposes. Allegations are identified as such until established through official findings.

This report is based on the sources listed above and reflects information available at the time of review; later developments may not be captured. Where a person is described as charged with or alleged to have done something, that allegation is unproven unless a conviction or a court or regulatory finding is stated. Headlights publishes journalism and commentary, not legal advice.

Want to write back?

Direct to my inbox.

ellie@useheadlights.com →