195 incidents on record · 2026 Headlights Incident reports by Ellie Harris · Melbourne
11 new this week Library last updated 12 September 2026
← The incident library
HD-INC-185
Criminal justice · Argentina · 2025 · A decision reserved to a judge was drafted with generative AI, and the appellate panel could not tell which reasoning was the judge's

It was reported that an Argentine court annulled a criminal sentence after a chatbot instruction was found in the judgment. Months later, that decision was overturned.

By Ellie Harris · Filed Sentence delivered 4 June 2025; annulled by the Camara Penal de Esquel on 15 October 2025

Alleged: Judiciary of Chubut Province developed or deployed the AI system implicated in this incident. Details are drawn from public reports; parties are presumed innocent of any wrongdoing not established by an official finding.

It was reported that an Argentine court annulled a criminal sentence after a chatbot instruction was found in the judgment. Months later, that decision was overturned.

What happened

It was reported that an Argentine appeals court annulled a criminal conviction in October 2025 after finding what appeared to be an instruction to a generative AI assistant left inside the written judgment.

The original judgment had been handed down in June and sentenced the defendant to two years and six months in prison. But the final document contained a line saying, in Spanish, that a section had been re-edited, had its citations removed and was ready to copy and paste.

That raised an obvious question: how much of the judgment had actually been written by the judge?

It was reported that the appeals court could not answer that from the judgment itself. It said there was too much uncertainty about what had been produced by generative AI and what had come from the judge. The court annulled the conviction and the trial, ordered a new trial before a different judge and referred the matter to the province’s Superior Tribunal.

But that decision was later overturned. It was reported in March 2026 that the Superior Tribunal found that the use of AI was not, by itself, enough to invalidate the judgment. Its view was that the focus should be on the reasoning in the judgment and whether there had been actual harm or an affected constitutional right, rather than simply on whether technology had been used to help write it.

What an auditable version would have shown

The AI use came to light because an instruction that appeared to be intended for the chatbot was left in the final judgment. Once it was found, there was no clear way to tell from the document how the tool had been used or how much of its output remained in the final version.

A record of the drafting process could have helped answer that. It could have shown what was sent to the AI assistant, what came back, what the judge changed and what the judge ultimately reviewed and approved.

The appeals court also reportedly raised concerns about confidentiality. It questioned whether information about the parties, witnesses and experts may have been entered into the AI assistant in breach of the court’s safeguards. The available reporting describes this as a concern raised by the court, rather than a finding that a confidentiality breach occurred.

Where the gap was

It was reported that the Superior Tribunal did not agree that the uncertainty around the use of AI was enough to throw out the judgment.

The higher court looked at the finished judgment instead. Its view was that the question was whether the judge’s reasoning could be reviewed and whether the defendant had actually been harmed. Simply finding evidence that AI had been used during drafting was not enough.

That left a fairly basic question unanswered: what had the AI actually done?

An AuthorityGate would record the point at which the person with the legal authority reviews and approves the decision. A ConductRecord would keep a record of what was produced with AI assistance and what the authorised person ultimately approved.

These are Headlights designs. Neither decides whether a judge should or should not use AI.

What governance should have looked like

It was reported that the appeals court was concerned about how much of the judgment came from AI and whether confidential information had been put into the tool.

Neither question should have to be worked out from a stray chatbot instruction accidentally left in a judgment.

If AI is being used to help draft a judicial decision, there should be a record of how it was used. That could show what was given to the tool, what it produced, what the judge changed and what the judge ultimately approved. There should also be a way to check whether confidential information was entered into the system.

That record would not have decided the legal argument between the two courts. But it would have answered a much simpler factual question before that argument even began: what role did the AI actually play in producing the judgment?

Failure Pattern: It was reported that a chatbot instruction was left inside a criminal judgment, and there was no clear way to determine from the final document how much of the judgment had been produced with AI assistance.

Governance Principle: If AI is used to help prepare a decision that only a particular person has the authority to make, there should be a record of what the AI did and what that person ultimately reviewed and approved.

The reference implementation of AuthorityGate and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.

Sources

The mailing list

Fresh incident reports every week. One email to match.

We add new incidents to the library regularly, and send a single short email each week with what's new. The library stays free and open; this is just how you keep up with it.

No tracking. Unsubscribe in one click.

The record

An auditable system would have produced a signed, tamper-evident record the moment this happened: what the system did, the version that did it, the basis it acted on, and the action taken, and Judiciary of Chubut Province could have produced it on demand.

This is the record the system as deployed did not produce in a signed, auditable form.

What this teaches
Capture what happened when it happens
What the system did, the version that did it, the basis it acted on, and the action taken, recorded at the moment, not reconstructed after.
Sign it, so no one has to trust the record-keeper
A tamper-evident entry. Edit it later and the signature breaks. The record does not ask for the benefit of the doubt.
Make it verifiable by anyone
A court, a regulator, a customer's lawyer can check the record themselves, without taking the company, or us, at our word.

Headlights summarises publicly reported AI incidents. All summaries are independently written, attributed to their original sources, and intended for research and educational purposes. Allegations are identified as such until established through official findings.

Last reviewed September 2026. This report is based on the sources listed above and reflects information available at the time of review; later developments may not be captured. Where a person is described as charged with or alleged to have done something, that allegation is unproven unless a conviction or a court or regulatory finding is stated. Headlights publishes journalism and commentary, not legal advice.

Want to write back?

Direct to my inbox.

ellie@useheadlights.com →