What happened
It was reported in October 2025 that a sitting US senator had asked Google’s Gemma model whether she had been accused of rape, and that the model returned a detailed account of an incident during a state election campaign, supported by links styled as news reports. Her office published the letter it sent to Google, stating that there had never been such an accusation, that no such person existed, that the campaign year was wrong, and that the links led to error pages and unrelated articles.
Google said Gemma was built for developers and researchers, was not meant for factual questions or for consumers, and removed it from its AI Studio interface while keeping it available through the programming interface. The senator replied that removing it from one interface does nothing to stop developers deploying it elsewhere. Separately, an activist who had found similar output about himself sued Google, and in July 2026 a court refused to dismiss the case while stating it was making no finding about the truth of the allegations.
What an auditable version would have shown
The people the output was about were the ones who discovered the problem. One ran the prompt in his own name and sued; the other’s office ran the prompt and published the result. The sources reviewed did not show how often the model produced claims like this, or what checks Google ran before returning them.
A record for each answer could show the sources the model used, whether those sources actually supported the claim, and whether the citations were checked before the answer appeared. That would give Google its own record of what happened, rather than leaving the person named in the output to discover it by running the prompt themselves.
Where the gap was
The senator’s letter states that the output presented a false factual claim about her and attached links styled as news reports that led to error pages and unrelated articles. A VerificationGate is designed to route a factual claim about a named person to a trusted source for checking rather than relying on model generation alone. A CitationVerifier is designed to test whether a citation supports the claim attached to it before an answer is delivered, and to record whether the check passed. A ConductRecord preserves the prompt, the retrieved documents, the checks and the output, which allows a disputed answer to be traced afterwards rather than reproduced by the person it concerns. None of these decides what a model should be permitted to say, and none resolves the questions before the Delaware court.
What governance should have looked like
Google’s stated position is that Gemma was built for developers and researchers and was not meant for factual assistance or for consumers. The senator argued that removing Gemma from one interface did not stop developers using it elsewhere. The sources reviewed do not settle that disagreement, and the proceedings are still unresolved.
Where a model is answering factual questions about real people, best practice would be to check those claims against a trusted source before showing them. Citations should also be checked to make sure they actually support what the model is saying, with a record kept of what was checked and what the checks found.
Failure Pattern: a model made a false claim about a real person and backed it up with links that looked credible but did not support what it said. The people named in the output found the problem, not the operator.
Governance Principle: when a model makes factual claims about real people, those claims should be checked against a trusted source. If it provides a citation, the organisation should also be able to show that the citation actually supports the claim.
The reference implementation of VerificationGate, CitationVerifier and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Letter to Sundar Pichai, 30 October 2025, carrying the prompt and the output (Office of Senator Marsha Blackburn)
- Blackburn demands answers from Google after Gemma manufactured fake criminal allegations against her (Office of Senator Marsha Blackburn, 31 October 2025)
- Blackburn holds Google’s feet to the fire after the company dodges accountability (Office of Senator Marsha Blackburn, 21 November 2025)
- Transcript of the Senate Commerce hearing of 29 October 2025 (Tech Policy Press)
- Google pulls Gemma from AI Studio, carrying Google’s statement in full (TechCrunch, 2 November 2025)
- Second report carrying the same Google statement (The Register, 3 November 2025)
- Starbuck v Google LLC, complaint filed 22 October 2025, Superior Court of Delaware
- Starbuck v Google LLC, opinion of 24 July 2026 denying the motion to dismiss