What happened
It was reported in Anthropic’s postmortem, published on 17 September 2025, that three separate bugs had affected the quality of Claude’s answers over several weeks. The problems overlapped.
Users began reporting poorer answers in early August. Anthropic said the first complaints were difficult to distinguish from normal variation in user feedback. It opened an investigation in late August, when the reports became more frequent and persistent.
The first problem began on 5 August. Some Claude Sonnet 4 requests were being sent to servers configured for a much longer context window. Anthropic said this initially affected 0.8% of requests. A routine load-balancing change on 29 August increased the number of requests sent to the wrong servers. By 31 August, 16% of Sonnet 4 requests were affected at the worst hour.
Anthropic said about 30% of Claude Code users who made requests during this period had at least one request sent to the wrong type of server. Follow-up requests tended to use the same server because routing was “sticky”. Some users were affected repeatedly.
A second bug appeared on 25 August. Anthropic said a configuration error on servers using Google’s TPU chips sometimes corrupted Claude’s answers. Users could receive Thai or Chinese characters in response to English questions. Code could contain obvious syntax errors. The problem affected Opus 4.1 and Opus 4 from 25 to 28 August. Sonnet 4 was affected from 25 August to 2 September.
The third bug followed a change in late August to how Claude selects each next word. Anthropic said this exposed an existing compiler bug. A workaround introduced in December 2024 had been hiding the problem. The August change removed that workaround. Anthropic confirmed the bug affected Haiku 3.5. It said some Sonnet 4 and Opus 3 requests might also have been affected.
Anthropic said it rolled back the second bug on 2 September. It deployed a fix for the routing problem on 4 September and completed the rollout by 18 September. The third bug was rolled back for Haiku 3.5 on 4 September and Opus 3 on 12 September. Sonnet 4 was also rolled back as a precaution, although Anthropic could not reproduce the problem on that model.
Anthropic said it never reduces model quality because of demand, time of day or server load.
What an auditable version would have shown
Anthropic’s postmortem described several ways the company checked Claude’s performance. These included benchmarks, safety evaluations, performance metrics and spot checks. Changes were also tested with small groups of users before wider release.
Despite these checks, Anthropic said its evaluations failed to capture the problems users were reporting. One reason was that Claude could often recover from individual mistakes, making the problems harder to detect.
Privacy controls created another challenge. Anthropic said engineers could not freely examine conversations with Claude, particularly when users had not submitted them as feedback. Anthropic said these controls protected user privacy but also prevented engineers from examining the interactions needed to reproduce the bugs.
On 29 August, complaints increased sharply. Anthropic said it did not immediately connect them to the load-balancing change made that day.
A useful record would have linked each response to the system that produced it. It would have shown which servers handled the request, which code version was running and what had recently changed. Engineers could then have compared the rise in complaints with changes made to the affected systems.
Where the gap was
It was reported in Anthropic’s postmortem that users began reporting problems in early August. The investigation that uncovered the bugs began in late August.
As Headlights sees it, the gap was between what users were reporting and what Anthropic’s internal checks could detect. During this period, the routing bug went from affecting less than 1% of Sonnet 4 requests to 16% at its worst hour.
Anthropic said the incidents exposed gaps it should have identified earlier. It also said it had relied too heavily on noisy evaluations.
A ConductRecord is designed to record what a system did, which version performed the action and what information it acted on. This would help a reviewer investigate a faulty output. It could also help identify system changes that may have contributed to the problem.
ConductRecord is a Headlights design.
What governance should have looked like
Anthropic said it had developed better evaluations to distinguish working systems from broken ones. It planned to improve them further. The company also said it would run quality checks continuously on production systems. It planned to develop tools to investigate user feedback faster without weakening privacy protections.
Anthropic said it had added tests for unexpected characters to its deployment process. It also switched the compiler operation behind the third bug to a more precise version.
When a model runs on several hardware platforms, best practice would be to check its response quality continuously on each platform. A sustained rise in user complaints should prompt an investigation. Organisations should not have to wait for internal tests to confirm a problem before investigating it.
Failure Pattern: three overlapping infrastructure bugs degraded Claude’s answers for weeks. Internal evaluations missed the problems. User reports became the main warning signal. The investigation began weeks after the first reports.
Governance Principle: an organisation serving an AI model should be able to trace an output to the system configuration that produced it. Persistent user complaints should prompt an investigation, even when internal tests show no problem.
The reference implementation of ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.