What happened
It was reported in the decision of Brazil’s Federal Court of Accounts (Tribunal de Contas da União, TCU) that the INSS handles claims for pensions and other social security benefits in two ways. Staff assess some claims, while computer systems process others automatically, without human involvement. According to the decision, 2.2 million claims were decided automatically in 2023, almost 70% more than the previous year.
The TCU examined a random sample of 192 claims refused automatically between January and May 2024, covering retirement, maternity pay and survivors’ benefits. In 21 cases, or 10.94%, the TCU disagreed with the refusal. It found that these refusals breached the legal requirement to consider documents submitted by claimants (Law 8.213/1991, article 29-A, paragraph 2, and Decree 3.048/1999, article 19, paragraph 1).
The TCU found inconsistencies in another 55 applications, or 28.64% of the sample. Although it upheld those refusals, it warned the INSS that the problems could lead to wrongful decisions and needed attention.
The INSS had not set a maximum acceptable error rate. The TCU therefore used 10%, the upper limit in international and Brazilian sampling standards, and found that the automatic error rate exceeded it. The TCU also found that 13.20% of the manual refusals reviewed by the INSS’s quality programme in 2023 were wrongful, again exceeding the limit.
What an auditable version would have shown
According to the decision, automatic refusals were being caused in part by mistakes claimants made when completing online applications. The system lacked checks that could catch those mistakes before processing the claims.
The TCU gave examples. Some claimants answered “no” when asked whether they had periods of special work, even though they had uploaded evidence of that work in the designated field. Others answered “no” to a question about military or public service while attaching a certificate proving it. The TCU said the question was difficult to understand.
Maternity pay applications showed a similar problem. In 17 of the 64 cases reviewed, claimants gave an incorrect answer about stopping work after childbirth. The TCU said at least some of these answers resulted from misunderstanding the question.
The decision records that the automation systems could not read the contents of attached documents. If a claimant failed to correct their work history on the confirmation screen, the system did not count those periods. That could result in an automatic refusal, even when the supporting documents had been attached.
The important record is one that shows which documents a claimant submitted, which documents the system actually considered and which rule led to the refusal. On the TCU’s findings, evidence could be sitting in an application without ever being considered in the automatic decision.
Where the gap was
It was reported in the decision that quality control for automatic analysis was not yet fully established. The INSS told the TCU that it had no policies or manuals for checking the quality of automatic decisions.
Errors were reported through an online feedback form, with reports emailed to one staff member who checked them daily. Problems also came through Teams, WhatsApp and email. The INSS said its process for handling automation errors was operating, but had not yet been documented.
Its existing quality programme covered manual decisions, not automatic ones. The INSS said a pilot to include automatic decisions was still being developed.
The TCU also recorded concerns from people familiar with the decisions. All the public defenders interviewed said automatic decisions were more open to challenge, as did 70% of the appeals council members interviewed. Of 707 INSS staff who answered an open question, 100 described problems with automatic analysis, including attached documents being ignored.
The TCU noted that 27.12% of applications containing filling errors had been submitted by claimants’ lawyers. It raised the possibility that at least some of those errors were deliberate. The TCU said this was another reason to invest in detecting errors early rather than relying only on clearer communication.
The TCU also found problems with incentives for staff making manual decisions. Productivity targets gave little credit for asking claimants to provide missing documents. In a staff survey, 70% disagreed that the INSS’s culture valued quality over meeting targets.
A ConductRecord is designed to show what information a system relied on, which rule it applied and the outcome of each decision. This would allow a sample of decisions to be checked against the law.
ConductRecord is a Headlights design.
What governance should have looked like
According to the decision, the TCU formally notified the INSS of its findings on automatic refusals and ordered changes to how claims were processed and checked.
The INSS was told to introduce checks that could identify and correct errors in applications before they affected processing. These included automatic consistency checks, warnings about possible mistakes and the use of artificial intelligence to correct errors that could be resolved.
The TCU also ordered the INSS to establish a quality review process for automatic decisions, similar to the one already used for manual decisions. It was told to introduce quality indicators and targets, make application forms and guidance clearer, and align staff productivity targets with the complexity of the work.
Where a public body uses automatic analysis to refuse benefits, best practice would be to ensure every document a claimant submits is either considered by the system or referred to a person. Automatic refusals should be checked regularly against the law, and claimants should receive a clear explanation of why their application was refused.
Failure Pattern: an automated benefits system refused claims without taking account of documents claimants had attached, while the agency had no formal process for checking the quality of automatic decisions.
Governance Principle: an agency that refuses benefits automatically should be able to show what evidence each claimant provided, what the system considered and which rule produced the decision.
The reference implementation of ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Acórdão 634/2025, Plenário, TC 008.309/2024-8 (Tribunal de Contas da União, session of 26 March 2025)
- TCU analisa indeferimentos indevidos no INSS (Tribunal de Contas da União, 26 March 2025)