What happened
It was reported in July 2025 that the Massachusetts Attorney General had entered into an Assurance of Discontinuance with a student loan refinancing company, accepted in lieu of litigation, under which the company agreed to pay 2.5 million dollars.
The Attorney General alleged that the company’s refinance model included an evaluation of the applicant’s cohort default rate as a weighted input until 13 September 2017. That figure is produced by the United States Department of Education and describes the average rate of loan defaults associated with specific higher education institutions, so it describes the college rather than the person applying. The Attorney General alleged that use of the subscore resulted in disparate impact in approval rates and loan terms, with Black and Hispanic applicants more likely to be penalised than White applicants. The instrument gives no figure for the disparity.
The Attorney General also alleged that until 30 June 2023 an applicant without at least a green card was automatically denied at the prescreen decline stage, before a full application was submitted and without any determination of creditworthiness or ability to repay.
The company denies the allegations. The instrument states that it is not an admission.
What an auditable version would have shown
The Attorney General alleged that until 2018 underwriters could override the model’s recommendation on approvals, denials and pricing, and that there was no written policy setting out how those exceptions would be made and no practice for recording them when they were made.
The obligations the company agreed to set out the records, testing and oversight it must now keep. They cover a model inventory listing the algorithms, the training data, the parameters and the fair lending testing results, and account level data tracking the underwriting inputs, the model prediction, how that prediction was used, the approval or denial, the price offered, and the later default and delinquency history. Documentation of underwriting decisions must be retained for four years, and fair lending testing must run annually on both the models and the automatic denial rules.
Those obligations address the allegations about undocumented exceptions and untested models. They do not establish that every record now required was previously absent.
Where the gap was
The Attorney General alleged that, in developing the refinance model, the company trained it using arbitrary, discretionary human selections of variables, without firm procedures for assessing whether those variables predicted default or ability to repay. It further alleged that the origination model was designed, in part or in whole, to replicate arbitrary and unfair refinance decisions.
A ConstraintGate is designed to check a standing rule before a variable enters a model. A MetricRecord is designed to count approvals, denials and pricing outcomes across groups. A ConductRecord keeps the inputs, the prediction, any departure from it by an underwriter, and the principal reasons given to the applicant in an adverse action notice.
What governance should have looked like
The company denies the allegations. It was reported that, as part of the settlement, it agreed to stop using school ranking and cohort default rates in its lending decisions, end the automatic immigration status rule, strengthen oversight of its algorithms and carry out regular fair lending testing.
The bigger governance issue is what should happen before a model like this is allowed to influence real lending decisions. If a lender wants to use information about where someone studied, there should be a clear reason why that information is relevant to that person’s ability to repay. The model should then be tested to see whether using it produces significantly different outcomes for different groups.
There should also be a record of what happened when the model was actually used. What information went in, what the model recommended, whether an underwriter followed or overrode that recommendation, and why the applicant ultimately received the decision they did. If patterns of unfair outcomes begin to appear, someone should be responsible for finding them and acting on them.
That does not mean an algorithm decides who should receive a loan. It means that when an algorithm helps make decisions that affect people’s lives, the organisation should be able to explain what it did, test what it is doing, and reconstruct what happened when something goes wrong.
Failure Pattern: it was alleged that an underwriting model carried a characteristic of the applicant’s college into decisions about the applicant, and that the model was not tested for whether that produced different outcomes by race.
Governance Principle: where an organisation uses a model to decide who gets credit and on what terms, it should be able to show what went into each decision, what the model predicted and whether the outcomes differ by group.
The reference implementation of ConstraintGate, ConductRecord and MetricRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Assurance of Discontinuance, the signed instrument carrying the allegations, the denial, the payment term and the agreed obligations (Office of the Attorney General of Massachusetts, 2025)
- Account of the settlement, carrying the announcement date (Consumer Finance Insights, 11 July 2025)
- Analysis of the settlement and the agreed governance obligations (Debevoise Data Blog, 20 July 2025)