What happened
It was reported that in 2020, a woman in her mid-fifties applied for an online tutoring position with iTutorGroup, a company recruiting American tutors to teach students overseas.
She entered her date of birth and was rejected almost immediately.
Curious about the decision, she applied again using the same qualifications but a different date of birth. This time, she was invited to an interview.
The difference was her age.
The EEOC alleged that iTutorGroup’s recruitment software had been programmed to automatically reject female applicants aged 55 and over and male applicants aged 60 and over.
No interview was needed. No recruiter had to make a judgement. The applicant’s date of birth was enough to trigger a rejection.
According to the EEOC, more than 200 qualified applicants were rejected because of their age.
The EEOC filed a lawsuit in May 2022. In September 2023, iTutorGroup agreed to pay 365,000 USD to settle the case. The company was also required to introduce anti-discrimination measures, provide training, invite affected applicants to reapply and comply with a five-year consent decree.
It became the EEOC’s first settlement involving discrimination through an automated hiring system.
What an auditable version would have shown
The software was not making a complicated prediction about someone’s ability to teach. It was applying an age-based rule.
An auditable system would have recorded which screening rules were used, who approved them and how they affected applicants.
A ConductRecord could have preserved the reason for each rejection. A MetricRecord could have compared hiring outcomes across age groups.
If every woman aged 55 or older was being rejected, that pattern should have been visible long before an applicant discovered it herself.
The EEOC’s case set out the age-based screening rule. It does not establish every detail of iTutorGroup’s internal logging or review processes.
Where the gap was
The software did what it was configured to do. The problem was the decision to use age as a reason to reject applicants.
Headlights identifies algorithmic discrimination and a rules-behaviour gap. Employment discrimination laws applied to the company, yet its automated recruitment process excluded applicants because of their age.
There is a human behaviour behind this worth examining.
Automation can make a decision appear neutral because a computer delivers it. Applicants receive a rejection without seeing the rule behind it, while employers can process thousands of applications without reviewing each decision individually.
But an automated decision is still a decision made on someone’s behalf.
In this case, the discrimination became visible because one applicant questioned the result and tried again with a different birthdate.
The behavioural interpretation is Headlights’ analysis. The discriminatory age filters were identified in the EEOC’s enforcement action.
What governance should have looked like
Before introducing an automated screening rule, iTutorGroup should have checked whether it complied with employment discrimination law.
Age-based rejection rules should have been identified and challenged before deployment.
Regular reviews of recruitment outcomes could also have revealed that older applicants were being excluded.
A MetricRecord could track rejection rates across age groups, while a ConductRecord could show which rules were responsible for individual decisions.
Any unusual pattern should trigger a human review, particularly where an entire age group is being rejected.
The lesson is straightforward. Removing a person from the hiring process does not remove the employer’s responsibility for the decision.
Failure Pattern: iTutorGroup’s recruitment software automatically rejected female applicants aged 55 and over and male applicants aged 60 and over. More than 200 qualified applicants were affected according to the EEOC, and the company settled for 365,000 USD.
Governance Principle: automated recruitment decisions must be checked for unlawful discrimination before deployment and monitored for discriminatory outcomes afterwards. Employers remain accountable for decisions made by their software.
The reference implementation of MetricRecord and ConductRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.