What happened
The New York State Comptroller reported in December 2025 that New York City’s agency responsible for enforcing its automated hiring law was missing potential breaches. Complaints were not reliably reaching the department, and state auditors found problems that its own reviews had missed.
New York City requires employers using certain automated hiring tools to check them for bias, publish the results and tell candidates when the tools will be used. The Department of Consumer and Worker Protection (DCWP) is responsible for enforcing those rules.
The Comptroller reported that when auditors called the city’s complaint line about automated hiring tools, most calls were sent to the wrong department or directed back to the employer. Very few reached DCWP.
The auditors also looked at employers whose websites DCWP had already reviewed. DCWP had identified one potential compliance problem. The state auditors reported finding at least 17 in the same group of employers.
Those findings were potential problems, not confirmed breaches of the law.
What an auditable version would have shown
The Comptroller reported that DCWP relied on complaints and company websites to find employers who might not be following the law. Its staff also lacked the technical expertise to assess automated hiring tools and had not sought help from the city’s technology office.
There was a problem with relying on complaints. If an employer did not tell a candidate that an automated tool had been used, the candidate might never know there was something to complain about.
DCWP needed a record of the complaints it received, where they went, which employers it reviewed and what it checked. That would have shown whether complaints were reaching the right people and whether its reviews covered the requirements of the law.
The auditors reported finding potential problems that DCWP had missed. A record of how each review was carried out would have helped establish why the department had reached different findings.
Where the gap was
The Comptroller reported that complaints about automated hiring tools were not reliably reaching DCWP. It also found that the department had identified far fewer potential compliance problems than the state auditors found when they examined the same employers.
The law requires employers to tell candidates when covered automated hiring tools will be used. But if an employer fails to give that notice, a candidate may have no idea that software played a part in assessing their application.
DCWP was relying on complaints from people who might not even know an automated hiring tool had been used.
A ConductRecord keeps a record of each decision an automated hiring tool contributes to, including the information used and the result.
A MetricRecord is designed to turn those records into signed aggregate figures, such as selection rates across different groups. An employer or auditor could use those figures to check whether a published bias audit reflects the decisions made using the tool.
ConductRecord and MetricRecord are Headlights designs. They could help employers show how their tools were used, but they would not fix the problems the Comptroller found with DCWP’s complaint handling and enforcement.
What governance should have looked like
The Comptroller reported that DCWP needed to improve how it received complaints, check published bias audits against the law’s requirements and look for potential breaches without waiting for someone to complain. The auditors also recommended that the department seek help from the city’s technology office.
DCWP agreed to implement or partially implement most of the recommendations. It rejected the recommendations to investigate why it had received so few complaints and to look for potential breaches through research and other leads. The department said that educating employers and candidates, alongside complaint-based enforcement, was the most efficient use of its resources.
Where an agency enforces rules on automated hiring tools, best practice would be to check that complaints reach the right department and regularly review whether employers are meeting the law’s requirements.
The agency should keep a record of which employers it reviewed, what it checked, what problems it found and what happened next. It should also be able to call on technical expertise when it needs to understand how a hiring tool works.
The Comptroller’s findings and DCWP’s response are public. The sources reviewed do not show whether the department completed the changes it agreed to make or whether a later review found that enforcement had improved.
Failure Pattern: the agency enforcing New York City’s law on automated hiring tools relied on complaints that were not reliably reaching it, and state auditors found at least 17 potential compliance problems in the same 22 companies where the agency had found one.
Governance Principle: a body that enforces rules on automated hiring tools should be able to show what it checked, against which requirement, and what it found, in a record that others can test.
The reference implementation of ConductRecord and MetricRecord is open source. It lives at github.com/saffronandindia/headlights-oss, Apache 2.0 licensed and free to install. The repository is public now.
Sources
- Enforcement of Local Law 144, Automated Employment Decision Tools, audit 2024-N-6 (Office of the New York State Comptroller, 2 December 2025)
- Audit report 2024-N-6, full report (Office of the New York State Comptroller)