New York City Local Law 144 went into effect on July 5, 2023, making it the first law in the United States to regulate the use of AI in hiring decisions. If you use automated tools to screen, score, or rank job candidates in NYC, this law applies to you.
But reading the actual law and the DCWP's final rules can feel like deciphering a foreign language. This guide breaks down what LL144 actually requires in plain English.
What Counts as an AEDT?
An Automated Employment Decision Tool (AEDT) is any computational process that uses machine learning, statistical modeling, data analytics, or artificial intelligence to substantially assist or replace discretionary decision-making in hiring or promotion.
This includes:
- Resume screening software that filters or ranks candidates
- AI scoring engines that assign numerical scores to applicants
- Chatbot assessments that evaluate candidate responses
- Video interview analytics that analyze facial expressions, tone, or word choice
- Any tool that produces a score, classification, or recommendation used to make hiring decisions
The key phrase is "substantially assists or replaces." If humans always make the final decision without relying on the tool's output, it might not qualify. But if the tool's score or recommendation influences who moves forward, it's likely covered.
Who Needs an Audit?
Any employer or employment agency that uses an AEDT to evaluate candidates or employees for positions located in New York City must comply. This applies even if the employer is headquartered elsewhere—if the job is in NYC, the law applies.
Note that LL144 regulates the use of AEDTs, not their development. The obligation falls on the employer or employment agency deploying the tool, not the vendor who built it. (Though smart vendors are getting their tools audited proactively to help their customers comply.)
What the Audit Must Include
The bias audit must be conducted by an independent auditor and must calculate:
- Selection rates for each demographic group (the percentage of applicants in each group who are selected)
- Impact ratios comparing each group's selection rate to the most-selected group
- Analysis by sex (male/female)
- Analysis by race/ethnicity (using EEO-1 categories)
- Intersectional analysis (sex × race/ethnicity combinations)
The audit must use data from the employer's own use of the tool. If the employer hasn't used the tool yet, they can use historical data from the vendor—but only if no employer-specific data is available.
The Four-Fifths Rule
The audit results are evaluated against the "four-fifths rule" (also called the 80% rule), established in the EEOC's Uniform Guidelines on Employee Selection Procedures.
Under this rule, if any group's selection rate is less than 80% of the highest group's selection rate, there's evidence of potential adverse impact. For example, if White candidates have a 40% selection rate and Black candidates have a 28% selection rate, the impact ratio is 70% (28/40)—below the 80% threshold.
LL144 doesn't automatically prohibit tools that fail the four-fifths rule, but it does require transparency. The results must be published, and employers should be prepared to justify continued use of tools showing disparate impact.
Public Posting Requirements
Employers must publicly post:
- The date of the most recent bias audit
- A summary of the audit results (selection rates and impact ratios)
- The distribution date of the AEDT (when it was deployed)
This information must be posted on the employer's website or made available upon request. The posting must remain active for at least six months.
Candidate Notice Requirements
Employers must notify candidates at least 10 business days before using an AEDT. The notice must include:
- That an AEDT will be used in the assessment
- What job qualifications or characteristics the tool evaluates
- Information about the data sources used
- Instructions for requesting an alternative selection process or accommodation
Penalties for Non-Compliance
Violations carry civil penalties:
- $500 for the first violation
- $500–$1,000 for the second violation within 24 months
- $1,000–$1,500 for each subsequent violation
Critically, each day an AEDT is used in violation constitutes a separate violation. If you use a non-compliant tool for a month, you could face 30 separate violations.
Getting Started
If you use any AI tools in your hiring process, start by inventorying what tools you have and determining if they qualify as AEDTs. Then ensure you have a recent (within 12 months) independent bias audit for each tool before using it.
Paritas can help. Our audits go beyond LL144's minimum requirements, providing not just the required statistics but also remediation recommendations and multi-jurisdiction compliance mapping.