The question we get most often from cautious HR leaders sounds something like: "If we deploy a custom pre-hire test, are we more or less exposed legally than if we just keep using the standardized battery we bought from a vendor?"
The answer is consistently "more defensible — if you build it right." But "if you build it right" is doing all the work in that sentence. To understand why, you need to understand the actual legal framework around hiring tests, and almost no HR team does. So let's walk through it.
The framework that matters: the Uniform Guidelines
In 1978, four federal agencies — the EEOC, the Department of Labor, the Department of Justice, and the Civil Service Commission — jointly published the Uniform Guidelines on Employee Selection Procedures. It's federal regulation, codified at 29 CFR Part 1607. Forty-seven years later, it's still the document every court reaches for when an employer's hiring test is challenged.
The Uniform Guidelines apply to any selection procedure used to make employment decisions. A "selection procedure" includes formal tests, interviews, reference checks, application forms, performance evaluations, and informal scoring systems. Anything that affects who gets the job.
The Guidelines don't ban any particular method. They impose one core requirement: if your selection procedure causes adverse impact on a protected group, you have to be able to prove it's job-related and consistent with business necessity. No proof, no defense.
Adverse impact and the four-fifths rule
Adverse impact means a selection procedure disadvantages a protected group at a meaningfully higher rate than the majority group. The Uniform Guidelines give a practical rule of thumb for when adverse impact is presumed: the four-fifths rule (sometimes called the 80% rule).
If the selection rate for any protected group is less than four-fifths (80%) of the selection rate for the group with the highest selection rate, the Guidelines consider that "adverse impact" exists. Here's how the math actually works:
A company hires for a care coordinator role. Their pre-hire test runs on a recent applicant cohort:
60% is below the 80% threshold. The four-fifths rule presumes adverse impact. The employer must now prove the test is job-related, or stop using it.
Two important nuances that get missed:
The four-fifths rule isn't a safe harbor. Passing 80% doesn't mean you're definitely fine. Courts have also accepted statistical-significance tests as evidence of adverse impact even when the four-fifths threshold is technically met.
The burden of proof shifts. Once adverse impact is shown, the employer has to demonstrate the test is "job-related and consistent with business necessity." If the employer succeeds, the burden flips back: the plaintiff can still win by showing a less-discriminatory alternative exists that the employer refused to use.
If your test causes adverse impact and you can't prove it's job-related, you don't get to keep using it.
"Job-related": the three forms of validity
"Job-related and consistent with business necessity" is the legal phrase. The Uniform Guidelines translate it into a technical requirement: you have to validate the procedure using one of three accepted methods.
For most hiring tests, the two practical paths to defensibility are content validity (drawn from observed job tasks) or criterion validity (statistically predicts actual performance). Both require something most companies don't actually have: a real job analysis.
What "job analysis" really means
A job analysis is a structured documentation of what a job actually requires — the tasks performed, the knowledge needed, the skills used, the conditions of work, the judgment calls made. Not a job description. A job description is what you write to attract candidates. A job analysis is what you write to prove what the job actually demands.
The Uniform Guidelines require that any validated test be tied to a job analysis. If you skip this step, your validation argument has nothing underneath it. A court will not accept "we believe this test measures the right things." They want documentation that the test was built from documented job requirements.
This is the single biggest reason most hiring tests aren't actually defensible: the company can't produce a job analysis to anchor the test to.
The mistake most HR teams make: assuming vendor validation transfers
When you buy a standardized pre-hire battery from a major vendor (SHL, Criteria, HireVue, Wonderlic), the vendor will hand you a validation report referencing studies showing the test's predictive validity across many jobs and industries.
Here's the catch: the vendor's validation evidence is for the test in general — not for your use of it. The Uniform Guidelines require the user to establish that the validation evidence applies to their use of the test, not just to the test in the abstract.
In court, this often comes down to: did you do a job analysis showing your role matches the jobs in the vendor's validation studies? If not, the vendor's validation report doesn't automatically save you.
A vendor's validation study is evidence. It is not a substitute for showing the test fits your jobs.
A practical checklist: is your hiring test defensible?
The 7 questions
- Have you measured adverse impact? If you've never run the four-fifths analysis on your test's outcomes by protected group, you have no idea whether you're exposed.
- Do you have a real job analysis — not a job description — that documents what the role actually requires?
- Can you tie every section / item / scoring rule of your test back to that job analysis? If not, the test is floating without a legal anchor.
- If you're relying on vendor validation, have you documented that your jobs match the jobs in the vendor's validation studies? Without that, the evidence doesn't automatically apply to you.
- Do you have an ongoing validation loop? Following up hires with performance data and updating cutoff scores. The Uniform Guidelines treat validation as continuing, not one-time.
- Are your scoring decisions, cutoff scores, and section floors documented and reviewed? Arbitrary cutoffs are one of the most common ways a test fails challenge.
- Have you reviewed all of this with an employment attorney? Not as the validator — as the person who'll defend you if it's challenged.
If you answered "no" or "not sure" to three or more of those, your current setup probably wouldn't survive serious scrutiny.
The bottom line
Off-the-shelf hiring tests aren't inherently more or less defensible than custom ones. What matters is whether the test you're using has been properly validated for your specific use, with a documented job analysis anchoring it. Most companies haven't done that work — for either flavor. That's the actual exposure.
The custom approach, done correctly, is usually more defensible than a generic battery — because the job analysis is real, the items are visibly tied to observed work, the criterion validation runs against your actual outcomes, and the whole chain of evidence is built for your specific roles.