Are AI interviews fair? The honest answer is that no vendor can promise a bias-free tool, and you should be suspicious of anyone who does. What you can do is measure outcomes on your own data and act on what you find. This guide shows how, using the four-fifths rule.
General information, not legal advice. The four-fifths rule comes from US employment guidance and is a screening rule of thumb, not an Australian legal test.
What "adverse impact" means
Adverse impact is when a selection step lets through noticeably fewer people from one group than another. It does not prove discrimination, and a clean result does not prove fairness. It is a signal that tells you where to look.
The four-fifths rule in one paragraph
Take the selection rate of the group with the highest rate (say 60 percent advanced to the next stage). Divide each other group's rate by that number. If a group's ratio is below 0.8, which is four-fifths, flag it for a closer look.
An example with made-up numbers: Group A advances 30 of 50 (60 percent). Group B advances 18 of 50 (36 percent). The ratio is 36 divided by 60, which is 0.6. That is below 0.8, so you investigate.
Why small groups need care
With only five or six people in a group, one extra person changes the percentage dramatically. Good tools mark small groups as "too small to judge" instead of giving a false sense of certainty. Critical Hive does this, and it never guesses anyone's gender, age or ethnicity: you supply group labels from your own HR data and only counts are logged.
How to run the check in Critical Hive
- Export the group labels you hold (for example from your HR system) as a CSV: email, group.
- Open the Compliance section, upload the CSV, choose the outcome to compare (for example "advanced by a human").
- Review the selection rates, the ratios and any flags.
See compliance and fairness in Critical Hive for the full feature.
What to do with a flag
- Check the data first. Are the labels accurate and the groups big enough?
- Look at the stage. Is the difference in the AI score, in human decisions, or in who completed the interview?
- Review the rubric. Does any question reward a style rather than a skill?
- Review the humans. Bias also lives in human decisions; a person-in-the-loop is not automatically fair.
- Document what you changed. An audit trail of decisions and fixes is worth more than a clean-looking number.
What the test cannot tell you
- Whether any individual was treated fairly.
- Anything about groups you did not label.
- Whether your job requirements themselves are fair.
Design choices that reduce risk
- The same questions and rubric for every candidate.
- Evidence next to every score, so a human can challenge it.
- No automatic rejection.
- A human interview option for candidates who ask.
- Regular re-testing as your candidate pool changes.
FAQ
Is the four-fifths rule a legal requirement in Australia?
No. It is a US guideline used widely as a screening heuristic. Australian discrimination law has its own tests, so take advice for your situation.
Can an AI interview be biased against accents or disability?
Any system can disadvantage people if it is poorly designed. Offer typed answers as well as speech, a practice question, and a human alternative, and review outcomes.
How often should I test?
Every time you change the rubric or the role, and at least quarterly on active roles.
Next step
Read about how Critical Hive keeps a person in charge or try the bias test on a pilot role. Start a free pilot.