Human Review of AI Shortlists: A Checklist That Holds Up

Your applicant tracking system ranks candidates, an AI interview tool scores them, and a recruiter signs off the shortlist. On paper, a human made the decision. In practice, the recruiter may have glanced at the top ten, agreed with the order and moved on.

That gap matters. If nobody can explain why a candidate was ranked low, or show that someone actually checked, the 'human in the loop' is decoration. This guide covers what genuine human review of AI shortlists looks like: what reviewers should question, what they should write down, and how to test the results for bias.

Why a sign-off is not the same as a review

A rubber stamp and a review can look identical in your system logs. Both show a named person approving a shortlist. The difference is whether that person engaged with the reasoning, could have disagreed, and sometimes did.

The Office of the Australian Information Commissioner (OAIC) has drawn the same line. In guidance on automated decision-making released on 30 September 2026, it includes a worked example, summarised by law firm Maddocks as at October 2026, in which an AI tool suggests staff bonuses. HR reviews the suggestions and a director signs off, yet the tool is still treated as informing the decision. According to Maddocks, the picture only changes where staff actually question the outputs, look for more evidence and record why they disagreed. Maddocks notes that tools ranking candidates or producing scores may still count even when a manager makes the final call.

That guidance sits alongside new privacy policy requirements starting on 10 December 2026, and recruitment firm SMS Personnel, summarising the OAIC fact sheet as at October 2026, says software that sorts candidate profiles and helps make hiring decisions is generally covered. This is general information, not legal advice, and you should get your own advice on how it applies to you.

The practical lesson holds regardless of the rules: if review leaves no trace of thinking, it will be hard to show it happened.

What makes an AI score reviewable

A reviewer can only question what they can see. Before you design a review process, check that each score gives the reviewer something to work with:

The criterion. What skill or behaviour was being assessed, in plain words.
The evidence. What the candidate actually said or submitted that led to the score, not a summary of a summary.
The scale. What a 2 means compared with a 4, ideally using a structured rubric such as the STAR interview rubric template.
The limits. What the tool did not assess, did not check, or could not see.

If a tool gives you a single number with no evidence behind it, your reviewer is being asked to trust rather than review. This is one reason Critical Hive pairs each score with the candidate's own words as evidence and lists what its integrity report did and did not check, so the reviewer can test the score against the answer.

Reviewer checklist: what to question and what to write down

Use this checklist for every shortlist that an AI tool has ranked or scored. It works for agencies and in-house teams.

Before reviewing

Confirm which tools touched the shortlist: ATS ranking, CV parsing, AI interview scoring, assessment platforms.
Know the criteria for the role and how they were weighted.
Set aside time to read evidence, not just scores.

While reviewing

Read below the line. Look at a sample of candidates just under the cut-off, not only the top of the list. That is where errors hurt most.
Check the evidence against the score. Does the candidate's answer support the rating? Would a colleague give the same score?
Look for proxy signals. Is a candidate marked down for phrasing, accent-related transcription errors, career breaks or unconventional job titles rather than for the skill itself?
Seek more evidence where it is thin. A short answer may reflect nerves or language, not ability. Check the CV, portfolio or references before deciding.
Be willing to disagree. If you never override the tool, ask whether you are reviewing or approving.

What to record

Who reviewed, and when.
Which candidates you looked at beyond the top of the list.
Every override: the original score, your decision, and one or two sentences explaining why.
Any pattern you noticed, such as a question that seems to disadvantage a group of candidates.

Those notes are what turn 'a person decided' into something you can show.

How to bias test shortlists with the four-fifths rule

Review catches individual errors. Bias testing catches patterns that no single reviewer would notice. The concern is real: University of Melbourne research led by Dr Natalie Sheard, reported in Pursuit in October 2025, found AI hiring systems put women, older workers, people with disability and people who speak English as a second language at risk of discrimination.

The four-fifths rule is a simple screening test:

Work out the selection rate for each group: candidates progressed divided by candidates in that group.
Find the group with the highest selection rate.
Divide each other group's rate by that highest rate.
If a ratio falls below 0.8 (four-fifths), treat it as a flag worth investigating.

A few cautions:

It is a flag, not a verdict. A low ratio tells you where to look, not what went wrong.
Small numbers mislead. With a handful of candidates, one decision can swing the ratio. Pool results across similar roles or periods before drawing conclusions.
Test your own data. A vendor's test on someone else's candidates tells you little about your roles and your applicant pool. Critical Hive, for example, runs the four-fifths rule on each customer's own data for this reason.
Test each stage. Check the ATS ranking, the interview scoring and the human shortlist separately, so you know where a gap opens.

You can only test groups you have data for, so think carefully about how you collect voluntary demographic information and how you explain its purpose to candidates.

Worked example: a SOC analyst shortlist

The numbers here are illustrative, not drawn from any real client.

An employer is hiring a security operations centre analyst. Sixty candidates complete an AI-scored interview. The recruiter reviews the ranked list.

The individual review. Instead of approving the top ten, the recruiter also reads five candidates just below the cut-off. One scored poorly on incident response. Reading the evidence, the recruiter sees the candidate described triaging alerts and escalating a ransomware incident, but used their previous employer's internal terms rather than standard ones. The recruiter checks the CV, confirms the experience, and moves the candidate onto the shortlist. The note reads: 'Score reflects terminology, not experience. Answer describes full triage and escalation. Overridden after CV check.'

The pattern check. At the end of the quarter, the team pools results across similar roles. Candidates who identified as native English speakers progressed at 30 per cent (12 of 40). Candidates who identified as speaking English as a second language progressed at 20 per cent (4 of 20). The ratio is 0.67, below 0.8.

What they do. They do not conclude the tool is biased on this alone. They review the questions where scores diverged, find one that rewards specific jargon, rewrite it to ask about actions rather than terms, and re-test next quarter. The override note above becomes part of the evidence for the change.

Find the scoring you did not know you had

Many teams only review the tool they bought as 'AI'. Maddocks observes, as at October 2026, that scoring tools are often built into recruitment platforms or HR systems and left off AI registers. Common examples include ATS match scores, auto-reject knockout questions and CV ranking features switched on by default.

Make a simple inventory: for each stage of your hiring process, list what ranks, filters or scores candidates, who configured it, and who reviews its output. SMS Personnel notes, as at October 2026, that the OAIC expects the business using the software, not the vendor, to meet its disclosure obligations, and vendors to give customers enough information to do so. Whatever your legal position, asking vendors for the criteria, evidence and limits behind their scores is good practice.

This is also no longer something you can wait for government to prescribe. The National AI Plan, launched on 2 December 2025, dropped the 10 mandatory guardrails for high-risk AI proposed in September 2024 and relies on existing laws instead, as reported by The Conversation. Fairness, accountability and transparency in hiring are largely up to employers and agencies to build themselves.

For agencies working with clients, our earlier piece on who decides and who tells candidates covers how to split these responsibilities. You can also read how we approach this in our AI transparency statement and on the compliance and fairness page.

FAQ

Does a recruiter approving the shortlist mean a human made the decision?

Not necessarily. A decision is genuinely human when the reviewer can see the evidence, questions it, seeks more where needed and records why they agreed or disagreed. Approval without engagement is a sign-off, not a review.

How often should we override AI scores?

There is no right number. But if reviewers never override, check whether they have enough evidence and time to disagree. Track override rates and reasons so you can spot both rubber-stamping and a tool that is regularly wrong.

Is the four-fifths rule enough to prove a tool is fair?

No. It is a screening test that highlights possible adverse impact. Use it to decide where to look, then examine questions, scoring and reviewer decisions. Small samples can produce misleading ratios.

Who is responsible if a vendor's tool is biased?

Lawyers quoted in coverage of the University of Melbourne research stress that the employer stays responsible even when the tool comes from a vendor. This is general information, not legal advice, so get your own advice for your situation.

Next step

If you want to see what evidence-backed scores and a reviewer workflow look like in practice, book a live demo and bring a real role to walk through.

Sources

This guide was drafted with AI assistance and passed automatic claims checks; it has not been reviewed by a person. General information only. Tell us if something looks wrong: contact us.