Hiring8 min read23 Oct 2026

AI Interviews for Agencies: Who Decides, Who Tells Candidates

A practical playbook for agencies and employers running AI interviews: who scores, who decides, what candidates are told and how to test fairness.

Your agency has started using an AI interview tool to screen candidates for a client. The tool belongs to a vendor, the role belongs to the client, and the shortlist goes out under your agency's name. When a rejected candidate asks "who actually decided this?", nobody in the chain has a clean answer.

That gap is now a practical problem, not a theoretical one. Candidates are asking more questions, clients want to know how shortlists are built, and the rules around automated decisions are changing. This playbook shows how to map who scores, who decides and who explains, so your AI interviews hold up when someone asks.

Why the "who does what" question matters now

Several developments have pushed this onto the agenda for Australian recruiters. The short version is below. It is general information, not legal advice, so check the detail with your own adviser.

  • Automated decision transparency. Law firm Allens reports that, as at October 2026, new privacy obligations requiring disclosure of certain automated decision-making commence on 10 December 2026. Allens and Corrs both note that the OAIC's final guidance lists recruitment software that sorts candidates and informs hiring decisions as in scope, and urges organisations to treat borderline cases as in scope.
  • Buying a tool still counts. Bird & Bird, writing in May 2026 about the OAIC's issues paper, notes the regulator's example that an organisation procuring third-party AI to screen job applications has arranged for the automated decision-making. Bird & Bird suggests checking vendor contracts for visibility, audit rights and disclosure cooperation.
  • No standalone AI guardrails federally. ABC News reported in December 2025 that the National AI Plan paused mandatory AI guardrails, with existing laws applying instead and an AI Safety Institute to advise on gaps.
  • States are moving. The Conversation reported in July 2026 that the Victorian government plans new laws aimed at preventing bias and discrimination in AI hiring decisions. It also noted that Victoria's Equal Opportunity Act does not currently require employers to make reasonable adjustments for people with disability in hiring.
  • More to come. As at August 2026, Responsible AI Australia reported that a federal Joint Select Committee on Artificial Intelligence is due to report by 30 November 2026.

You don't need to become a privacy lawyer. You do need a process where every step has a named owner and a plain explanation.

The three parties in an agency AI interview

When an agency runs AI interviews for a client, three parties usually touch the process. Write down who does what before you go live.

The vendor

  • Supplies the interview software, question delivery and scoring model.
  • Should be able to explain, in plain English, what the tool measures and what it does not.
  • Should give you access to evidence behind scores and data for fairness testing.

The agency

  • Designs or approves the questions and rubric with the client.
  • Invites candidates and is usually their main point of contact.
  • Reviews AI outputs and decides who goes on the shortlist.

The client (employer)

  • Defines the role requirements and what "good" looks like.
  • Makes the final hiring decision.
  • Owns any further interviews, offers and onboarding.

The weak point is almost always the hand-offs. If the vendor scores, the agency forwards scores without reading them, and the client hires off the ranking, then in practice the software decided. Fixing that is a process job.

Step by step: map your AI interview process before launch

  1. List every decision point. Invitation, interview completion, scoring, shortlist, client interview, offer. Mark which ones involve software.
  2. Name a human owner for each. One person, by role, who can explain the decision.
  3. Decide what the AI does and does not do. For example: it asks structured questions and suggests rubric scores; it does not reject anyone on its own.
  4. Agree the rubric with the client. Use behavioural questions tied to the role. Our STAR interview rubric template is a practical starting point.
  5. Check what the vendor can show you. Evidence for each score, what integrity checks ran, and data you can use for fairness testing.
  6. Write the candidate explanation. What happens, who reviews, how to ask for a review or an adjustment.
  7. Set a review cadence. Monthly is reasonable for active campaigns: check pass-through rates, complaints and adjustment requests.

What to tell candidates

Candidates mostly want to know three things: is a person involved, what is being assessed, and what can they do if something goes wrong. Keep it short and put it in the invitation, not just a policy page.

Here is example wording you can adapt. It is a communication template, not a legal clause, so have it reviewed alongside your privacy policy.

This interview is delivered through an AI interview platform on behalf of [Agency] for a role with [Client]. The platform asks every candidate the same structured questions and suggests scores against a rubric agreed with [Client]. A recruiter at [Agency] reviews your answers and the suggested scores before any shortlisting decision. [Client] makes the final hiring decision. If you need an adjustment, such as extra time or a different format, or you want a person to review an outcome, contact [name, email].

Notice the adjustment line. Given The Conversation's point about reasonable adjustments, offering them up front is simply good practice for candidate experience, whatever the legal position in your state. Our candidate transparency page shows how we explain this ourselves.

Keep a person in the decision, with evidence

Structured interviews work because every candidate gets the same questions and is judged against the same rubric. AI can make that consistent at scale. It should not quietly replace the judgement step.

Three habits make human review real rather than a rubber stamp:

  • Read the evidence, not just the number. A score without the candidate's own words behind it can't be checked.
  • Record overrides. If a recruiter moves a candidate up or down, note why in one line.
  • Know what wasn't checked. Integrity checks have limits. Detecting AI-written answers or deepfakes is not reliable, so treat flags as prompts for a conversation, not verdicts.

This is how Critical Hive is built: a person always makes the hiring decision, scores come with the candidate's own words as evidence, and the integrity report lists what was and was not checked. More on that on our fraud and integrity page.

Interestingly, PwC's 2026 AI Jobs Barometer (around June 2026) quotes a PwC Australia partner saying recruiters favour candidates who can exercise judgement, take clear accountability and oversee high-stakes decisions. The same standard is worth applying to your own process.

Test fairness on your own data

Vendor claims about fairness are not a substitute for checking your own outcomes. A common screening test is the four-fifths rule: compare the pass rate of each group with the group that has the highest pass rate. If a group's rate is below 80% of the highest, investigate.

It is a signal to look closer, not proof of discrimination or of fairness. Small numbers can swing the result, so look at trends over time. Critical Hive runs this test on each customer's own data; details are on our compliance and fairness page.

Worked example: a SOC analyst campaign

The figures below are illustrative, not real data.

An agency is hiring SOC analysts for a client. 150 candidates complete an AI interview. The tool asks six structured questions and suggests rubric scores.

  • Owner map: vendor scores; agency recruiter reviews every suggested score with transcript evidence; client hiring manager makes the decision after a live panel.
  • Overrides: the recruiter lifts four candidates whose answers showed strong incident-handling experience the rubric under-weighted, and notes why. The rubric is updated with the client for the next round.
  • Fairness check: Group A, 40 of 100 advance (40%). Group B, 15 of 50 advance (30%). 30 divided by 40 is 0.75, below 0.8. The agency reviews the questions and finds one relies on jargon from a specific vendor's tooling. It is reworded and pass rates are monitored.
  • Candidate query: a rejected candidate asks for a review. The named recruiter re-reads the answers, confirms the outcome, and explains which rubric areas were weaker.

Nothing here is complicated. It just requires that every step has an owner.

Checklist before you switch AI interviews on

  • Every decision point has a named human owner.
  • The AI's role is written down, including what it does not do.
  • Rubric agreed with the client and tied to the role.
  • Vendor provides score evidence, integrity check details and data for fairness testing.
  • Contract covers visibility, audit rights and cooperation on candidate explanations.
  • Candidate invitation explains the process, the human review and how to request adjustments.
  • A review route exists with a real contact.
  • Overrides are recorded.
  • Four-fifths check scheduled, with a plan for what you'll do if it flags.
  • Your privacy policy and candidate notices reviewed by an adviser before 10 December 2026.

If you work across several clients, our recruitment agencies page covers how multi-client set-ups work.

FAQ

Does a human reviewing AI scores mean the tool is out of scope?

Not necessarily. Allens and Corrs report that the OAIC urges organisations to treat borderline cases as in scope. Human review is still worth doing for quality and trust, but get advice on your disclosure obligations. This is general information, not legal advice.

Who should explain the process to candidates: agency, client or vendor?

Whoever the candidate deals with should be able to explain it, which is usually the agency. Agree the wording with the client and make sure the vendor contract supports you with the detail you need.

Can a candidate ask for a person to review an AI interview outcome?

They should be able to. Give a named contact in the invitation and have the reviewer read the candidate's actual answers, not just the score.

Can AI interview tools reliably catch cheating or AI-written answers?

No. Integrity checks can surface things worth a conversation, but detection of AI-written text or deepfakes is not reliable. Know what was and wasn't checked, and let a person decide.

Next step

See how human review, score evidence and integrity reporting fit together in practice by booking a live demo.

Sources

This guide was drafted with AI assistance and passed automatic claims checks; it has not been reviewed by a person. General information only. Tell us if something looks wrong: contact us.

KEEP READING

Interview with confidence.
Know who you're hiring.

Book a Critical Hive demo How Critical Hive works