Put a hiring manager, a recruiter and a site lead in a room and ask whether an applicant is qualified. All three will answer quickly, and they will often agree. Then ask each of them what they checked, and the agreement falls apart.

The criteria were the same. The words were the same. What each person meant by them was different, and nobody noticed until the shortlist came back and somebody said it was wrong.

This post is about that gap, why the research on structured hiring keeps pointing at it, and how turning criteria into observable evidence closes most of it.

Three readers took the same word three ways

Take a brief for a warehouse shift lead with a line that says “experienced forklift operator”. Each reader brings their own definition to it.

• The hiring manager reads it as judgement. They want someone who knows when not to lift a pallet, and they will know it when they see it.

• The recruiter reads it as a threshold. Two years on the CV and a valid certificate means the box is ticked.

• The site lead reads it as fit for their floor. They want someone who has driven a reach truck in a narrow-aisle racking system on nights, because that is what the job is.

None of them is wrong. They have each filled the word with the part of the job they care about most. The trouble is that the brief says one thing and the three of them are screening for three.

The same pattern shows up with almost every soft criterion. “Good communicator” means calm on the phone to one person and clear in a written handover to another. “Reliable” means no missed shifts to a site lead and a clean reference to a recruiter.

Where the disagreement hides

The disagreement is invisible for most of the process, and that is what makes it expensive.

Each reader screens their share of the pile by their own definition. Everyone feels consistent, because each of them is, by their own standard. The difference only surfaces at the end, when a site lead looks at twelve names out of sixty and finds that half of them have never worked a night.

By then the cost is already paid. Applicants were declined who met the site lead’s bar, and some went through who did not. The shortlist gets rebuilt, the role stays open longer, and the argument turns on taste because nobody wrote down what they were checking.

Structured hiring research has made a version of this point for a long time. Applicants judged against the same written criteria are compared more fairly than applicants picked on a quick read. The part that gets less attention is the word “same”. A criterion written down is only shared if everyone reads it the same way.

Turn each criterion into something you could observe

The fix is to rewrite every criterion as evidence a reviewer could point to. If two people cannot look at the same answer and agree whether it meets the bar, the criterion is still an opinion.

A useful test for each line in a brief:

1. What would the applicant have done, said or shown if this were true?

2. Where would that evidence appear: an answer, a certificate, a reference or a work history?

3. What would a weak version of the same evidence look like?

A worked example

Here is “experienced forklift operator” before and after.

Before: Experienced forklift operator.

After: Holds a valid counterbalance and reach truck certificate. Has worked a reach truck in narrow-aisle racking, on at least some night shifts. Can describe a time they stopped a lift because the load or the aisle was unsafe, and what they did next.

Each part of the second version maps to one of the three readers. The certificate is the recruiter’s threshold, stated as a document. The racking and nights are the site lead’s floor, stated as work history. The stopped lift is the hiring manager’s judgement, stated as a question with an answer somebody can read.

Nobody had to give anything up. They had to say out loud what they meant, and write it where the others could see it.

Write down what counts, and what does not

Once the criteria are observable, the scorecard can carry them. In Yardstick each criterion in the brief becomes a line on a custom scorecard, and each rating links to the passage it came from. A reviewer who disagrees can open the answer and see what earned the mark.

Two habits keep the shared meaning intact:

• Name the evidence type on each line. “Certificate”, “answer to question 3”, “reference”. A reviewer then knows where to look, and an applicant without that evidence is marked as unconfirmed rather than failed.

• Write down what must not count. The name of a previous employer, a gap in a CV, a university. Bias controls in Yardstick are set per role, and the brief is where you decide them. Written down, they are choices the team made together, rather than habits one reader brought along.

Keep the vocabulary between openings

A shared definition is worth more the second time. Harbourline Freight wrote one brief per role across four terminals, and the plan for the next season is a single rubric library all four sites draw from. The same words then mean the same thing in Gothenburg and in Turku.

The vocabulary will still drift. A site changes its racking, a regulator changes a certificate, a hiring manager leaves. Treat the brief as a draft you revise after each shortlist, and read the evidence behind the names that surprised you.

None of this decides who gets hired. A person still reads the shortlist, still runs the interview and still makes the call. What a shared vocabulary changes is that the three people in the room are finally arguing about the same thing.