Graduate intakes arrive all at once and are read by a rotating cast of interviewers who have day jobs. The rubric drifts from reviewer to reviewer, and nobody can see it drifting because no two people read the same application.

Solberg’s graduate programme takes on new analysts twice a year, drawn from an intake that regularly runs past six hundred applications. Every one of those applications needs to be read against the same standard, by people who are also doing their own jobs that week.

Each reviewer brought their own standard

Solberg’s intake was assessed by whoever could be freed up that week. Each of them was competent, and each weighted the criteria slightly differently. One leaned on the degree, one on the internship, one on how the candidate came across on the page.

Aggregate that across a few hundred applications and the shortlist reflects the reviewers as much as the applicants. Two candidates with near-identical records could land on opposite sides of the cut, depending only on which of the twelve reviewers happened to read their file.

The programme runs across four desks (markets, research, operations and technology), each with its own idea of a strong graduate application. None of it was written down anywhere the others could see.

Every reviewer now scores against the same rubric

Solberg uses Yardstick Agents to apply a single agreed rubric across the whole intake, so differences between candidates are differences in the answers rather than in who read them.

The rubric is agreed by the hiring group before the intake opens and is not changed mid-cycle. Where it turns out to be wrong, it is changed for the next intake and the change is recorded with the reason. Each of the four desks still sets its own weighting for its own criteria. Every application on that desk is read against the same weighting, not a version of it.

Scores carry the evidence behind them, so a reviewer checking a shortlist later sees what the candidate actually wrote, not just a number a colleague produced weeks earlier.

Reviewers still interview

The structured round produces a comparable shortlist; the interviews are still run by people and the offer is still theirs to make. What the reviewers no longer do is the part where consistency was impossible by hand: reading six hundred applications to the same standard, in the same week, without drifting.

A rubric that holds across the intake

Score agreement across reviewers now runs at 94%, measured by having a second reviewer re-score a sample of the same applications. Before the rubric was fixed, that figure was not being measured at all, because there was no shared standard to check reviewers against.

Graduate applicants are screened against one rubric: roughly 640 applications each intake, all read to the same weighting on their desk. The structured interview round has also shortened, running 30% less time than before. The arguing about criteria now happens before the interviews, not during them.

What still takes a human eye is the final reference call before an offer goes out. Solberg has not automated that, and has no plan to.

Applying the same test to lateral hires

Solberg is reviewing whether the same rubric-per-desk approach should extend to lateral hires, where the reviewer pool is even smaller and the drift risk is higher.