What research says about choosing the right hire

What validity means and what the 1998 review found

Predictive validity is the correlation between how candidates score on a hiring method and how they later perform in the job: zero means no relationship and 1.0 a perfect one. Schmidt and Hunter's 1998 article in Psychological Bulletin summarised 85 years of meta-analytic findings and tabled the validity of 19 selection procedures for overall job performance. Later authors note that the 1998 article presented only values corrected for measurement error in the criterion and for restriction of range.^[1][2]

In that table, general mental ability scored .51, work sample tests .54, structured employment interviews .51, unstructured interviews .38 and reference checks .26. Near the bottom were years of education at .10, graphology at .02 and age at -.01. The authors also reported the gain from adding a second predictor to a general mental ability test: adding a structured interview raised the combined multiple correlation to .63, a gain of .12.^[1]

Structured and unstructured interviews

Schmidt and Hunter drew the distinction plainly. An unstructured interview has no fixed format or set of questions, the interviewer often asks different candidates different things, and responses are usually not scored individually but summed up in an overall impression. A structured interview is the opposite on all counts, with questions usually set by analysis of the job. They reported an average validity of .51 for the structured interview and .38 for the unstructured one, and noted that structured interviews cost more to construct and use.^[1]

The 2022 revision lowered the numbers. Sackett and colleagues placed structured interviews first, with a mean validity of .42. Their 2023 follow-up says the unstructured interview estimate was .19 lower than previously reported, which against the 1998 value of .38 implies about .19; that subtraction is this guide's arithmetic, not a quoted figure. The follow-up also reports that structured interviews have a standard deviation of .19 and an 80 per cent credibility interval from .18 to .66, which the authors say is better read as .42 plus or minus .24.^[3][2][1]

Work samples and reference checks

A work sample asks candidates to perform a sample of the job itself. Schmidt and Hunter gave work sample tests the highest value in their table, .54. The 2023 follow-up says work sample validity was .21 lower in the revised estimates, which against the 1998 value implies about .33 by this guide's own subtraction, not a quoted figure. It adds that job-specific predictors such as structured interviews, job knowledge tests and work samples tend to fare better than general measures of ability and personality, but that work samples and job knowledge tests are not generally suitable for entry-level hiring where skills are built after hire.^[1][2]

Reference checks scored .26 in the 1998 table, below structured interviews and work samples. The article states that this value comes from earlier research by Hunter and Hunter, published in 1984. The 2023 paper consulted for this guide does not give a revised reference-check figure in its text, so none is stated here. This guide did not open the full 2022 article, so it cannot say whether a revised figure appears there.^[1][2]

The 2022 revision and the debate

The 2022 paper argued that the artifact distributions used to correct for restriction of range in earlier meta-analyses often overcorrected, so that the validity of many procedures had been substantially overestimated. Its authors report that most of the methods that ranked high before stay high, but with mean estimates reduced by .10 to .20. Their 2023 follow-up gives general mental ability a revised .31, no longer the stand-out predictor. It also reports ongoing work in which averaging over all combinations of one to five predictors gives .51 on prior values from Roth and colleagues and .47 on the revised ones, and concludes that the work does not challenge the value of selection practice.^[3][2]

The revision is contested. Ones and Viswesvaran argue that range restriction corrections in concurrent validation studies are appropriate, describe the revision's case as conceptual rather than empirical, and disagree that the operational validity of cognitive ability is much lower than reported. Sackett and colleagues reply that range restriction can matter in concurrent studies, for example where a predictor correlates above .70 with the hiring composite and the selection ratio is .10 or lower, but that this is implausible on average. They also cite a meta-analysis of 21st-century studies giving general mental ability .16 observed and .23 corrected.^[4][2]

Reading the numbers with care

Three cautions follow. First, coefficients are averages across jobs and settings, and the 2023 paper stresses the spread around them rather than the mean alone. Second, most apply to a single predictor used alone, whereas the authors note that composites of predictors reach overall validity close to what earlier work indicated. Third, the 2022 paper also reports White-Black mean differences for each predictor, a reminder that validity is one of several trade-offs in designing a selection system.^[2][3]

For a recruiter, the research speaks more to method than to any single tool. The features the authors single out are questions built from a job analysis and tasks that sample the work itself. They also note that several top predictors need candidates with prior experience, and that structured interviews are not easily used in high-volume screening. This guide located no UAE-specific validity evidence in the sources read, so the figures describe the research literature generally.^[2]

Questions