12630: What Do We Know About Job Knowledge Tests? A 30-Year Update of Validity Evidence (session 25852)

Designated as a “Best Paper” by HR
view abridged “best paper” (if available)

Job knowledge tests (JKTs) are widely used for selection and promotion in organizations, as well as for certification in professions such as law, medicine, and education. Despite this, evidence regarding the criterion-related validity of JKTs is based primarily on a very brief meta-analysis published more than 30 years ago (Dye et al., 1993). The present study provides an important update regarding JKTs, as well as some of the first meta-analytic evidence concerning the predictive validity of knowledge-based professional certification tests. Analyses of data from 87 independent samples revealed overall mean operational validity estimates of .30 for job performance (k = 62, SDr = .14), .35 for training performance (k = 28, SDr = .07), and -.09 for turnover (k = 9, SDr = .02). Operational validities were larger for job-specific JKTs (.36) than for organization- and occupation-specific tests (.19 and .21, respectively). In addition, JKTs used for selection or promotion purposes were associated with higher validity (.47) than tests used for research (.31) or certification purposes (.21). Finally, we tested the mediating role of job knowledge between other antecedents and job performance. Results suggested that job knowledge fully explains relations between cognitive ability and job performance, partially explains relations between job experience and performance, and does not explain relations between conscientiousness and performance. Overall, these results provide a much-needed update regarding the use of JKTs for decision making, as well as the role job knowledge plays in other prominent predictors of job performance.

view paper (if available)