Validity generalization and situational specificity: An analysis of the prediction of first-year grades in law school

Abstract

Results from 726 validity studies were analyzed to determine the degree of validity generalization of the Law School Admission Test for predicting first-year grades in law school. Four validity generalization procedures were used and their results compared. As much as 70% of the variance in observed validity coefficients could be accounted for by differences in the within-study variability of LSAT scores, simple sampling error, and between-study differences in criterion reliability. The 90% credibility value for the true validities was estimated to be .45, and the average true validity was estimated to be .54. Despite the substantial degree of validity generalization, law school and the year the study was conducted explained significant portions of the residual variance in validities. Thus, some degree of situational specificity of validity remained.