Estimating measures of pass-fail reliability from parallel half-tests

Abstract

Two methods are derived for estimating measures of pass-fail reliability. The methods require only a single test administration and are computationally simple. Both are based on the Spearman-Brown formula for estimating stepped-up reliability. The non-distributional method requires only that the test be divisible into parallel half-tests; the normal method makes the additional assumption of normally distributed test scores. Bias for the two procedures is investigated by simulation. For nearly normal test score distributions, the normal method performed slightly better than the non-distributional method, but for moderately to severely skewed or symmetric platykurtic test score distributions the non-distributional method was superior. Test results from a licensure examination are used to illustrate the methods. Index terms: Cohen’s kappa, licensure examinations, pass-fail reliability, reliability, Spearman-Brown formula.