Estimation of reliability coefficients using the test information function and its modifications

Abstract

The reliability coefficient and the standard error of measurement in classical test theory are not properties of a specific test, but are attributed to both a specific test and a specific trait distribution. In latent trait models, or item response theory, the test information function (TIF) provides more precise local measures of accuracy in trait estimation than are available from the reliability coefficient. The reliability coefficient is still widely used, however, and is popular because of its simplicity. Thus, it is worthwhile to relate it to the TIF. In this paper, the reliability coefficient is predicted from the TIF, or two modified TIF formulas, and a specific trait distribution. Examples demonstrate the variability of the reliability coefficient across different trait distributions, and the results are compared with empirical reliability coefficients. Practical suggestions are given as to how to make better use of the reliability coefficient. Index terms: adaptive testing, bias, classical test theory, item information function, latent trait models, maximum likelihood estimation, reliability coefficient, standard error of measurement, test information function, trait estimation.