Hypothesis Testing for Adaptive Measurement of Individual Change

Abstract

The significance of individual change has been an important topic in psychology and related fields. This study investigated performance of five hypothesis testing methods-Z, likelihood ratio, score test, and Kullback-Leibler divergence test with uniform and normal prior distributions -"and three item selection methods-Fisher information, Kullback-Leibler information and a modified Kullback-Leibler information-as an extension of Finkelman et al.'s (2010) methods to determine the significance of individual change in the context of adaptive measurement of change (AMC). Comparisons between methods were made based on observed Type I error rates and power. Monte Carlo simulation was conducted with the level of item discriminations, bank information shape, bank size, and test length varied. Overall, the Z statistic displayed a better balance of Type I error rates and power than the other four statistics under various conditions. The efficiency of variable-length AMC was evaluated compared to fixed-length AMC based on the number of items saved as well as the precision of decisions.