Measuring Intra-Individual Change at Two or More Occasions With Hypothesis Testing Methods

Abstract

The present study proposed six new omnibus hypothesis tests – F1, F2, LR, ST and two chi-squared based statistics – to measure psychometric significance of individual change when an individual is measured at two or more occasions. The hypothesis tests were evaluated on criteria of Type I error, power, and agreement between the methods in the adaptive measurement of change (AMC) framework. This study expanded on AMC research by Finkleman, Weiss and Kim-Kang (2010) and Lee (2015), by introducing more generalized methods for multi-occasion case. The omnibus tests were evaluated under various discrimination, bank type, and change conditions. The simulation results showed the LR test to achieve an optimum balance between Type I error and power. The hypothesis tests were found to be robust under most testing conditions. The tests were successfully applied to K-12 math data. The proposed methods are applicable under a variety of testing conditions in which IRT-based item parameters have been established.