The effect of test length and IRT model on the distribution and stability of three appropriateness indexes

Abstract

The extent to which three appropriateness indexes-Z, ECIZ₄, and W (a variation of Wright’s person-fit statistic)-are well-standardized was investigated in a monte carlo study. To assess the effects of the item response theory (IRT.) model and test length on the distribution of the indexes and their cutoff values at three false positive rates, nonaberrant response patterns were generated. ECIZ₄ most closely approximated a normal distribution, showing less skewness and kurtosis than Z, and W. The ECIZ₄ cutoff values were affected less by test length and the IRT model than were Z, and W. In contrast, the distribution of W was the least stable over replications, and its cutoff values varied greatly depending on the IRT model and test length. Index terms: appropriateness measurement, caution index, item response theory (person fit), person-fit statistics, unusual response patterns.