Comparison of equipercentile and item response theory equating when the scaling test method is applied to a multilevel achievement battery

Abstract

Test publishers generally choose an anchor or scaling test approach to the development of a growth scale for a multilevel achievement battery. Although some studies have been conducted comparing traditional equipercentile equating procedures with item response theory models using the anchor test (overlapping items) approach, to date there is no evidence on the comparability of equating procedures when the scaling test approach is used. The purpose of this study was to compare the equipercentile, Rasch, one-parameter modified logistic, and two-parameter logistic item response theory procedures in the equating of a multilevel achievement test battery using the scaling test approach. Since the equipercentile method has been widely used by test publishers, it was chosen as a standard for comparison of the experimental results. Individual item pseudo-guessing parameters were specified for the one-parameter modified logistic and two-parameter logistic item response theory models based on the proportion of students in the national standardization sample selecting the least attractive distractor for the item. Two grades—fourth and eighth—and two subtests—reading and mathematics— were selected for analysis. The results of the study suggest that for a small-sample situation in which the scaling test approach has been applied to a multilevel achievement battery, the one-parameter modified and two-parameter item response theory methods (as modified in this study) appear to be viable alternatives to the equipercentile procedure.