Problems related to the use of conventional and item response theory equating methods in less than optimal circumstances

Abstract

This paper focuses on a discussion of how various equating methods are affected by (1) sampling error, (2) sample characteristics, and (3) characteristics of anchor test items. Studies that examine the effect of analytic techniques for smoothing or modeling marginal and bivariate frequency distributions on the accuracy of equipercentile equating are reviewed. A need for simulation and empirical studies designed to evaluate the effectiveness of analytic smoothing techniques for recovering the underlying distribution when sample size, test length, and distributional shape are varied is identified. Studies that examine the question of whether an equating transformation remains the same regardless of the group used to define it are also reviewed. The results of some studies suggested that this may not be a problem for forms of a homogeneous test constructed to be similar in all respects. Results of other studies indicated that examinees who take a test on different administration dates may vary in systematic ways and thus affect equating results. Finally, studies which examine the characteristics of anchor test items are reviewed. It is concluded that whenever groups differ in level and dispersion of ability, special care must be taken to assure that the anchor test is a miniature of the total test.