Modeling guessing behavior: A comparison of two IRT models

Abstract

This study compared the fit of the three-parameter model to that of the Ability Removing Random Guessing (ARRG) model (Waller, 1973) on data from a wide range of tests of cognitive ability in three representative samples. Although both models were designed to remove only the effects of random guessing, the results of this study indicated that the three-parameter model also makes an adjustment for partial-knowledge guessing. Fit of the three-parameter model with guessing parameters estimated at a constant value of 1 divided by the number of alternatives was compared to fit with individually estimated guessing parameters. The latter were found to produce fit far superior to those estimated at a constant value. A solution to the convergence problems often encountered with the three-parameter model is discussed. Index terms: Ability Removing Random Guessing model, convergence in three-parameter estimation procedures, item response theory, maximum likelihood estimation, partial-knowledge guessing, random guessing.