Operational characteristics of adaptive testing procedures using the graded response mode

Abstract

The purpose of the present research was to develop general guidelines to assist practitioners in setting up operational computerized adaptive testing (CAT) systems based on the graded response model. Simulated data were used to investigate the effects of systematic manipulation of various aspects of the CAT procedures for the model. The effects of three major variables were examined: item pool size, the stepsize used along the trait continuum until maximum likelihood estimation could be calculated, and the stopping rule employed. The findings suggest three guidelines for graded response CAT procedures: (1) item pools with as few as 30 items may be adequate for CAT; (2) the variable-stepsize method is more useful than the fixed-stepsize methods; and (3) the minimum-standard-error stopping rule will yield fewer cases of nonconvergence, administer fewer items, and produce higher correlations of CAT θ estimates with full-scale estimates and the known θs than the minimum-information stopping rule. The implications of these findings for psychological assessment are discussed. Index terms: computerized adaptive testing, graded response model, item response theory, polychotomous scoring.