Simulated data were used to investigate the performance
of modified versions of the Mantel-Haenszel
method of differential item functioning (DIF) analysis
in computerized adaptive tests (CATs). Each simulated
examinee received 25 items from a 75-item pool. A
three-parameter logistic item response theory (IRT)
model was assumed, and examinees were matched on
expected true scores based on their CAT responses and
estimated item parameters. The CAT-based DIF statistics
were found to be highly correlated with DIF statistics
based on nonadaptive administration of all 75 pool items
and with the true magnitudes of DIF in the simulation.
Average DIF statistics and average standard errors also
were examined for items with various characteristics.
Finally, a study was conducted of the accuracy with
which the modified Mantel-Haenszel procedure could
identify CAT items with substantial DIF using a classification
system now implemented by some testing
programs. These additional analyses provided further
evidence that the CAT-based DIF procedures performed
well. More generally, the results supported the use of
IRT-based matching variables in DIF analysis. Index
terms: adaptive testing, computerized adaptive testing,
differential item functioning, item bias, item response
Zwick, Rebecca; Thayer, Dorothy T.; Wingersky, Marilyn.
A Simulation Study of Methods for Assessing Differential Item Functioning in Computerized Adaptive Tests.
Retrieved from the University of Minnesota Digital Conservancy,
Content distributed via the University of Minnesota's Digital
Conservancy may be subject to additional license and use
restrictions applied by the depositor.