Two nonparametric procedures for detecting differential
item functioning (DIF)-the Mantel-Haenszel
(MH) procedure and the simultaneous item bias (SIB)
procedure-were compared with respect to their Type I
error rates and power. Data were simulated to reflect
conditions varying in sample size, ability distribution
differences between the focal and reference groups, proportion
of DIF items in the test, DIF effect sizes, and type
of item. 1,296 conditions were studied. The SIB and MH
procedures were equally powerful in detecting uniform
DIF for equal ability distributions. The SIB procedure
was more powerful than the MH procedure in detecting
DIF for unequal ability distributions. Both procedures
had sufficient power to detect DIF for a sample size of
300 in each group. Ability distribution did not have a
significant effect on the SIB procedure but did affect the
MH procedure. This is important because ability distribution
differences between two groups often are found in
practice. The Type I error rates for the MH statistic were
well within the nominal limits, whereas they were
slightly higher than expected for the SIB statistic. Comparisons
between the detection rates of the two procedures
were made with respect to the various factors.
Index terms: differential item functioning, Mantel-
Haenszel statistic, power, simultaneous item bias statistic,
SIBTEST, Type I error rates.
Narayanan, Pankaja & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18, 315-328. doi:10.1177/014662169401800403
Narayanan, Pankaja; Swaminathan, H..
Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning.
Retrieved from the University of Minnesota Digital Conservancy,
Content distributed via the University of Minnesota's Digital Conservancy may be subject to additional license and use restrictions applied by the depositor.