The influence of conditioning scores in performing DIF analyses

Abstract

The effect of the conditioning score on the results of differential item functioning (DIF) analyses was examined. Most DIF detection procedures match examinees from two groups of interest according to the examinees’ test score (e.g., number correct) and then summarize the performance differences across trait levels. DIF has the potential to occur whenever the conditioning criterion cannot account for the multidimensional interaction between items and examinees. Response data were generated from a two-dimensional item response theory model for a 30-item test in which items were measuring uniformly spaced composites of two latent trait parameters, θ₁ and θ₂. Two different DIF detection methods- the Mantel-Haenszel and simultaneous item bias (SIBTEST) detection procedure-were used for three different sample size conditions. When the DIF procedures were conditioned on the number-correct score or on a transformation of θ₁ or θ₂, differential group performance followed hypothesized patterns. When the conditioning criterion was a function of both θ₁ and θ₂ (i.e., when the complete latent space was identified), DIF, as theory would suggest, was eliminated for all items. Index terms: construct validity, differential item functioning, item bias, Mantel-Haenszel procedure, SIBTEST.