Between Dec 19, 2024 and Jan 2, 2025, datasets can be submitted to DRUM but will not be processed until after the break. Staff will not be available to answer email during this period, and will not be able to provide DOIs until after Jan 2. If you are in need of a DOI during this period, consider Dryad or OpenICPSR. Submission responses to the UDC may also be delayed during this time.

Browsing by Author "Millsap, Roger E."

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning
    (1994) Uttaro, Thomas; Millsap, Roger E.
    The performance of the Mantel-Haenszel odds-ratio estimator and X² significance test were investigated using simulated data. Multiparameter logistic item response theory models were used to generate item scores for 20- and 40-item tests for 500 reference group and 500 focal group examinees. The difficulty, discrimination, and guessing parameters, and the difference in the group trait level averages were varied and combined factorially. Within each cell of the design, 200 replications were completed under both differential item functioning (DIF) and no-DiF conditions. The empirical X² Type I and Type II error rates, and the average of the odds-ratio estimates, were analyzed over the 200 replications. Under no-DiF conditions, inflated X² Type I error rates and misestimated odds-ratio values were found for the 20-item test and resulted from interactions between item parameter values and trait differences. For the 40-item test, Type I error rate inflation disappeared but odds-ratios still were misestimated. Under DIF conditions, Type II error rates were not inflated, but odds-ratios were misestimated, due to parameter X trait level interactions for both test lengths. The results demonstrate the importance of using both the odds-ratio and the significance test in interpreting the presence or absence of DIF. In addition, the accuracy under the DIF conditions depended on the size and uniformity Of DIF. Index terms: differential item functioning, item bias, item response theory, Mantel-Haenszel procedure, measurement bias, odds-ratio, simulation.
  • Thumbnail Image
    Item
    Inferential conditions in the statistical detection of measurement bias
    (1992) Millsap, Roger E.; Meredith, William
    Measurement bias in an observed variable Y as a measure of an unobserved variable W exists when the relationship of Y to W varies among populations of interest. Bias is often studied by examining population differences in the relationship of Y to a second observed measure Z that serves as a substitute for W. Whether the results of such studies have implications for measurement bias is addressed by first defining two forms of invariance- one corresponding to the relationship of Y to the unmeasured W, and one corresponding to the relationship of Y to the observed Z. General theoretical conditions are provided that justify the inference of one form of invariance from the other. The implications of these conditions for bias detection in two broad areas of application are discussed: differential item functioning and predictive bias in employment and educational settings. It is concluded that the conditions for inference are restrictive, and that bias investigations that rely strictly on observed measures are not, in general, diagnostic of measurement bias or the lack of bias. Some alternative approaches to bias detection are discussed. Index terms: differential item functioning, invariance, item bias, item response theory, measurement bias, predictive bias.
  • Thumbnail Image
    Item
    Methodology review: Statistical approaches for assessing measurement bias
    (1993) Millsap, Roger E.; Everson, Howard T.
    Statistical methods developed over the last decade for detecting measurement bias in psychological and educational tests are reviewed. Earlier methods for assessing measurement bias generally have been replaced by more sophisticated statistical techniques, such as the Mantel-Haenszel procedure, the standardization approach, logistic regression models, and item response theory approaches. The review employs a conceptual framework that distinguishes methods of detecting measurement bias based on either observed or unobserved conditional invariance models. Although progress has been made in the development of statistical methods for detecting measurement bias, issues related to the choice of matching variable, the nonuniform nature of measurement bias, the suitability of current approaches for new and emerging performance assessment methods, and insights into the causes of measurement bias remain elusive. Clearly, psychometric solutions to the problems of measurement bias will further understanding of the more central issue of construct validity. The continuing development of statistical methods for detecting and understanding the causes of measurement bias will continue to be an important scientific challenge. Index terms: bias detection, differential item functioning, item bias, measurement bias, test bias.
  • Thumbnail Image
    Item
    Tolerance intervals: Alternatives to credibility intervals in validity generalization research
    (1988) Millsap, Roger E.
    In validity generalization research, the estimated mean and variance of the true validity distribution are often used to construct a credibility interval, an interval containing a specified proportion of the true validity distribution. The statistical interpretation of this interval in the literature has varied between Bayesian and classical (frequentist) viewpoints. Credibility intervals are here discussed from the frequentist perspective. These are known as "tolerance intervals" in the statistical literature. Two new methods for constructing a credibility interval are presented. Unlike the current method of constructing the credibility interval, tolerance intervals have known performance characteristics across repeated applications, justifying confidence statements. The new methods may be useful in validity generalization research involving a small or moderate number of validation studies. Index terms: Bayesian statistics, Credibility intervals, Metaanalysis, Tolerance intervals, True validity distribution, Validity generalization.