Examining Power and Type 1 Error for Step and Item Level Tests of Invariance: Investigating the Effect of the Number of Item Score Levels
2017-05
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
Examining Power and Type 1 Error for Step and Item Level Tests of Invariance: Investigating the Effect of the Number of Item Score Levels
Authors
Published Date
2017-05
Publisher
Type
Thesis or Dissertation
Abstract
Within polytomous items, differential item functioning (DIF) can take on various forms due to the number of response categories. The lack of invariance at this level is referred to as differential step functioning (DSF). The most common DSF methods in the literature are the adjacent category log odds ratio (AC-LOR) estimator and cumulative category log odds ratio estimator (CU-LOR). Although the study of DSF may be helpful when opposing DIF effects within an item can go undetected or for informing what part of a multi-step item may need improvement, research regarding DSF procedures is limited. The effect of number of item score levels has not been investigated with regard to the relationship between DSF and traditional DIF methods, including differences in statistical behavior. This study investigates the effect of the number of item score levels on power and Type I error of the following DSF methods: AC-LOR, CU-LOR as well as DIF methods: Mantel (chi-square) Test, Liu Agresti, Generalized Mantel-Haenszel, and Simultaneous Step Level test (SSL). This study also examined which statistical procedures are most effective for adjusting per comparison Type I errors for the SSL method: Dunn-Bonferroni, Benjamini and Hochberg, or Holm’s. Conditions varied included (a) sample size ratio, (b) number of item score levels, (c) generating model, (d) impact, and (e) DSF pattern. Results suggest that altering the number of score levels did not have an effect on the DSF/DIF detection methods. When considering both statistical and practical significance of factors affecting power, the pattern of DSF was the most important effect. Additionally, the Dunn-Bonferroni adjustment was adequate when using the SSL method. The SSL method performed well compared to the other DIF methods and should be considered for simultaneously detecting both DSF and DIF. The significance of these results as well as limitations and future directions are discussed.
Keywords
categorical data
differential item functioning
differential step functioning
nonparametric tests
odds ratio
test bias
Description
University of Minnesota Ph.D. dissertation. May 2017. Major: Educational Psychology. Advisor: Ernest Davenport. 1 computer file (PDF); x, 141 pages.
Related to
Replaces
License
Collections
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Ayodele, Alicia. (2017). Examining Power and Type 1 Error for Step and Item Level Tests of Invariance: Investigating the Effect of the Number of Item Score Levels. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/191406.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.