An Automated Test Assembly Approach Using Item Response Theory to Enhance Evidence of Measurement Invariance
2022-06
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
An Automated Test Assembly Approach Using Item Response Theory to Enhance Evidence of Measurement Invariance
Authors
Published Date
2022-06
Publisher
Type
Thesis or Dissertation
Abstract
When creating a new test to measure a latent trait, test developers must select items that together demonstrate desirable psychometric properties. Automated test assembly (ATA) algorithms allow test developers to systematically compare possible item combinations based on the test’s goals. ATA algorithms afford flexibility to incorporate various psychometric criteria for evaluating a new test. However, few algorithms have integrated analyses for item- and test-level bias, particularly within the item response theory framework. This dissertation proposes an approach that balances common indices of test score precision and model fit while simultaneously accounting for differing measurement models between two groups. Three Monte Carlo studies were designed to evaluate the proposed method (termed “Unbiased-ATA”). The first study found that in many testing scenarios, Unbiased-ATA appropriately constructed tests with evidence of measurement invariance (MI), item fit, and test information function alignment. Importantly, Unbiased-ATA’s performance depended on the accuracy of both the DIF detection method and item parameter estimation. The second study revealed that differentially weighting the Unbiased-ATA objective function criteria did not substantially affect the method’s performance. The final study found that Unbiased-ATA produced tests with stronger psychometric properties than an objective function based solely on test score precision. Yet adding a criterion for item-level MI did not noticeably improve tests’ psychometric strength above and beyond a criterion for test-level MI. Future directions for integrating ATA, test bias, and test fairness more broadly in psychological and educational measurement are discussed.
Description
University of Minnesota Ph.D. dissertation. 2022. Major: Psychology. Advisor: Niels Waller. 1 computer file (PDF); 240 pages.
Related to
Replaces
License
Collections
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Cooperman, Allison. (2022). An Automated Test Assembly Approach Using Item Response Theory to Enhance Evidence of Measurement Invariance. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/241639.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.