Browsing by Subject "Item Response Theory"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    A Comparison of Item Selection Methods and Stopping Rules in Multi-category Computerized Classification Testing
    (2022-12) Suen, King Yiu
    Computerized classification testing (CCT) aims to classify people into one of two or more possible categories while maximizing accuracy and minimizing test length. Two key components of CCT are the item selection method and the stopping rule. The current study used simulation to compare the performance of various item selection methods and stopping rules for multi-category CCT in terms of average test length (ATL) and percentage of correct classifications (PCC) under a wide variety of conditions. Item selection methods examined include selecting items to maximize the Fisher information at the ability estimate, Fisher information at the nearest cutoff, and the sum of Fisher information of all cutoffs weighted with the likelihood function. The stopping rules considered were a multi-hypothesis sequential probability ratio test (mSPRT) and a multi-category generalized likelihood ratio test (mGLR), combined with three variations of stochastic curtailment methods (SC-Standard, SC-MLE and SC-CI). Manipulated conditions included the number of cutoffs, the distribution of the examinees’ abilities, the width of the indifference region, the shape of the item bank information function, and whether the items were calibrated with estimation error. Results suggested that the combination of mGLR and SC-MLE consistently had the best balance of ATL and PCC. The three item selection methods performed similarly across all conditions.
  • Thumbnail Image
    Item
    Estimating a noncompensatory IRT model using a modified metropolis algorithm.
    (2009-12) Babcock, Benjamin Grant Eugene
    Two classes of dichotomous multidimensional item response theory (MIRT) models, compensatory and noncompensatory, are reviewed. After a review of the literature, it is concluded that relatively little research has been conducted with the noncompensatory class of models. A monte-carlo simulation study was conducted exploring the estimation of a 2-parameter noncompensatory IRT model. The estimation method used was a modification of the Metropolis-Hastings algorithm that used multivariate prior distributions to help determine whether or not a newly sampled value was retained or rejected. Results showed that the noncompensatory model required a sample size of 4,000 people, 6 unidimensional items per dimension, and latent traits that are not highly correlated, for acceptable item parameter estimation using the modified Metropolis method. It is then argued that the noncompensatory model might not warrant further research due to the great requirements for acceptable estimation. The multidimensional interactive IRT model (MIIM) is proposed, which is more flexible than previous multidimensional models and explicitly accounts for correlated latent traits by using an interaction term within the logit. Item response surfaces for the MIIM model can be shaped either like compensatory or noncompensatory IRT model response surfaces.
  • Thumbnail Image
    Item
    A Restricted Bi-factor Model of Subdomain Relative Strengths and Weaknesses
    (2015-08) CHANG, YU-FENG
    There are increasing demands to report subscores in educational and psychological assessments. Subscores provide unique information about examinees (Sinharay, Puhan & Haberman, 2011). However, there has been much debate about reporting subscores because subscores require meeting certain standards and psychometric qualities as a prerequisite to reporting them. Because there is an increasing need for improving the methods of estimating subscores, multidimensional item response theory (MIRT) is one of the methods to estimate subscores. One MIRT model is the item bi-factor model, which includes a general dimension on which all items load and specific dimensions corresponding to the subdomains from which the items come (Holzinger & Swineford’s, 1937; Gibbons & Hedeker, 1992). However, there is a challenge to interpreting the specific dimension scores in the item bi-factor model while the general dimension score is readily interpreted. The specific dimension scores are residuals from the general factor and residuals can be difficult to interpret. To solve this issue, a restricted bi-factor model was proposed in this paper. This paper contains a real data study and a simulation study to evaluate this model. The results of two studies, interpretation of the model, and practical application of the model were discussed.