Browsing by Subject "Multidimensional"

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Application of the bifactor model to computerized adaptive testing.
    (2011-01) Seo, Dong Gi
    Most CAT has been studied under the framework of unidimensional IRT. However, many psychological variables are multidimensional and might benefit from using a multidimensional approach to CAT. In addition, a number of psychological variables (e.g., quality of life, depression) can be conceptualized as being consistent with a bifactor model (Holzinger & Swineford, 1937) in which there is a general dimension and some number of subdomains with each item loading on only one of those domains. The present study extended the work on the bifactor CAT of Weiss & Gibbons (2007) in comparison to a fully multidimensional bifactor method using multidimensional maximum likelihood estimation and Bayesian estimation for the bifactor model (MBICAT algorithm). Although Weiss and Gibbons applied the bifactor model to CAT (BICAT algorithm), their methods for item selection and scoring were based on unidimensional IRT methods. Therefore, this study investigated a fully multidimensional bifactor CAT algorithm using simulated data. The MBICAT algorithm was compared to the two BICAT algorithms under three different factors: the number of group factors, the group factor discrimination condition, and estimation method. A fixed- test length was used as the termination criterion for the CATs for Study 1. The accuracy of estimates using the BICAT algorithm and the MBICAT algorithm was evaluated with the correlation between true and estimated scores , the root mean square error (RMSE), and the observed standard error (OSE). Two termination criteria (OSE = .50 and .55) were used to investigate efficiency of the MBICAT for Study 2. This study demonstrated that the MBICAT algorithm worked well when latent scores on the secondary dimension were estimated properly. Although the MBICAT algorithm did not improve the accuracy and efficiency for the general factor scores compared to two BICAT algorithms, the MBICAT showed an improvement of the accuracy and efficiency for the group factors. In the two BICAT algorithms, the use of differential entry on the group factors did not make a difference compared to initial item at trait of 0 for both the general factor and group factor scales (Gibbons, et al., 2008) in terms of accuracy and efficiency.
  • Thumbnail Image
    Item
    Between-person and within-person subscore reliability: comparison of unidimensional and multidimensional IRT models
    (2013-06) Bulut, Okan
    The importance of subscores in educational and psychological assessments is undeniable. Subscores yield diagnostic information that can be used for determining how each examinee's abilities/skills vary over different content domains. One of the most common criticisms about reporting and using subscores is insufficient reliability of subscores. This study employs a new reliability approach that allows the evaluation of between-person subscore reliability as well as within-person subscore reliability. Using this approach, the unidimensional IRT (UIRT) and multidimensional IRT (MIRT) models are compared in terms of subscore reliability in simulation and real data studies. Simulation conditions in the simulation study are subtest length, correlations among subscores, and number of subtests. Both unidimensional and multidimensional subscores are estimated with the maximum a posteriori probability (MAP) method. Subscore reliability of ability estimates are evaluated in light of between-person reliability, within-person reliability, and total profile reliability. The results of this study suggest that the MIRT model performs better than the UIRT model under all simulation conditions. Multidimensional subscore estimation benefits from correlations among subscores as ancillary information, and it yields more reliable subscore estimates than unidimensional subscore estimation. The subtest length is positively associated with both between-person and within-person reliability. Higher correlations among subscores improve between-person reliability, while they substantially decrease within-person reliability. The number of subtests seems to influence between-person reliability slightly but it has no effect on within-person reliability. The two estimation methods provide similar results with real data as well.
  • Thumbnail Image
    Item
    Estimating a noncompensatory IRT model using a modified metropolis algorithm.
    (2009-12) Babcock, Benjamin Grant Eugene
    Two classes of dichotomous multidimensional item response theory (MIRT) models, compensatory and noncompensatory, are reviewed. After a review of the literature, it is concluded that relatively little research has been conducted with the noncompensatory class of models. A monte-carlo simulation study was conducted exploring the estimation of a 2-parameter noncompensatory IRT model. The estimation method used was a modification of the Metropolis-Hastings algorithm that used multivariate prior distributions to help determine whether or not a newly sampled value was retained or rejected. Results showed that the noncompensatory model required a sample size of 4,000 people, 6 unidimensional items per dimension, and latent traits that are not highly correlated, for acceptable item parameter estimation using the modified Metropolis method. It is then argued that the noncompensatory model might not warrant further research due to the great requirements for acceptable estimation. The multidimensional interactive IRT model (MIIM) is proposed, which is more flexible than previous multidimensional models and explicitly accounts for correlated latent traits by using an interaction term within the logit. Item response surfaces for the MIIM model can be shaped either like compensatory or noncompensatory IRT model response surfaces.
  • Thumbnail Image
    Item
    Multidimensional mastery testing with CAT
    (2013-12) Nydick, Steven Warren
    Computerized mastery testing (CMT) is a subset of computerized adaptive testing (CAT) with the intent of assigning examinees to one of two, mutually exclusive, categories. Most mastery testing algorithms have been designed to classify examinees on either side of a cut-point in one dimension, but many psychological attributes are inherently multidimensional. Little psychometric work has generalized these unidimensional algorithms to multidimensional traits. When classifying examinees in multidimensional space, practitioners must choose a cut-point function that separates a mastery region from a non-mastery region. The possible cut-point functions include one in which a linear combination of ability across dimensions must exceed a threshold and one in which each ability must exceed a threshold irrespective of any other ability. Moreover, two major components of every classification test are choosing successive questions and determining when a classification decision should be made. One frequently used stopping rule in unidimensional mastery testing is the Sequential Probability Ratio Test (SPRT), in which a classification is made either when the log-likelihood test statistic is sufficiently large or when the maximum number of items has been reached. Due to inefficiencies in the SPRT, alternative algorithms have been proposed, such as the Generalized Likelihood Ratio (GLR), and the SPRT with Stochastic Curtailment (SCSPRT). The current study explores properties of unidimensional classification testing algorithms, generalizes unidimensional methods to multidimensional mastery tests, and then tests many of the multidimensional procedures. Most of the multidimensional algorithms yield relatively efficient and accurate multidimensional classifications. However, some multidimensional classification problems, such as classifying examinees with respect to a linear classification bound function, are more robust to poor choices in the item bank or adaptive testing algorithms. Based on results from the main study in this thesis, a follow-up study is proposed to better combine sequential classification methods with those based on directly quantifying incorrect classifications. I conclude by discussing consequences of the results for practitioners in realistic mastery testing situations.