Grid Multi-classification Adaptive Classification Testing with Multidimensional Polytomous Items

Abstract

The adaptive classification testing (ACT) is a form of computerized adaptive testing (CAT) which is developed to efficiently classify examinees into multiple groups based on predetermined classification cutoff points. All existing multidimensional ACT studies handled multidimensional classifications in a unidimensional space by performing classification on a composite of multiple traits. However, classification along separate dimensions is sometimes preferred because it provides clearer information regarding a person’s relative standing along each dimension. This type of classification is referred to as grid classification, as each examinee is classified into one of the grids encircled by cutoff scores (lines/surfaces) on different dimensions. Complications arise when there is more than one cutoff along each dimension. In order to perform grid classification using ACT, two termination criteria, sequential probability ratio test (SPRT) and confidence interval (CI) were adopted from one-dimensional classification in the between-item multidimensional test. In addition, two new termination criteria for grid multiclassification ACT were developed, namely, the grid classification generalized likelihood ratio (GGLR) and the simplified generalized likelihood ratio (SGLR). Two item selection methods, D-optimal and multidimensional mutual information (MMI) were borrowed from measurement CAT. A new item selection method, i.e., the posterior weighted D-optimal on cutoff points (PWCD-optimal), was also proposed. Three simulation studies were conducted to evaluate the performance of ACT in multidimensional grid classification. Results show ACT resulted in up to 20% shorter ATLs than the measurement CAT-based two-step approach, thus is more efficient in grid multiclassification. Specifically, SPRT and CI outperformed the two new termination criteria (GGLR and SGLR), and D-optimal and PWCD-optimal outperformed MMI based on the normally distributed population. Meanwhile, the classification was harder when examinees were closer to the cutoff scores. PWCD-optimal and D-optimal lead to stable test length and classification accuracy. As SPRT and CI resulted in lower classification accuracy for examinees that are close to the cutoff points thus are hard to be classified, the overall high efficiency of SPRT and CI in study 1 can be largely attributed to the large proportion of examinees that are far from the cutoff points in the normally distributed population.