Stochastic Curtailment: A New Approach to Improve Efficiency in Computerized Adaptive Tests

Abstract

Stochastic curtailment (SC) is a statistical procedure that was originally developed to enhance the efficiency of clinical trials. It has been applied to psychological testing, but to sequential mastery testing only (Finkelman, 2008, 2010). This study adapted the method to detect low-precision examinees (i.e., examinees whose final standard error of measurement (FSEM) at the end of a full-length test could not reach the pre-specified SEM termination level) in measurement computerized adaptive tests (CATs). Using central limit approximations, the study developed a method to estimate the distribution of test information at maximum test length and the corresponding FSEM. The study also developed a hypothesis testing procedure to implement SC. Using monte-carlo simulations, the study found that (1) the FSEM estimation procedure performed well in the middle range of ? values but less so at extreme ? values; (2) the SC procedure had good predictive accuracy, with excellent performance on positive predictive values and good performance on true positive rates and false positive rates; (3) the reduction in test length was substantial. Overall, the study showed that SC is a promising procedure to identify low-precision examinees and enhance efficiency in measurement CATs. A guide on implementing SC is provided.