Psychometric properties of finite-state scores versus number-correct and formula scores: A simulation study

Abstract

As developed by García-Pérez (1987), finite-state scores are nonlinear transformations of the proportions of conventional multiple-choice responses that are correct, incorrect, and omitted. They estimate the proportions of item alternatives which the examinees had the knowledge needed to classify (as correct or incorrect) before seeing them together in the items. The present study used simulation techniques to generate conventional test responses and to track the proportions of alternatives the examinees could classify independently before taking the test and the proportions they could classify after taking the test. Then the finite-state scores were computed and compared with these actual values and with number-correct and formula scores based on the conventional responses. Highly favorable results were obtained leading to recommendations for the use of finite-state scores. These results were almost the same when the simulation proceeded according to the model and when it was based on a naturalistic process completely independent of the model. Hence the scoring procedures on which finite-state scores are based are both accurate and robust. Index terms: applied measurement models, examinee behavior, finite-state scores, guessing, multiple-choice tests, test scoring.