Using Bayesian decision theory to design a computerized mastery test

Abstract

A theoretical framework for mastery testing based on item response theory and Bayesian decision theory is described. The idea of sequential testing is developed, with the goal of providing shorter tests for individuals who have clearly mastered (or clearly not mastered) a given subject and longer tests for those individuals for whom the mastery decision is not as clear-cut. In a simulated application of the approach to a professional certification examination, it is shown that average test lengths can be reduced by half without sacrificing classification accuracy. Index terms: Bayesian decision theory, computerized mastery testing, item response theory, sequential testing, variable-length tests.