Ordering power of separate versus grouped true-false tests: Interaction of type of test with knowledge levels of examinees

Abstract

The ordering power of an objective test was defined in terms of the probability that this test led to the correct ranking of examinees. A comparison of the relative ordering power of separate and grouped-items true-false (T-F) tests indicated that neither type of test was uniformly superior to the other across all levels of knowledge of examinees. Instead, separate-items T-F tests were found to be superior in discriminating among examinees with medium and high levels of knowledge, and grouped-items T-F tests with two and three items per cluster were found to be superior for discriminating among examinees with low levels of knowledge. These findings do not support blanket recommendations such as Ebel’s (1978) that "test constructors should avoid constructing items in multiple-choice form which are essentially collections of T-F statements" (p. 43) or that, in general, "it is better to present such statements as independent T-F items" (p. 43). Rather, they are similar to Lord’s (1977) findings concerning the relative efficiency of multiple-choice tests with different numbers of options per question for examinees of differing ability levels.