Browsing by Author "Frary, Robert B."

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    The effect of misinformation, partial information, and guessing on expected multiple-choice test item scores
    (1980) Frary, Robert B.
    Six response/scoring methods for multiple-choice tests are analyzed with respect to expected item scores under various levels of information and misinformation. It is shown that misinformation always and necessarily results in expected item scores lower than those associated with complete ignorance. Moreover, it is shown that some response/ scoring methods penalize all conditions of misinformation equally, and others have varying penalties according to the number of wrong choices the misinformed examinee has categorized with the correct choice. One method exacts the greatest penalty when a specific wrong choice is believed correct ; two other methods provide the maximum penalty when the examinee is confident only that the correct choice is incorrect. Partial information is shown to yield substantially different expected item scores from one method to another. Guessing is analyzed under the assumption that examinees guess whenever it is advantageous to do so under the scoring method used and that these conditions would be made clear to the examinee. Additional guessing is shown to have no effect on expected item scores in some cases, though in others it is shown to lower the expected item score. These outcomes are discussed with respect to validity and reliability of resulting total scores and also with respect to test content and examinee characteristics.
  • Thumbnail Image
    Item
    Psychometric properties of finite-state scores versus number-correct and formula scores: A simulation study
    (1989) García-Pérez, Miguel A.; Frary, Robert B.
    As developed by García-Pérez (1987), finite-state scores are nonlinear transformations of the proportions of conventional multiple-choice responses that are correct, incorrect, and omitted. They estimate the proportions of item alternatives which the examinees had the knowledge needed to classify (as correct or incorrect) before seeing them together in the items. The present study used simulation techniques to generate conventional test responses and to track the proportions of alternatives the examinees could classify independently before taking the test and the proportions they could classify after taking the test. Then the finite-state scores were computed and compared with these actual values and with number-correct and formula scores based on the conventional responses. Highly favorable results were obtained leading to recommendations for the use of finite-state scores. These results were almost the same when the simulation proceeded according to the model and when it was based on a naturalistic process completely independent of the model. Hence the scoring procedures on which finite-state scores are based are both accurate and robust. Index terms: applied measurement models, examinee behavior, finite-state scores, guessing, multiple-choice tests, test scoring.