Several studies have compared different judgmental
methods of setting passing scores by estimating item
difficulties for the minimally competent examinee.
Usually, a direct method of estimating item difficulties
has been compared with an indirect method suggested
by Nedelsky (1954). Nedelsky’s method has usually
resulted in a substantially lower cutoff score than that
arrived at with a direct method. Two studies were carried
out for the purpose of comparing a direct method
of setting passing scores with an indirect method that
allowed judges to estimate the probability of the minimally
competent examinee eliminating each incorrect
alternative. In Study 1 a sample of 52 first-level supervisors
used both methods to estimate passing scores
on a content-oriented selection test for building maintenance
specialists. In Study 2 a sample of 62 first-level
supervisors used both methods to estimate passing
scores on an entry level auto mechanics test. Results
of both studies showed that the variance component
for method was relatively small and that for
raters was relatively large. Reliability estimates of
judgments and correlations between judged difficulties
and empirical difficulties showed the Angoff (1971)
approach to be slightly superior. Results showed no
particular advantage to using an indirect approach for
estimating minimal competence.
Recently, the problem of setting passing scores
Reilly, Richard R, Zink, Donald L & Israelski, Edmond W. (1984). Comparison of direct and indirect methods for setting minimum passing scores. Applied Psychological Measurement, 8, 421-429. doi:10.1177/014662168400800406
Reilly, Richard R.; Zink, Donald L.; Israelski, Edmond W..
Comparison of direct and indirect methods for setting minimum passing scores.
Retrieved from the University of Minnesota Digital Conservancy,
Content distributed via the University of Minnesota's Digital Conservancy may be subject to additional license and use restrictions applied by the depositor.