A large number of seemingly diverse coefficients
have been proposed as indices of dependability, or
reliability, for domain-referenced and/or mastery
tests. In this paper it is shown that most of these
indices are special cases of two generalized indices
of agreement-one that is corrected for chance and
one that is not. The special cases of these two indices
are determined by assumptions about the nature
of the agreement function or, equivalently, the
nature of the loss function for the testing procedure.
For example, indices discussed by Huynh
(1976), Subkoviak (1976), and Swaminathan,
Hambleton, and Algina (1974) employ a threshold
agreement, or loss, function; whereas, indices discussed
by Brennan and Kane (1977a, 1977b) and
Livingston (1972a) employ a squared-error loss
function. Since all of these indices are discussed
within a single general framework, the differences
among them in their assumptions, properties, and
uses can be exhibited clearly. For purposes of comparison,
norm-referenced generalizability coefficients
are also developed and discussed within this
Kane, Michael T & Brennan, Robert L. (1980). Agreement coefficients as indices of dependability for domain-referenced tests. Applied Psychological Measurement, 4, 105-126. doi:10.1177/014662168000400111
Kane, Michael T.; Brennan, Robert L..
Agreement coefficients as indices of dependability for domain-referenced tests.
Retrieved from the University of Minnesota Digital Conservancy,
Content distributed via the University of Minnesota's Digital Conservancy may be subject to additional license and use restrictions applied by the depositor.