A computer simulation study was designed to investigate
the extent to which the interrater reliability
of a clinical scale is affected by the number of categories
or scale points (2, 3, 4, ... ,100). Results indicate
that reliability increases steadily up to 7 scale
points, beyond which no substantial increases occur,
even when the number of scale points is increased to
as many as 100. These findings hold under the following
conditions: (1) The research investigator has insufficient
a priori knowledge to use as a reliable guideline
for deciding on an appropriate number of scale
points to employ, and (2) the dichotomous and ordinal
categories being considered all have an underlying
metric or continuous scale format.
Cicchetti, Domenic V, Showalter, Donald & Tyrer, Peter J. (1985). The effect of number of rating scale categories on levels of interrater reliability: A monte carlo investigation. Applied Psychological Measurement, 9, 31-36. doi:10.1177/014662168500900103
Cicchetti, Domenic V.; Showalter, Donald; Tyrer, Peter J..
The effect of number of rating scale categories on levels of interrater reliability: A monte carlo investigation.
Retrieved from the University of Minnesota Digital Conservancy,
Content distributed via the University of Minnesota's Digital Conservancy may be subject to additional license and use restrictions applied by the depositor.