Between Dec 19, 2024 and Jan 2, 2025, datasets can be submitted to DRUM but will not be processed until after the break. Staff will not be available to answer email during this period, and will not be able to provide DOIs until after Jan 2. If you are in need of a DOI during this period, consider Dryad or OpenICPSR. Submission responses to the UDC may also be delayed during this time.

Browsing by Author "Cicchetti, Domenic V."

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Comparison of the null distributions of weighted kappa and the C ordinal statistic
    (1977) Cicchetti, Domenic V.; Fleiss, Joseph L.
    It frequently occurs in psychological research that an investigator is interested in assessing the extent of interrater agreement when the data are measured on an ordinal scale. This monte carlo study demonstrates that the appropriate statistic to apply is weighted kappa with its revised standard error. The study also demonstrates that the minimal number of cases required for the valid application of weighted kappa varies between 20 and 100, depending upon the size of the ordinal scale. This contrasts with a previously cited large sample estimate of 200. Given the difficulty of obtaining sample sizes this large, the latter finding should be of some comfort to investigators who use weighted kappa to measure interrater consensus.
  • Thumbnail Image
    Item
    The effect of number of rating scale categories on levels of interrater reliability: A monte carlo investigation
    (1985) Cicchetti, Domenic V.; Showalter, Donald; Tyrer, Peter J.
    A computer simulation study was designed to investigate the extent to which the interrater reliability of a clinical scale is affected by the number of categories or scale points (2, 3, 4, ... ,100). Results indicate that reliability increases steadily up to 7 scale points, beyond which no substantial increases occur, even when the number of scale points is increased to as many as 100. These findings hold under the following conditions: (1) The research investigator has insufficient a priori knowledge to use as a reliable guideline for deciding on an appropriate number of scale points to employ, and (2) the dichotomous and ordinal categories being considered all have an underlying metric or continuous scale format.
  • Thumbnail Image
    Item
    Inference about weighted kappa in the non-null case
    (1978) Fleiss, Joseph L.; Cicchetti, Domenic V.
    The accuracy of the large sample standard error of weighted kappa appropriate to the non-null case was studied by computer simulation. Results indicate that only moderate sample sizes are required to test the hypothesis that two independently derived estimates of weighted kappa are equal. However, in most instances the minimal sample sizes required for setting confidence limits around a single value of weighted kappa are inordinately large. An alternative, but as yet untested procedure for setting confidence limits, is suggested as being potentially more accurate.
  • Thumbnail Image
    Item
    Testing the normal approximation and minimal sample size requirements of weighted kappa when the number of categories is large
    (1981) Cicchetti, Domenic V.
    The results of this computer simulation study indicate that the weighted kappa statistic, employing a standard error developed by Fleiss, Cohen, and Everitt (1969), holds for a large number of k categories of classification (e.g., 8 ≤ k ≤ 10). These data are entirely consistent with an earlier study (Cicchetti & Fleiss, 1977), which showed the same results for 3 ≤ k ≤ 7. The two studies also indicate that the minimal N required for the valid application of weighted kappa can be easily approximated by the simple formula 2k². This produces sample sizes that vary between a low of about 20 (when k = 3) to a high of about 200 (when k = 10). Finally, the range 3 ≤ k ≤ 10 should encompass most extant clinical scales of classification.