The power of significance tests based on difference
scores is indirectly influenced by the reliability
of the measures from which differences are obtained.
Reliability depends on the relative magnitude of
true score and error score variance, but statistical
power is a function of the absolute magnitude of
these components. Explicit power calculations
reaffirm the paradox put forward by Overall &
Woodward (1975, 1976)-that significance tests of
differences can be powerful even if the reliability
of the difference scores is 0. This anomaly arises
because power is a function of observed score
variance but is not a function of reliability unless
either true score variance or error score variance is
constant. Provided that sample size, significance
level, directionality, and the alternative hypothesis
associated with a significance test remain the same,
power always increases when population variance
decreases, independently of reliability. Index
terms: difference scores, error of measurement, power,
significance tests, t test, test reliability, true scores.
Zimmerman, Donald W, Williams, Richard H & Zumbo, Bruno D. (1993). Reliability of measurement and power of significance tests based on differences. Applied Psychological Measurement, 17, 1-9. doi:10.1177/014662169301700101
Zimmerman, Donald W.; Williams, Richard H.; Zumbo, Bruno D..
Reliability of measurement and power of significance tests based on differences.
Retrieved from the University of Minnesota Digital Conservancy,
Content distributed via the University of Minnesota's Digital Conservancy may be subject to additional license and use restrictions applied by the depositor.