Evaluating reading diagnostic tests: An application of confirmatory factor analysis to multitrait-multimethod data

Abstract

Diagnostic reading tests, in contrast to achievement tests, claim to measure specific components of ability hypothesized to be important for diagnosis or remediation. A minimal condition for demonstrating the construct validity of such tests is that they are able to differentiate validly between the reading traits that they claim to measure (e.g., comprehension, sound discrimination, blending). This condition is rarely tested, but multitrait-multimethod (MTMM) designs are ideally suited for this purpose. This is demonstrated in two studies based on the 1966 version of the Stanford Diagnostic Reading Test (SDRT). In each study, the application of the Campbell-Fiske guidelines and confirmatory factor analysis (CFA) to the MTMM data indicated that the SDRT subscales could be explained in terms of a method/halo effect and a general reading factor that was not specific to any of the subscales; this refutes the construct validity of the 1966 version of the SDRT as a diagnostic test. Other diagnostic tests probably suffer the same weakness and should also be evaluated in MTMM studies.