Correlated effects in generalizability studies

Abstract

The analytical model typically used to perform generalizability analysis assumes that design effects are uncorrelated. Often, the assessment of behavioral data involves designs that employ multiple occasions or repeated trials (as in many observational and rating studies). In these cases, design effects may be serially correlated. The implications of serially correlated effects on the results of generalizability analyses are discussed. Simulated data are provided that demonstrate the biases that serially correlated effects introduce into the results. Index terms: correlated effects, estimation of variance components, generalizability theory, observational studies, repeated trials, serial correlation.