Sources of Variance In Reading Comprehension Research: the Role of Measures and Interventions

Abstract

The purpose of this study was to examine if differences in reading comprehension measures’ response formats were associated with differential outcomes for reading comprehension interventions. Specifically, this study used meta-analysis to evaluate the overall treatment effect of reading comprehension interventions, the association between a measure’s response format and measured intervention outcomes, and whether specific intervention effects varied based on the measure’s response format. A systematic review of the literature identified 66 published and unpublished research reports and studies conducted since 2000. All studies administered a reading comprehension intervention for students in the primary grades and measured the effects using a reading comprehension measure. Meta-analytic findings suggested an overall positive effect of reading comprehension interventions for both intervention to control group comparisons at posttest (Hedge’s g = 0.20) and pretest to posttest comparisons in the intervention group (Hedge’s g = 0.71). The response format of a reading comprehension measure, specifically retell/summary formats, was significantly associated with intervention outcomes, even after controlling for purposively selected variables. Findings also indicated that improving background knowledge and multicomponent interventions were significantly associated with performance on measures of reading comprehension with retell/summary response formats. The results of this study provide additional evidence that measures using the retell/summary response formats value reading comprehension differently, specifically in the context of interventions. Findings may also be used to caution against the interchangeable use of retell/summary formats with other measures of reading comprehension.