An Evaluation of the Accuracy of Time Series Interpretations of CBM-R Progress Monitoring Data

Abstract

Curriculum based measurement of reading (CBM-R) is used to monitor the effects of academic interventions for individual students. Decisions to continue, modify, or terminate instructional programs are made by interpreting patterns of observations collected across time. Educators visually analyze or apply decision rules to evaluate student progress. Despite the popularity of CBM-R as a progress monitoring tool, there is a paucity of research evaluating the accuracy of visual analysis and decision rules. Inaccurate interpretations undermine the use of CBM-R as a progress monitoring tool because educators may continue ineffective interventions or prematurely terminate effective interventions. The accuracy of visual analysis and decision rules were investigated in this project. In Study 1 a large extant dataset was analyzed to identify measurement characteristics of CBM-R progress monitoring data. In Study 2 the accuracy of visual analysis and decision rules were evaluated by comparing responses from visual analysts and decision rules with responses of an expert panel. One hundred eight progress monitoring graphs were evaluated in Study 2. The manner in which progress monitoring graphs differed was informed by the results of Study 1. The results of this project suggest evaluation method, number of weeks data are collected, variability of observations, and whether the student is making adequate progress influence the probability of correct decisions. Educators and researchers can improve the probability of correct decisions by visually analyzing progress monitoring graphs with a goal line and trend line, minimizing variability, and collecting data for longer than six weeks. The implications of the findings, limitations, and needs for future research are discussed.