Mathematics Computation: Generalizability and Dependability of Student Performance By Sample Size

Abstract

The purpose of this study was to estimate the utility of general outcome measures and targeted skill measures to answer questions related to the mastery of mathematics computation skills. Specifically, this study used skills analysis to analyze student performance on curriculum-based measures in mathematics (CBM-M) with special attention to skill. Participants included 488,572 students in second and third grade across a national sample of participants. Generalizability theory was used to investigate the reliability of student mathematics computation performance samples. Generalizability studies were conducted to estimate the amount of variance in student performance associated with person, form, item (problem), and the interactions between each of these facets. Decision studies were conducted to determine reliability and standard error of measurement (SEM) estimates for various student performance samples, in terms of both rank order reliability and absolute score reliability. The results of this study provide an estimate of the size of a performance sample required to make reliable and valid decisions to guide instructional planning.