Measuring speed of numerical reasoning.

Abstract

If numerical reasoning items are administered with time limits, will two dimensions be required to account for the responses, a numerical ability dimension and a speed dimension? If we want to know how quickly a person solves a problem, how can we obtain a reliable measure of speed? This study reanalyzed the data collected by Semmes, Davison, & Close (2009) in which one hundred and eighty-one college students answered 74 numerical reasoning items. Every item was administered with and without a time limit by half of the students. Three two-dimensional models were fit to item responses under self-paced and experimenter-paced conditions and response times under self-paced administrations. The best fitting model suggested that, other than the Level dimension, a second Speed dimension was needed to account for variation in numerical reasoning performance under experimenter-paced administration. After adding response time to the model, we saw a significant increase in the reliability estimate for the Speed factor compared to prior research with the same data, but estimating speed scores using only the experimenter-paced responses (Semmes et al., 2009). The validity of the Speed dimension was supported by its unique contribution to the prediction of ACT scores after controlling for the variation accounted for by the Level dimension. An alternative method of measuring Speed is mentioned. Some previous research using response times for other purposes besides measurement of speed are also discussed.