Modeling Response Processes in Early Literacy Measures

Abstract

Response processes are an important component of validity to support the use and interpretation of test scores. Response processes information can provide insight into how students engage with assessment tasks and the type of errors made when solving items, as well as allow for the study of cognitive properties in items that may be associated with item parameters, and potential differences in interpretation across relevant groups of examinees. Empirical data on response processes typically come from information based on think-aloud protocols, response time information, response process logs from computer-administered tests, eye-tracking studies, and item responses themselves. However, findings from empirical studies are rarely provided as part of the evidence to support the validity argument. The purpose of this study was to examine the applicability of explanatory IRT modeling in detecting the impact of response processes, as modeled by item and person characteristics, on the functioning of a Spanish early literacy measure designed for Spanish-English Dual Language Learner (SE-DLL) preschool students. Specifically, I examined three construct-relevant properties in the response options of multiple-choice items of a phonological awareness measure, and how such properties may elicit different responses in students from two different language of exposure backgrounds, Spanish vs. both Spanish and English. Results from this study included main effects for an item property and differences in performance across the two subgroups. In this case, the results provided validity evidence based on response processes that can serve as guidance for test development and item design, and potentially contribute to existing theories of DLL language development.