Toward the fair and valid use of curriculum-based measurement for students with intensive writing needs and linguistically diverse backgrounds.

Abstract

Curriculum-based measurement (CBM) is a valuable assessment method for students with intensive learning needs, including in writing. Despite research on CBMs in writing, insufficient attention has been given to linguistic diversity, especially among young or beginning writers, leaving uncertainty about whether current assessment practices with writing CBMs provide reliable and valid information on multilingual students’ early writing development in English. The purpose of this study was to evaluate the measurement invariance of Word Dictation, a specific CBM writing task designed to assess English transcription (spelling and handwriting) skills at the word level, across two groups: multilingual and English-monolingual students, both with intensive needs in writing. This study used data obtained from a large multi-site, multi-year randomized control trial, which investigated the effects of a professional development program designed to support elementary teachers in implementing data-based instruction for struggling early writers. This study analyzed Word Dictation responses from 349 students, including 67 multilingual and 282 English-monolingual students, primarily in Grades 1–3. In evaluating measurement invariance at both item and assessment levels, I used different scoring metrics, specifically words spelled correctly and correct letter sequences, and employed a range of analytical methods, including those based on item response theory and classical test theory frameworks. Within the current study context, I identified a few items as potentially displaying differential item functioning, but the magnitude was overall small. The direction of the detected differential item functioning was not systematic, with some items favoring the multilingual student group and others favoring the English-monolingual student group. When comparing the scoring metrics, some differential item functioning observed in words spelled correctly was mitigated when using correct letter sequences. At the assessment level, Word Dictation did not function differently across the two student groups. These findings build upon existing knowledge of the technical adequacy of Word Dictation, incorporating varying scoring metrics, and further expand by providing evidence of validity and fairness for multilingual students learning to write in English. The discussion also highlights significant limitations of this study. These include grouping of students with diverse native languages into a single category of multilingual students, analyzing subsets of items rather than the entire set, and limitations in the specific analytic approach, such as limitations in selecting anchor items for analyses using correct letter sequences. With these limitations in mind, I discuss future research directions and implications for educators using Word Dictation to better serve linguistically diverse students requiring intensive support in developing their writing skills in English.