Technical characteristics of e-based vs. paper-pencil CBM tasks for students who are deaf and hard of hearing

Abstract

To increase literacy outcomes for students who are deaf and hard of hearing, professionals need to provide high quality instruction that is informed by accurate and valid assessment data. This study compared the reliability and validity of student scores from paper-pencil and e-based assessments, “Maze” and “Slash.” Forty (N=40) students, who were deaf or hard of hearing and read between the second and fifth grade reading level, participated. Twenty-one teachers of students who are deaf and hard of hearing also participated. For Maze, alternate form reliability coefficients obtained from correct scores and correct scores adjusted for guessing ranged from .61 to .84 (ps < .01); criterion-related validity ranged from .33 to .67 (majority of ps < .01). These findings are generally consistent with findings from previous research. For Slash, alternate form reliability coefficients obtained from correct scores ranged from .50 to .75 (ps < .01); criterion-related validity ranged from .25 to .72. The extent to which testing modifications delivered in an electronic-based (e-based) format influenced student scores was also examined. Differences between paper-pencil and e-based conditions were generally non-significant for Maze; significant differences between conditions for Slash favored the paper-pencil condition. Overall, findings suggest that Maze holds promise for use with students who are deaf and hard of hearing in both conditions, with inconclusive results for Slash. Future research is needed to explore the impact of providing testing modifications through e-based progress monitoring tools.