Exploring a Framework for Consequential Validity for Performance-Based Assessments

Abstract

This study explores a new comprehensive framework for understanding elements of validity, specifically for performance assessments that are administered within specific and dynamic contexts. The adoption of edTPA is a good empirical case for examining the concept of consequential validity because this assessment has been implemented at the state level over several years in Minnesota, adding a much needed large-scale case to the literature on situated assessment validity (Moss, 1996). By drawing on the work of several measurement theorists (Frederiksen & Collins, 1989; Linn, Baker, & Dunbar, 1991; Uhlenbeck, Verloop, & Beijaard, 2002), I have constructed a framework of six dimensions for exploring the concept of consequential validity: 1) educational consequence, 2) meaningfulness, 3) directness, 4) transparency, 5) fairness, and 6) usability. This framework serves both as an analytic frame for data analysis and as our effort to further define and synthesize what is meant by consequential validity within the measurement and assessment field. This study used three data collection strategies: individual and group interviews with teacher candidates and teacher educators and a survey of teacher educators. 23 teacher candidates and 11 teacher educators were interviewed and they represented the edTPA participants at the University of Minnesota. In addition, 22 teacher educators from other institutions participated in a survey across Minnesota. Findings from this study indicate that teacher candidates and teacher educators identify both positive and negative consequences of edTPA based on its design and expectations. However, the consequences of edTPA were strongly affected by local implementation contexts. By using the six dimensions within the proposed framework, this study uncovers links between consequences and contexts. This suggests that implementation of a new assessment that is meant to be educative may also rely on developing a particular “assessment culture” in which data interpretation and use is a more commonly accepted practice.