Ten Core Competencies Assessment Scales (CCAS): Validity Evidence for Measures of Core Competencies in Liberal Arts Education

Abstract

It is generally accepted that college education in general, and liberal arts education in particular, helps students develop important career and world readiness competencies (Boyer 2030 Commission, 2023). There is, however, no well established instrument supporting a scalable approach to the reliable and valid assessment of students' development of these competencies. In a series of eight studies involving data from more than 55,000 undergraduate students, we report on the development of an assessment instrument for ten core career and world readiness competencies, from theoretical conceptualization to demonstrating predictive validity of the instrument. Specifically, we collected four sources of validity evidence that support score interpretations and uses: (a) validity evidence based on the assessment content, (b) validity evidence based on response processes, (c) validity evidence based on internal structure, and (d) validity evidence based on association with other variables. Stated differently, we evaluated the psychometric properties of the assessment scales, demonstrated their factor structures, score reliability, and measurement invariance across demographic groups. We then present empirical evidence that shows scores obtained from the scales are not only reliable, but also have convergent, discriminant, and predictive validity. We conclude that score interpretations from the ten Core Competency Assessment Scales developed as part of the RATE™ (reflect, articulate, translate, evaluate) tool are a reliable, valid, and scalable solution for institutions interested in assessing their students’ competency development to aid student learning and document institutional effectiveness.