Standardized Bilingual Assessments: A Means To Reduce Construct-Irrelevant Variance and Ethnic/Racial Stereotype Threat

Abstract

Every academic year, culturally and linguistically diverse students (CLDs) are misassigned and misidentified to special education, remedial classes, or less-efficient educational settings partially based on assessments that are not developed in the CLDs’ native languages. Latino students are not only misassigned in higher proportions to lower tier math classes, they continue to show assessment scores that lag behind their monolingual White peers by sizable effect sizes. These differences are even more prominent for Latino students who live in homes where Spanish is the primary language. Math assessments in the United States are usually developed with norming samples of White students, and include language that is complex to CLDs. Moreover, significantly more Latino students are taking high stake assessments than a decade ago, but, there are no signs of improving the performance gaps with their White peers. The unintentional language complexity introduces a source of construct-irrelevant variance when assessing mathematical ability of Latino students because math ability is not the only construct being measured. The language complexity dimension is not as readily accessible to Latino students—especially English learners—as it is to their White peers. To make matters worse, it is clear from the growing body of research on stereotype threat that negative stereotypes systematically and negatively affect how students of color perform on intellectual, cognitive, and scholastic assessments. In order to maximize the optimal testing conditions for CLDs, while minimizing construct-irrelevant factors, a bilingual assessment was used in this study to investigate whether it would (a) reduce measurement invariance, (b) lessen the stereotype threat-activating cues of the diagnostic assessment, and (c) preserve the construct-related validity and reliability standards of the assessment. A 2 (original language of assessment form) x 2 (language adaptation) x 3 (ethnic/racial stereotype threat) x 3 (gender stereotype threat) between-subjects quasi-experimental factorial design was used. A total of 449 Latino participants from grades 10, 11, and 12 partook in the study; 54.8% male; 61.9% second-generation Latino; and, 26.9% English learners. Results of the study highlighted that the complex (English-only) language increased the difficulty of the assessments by introducing construct-irrelevant variance, posing a serious threat to validity. Specifically, participants who received the bilingual form of the English-only assessment scored on average .38 logits higher than those who received the English-only form (Cohen’s d = 0.33, 95% CI[0.08, 0.57]). This language adaptation effect was still statistically significant, b = 0.37, t(204) = 2.46, p < .05, after controlling for generational status, gender, EL status, grade, English proficiency, Spanish proficiency, grade point average, Students Like Learning Mathematics factor scores, and ethnic/racial and gender stereotype threat in a multiple regression analysis, explaining 2.12% of additional variance. It was also found that bilingual assessments reduce ethnic/racial stereotype threat (ERST) effects (g = 0.48 [0.04, 0.91] and g = 0.45 [0.02, 0.88], when ERST was activated or in a control condition, respectively). Lastly, the construct-related validity and reliability evidence highlighted that bilingual assessment forms do not have an adverse impact on the construct being measured. Utilizing a bilingual assessment to measure mathematical ability of CLDs would indirectly improve the measurement qualities of a mathematical assessment, that is, higher classical item difficulty values, implying a more accurate measurement of math ability. Not to mention the positive outcomes that could be seen in primary and secondary classrooms where ELs would not be held back in subject areas that, possibly, they had already mastered in their home countries, or, simply, would not be misassigned to special education services or remedial classes. These positive outcomes would also be evident when EL juniors and seniors who have recently arrived to the United States would be able to show their knowledge of mathematics on national assessments such as the SAT and ACT, paving way to possible scholarships and fellowships based on their mathematical ability. In all, the use of bilingual assessments will provide equitable and fair access to learning opportunities for our culturally and linguistically diverse students.