Quantifying the Biasing Effect of Rapid Guessing on Estimates of Coefficient Alpha

Abstract

An underlying assumption of coefficient alpha is that random error is uncorrelated; however, this assumption is violated when examinees engage in similar construct-irrelevant behaviors across items. One construct-irrelevant behavior that has gained increased attention in the literature is rapid guessing (RG), which occurs when examinees answer quickly with intentional disregard for item content. To examine the extent that estimates of coefficient alpha are biased due to RG, a simulation study was conducted in which the ability characteristics of rapid responders and the percentage and pattern of RG were manipulated. After controlling for test length and difficulty, results indicated that RG characteristics had a practically negligible impact on estimates of coefficient alpha, with the average degree of bias found to range from -.05 to .02 for upwards of 30% of RG responses in the data. This negligible effect was supported in a meta-analytic investigation, which observed a difference in coefficient alpha of .07 when comparing filtered (i.e., RG responses removed) and unfiltered (i.e., RG responses left in the data) datasets across five studies and 12 effect sizes. These findings suggest that estimates of coefficient alpha are largely robust to violations of the assumption that random error is uncorrelated due to construct-irrelevant behaviors such as RG.