Individual inconsistency: Implications for test reliability and behavioral predictability

Abstract

The nature of individual inconsistency in performance on trait measurements is an important topic in psychometrics because of its direct relevance to measurement reliability. Several studies have supported short-term inconsistency as a systematic source of variation among individuals by finding some evidence for generalizability and relationship to behavioral predictability. However, these findings are questionable, since these studies confounded change with short-term fluctuation in their response inconsistency measure. The current research separates these two sources of inconsistency in a reanalysis of the data from one major study on short-term consistency and finds little evidence for generalizability or a relationship to behavioral predictability. These results support the popular assumption that measurement error from short-term fluctuations is not due to systematic individual differences in response consistency, as well as supporting a more limited definition of the individual inconsistency construct.