A comparison of factor score indeterminacy when using sum score and common factor score estimators

Abstract

Social scientists routinely develop psychological instruments to measure individual differences using factor analysis without addressing the factor score indeterminacy (FSI) issue. The FSI, quantified as the correlation between latent traits and their estimates, is highly relevant for test development as it can affect the measurement and interpretation of latent traits. To better understand in what data-model conditions and to what extent FSI can affect trait measurement, I conducted a comprehensive simulation study that examined FSI when different factor score estimation methods were used. These methods included the unit-weighted, Thurstone’s, Bartlett’s, and ten Berge’s scores. In total, I simulated 47,520 data sets by varying the (a) salient factor size, (b) number of factor indicators, (c) probability of factor cross-loadings, (d) factor correlation size, (e) model approximation error, (f) sample size, and (g) shape of the item response distribution. My results showed that when trait measurements included many items with high factor loadings, the effect of FSI was generally minimized. In contrast, factor models with few small-loading items had large FSI. Based on these and other results, I provide several recommendations for reporting FSI estimates when using unit and non-unit scores in applied research.