Methodology review: Assessing unidimensionality of tests and items

Abstract

Various methods for determining unidimensionality are reviewed and the rationale of these methods is assessed. Indices based on answer patterns, reliability, components and factor analysis, and latent traits are reviewed. It is shown that many of the indices lack a rationale, and that many are adjustments of a previous index to take into account some criticisms of it. After reviewing many indices, it is suggested that those based on the size of residuals after fitting a two- or three-parameter latent trait model may be the most useful to detect unidimensionality. An attempt is made to clarify the term unidimensional, and it is shown how it differs from other terms often used interchangeably such as reliability, internal consistency, and homogeneity. Reliability is defined as the ratio of true score variance to observed score variance. Internal consistency denotes a group of methods that are intended to estimate reliability, are based on the variances and the covariances of test items, and depend on only one administration of a test. Homogeneity seems to refer more specifically to the similarity of the item correlations, but the term is often used as a synonym for unidimensionality. The usefulness of the terms internal consistency and homogeneity is questioned. Unidimensionality is defined as the existence of one latent trait underlying the data.