A new method for evaluating the content
representation of a test is illustrated. Item similarity
ratings were obtained from content domain experts
in order to assess whether their ratings corresponded
to item groupings specified in the test
blueprint. Three expert judges rated the similarity
of items on a 30-item multiple-choice test of study
skills. The similarity data were analyzed using a
multidimensional scaling (MDS) procedure followed
by a hierarchical cluster analysis of the MDS
stimulus coordinates. The results indicated a strong
correspondence between the similarity data and the
arrangement of items as prescribed in the test
blueprint. The findings suggest that analyzing item
similarity data with MDS and cluster analysis can
provide substantive information pertaining to the
content representation of a test. The advantages
and disadvantages of using MDS and cluster
analysis with item similarity data are discussed.
Index terms: cluster analysis, content validity,
multidimensional scaling, similarity data, test
Sireci, Stephen G & Geisinger, Kurt F. (1992). Analyzing test content using cluster analysis and multidimensional scaling. Applied Psychological Measurement, 16, 17-31. doi:10.1177/014662169201600102
Sireci, Stephen G.; Geisinger, Kurt F..
Analyzing test content using cluster analysis and multidimensional scaling.
Retrieved from the University of Minnesota Digital Conservancy,
Content distributed via the University of Minnesota's Digital Conservancy may be subject to additional license and use restrictions applied by the depositor.