Issues of validity for criterion-referenced measures

Abstract

It has sometimes been assumed that validity of criterion-referenced tests is guaranteed by the definition of the domain and the process used to generate items. These are important considerations for content validity. It is argued that the proper focus for content validity is on the items of a test rather than on examinee responses to those items. Content validity is important for criterion-referenced measures, but it is not sufficient. This claim is discussed and the case is made that interpretations and uses of criterion-referenced tests require support of other kinds of evidence and logical analysis. The inferences that are made should dictate the kinds of evidence and logical arguments that are needed to support claims of validity. Illustrations of aspects of the validation process are provided in two concrete examples.