Effects of test preparation on the validity of a graduate admissions test

Abstract

Test score improvement has been the major concern in nearly all the extant studies of special preparation, or "coaching," for tests. Recently, however, logical analyses of the possible outcomes and implications of special test preparation (Anastasi, 1981; Cole, 1982; Messick, 1981) have suggested that the issue of test score effects is but one aspect of the controversy surrounding coaching; the impact of special preparation on test validity is an equally germane consideration. Although the assumption is sometimes made that coaching can serve only to dilute the construct validity and impair the predictive power of a test, some kinds of special preparation may, by reducing irrelevant sources of test difficulty, actually improve both construct validity and predictive validity. This study examined the relationships of both internal and external criteria to Graduate Record Examination (GRE) candidates’ performance on several analytical ability item types, obtained under several test preparation conditions. The purpose was to assess the effects of these various preparations on test reliability and validity. The preparation conditions were those previously shown to be effective, in varying degrees, in improving examinee performance on two of three analytical item types (Powers & Swinton, 1982, 1984). The data for this study were those collected by Powers & Swinton (1982, 1984). The results suggest that GRE analytical ability scores may relate more strongly to academic performance after special test preparation than under more standard conditions and that they may relate less to measures of other cognitive abilities (verbal and quantitative scores). No consistent effects were detected on either the internal consistency or the convergent validity of the analytical measure.