The reliability and validity of objective indices of moral development

Abstract

The present paper addresses three issues surrounding Rest’s Defining Issues Test, an objective test of moral development based on Kohlberg’s six-stage theory of moral development. Those issues are (1) the stability of test scores over time; (2) correlation of scores with Kohlberg’s interview measure of moral development; and (3) the insensitivity of its scoring procedure, which ignores responses to all items keyed to lower stages. In two age heterogeneous samples, total score test-retest reliabilities were generally in the high .70’s or low .80’s, regardless of which of several scoring schemes was used. In another age heterogeneous sample, the correlation with scores on Kohlberg’s test was .70; but in two age homogeneous samples, the correlations were about .35 and .20. These validity coefficients suggest that (1) the common variance shared by Rest’s and Kohlberg’s tests in age heterogeneous samples can be attributed to the fact that scores on both tests increase with age and (2) the two tests cannot be considered equivalent measures of the same construct differing only in format. Results also indicated that an empirically weighted scoring scheme is more sensitive to longitudinal change than is Rest’s P score. This sensitivity to longitudinal trends is an important property for tests such as Rest’s which claim to be developmental and are frequently used to assess educational change. The empirically weighted sum had a significantly higher test-retest reliability (p < .05) than did a simple sum of item responses, and it had a significantly higher correlation with Kohlberg’s measure than did a theoretically weighted sum.