Problems with individual difference measures based on some componential cognitive paradigms

Abstract

This article demonstrates that slope and ratio scores may have the same psychometric difficulties—low reliability—as difference scores. Empirically, direct measures and derived scores from Baron’s, Collins’, Meyer’s, and Posner’s cognitive paradigms were examined in terms of their reliabilities and cross-correlations. Reliabilities of the direct measures and their intercorrelations were high. The derived measures, which were slope, ratio, and difference scores, had reliabilities near zero and, therefore, their cross-correlations were also low. It is concluded that derived scores, although intuitively appealing as measures of mental operations, may have inherent psychometric difficulties that render them of little value for differential prediction. Index terms: cognitive paradigms, difference scores, individual differences, prediction, ratio scores, reliability, slope scores.