Scoring method and the detection of person misfit in a personality assessment context

Abstract

The purpose of this research was to explore psychometric issues pertinent to the application of an IRT-based person-fit (response aberrancy) detection statistic in the personality measurement domain. Monte carlo data analyses were conducted to address issues regarding lz the person-fit statistic. The major issues explored were characteristics of the null distribution of lz and its power to identify nonfitting response patterns under different scoring strategies. There were two main results. First, the lz index null distribution was not well standardized when item parameters of personality scales were used; the lz null distribution variance was significantly less than the hypothesized value of 1.0 under several conditions. Second, the power of lz to detect response misfit was affected by the scoring method. Detection power was optimal when a biweight estimator of θ was used. Recommendations are made regarding proper implementation of person-fit statistics in personality measurement. Index terms: appropriateness measurement, item response theory, lz statistic, person fit, personality assessment, response aberrancy, scoring methods, two-parameter model.