Applied Psychological Measurement, Volume 12, 1988

Persistent link for this collection

https://hdl.handle.net/11299/103300

Search within Applied Psychological Measurement, Volume 12, 1988

Browse

Now showing 1 - 20 of 34

Louis Guttman 1916-1987
(1988)
An iterative procedure for linking metrics and assessing item bias in item response theory
(1988) Candell, Gregory L.; Drasgow, Fritz
The presence of biased items may seriously affect methods used to link metrics in item response theory. An iterative procedure designed to minimize this methodological problem was examined in a monte carlo investigation using the two-parameter item response model. The iterative procedure links the scales of independently calibrated parameter estimates using only those items identified as unbiased. Two methods for transforming parameter estimates to a common metric were incorporated into the iterative procedure. The first method links scales by equating the first two moments of the distributions of estimated item difficulties. The second method determines the linking transformation by minimizing differences across IRT characteristic curve estimates. Results indicate that iterative linking provides a substantial improvement in item bias detection over the noniterative approach. Index terms: Item bias, Item response theory, Iterative method, Linking, Metric linking, Two-parameter item response model.
Set correlation and contingency tables
(1988) Cohen, Jacob
Set correlation is a realization of the general multivariate linear model, can be viewed as a multivariate generalization of multiple correlation analysis, and may be employed in the analysis of multivariate data in any form. Set correlation supplements the four methods for analyzing two-way contingency tables described by Zwick and Cramer (1986), and its application to their example is illustrated. It gives the same results for the overall association, and in addition, by the use of nominal scale coding and partialling, it assesses specific hypotheses about the details of the association. Set correlation includes measures of strength of association (including correlations and proportions of variance), significance tests and estimation, power analysis, and computer programs to implement the calculations. Index terms: canonical analysis, contingency table analysis, correspondence analysis, general multivariate linear model, multivariate analysis of variance, Pearson chi-square, set correlation.
An alternative procedure for assessing convergent and discriminant validity
(1988) Lehmann, Donald R.
This paper presents a simple procedure for establishing convergent and discriminant validity. The method uses ordinary least-squares regression (OLS) with the correlations between measures as the dependent variable. Illustrations indicate that the method provides reasonable answers. Index terms: convergent validity, discriminant validity, multiple regression.
Measuring attitudes with a threshold model drawing on a traditional scaling concept
(1988) Rost, Jürgen
This paper presents a generalized Rasch model for measuring attitudes which is based on the concepts of Thurstone’s method of successive intervals. The model combines the rating scale and the dispersion model proposed by Andrich and a submodel of the partial credit model proposed by Masters. An estimation procedure for unconditional maximum likelihood (ML) estimates is outlined. A recursion formula for the symmetric functions, which is needed for conditional ML procedures, is given. The benefits of the model are illustrated with a study on students’ interest in physics. The fit of different threshold models can be compared using conditional likelihood values and conditional likelihood ratio tests. Index terms: attitude measurement, conditional likelihood ratio test, partial credit model, Rasch model, rating scales, successive intervals, threshold model.
The item log-likelihood surface for two- and three-parameter item characteristic curve models
(1988) Baker, Frank B.
This article investigated the form of item log-likelihood surface under two- and three-parameter logistic models. Graphs of the log-likelihood surfaces for items under two-parameter and three-parameter (with a fixed value of c) models were very similar, but were characterized by the presence of a ridge. These graphs suggest that the task of finding the maximum of the surface should be roughly equivalent under these two models when c is fixed in the three-parameter model. For two items, the item log-likelihood surface was plotted for several values of c to obtain the contour line of the maxima. For an item whose value of Lord's b − 2/a index was less than the criterion value, the contour line was relatively flat. The item having an index value above the criterion value had a contour line with a very sharp peak. Thus, under a three-parameter model, finding the maximum of the item log-likelihood is more difficult when the criterion for Lord’s index is not met. These results confirm that the LOGIST program procedures used to locate the maximum of the likelihood function are consistent with the form of the item log-likelihood surface. Index terms: estimation, item parameter; likelihood surfaces; LOGIST procedures; log-likelihood; maximum likelihood estimation.
An approach to assessing unidimensionality revisited
(1988) Bejar, Isaac I.
A reanalysis of data from Hambleton and Rovinelli (1986) argues that the methods suggested by Bejar (1980) are a valuable descriptive tool for assessing the unidimensionality assumption when a priori information is available about possible response factors. Index terms: achievement testing, item response theory, unidimensionality.
Unidimensionality versus statistical accuracy: A note on Bejar's method for detecting dimensionality of achievement tests
(1988) Liou, Michelle
A simulation investigated use of the difficulty parameter (Bejar, 1980) to evaluate item unidimensionality. Artificial tests were designed to be nonequivalent in both length and dimensionality. Simulated item responses to the tests were analyzed with the LOGIST computer program. Two indices were calculated: the slope of the principal axis between the content-areabased item difficulty estimates and corresponding total- test-based estimates, and the correlation between the two sets of estimates. Results show that the magnitude of the correlation coefficient provides no information about dimensionality of a set of test items. The slope of the principal axis, on the other hand, is sensitive to multidimensionality in the data as well as test length. The size of the slope adequately detects the dimensionality of items for relatively long tests. Index terms: equating, item difficulty parameters, item response theory, unidimensionality.
IRT item bias detection procedures: Issues of model misspecification, robustness, and parameter linking
(1988) Lautenschlager, Gary J.; Park, Dong-gun
This article examines the consequences of employing IRT item bias detection procedures with multidimensional IRT item data. Parameter linking methods used in previous studies of item bias were investigated in a simulation that minimized the need for such linking. The results illustrate shortcomings of two linking methods that have been employed in IRT item bias detection studies. The effectiveness of these methods depended on several factors, including the number of biased items in a fixed-length test, whether bias was against only one group or more than one group, and the correlation between the two latent abilities. The findings indicated that some current IRT-based statistical procedures for detecting item bias were not generally effective at differentiating biased from unbiased items. Index terms: item bias, item response theory, multidimensional IRT data, parameter linking, reverse bias, statistical artifacts.
Detecting and interpreting local item dependence using a family of Rasch models
(1988) Wilson, Mark
This paper describes a method for detecting and interpreting disturbances of the local independence assumption among items that share common stimulus material or other substantive features. Dichotomous and polytomous Rasch models are used in an example to analyze Structure of the Learning Outcome (SOLO) superitems and examine the results for local independence problems. The results indicate that some disturbances were present among particular subsets of the items. Index terms: local independence, partial credit model, one-parameter logistic model, Rasch model, rating scale model.
Acquiescence in components analysis and multidimensional scaling of self-rating items
(1988) Davison, Mark L.; Srichantra, Niyada
Earlier work has shown that when multidimensional scaling (MDS) is applied to item intercorrelations, metric MDS implicitly subtracts the standardized person mean (SPM) from responses. As a result, when a metric or nonmetric MDS solution is compared to a components solution, the components solution often contains one component with no counterpart among the scaling dimensions. If self-report items form a balanced scale and negatively worded items are not reverse scored, the SPM is closely related to several concepts of acquiescence and disacquiescence. In this paper, MDS and components solutions are compared using two balanced self-report item sets. In the Likert self-report attitude item set, the MDS and components solutions were essentially the same. In a set of affective well-being items, the components solution contained a general component with no counterpart among the scaling dimensions. Scores along the general component were substantially correlated with measures of acquiescence and disacquiescence. Results in the second dataset suggest that when the self-report items are balanced and the negatively worded items have not been reverse scored, MDS and components solutions can differ largely with respect to a component closely associated with certain measures of acquiescence. Index terms: acquiescence, attitude measurement, factor analysis, multidimensional scaling, personality measurement, response bias, self-report items.
Theories, models, and standard systems of measurement
(1988) Aftanas, Marion S.
Measurement theories in psychology may be classified in terms of whether they begin from a general measurement framework or from a specific area of measurement. Points of contact between theories and different specific measurement areas have been limited by the choice of focus in discussions of general measurement theories and specific theories or models. This presentation outlines a metatheoretical framework that begins with the obvious common factor in all areas, the standard system of measurement. Just as a standard is a commonly accepted unit of measurement, a standard system is a commonly accepted mechanism of measurement for a given area. The concept of a standard system amplifies general definitions of measurement and clarifies metatheoretical statements concerning the requirements for measurement. Differences between measurement situations may be categorized by the type of standard system used and by features of the attribute measured. Identification of different standard systems and elements of the measurement process provides a focus for comparisons between measurement theories and models in different measurement situations. Index terms: applied measurement models, comparison of measurement models, definition of measurement, measurement process, measurement theory, metatheoretical framework for applied measurement models, standard systems of measurement.
The efficacy of unconditional maximum likelihood bias correction: Comment on Jansen, Van den Wollenberg, and Wierda.
(1988) Wright, Benjamin D.
Index terms: Item response theory, item parameter estimation; Item response theory, one-parameter logistic model; Maximum likelihood estimation, conditional; Maximum likelihood estimation, unconditional; Minimum chi-square estimation; One-parameter logistic model; Rasch model.
Consistency of Rasch model parameter estimation: A simulation study
(1988) Van den Wollenberg, Arnold L.; Wierda, Folkert W.; Jansen, Paul G. W.
It is shown in this paper that the unconditional or simultaneous maximum likelihood estimation procedure for the one-parameter logistic model gives rise to biased estimators. This bias cannot be removed by a correction factor (K - 1)/K (where K is the number of items), contrary to the contention of several authors. The bias is dependent not only on the number of items, but also on the distribution of the item parameters, which makes correcting for bias practically impossible. Furthermore, it is shown that the minimum chi-square estimation procedure, as introduced by Fischer, results in unbiased estimates. In addition, this method is computationally fast, so that it seems to be a good alternative for CML estimation when the latter method meets practical impediments. Index terms: Maximum likelihood estimation, conditional; Maximum likelihood estimation, unconditional; Minimum chi-square estimation; One-parameter logistic model; Rasch model.
Correcting unconditional parameter estimates in the Rasch model for inconsistency
(1988) Jansen, Paul G. W.; Van den Wollenberg, Arnold L.; Folkert W., Folkert W.
Results of simulation studies indicate that the unconditional maximum likelihood method is commonly regarded as an appropriate substitute for the theoretically superior conditional method for estimating the parameters of the Rasch model. To this end, the unconditional estimates are "corrected" by a factor (K - 1)/K, where is the number of items. In this paper, the simulation study of Wright and Douglas (1977b), which seemed to corroborate this correction term, is critically discussed. It appears to contain a puzzling assumption, and to rest on inadequate logic. Accordingly, there is a need for new simulation studies on the validity of the correction term (K − 1)/K for unconditional maximum likelihood estimation in the Rasch model. Index terms: Item response theory, item parameter estimation; Item response theory, one-parameter logistic model; Maximum likelihood estimation, unconditional; One-parameter logistic model; Rasch model.
Exploiting auxiliary information about items in the estimation of Rasch item difficulty parameters
(1988) Mislevy, Robert J.
Standard procedures for estimating the item parameters in IRT models make no use of auxiliary information about test items, such as their format, their content, or the skills they require for solution. This paper describes a framework for exploiting this information, thereby enhancing the precision and stability of item parameter estimates and providing diagnostic information about items’ operating characteristics. The principles are illustrated in a context for which a relatively simple approximation is available: empirical Bayesian estimation of Rasch item difficulty parameters. Index terms: Bayesian estimation, Collateral information, Empirical Bayesian estimation, Exchangeability, Hierarchical models, Item response theory, Linear logistic test model, Rasch model item parameters.
Full-information item factor analysis
(1988) Bock, R. Darrell; Gibbons, Robert; Muraki, Eiji
A method of item factor analysis based on Thurstone’s multiple-factor model and implemented by marginal maximum likelihood estimation and the EM algorithm is described. Statistical significance of successive factors added to the model is tested by the likelihood ratio criterion. Provisions for effects of guessing on multiple-choice items, and for omitted and not-reached items, are included. Bayes constraints on the factor loadings are found to be necessary to suppress Heywood cases. Numerous applications to simulated and real data are presented to substantiate the accuracy and practical utility of the method. Index terms: Armed Services Vocational Aptitude Battery, Beta prior, E M algorithm, Item factor analysis, TESTFACT, Tetrachoric correlation.
Estimation of the squared cross-validity coefficient in the context of best subset regression
(1988) Kennedy, Eugene
A monte carlo study was conducted to examine the performance of several strategies for estimating the squared cross-validity coefficient of a sample regression equation in the context of best subset regression. Data were simulated for populations and experimental designs likely to be encountered in practice. The results indicated that a formula presented by Stein (1960) could be expected to yield estimates as good as or better than cross-validation, or several other formula estimators, for the populations considered. Further, the results suggest that sample size may play a much greater role in validity estimation in subset selection than is true in situations where selection has not occurred. Index terms: Best subset regression, Cross-validity coefficient, Multiple regression, Predictive validity, Variable selection.
The longitudinal Guttman simplex: A new methodology for measurement of dynamic constructs in longitudinal panel studies
(1988) Collins, Linda M.; Cliff, Norman; Dent, Clyde W.
Traditional psychometric procedures can be inadequate for the measurement of dynamic constructs in longitudinal panel studies. This paper introduces an alternative based on the longitudinal Guttman simplex (LGS) model,a measurement model developed especially for dynamic constructs measured longitudinally. The LGS is a model of cumulative, unitary development. It is cumulative in the sense that as persons acquire new skills (or abilities, or opinions), earlier obtained skills are retained; it is unitary in the sense that all persons progress through a sequence of skills in the same skill order. CL, a consistency index that gives the researcher a measure of the extent to which the LGS model axioms are obeyed in a given dataset, is introduced. By making use of this consistency index, the researcher can develop scales uniquely sensitive to cumulative, unitary development. LGSCLUS, an exploratory procedure to find longitudinal Guttman scales in empirical datasets, is described. An artificial data study is reported, the purpose of which was to test the performance of LGSCLUS under controlled conditions. The artificial data study showed that, in general, LGSCLUS recovers longitudinal Guttman scales with a high degree of accuracy. There remains a need for measurement procedures for dynamic constructs exhibiting types of development other than cumulative and unitary. Index terms: Dynamic constructs, Guttman simplex, Longitudinal panel studies, Mathematical models, Measurement theory, Scaling, Three-set data.
A zero-one programming approach to Gulliksen's matched random subtests method
(1988) Van der Linden, Wim J.; Boekkooi-Timminga, Ellen
Gulliksen’s matched random subtests method is a graphical method to split a test into parallel test halves. The method has practical relevance because it maximizes coefficient α as a lower bound to the classical test reliability coefficient. In this paper the same problem is formulated as a zero-one programming problem, the advantage being that it can be solved by computer algorithms that already exist. It is shown how the procedure can be generalized to split tests of any length. The paper concludes with an empirical example comparing Gulliksen’s original hand-method with the zero-one programming version. Index terms: Classical test theory, Gulliksen’s matched random subtests method, Item matching, Linear programming, Parallel tests, Test reliability, Zero-one programming.