Sensitivity of marginal maximum likelihood estimation of item and ability parameters to the characteristics of the prior ability distributions

Abstract

The sensitivity of marginal maximum likelihood estimation of item and ability (θ) parameters was examined when the prior θ distributions are not matched to the underlying θ distributions. Thirty sets of 45-item test data were generated by specification of three types of underlying θ distributions. They were then analyzed with PC-BILOG. Appropriate specification of the prior θ distribution increased the accuracy of estimation for item and θ parameters when the sample size was large. With a small dataset, the appropriate specification of the prior increased the accuracy of θ parameter estimation, but it did not have that effect on item parameter estimation. Only with a large dataset and matched underlying and prior θ distributions did increasing the number of quadrature points improve the accuracy of estimation of the item parameters. However, the accuracy of θ estimation was increased by increasing the number of quadrature points, regardless of sample size and appropriateness of the prior θ distribution. The number of examinees had an important effect on the accuracy of item parameter estimation. Index terms: ability distribution, BILOG, item response theory, marginal maximum likelihood estimation, parameter estimation, quadrature points.