This study was conducted to determine the degree
to which subject matter experts could predict the difficulty
and discrimination of items on the Test of Standard
Written English. It was concluded that despite an
extended training period the raters did not approach a
high level of accuracy, nor were they able to pinpoint
the factors that contribute to item difficulty and discrimination.
Further research should attempt to uncover
those factors by examining the items from a linguistic
and psycholinguistic perspective. It is argued
that by coupling linguistic features of the items with
subject matter ratings it may be possible to attain more
accurate predictions of item difficulty and discrimination.