The effect of numbers of experts and common items on cutting score equivalents based on expert judgment
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
The effect of numbers of experts and common items on cutting score equivalents based on expert judgment
Authors
Published Date
1991
Publisher
Type
Article
Abstract
The effect of different numbers of experts and
common items on the scaling of cutting scores
derived by experts’ judgments was investigated.
Four test forms were created from each of two
examinations; each form from the first examination
shared a block of items with one form from
the second examination. Small groups of experts
set standards on each using a modification of
Angoff’s (1971) method. Cutting score equivalents
were estimated for the matched forms using different
group sizes and numbers of common items;
they were compared with cutting score equivalents
based on score equating. Results showed that a
reduction in error is associated with using more
experts or having more items in common between
the two forms. For 25 or more common items and
five or more judges, the error was about one item
on a 100-item test. More than five experts or 25
common items made only a very small difference
in error. Index terms: cutting scores, equating,
expert judgment, standard setting.
Keywords
Description
Related to
Replaces
License
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Norcini, John, Shea, Judy & Grosso, Louis. (1991). The effect of numbers of experts and common items on cutting score equivalents based on expert judgment. Applied Psychological Measurement, 15, 241-246. doi:10.1177/014662169101500303
Suggested citation
Norcini, John; Shea, Judy; Grosso, Louis. (1991). The effect of numbers of experts and common items on cutting score equivalents based on expert judgment. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/114410.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.