The internal and external optimality of decisions based on tests
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
The internal and external optimality of decisions based on tests
Published Date
1979
Publisher
Type
Article
Abstract
In applied measurement, test scores are usually
transformed to decisions. Analogous to classical
test theory, the reliability of decisions has been defined
as the consistency of decisions on a test and a
retest or on two parallel tests. Coefficient kappa
(Cohen, 1960) is used for assessing the consistency
of decisions. This coefficient has been developed for
assessing agreement between nominal scales. It is
argued that the coefficient is not suited for assessing
consistency of decisions. Moreover, it is argued
that the concept consistency of decisions is not
appropriate for assessing the quality of a decision
procedure. It is proposed that the concept consistency
of decisions be replaced by the concept
optimality of the decision procedure. Two types of
optimality are distinguished. The internal optimality
is the risk of the decision procedure with respect
to the true score the test is measuring. The external
optimality is the risk of the decision procedure with
respect to an external criterion. For assessing the
optimality of a decision procedure, coefficient delta
(van der Linden & Mellenbergh, 1978), which can
be considered a standardization of the Bayes risk or
expected loss, can be used. Two loss functions are
dealt with: the threshold and the linear loss functions.
Assuming psychometric theory, coefficient
delta for internal optimality can be computed from
empirical data for both the threshold and the linear
loss functions. The computation of coefficient delta
for external optimality needs no assumption of psychometric
theory. For six tests coefficient delta as
an index for internal optimality is computed for
both loss functions; the results are compared with
coefficient kappa for assessing the consistency of
decisions with the same tests.
Keywords
Description
Related to
Replaces
License
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Mellenbergh, Gideon J & Van der Linden, Wim J. (1979). The internal and external optimality of decisions based on tests. Applied Psychological Measurement, 3, 257-273. doi:10.1177/014662167900300214
Suggested citation
Mellenbergh, Gideon J.; Van der Linden, Wim J.. (1979). The internal and external optimality of decisions based on tests. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/99629.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.