Browsing by Author "Reilly, Richard R."

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Comparison of direct and indirect methods for setting minimum passing scores
    (1984) Reilly, Richard R.; Zink, Donald L.; Israelski, Edmond W.
    Several studies have compared different judgmental methods of setting passing scores by estimating item difficulties for the minimally competent examinee. Usually, a direct method of estimating item difficulties has been compared with an indirect method suggested by Nedelsky (1954). Nedelsky’s method has usually resulted in a substantially lower cutoff score than that arrived at with a direct method. Two studies were carried out for the purpose of comparing a direct method of setting passing scores with an indirect method that allowed judges to estimate the probability of the minimally competent examinee eliminating each incorrect alternative. In Study 1 a sample of 52 first-level supervisors used both methods to estimate passing scores on a content-oriented selection test for building maintenance specialists. In Study 2 a sample of 62 first-level supervisors used both methods to estimate passing scores on an entry level auto mechanics test. Results of both studies showed that the variance component for method was relatively small and that for raters was relatively large. Reliability estimates of judgments and correlations between judged difficulties and empirical difficulties showed the Angoff (1971) approach to be slightly superior. Results showed no particular advantage to using an indirect approach for estimating minimal competence. Recently, the problem of setting passing scores
  • Thumbnail Image
    Item
    Validation and comparison of homogeneous and occupational interest scales
    (1979) Reilly, Richard R.; Echternacht, Gary J.
    Occupational interest scales and homogeneous scales were developed and compared in a double cross-validation design. A sample of 3,072 Air Force personnel was randomly divided for study purposes. Homogeneous scales were developed in each half-sample using a backward item selection procedure. Occupational scales were developed by selecting items which discriminated satisfied personnel within a career field from a "men-in-general" group. A Bayesian classification procedure was then used to cross-validate the scales developed in each half-sample. The results indicated that the occupational scales were slightly superior in correctly classifying satisfied personnel versus men-in-general. However, correlations between all interest scales and job satisfaction indicated generally higher validity for relevant homogeneous scales than for occupational scales. It was concluded that where prediction of group membership or occupational choice is desired, occupational keying should probably be the preferred approach. However, when concern is with a broader range of criteria, homogeneous scales may offer more general utility.