Between Dec 19, 2024 and Jan 2, 2025, datasets can be submitted to DRUM but will not be processed until after the break. Staff will not be available to answer email during this period, and will not be able to provide DOIs until after Jan 2. If you are in need of a DOI during this period, consider Dryad or OpenICPSR. Submission responses to the UDC may also be delayed during this time.

Browsing by Author "Brennan, Robert L."

Now showing 1 - 7 of 7
  • Thumbnail Image
    Item
    Agreement coefficients as indices of dependability for domain-referenced tests
    (1980) Kane, Michael T.; Brennan, Robert L.
    A large number of seemingly diverse coefficients have been proposed as indices of dependability, or reliability, for domain-referenced and/or mastery tests. In this paper it is shown that most of these indices are special cases of two generalized indices of agreement-one that is corrected for chance and one that is not. The special cases of these two indices are determined by assumptions about the nature of the agreement function or, equivalently, the nature of the loss function for the testing procedure. For example, indices discussed by Huynh (1976), Subkoviak (1976), and Swaminathan, Hambleton, and Algina (1974) employ a threshold agreement, or loss, function; whereas, indices discussed by Brennan and Kane (1977a, 1977b) and Livingston (1972a) employ a squared-error loss function. Since all of these indices are discussed within a single general framework, the differences among them in their assumptions, properties, and uses can be exhibited clearly. For purposes of comparison, norm-referenced generalizability coefficients are also developed and discussed within this general framework.
  • Thumbnail Image
    Item
    A comparison of the Nedelsky and Angoff cutting score procedures using generalizability theory
    (1980) Brennan, Robert L.; Lockwood, Robert E.
    Nedelsky (1954) and Angoff (1971) have suggested procedures for establishing a cutting score based on raters’ judgments about the likely performance of minimally competent examinees on each item in a test. In this paper generalizability theory is used to characterize and quantify expected variance in cutting scores resulting from each procedure. Experimental test data are used to illustrate this approach and to compare the two procedures. Consideration is also given to the impact of rater disagreement on some issues of measurement reliability or dependability. Results suggest that the differences between the Nedel sky and Angoff procedures may be of greater consequence than their apparent similarities. In particular, the restricted nature of the Nedelsky (inferred) probability scale may constitute a basis for seriously questioning the applicability of this procedure in certain contexts.
  • Thumbnail Image
    Item
  • Thumbnail Image
    Item
    Linear equating models for the common-item nonequivalent-populations design
    (1987) Kolen, Michael J.; Brennan, Robert L.
    The Tucker and Levine equally reliable linear methods for test form equating in the common-item nonequivalent- populations design are formulated in a way that promotes understanding of the methods. The formulation emphasizes population notions and is used to draw attention to the practical differences between the methods. It is shown that the Levine method weights group differences more heavily than the Tucker method. A scheme for forming a synthetic population is suggested that is intended to facilitate interpretation of equating results. A procedure for displaying form and group differences is developed that also aids interpretation.
  • Thumbnail Image
    Item
    Some practical issues in equating
    (1987) Brennan, Robert L.; Kolen, Michael J.
    The practice of equating frequently involves not only the choice of a statistical equating procedure but also consideration of practical issues that bear upon the use and/or interpretation of equating results. In this paper, major emphasis is given to issues involved in identifying, quantifying, and (to the extent possible) eliminating various sources of error in equating. Other topics considered include content specifications and equating, equating in the context of cutting scores, reequating, and the effects of a security breach on equating. To simplify discussion, some issues are treated from the linear equating perspective in Kolen and Brennan (1987).
  • Thumbnail Image
    Item
    "Technical and practical issues in equating: A discussion of four papers": Reply
    (1987) Brennan, Robert L.; Kolen, Michael J.
    We would like to thank Angoff (1987) for his thoughtful and extensive review of the Kolen and Brennan (1987) and Brennan and Kolen (1987) papers. His comments were very helpful to us in clarifying our thinking about a number of issues. Although we find ourselves in agreement with most of his comments, there are two issues that we believe merit further consideration-synthetic population weights and the circular equating paradigm. In retrospect, our initial discussion of these topics probably should have been more extensive. We hope that the following reply will clarify our position with respect to these two issues.
  • Thumbnail Image
    Item
    A variance components model for measurement procedures associated with a table of specifications
    (1982) Jarjoura, David; Brennan, Robert L.
    Although many tests are developed according to a table of specifications, the literature contains little guidance to measurement specialists for considering the measurement properties of tests developed in this manner. Rather, most of the literature makes the simplistic assumption that the entire set of items are drawn from (or represent) a common undifferentiated domain or universe. This paper presents a variance components model for many measurement procedures that are associated with a table of specifications. In addition, simple procedures are provided for estimating the model parameters (variance components and category means) and functions of them (e.g., composite universe score variance and error variances).