Browsing by Subject "differential item functioning"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Differential Item Functioning and Measurement Invariance of Self- and Proxy-Reports: An Evaluation of Objective Quality of Life Measures for People with Intellectual and Developmental Disabilities
    (2015-01) Hepperlen, Renee
    Abstract The field of intellectual disabilities and developmental disabilities (ID/DD) uses objective quality of life indicators for policy and program development (Verdugo, Schalock, Keith, & Stancliffe, 2005). An ongoing concern in this field is the assessment of quality of life for people who are unable to answer for themselves. In these instances, a proxy-respondent, or someone who knows the person with ID/DD well, will respond on his/her behalf. Research examining the efficacy of using proxy-respondents has yielded mixed results. While some studies failed to show statistically significant differences in responses (McVilly, Burton-Smith, & Davidson, 2000; Rapley, Ridgway, & Beyer, 1998; Stancliffe, 1999), other research has found meaningful differences between matched pairs of self- and proxy-respondents (Rapley et al., 1998). A principle limitation of these previous studies is the reliance on simplistic analytic methods, such as a t-test or correlation to determine if similarities existed between these matched groups. Methodologically, the previous studies on self- and proxy-respondents used t-tests and correlations to examine the relationship between self- and proxy-responses. The present study extends this body of research through the use of differential item functioning and measurement invariance to examine the use of self- and proxy-respondents. Specifically, this study examined the internal structure of the three objective quality of life measures on the National Core Indicators, including the Community Inclusion, Life Decisions, and Everyday Choices scales. Study findings revealed that several items function differently for these two groups when comparing these respondents based on the total score of the scale, which implies that construct-irrelevant differences impacted some item responses (American Educational Research Association [AERA], American Psychological Association [APA], National Council on Measurement in Education [NCME], 1999). In addition, an examination of measurement invariance established that metric invariance fits these data well, meaning that it is not possible to compare these two groups. These findings have policy-and program-evaluation implications, since construct irrelevance (AERA, APA, & NCME, 1999) indicates that for the items identified as functioning differently for these groups, responses also include another construct that is separate from the construct that the scale intends to measure. With these differences, it becomes more difficult to conclude that changes in outcome can be attributed to program social justice implications, since differential item functioning and measurement invariance assessments relate to fairness in testing (Huggins, 2013). When items function differently for groups, this means that respondents find these items difficult, which makes full participation challenging. When individuals find items confusing or hard, then responses may not accurately reflect their experiences. These findings have implications for policy and practice, since policy makers and practitioners use these scales to make program decisions for people with ID/DD.
  • Thumbnail Image
    Item
    Examining Power and Type 1 Error for Step and Item Level Tests of Invariance: Investigating the Effect of the Number of Item Score Levels
    (2017-05) Ayodele, Alicia
    Within polytomous items, differential item functioning (DIF) can take on various forms due to the number of response categories. The lack of invariance at this level is referred to as differential step functioning (DSF). The most common DSF methods in the literature are the adjacent category log odds ratio (AC-LOR) estimator and cumulative category log odds ratio estimator (CU-LOR). Although the study of DSF may be helpful when opposing DIF effects within an item can go undetected or for informing what part of a multi-step item may need improvement, research regarding DSF procedures is limited. The effect of number of item score levels has not been investigated with regard to the relationship between DSF and traditional DIF methods, including differences in statistical behavior. This study investigates the effect of the number of item score levels on power and Type I error of the following DSF methods: AC-LOR, CU-LOR as well as DIF methods: Mantel (chi-square) Test, Liu Agresti, Generalized Mantel-Haenszel, and Simultaneous Step Level test (SSL). This study also examined which statistical procedures are most effective for adjusting per comparison Type I errors for the SSL method: Dunn-Bonferroni, Benjamini and Hochberg, or Holm’s. Conditions varied included (a) sample size ratio, (b) number of item score levels, (c) generating model, (d) impact, and (e) DSF pattern. Results suggest that altering the number of score levels did not have an effect on the DSF/DIF detection methods. When considering both statistical and practical significance of factors affecting power, the pattern of DSF was the most important effect. Additionally, the Dunn-Bonferroni adjustment was adequate when using the SSL method. The SSL method performed well compared to the other DIF methods and should be considered for simultaneously detecting both DSF and DIF. The significance of these results as well as limitations and future directions are discussed.