Browsing by Subject "Validation"

Now showing 1 - 9 of 9
  • Thumbnail Image
    Item
    Developing and validating an instrument to measure college students' inferential reasoning in statistics: an argument-based approach to validation
    (2012-06) Park, Jiyoon
    The purpose of this study was to develop and validate an assessment to measure college students' inferential reasoning in statistics. This proposed assessment aims to help statistics educators guide and monitor students' developing ideas of statistical inference. Within the two-stage cycle, the formative and summative stages, this study first built arguments for the use of assessment and score interpretations, and verified inferences made from those arguments. The five claims were used to examine the plausibility of the validity arguments: 1) The test measures students' level of statistical inferential reasoning in two aspects--informal statistical inference and formal statistical inference; 2) The test measures statistical inferential reasoning in the representative test domains; 3) The test produces scores with sufficient precision to be meaningfully reported; 4) The test is functional for the purposes of formative assessment; and 5) The test provides information about students' level of statistical inferential reasoning in the realms of informal and formal statistical inference. Using a mixed-methods study design, different types of validity evidence were gathered and investigated. Three content experts provided their evaluation of the test blueprint and assessment, based on their qualitative reviews. For the revised assessment resulting from the experts' feedback, cognitive interviews were conducted with nine college students using think-aloud protocols, whereby the students verbalized their reasoning as they reached an answer. A pilot-test administered in a classroom provided preliminary information of the psychometric properties of the assessment. The final version of the assessment was administered to 2,056 students in 39 higher education institutions across the United States. For the data obtained from this large-scale assessment, a unidimensional model in confirmatory factor analysis and the Graded Response Model in item response theory were employed to examine the arguments regarding the internal structure and item properties. The results suggest that the AIRS is unidimensional with appropriate levels of item difficulty and information. The pedagogical implications for the use of the AIRS test are discussed with regard to the areas where students showed difficulties in the domain of statistical inference.
  • Thumbnail Image
    Item
    Development of a novel instrument to evaluate interdisciplinary problem-solving skills of Chemistry and Biochemistry Majors.
    (2021-07) Jayathilaka, Adeesha
    Real-world applications of science are often interdisciplinary, involving overlapping concepts from multiple STEM fields. However, undergraduates majoring in science learn these concepts in separate courses, one for each field, and their performance is then evaluated by asking them to solve problems in each subdiscipline separately. This approach is often not effective in explicitly relating the subdisciplines of STEM to interdisciplinary applications in industry or academic research. Therefore, our siloed approach to teaching and evaluating STEM students leave us with an unassessed, but very important question: How well can STEM graduates solve problems at the intersections between STEM subfields? To begin addressing this question, we first turn to the intersections and interdisciplinary problems that exist within our own field: Chemistry and Biochemistry. To assess students’ interdisciplinary problem-solving skills among the subdisciplines of chemistry, we have developed a novel instrument: Interdisciplinary Problem-Solving Quiz (IPSQ). Each item in the instrument is designed so that it requires understanding in principles of multiple branches of chemistry and utilizes problem-solving skills necessary in real-world applications and research. To confirm the validity of the instrument, four validation studies were conducted. Feedback obtained from validation studies was used to optimize the instrument. During the scope of the study, content validation and face validation were achieved. Further validation studies are recommended to achieve the construct validation of the IPSQ. It is anticipated to administer the validated instrument on graduating seniors, gather information to interpret the output of undergraduate chemistry education, and introduce appropriate reforms to the undergraduate curriculum.
  • Thumbnail Image
    Item
    An initial study to develop instruments and validate the Essential Competencies for Program Evaluators (ECPE)
    (2012-06) Wilcox, Yuanjing
    Contemporary program evaluation emerged in the 1960s with the unprecedented Great Society programs. Over the past half of a century, program evaluation has clearly developed, yet it has not developed into a full-fledged profession. Unlike many established professions, program evaluation does not have certification or licensure for evaluators and cannot exclude unqualified persons from the profession. Evaluator competencies are one of the underpinning foundations of professional designation. Evaluator competencies demonstrate the knowledge, skills, and attitudes required in program evaluators. The Essential Competencies for Program Evaluators, developed by King et al. (2001) and revised by Stevahn et al. (2005), were one of the first efforts at specifying evaluator competencies. The purpose of this study was to develop instruments (a web-based survey and interview protocol) to initially validate the Essential Competencies for Program Evaluators using Messick's (1989, 1995a, 1995b) concept of unitary validity as a framework. The validated evaluator competencies can be used to enhance the professionalization of the field of evaluation.
  • Thumbnail Image
    Item
    Integration and validation in hybrid-online teacher preparation: a case study of persistence in a Native American Special Education Licensure Program
    (2013-12) Patterson, Donna Rose
    This study explored persistence through the experience of professional studies students in a special education licensure program. The context of the study was a graduate level teacher preparation program delivered in a hybrid format of face-to-face and online learning environments. The goal of the program was to prepare teachers from a Native American perspective to work effectively with students with disabilities. The study focus was: the academic and social integration of students; the in- and out-of-class validating agents that fostered academic and interpersonal development; the perceived sense of preparedness to work Native American youth who are identified with a disability, and the bases of these perceptions; and differences between the Native American students and the non-Native students in their experience of integration, validation and preparedness. This study used qualitative methodology including program evaluation findings, individual interviews, a written survey, focus group and analysis of online postings. The participants in the study were 13 non-traditional graduate professional studies students. Results of this study suggested that both the online and the face-to-face learning environments contributed to academic and social integration. These integrating experiences were essential in creating a vibrant and supportive learning environment, and in supporting persistence. Validation was an important factor in developing a sense of belonging in the community, and in fostering self-efficacy as future special education teachers in diverse settings. Validating experiences came from both the face-to-face and online learning environments. Participants felt well-prepared to be a special education teacher in several skill areas: working with students and families, cultural awareness, ability to be a valued part of an effective team, and instructional strategies. Two areas of skills and knowledge were identified in which students did not feel well prepared: special education assessment and the paperwork associated with meeting the legal mandate of special education services. Differences between Native American and non-Native participants included the quality and quantity of giving and seeking support, reaction to challenges, and identified priorities. Key words: persistence, non-traditional, nontraditional, Native American, American Indian, hybrid, online, hybrid-online, integration, validation, professional studies, teacher preparation, special education, disability, disabilities.
  • Thumbnail Image
    Item
    A Lateral Dynamic Model of a Tractor-Trailer: Experimental Validation
    (1996-11) Alexander, Lee; Donath, Max; Hennessey, Michael; Morellas, Vassilios; Shankwitz, Craig
    The SAFETRUCK program focuses on preventing accidents on rural highways, especially those associated with run-off-the-road incidents and driver fatigue, by giving the vehicle the ability to steer to the side of the road and come to a safe stop if the driver falls asleep or is otherwise incapacitated. Researchers have equipped a Navistar 9400 series class 8 truck tractor with the sensors and control computers necessary to perform this task. Designing the controller that will steer the truck requires a mathematical model of the lateral response of the truck to steering inputs. In this project, researchers developed a lateral dynamic model by incorporating second order dynamics into the steering axle tires. Simulation of the resulting models indicated dynamic behavior that was close to the experimental data for speeds between 15 and 30 miles per hour. This is the first time that a lateral dynamic model of a truck has been experimentally verified. Both models, however, resulted in experimentally determined values for steering axle cornering stiffness that were considerably smaller than published values for the Goodyear G 159 tires on the truck.
  • Thumbnail Image
    Item
    Practical Methods for Analyzing Pedestrian and Bicycle Use of a Transportation Facility
    (Minnesota Department of Transportation Office of Research Services, 2010-02) Somasundaram, Guruprasad; Morellas, Vassilios; Papanikolopoulos, Nikolaos P.
    The objective of the project is to analyze existing technologies used for the process of generating counts of bicycles and pedestrians in transportation facilities such as walk and bicycle bridges, urban bicycle routes, bicycle trails etc. The advantages and disadvantages of each existing technology which is being applied to counting has been analyzed and some commercially available products were listed. A technical description of different methods that were considered for vision based object recognition is also mentioned along with the reasons as to why such methods were overlooked for our problem. Support Vector Machines were used for classification based on a vocabulary of features built using interest point detectors. After finalizing the software and hardware, five sites were picked for filming and about 10 hours of video was acquired in all. A portion of the video data was used for training and the remainder was used for testing the algorithm’s accuracy. Results of counts are provided and an interpretation of these results is provided in this report. Upon detailed analysis the reasons for false counts and undercounting in some cases have been identified and current work concerns dealing with these issues. Changes are being made to the system to improve the accuracy with the current level of training and make the system available for practitioners to perform counting.
  • Thumbnail Image
    Item
    Predicting the effects of climate change on water yield and forest production in the northeastern United States
    (1995) Aber, John D; Ollinger, Scott V; Federer, C. Anthony; Reich, Peter B; Goulden, Michael L; Kicklighter, David W; Melillo, Jerry M; Lathrop, Richard G Jr
    Rapid and simultaneous changes in temperature, precipitation and the atmospheric concentration of CO2 are predicted to occur over the next century. Simple, well-validated models of ecosystem function are required to predict the effects of these changes. This paper describes an improved version of a forest carbon and water balance model (PnET-II) and the application of the model to predict stand- and regional-level effects of changes in temperature, precipitation and atmospheric CO2 concentration. PnET-II is a simple, generalized, monthly time-step model of water and carbon balances (gross and net) driven by nitrogen availability as expressed through foliar N concentration. Improvements from the original model include a complete carbon balance and improvements in the prediction of canopy phenology, as well as in the computation of canopy structure and photosynthesis. The model was parameterized and run for 4 forest/site combinations and validated against available data for water yield, gross and net carbon exchange and biomass production. The validation exercise suggests that the determination of actual water availability to stands and the occurrence or non-occurrence of soil-based water stress are critical to accurate modeling of forest net primary production (NPP) and net ecosystem production (NEP). The model was then run for the entire NewEngland/New York (USA) region using a 1 km resolution geographic information system. Predicted long-term NEP ranged from -85 to +275 g C m-2 yr-1 for the 4 forest/site combinations, and from -150 to 350 g C m-2 yr-1 for the region, with a regional average of 76 g C m-2 yr-1. A combination of increased temperature (+6*C), decreased precipitation (-15%) and increased water use efficiency (2x, due to doubling of CO2) resulted generally in increases in NPP and decreases in water yield over the region.
  • Thumbnail Image
    Item
    Synthesis and validation of flight control for UAV.
    (2009-11) Paw, Yew Chai
    Unmanned Aerial Vehicles (UAVs) are widely used worldwide for a board range of civil and military applications. There continues to be a growing demand for reliable and low cost UAV systems. This is especially true for small-size mini UAV systems where majority of systems are still deployed as prototypes due to their lack of reliability. Improvement in the modeling, testing and flight control for the small UAVs would increase their reliability during autonomous flight. The traditional approach used in manned aircraft and large UAV system synthesizing, implementing and validating the flight control system to achieve desired objectives is time consuming and resource intensive. This thesis aims to provide an integrated framework with systematic procedures to synthesize and validate flight controllers. This will help in the certification of UAV system and provide rapid development cycle from simulation to real system flight testing. The effectiveness of the approach is demonstrated by applying the developed framework on a small UAV system that was developed at the University of Minnesota. The thesis is divided into four main parts. The first part is mathematical modeling of the UAV nonlinear simulation model using first principle theory. Flight test system identification technique is used to extract model and model uncertainty parameters to update the nonlinear simulation model. The nonlinear simulation model developed must be able to simulate the actual UAV flight dynamics accurately for real-time simulation and robust control design purposes. Therefore it is important to include model uncertainties into the nonlinear simulation model developed, especially in small UAV system where its dynamics is less well understood than the full-size aircraft. The second part of the work provides the approach and procedures for uncertainty modeling into the nonlinear simulation model such that realization of linear uncertain model is possible. The third part of work describes the flight control design and architecture used in the UAV autopilot system. Classical and model-based control synthesis approaches are presented for roll angle tracking controller to demonstrate the controller synthesis approaches and practical controller implementation issues on the embedded flight computer system. The last part of work blends in all the previous works into the integrated framework for testing and validation of the synthesized controllers. This involves software-in-the-loop, processor-in-the-loop and flight testing of the synthesized controllers using the integrated framework developed.
  • Thumbnail Image
    Item
    Validating Movement-Based Sleep Assessment in a Pediatric Sample with Developmental Disabilities and Associated Motor Impairment and Intellectual Disability
    (2022-04) Merbler, Alyssa
    Individuals with developmental disabilities (DD) have increased rates of sleep disorders compared to their typically-developing peers. Poor sleep can have serious negative consequences for the individual and their family’s health and quality of life, underscoring the need for appropriate sleep assessment and intervention. Objective measures of sleep offer advantages over commonly-used proxy-report sleep measures, such as the ability to directly measure sleep as it is happening and not requiring recall. Polysomnography (PSG) is the current gold-standard, but it has a high resource and personnel burden, limiting its availability and utility for the number of children with DD who need sleep assessment. Actigraphy (ACT) is an alternative objective, non-invasive, cost-effective tool that measures movement via a watch-like device to score sleep and wake. ACT has been extensively validated against PSG in typically-developing adults and children, but there is limited validation in DDs. The purpose of this study was to conduct an initial validation of ACT in the largest sample of children with DD and associated motor impairment and intellectual disability to date. Fifty-one participants wore an ACT device while undergoing standard-of-care PSG. Results indicated that the low movement threshold setting of ACT analyses (i.e. less movement required to score wake) had the best specificity without sacrificing excellent sensitivity, compared to the medium and high thresholds. Overall, ACT had high ability to detect true sleep (0.93), but weak ability to detect true wake (0.45). This pattern is consistent with the broad ACT validation literature. However, these data show wide variability in agreement, indicating ACT shows high agreement for some but poor agreement for others. Summary-level comparisons and Bland-Altman plots showed ACT slightly over estimated onset latency and sleep efficiency, over estimated total sleep time, but underestimated wake after sleep onset. There was wide variability in individual differences between PSG and ACT. There may be relationships with motor impairment and age, but future work with a larger sample is needed to elucidate these potential relations. This work is necessary to provide guidance on ACT’s use in DD populations with associated motor impairment and intellectual disability.