Browsing by Subject "Model validation"

Now showing 1 - 2 of 2
  • Thumbnail Image
    Item
    Test platforms for model-based flight research
    (2013-09) Dorobantu, Andrei
    Demonstrating the reliability of flight control algorithms is critical to integrating unmanned aircraft systems into the civilian airspace. For many potential applications, design and certification of these algorithms will rely heavily on mathematical models of the aircraft dynamics. Therefore, the aerospace community must develop flight test platforms to support the advancement of model-based techniques. The University of Minnesota has developed a test platform dedicated to model-based flight research for unmanned aircraft systems. This thesis provides an overview of the test platform and its research activities in the areas of system identification, model validation, and closed-loop control for small unmanned aircraft.
  • Thumbnail Image
    Item
    Validating a computational model of patient illness: the Simcare Patient Model.
    (2012-08) McCabe, Ryan M.
    The SimCare Patient Model is a computational model of individuals with type 2 diabetes. The model represents a patient as a sequence of health states that respond to treatments over varying intervals of time. It was originally constructed as a “clinical” model of an “individual patient” with type 2 diabetes so that a physician could access the model by querying the patient state for information, ordering specific treatments for the simulated patient and scheduling the next clinical encounter. A software implementation of the model, generated by previous research (Dutta, et al. 2005), has been used as a training tool for medical residents and primary care physicians (O'Connor, Sperl-Hillen, et al., Simulated Physician Learning Intervention to Improve Safety and Quality of Diabetes Care: A Randomized Trial 2009), a guideline and protocol simulator as well as a tool for identifying optimal treatments under given constraints (McCabe, et al. 2008). This thesis contributes to the understanding of computational model validation in three ways, by: conducting a two-part validation of a model of patient illness, generating a conceptual model so that explanations can be generalized from simulations, and developing an N=1 approach to validating meaningful variation over time in individual patients with chronic disease. The validation is a two-part study of the SimCare Patient Model. The first part is a conceptual validation that defines what aspects of a real-world problem are being modeled and why. How these aspects are represented in the model as sets of variables and functions is also defined. The conceptual validation provides transparency as to the workings of the model, a basis for generalizing explanations related to model predictions or emergent behavior, and the relevant contexts for model utilization. The second part is an operational validation that conducts two sets of simulation experiments to compare model predictions to observed values. Each set of experiments is used to characterize model accuracy in different contexts: The simulation of aggregated outcomes of cohorts of patients responding to treatment protocols in controlled trials and of meaningful variation in individual patients responding to treatments in a clinical care setting. The first set of experiments compares the simulated results of three published randomized clinical trials – each with a different focus on a main aspect of treatment of patients with type 2 diabetes – using three different cohort measures: nominal intermediate health outcomes, relative intermediate health outcomes and cardiovascular disease event rates. One trial has also been simulated by multiple, alternative type 2 diabetes models and provides a basis for comparison of these models with SimCare. The second set of experiments compares actual treatments and outcomes drawn from de-identified electronic health records in a clinical care database to a range of simulated responses from identical synthetic patients and treatments over the course of a year, one patient at a time (N=1). The contributions of this thesis can be organized into three related parts, 1) a two-part validation study of a computational model of patient illness, 2) a conceptual model to be used as the basis for generating explanations for model behavior, and 3) a novel form of operational validation using an N=1 experimental approach to measure meaningful variation in individual patients over time. The validation is presented to satisfy the interests of two overlapping research communities – those interested in the content of the model: the healthcare research community; those interested in computational modeling and validation techniques: the computer science community. The validation study is divided into a conceptual validation and an operational validation. The conceptual validation establishes the set of relevant theories identified in the natural system to be represented in the model. These theories enable the explanations of the model to be generalized and learned from, and they define the intent and contexts for relevant uses of the model. The operational validation performs two types of simulation studies that characterize the outputs of the model under two different real-world contexts. The first set of experiments compares the simulation of populations of individuals under treatment protocols to the outcomes of three well-known clinical trials in the diabetes community. This distinguishes the model as being able to simulate controlled trials to the extent that a population of individuals can be generated and treatment protocols defined. In the second set of simulation experiments, a series of N=1 trials are conducted using retrospective, outpatient clinical care data to demonstrate that SimCare accurately represents meaningful variation over time in individuals being treated for diabetes in a clinical (i.e., less controlled) setting over time. In this setting, meaningful variation is defined as the non-random, clinically relevant variation in outcomes that can emerge over time given a specific course of treatment and an initial patient state. For example, if a physician were to treat two simulated patients via model software, and each patient had identical, observable initial states and received same treatments, the physician would not expect the two patients to exhibit identical responses to the treatments. This variation that exists in the real world of clinical care and causes an individual patient to exhibit meaningful differences in outcomes over time is an intentional part of the SimCare model and requires its own validation study. This distinguishes the model as being one of individual patients (rather than population-based) representing common, primary care encounters (rather than pre-screened patients under controlled protocols). The results of the conceptual validation show that the SimCare model is clinically transparent and capable of generating explanations related to treatment outcomes. The operational validation shows that the SimCare model is able to capture meaningful and typical variation both in individual patients over time and across sets of patients in controlled cohorts.