Browsing by Subject "Data processing"

Now showing 1 - 6 of 6
  • Thumbnail Image
    Item
    Cell-culture Data Pipeline Python package (CDPpy) for processing and analyzing cell culture datasets
    (2024-08-05) Lu, Yen-An; Fukae, Yudai; Hu, Wei-Shou; Zhang, Qi; qizh@umn.edu; Zhang, Qi; University of Minnesota Decision Discovery and Optimization Lab; University of Minnesota Cellular Engineering Lab
    CDPpy (Cell-culture Data Pipeline Python package) is an open-source library designed for the analysis of fed-batch cell culture data from multiple experiments and cell lines. The package features the functions of a data processing pipeline and visualization toolbox. The processing pipeline reads raw data from Excel files following a fixed template, derives variables such as cumulative substrate consumption and various specific rates, and exports the processed dataset into an Excel file. The specific rates show changing cellular activities over time in culture, providing insights for process optimization. The visualization toolbox enables users to analyze process profiles across experimental runs and cell lines, aiding in future experimental design. In this repository, we include the source code for the package, an instruction for package setup, and a Jupyter notebook that provides step-by-step guidelines for data processing and visualization using an example dataset. The updated version will be announced in the GitHub repository: https://github.com/ddolab/CDPpy in the future.
  • Thumbnail Image
    Item
    Data Processing Methods And Their Effects On The Limits Of Agreement And Reliability Of Automated Submaximal Threshold Calculations
    (2023-09) Hesse, Anton
    Cardiovascular exercise intensity constitutes an important component of exercise prescription, but its individualization poses challenges. Previous research underscores the greater efficacy of physiological threshold-based cardiovascular exercise intensity prescription, when compared to standardized percentages of maximum heart rate, VO2, or workload. However, identifying these thresholds usually entails pre-processing steps like outlier elimination, interpolation, and data averaging. Although diverse algorithms exist to pinpoint these thresholds, the influence of prior data processing steps on algorithm-derived thresholds remains unclear. Through a scoping review, we gathered articles from studies that collected breath-by-breath gas exchange data during exercise in humans. We assessed the reporting prevalence and the nature of outlier removal, interpolation, and data-averaging methods. Approximately 5% of articles described outlier removal and interpolation details in their methods, while 2/3 reported data averaging. We developed an open-source R package, ”gasExchangeR,” to assess the effects of data processing choices on algorithm-derived thresholds. We included multiple threshold-detection algorithms from previous research in this package and validated them against simulated and human exercise tests. Most algorithms performed well under low simulated noise conditions but had higher relative and absolute error than visual detection. Leveraging the gasExchangeR package, algorithm-derived thresholds were computed across varied outlier removal limits, averaging durations, and algorithms using 350 exercise tests. A similar analysis was performed with 17 participants to assess the effect of these parameters on the test-retest reliability of algorithm-derived thresholds. The outcomes exhibited generally negligible main effects and interactions between outlier removal limit, averaging duration, and algorithm selection on average threshold values. Nevertheless, some statistically significant differences were observed. The 95% limits of agreement (LOA) among diverse data processing and algorithm combinations exceeded the expected measurement error in VO2. Linear regression highlighted algorithmic comparisons as the primary contributor to LOA variance. Specific algorithm types yielded statistically significant ICC values more frequently. These findings indicate that manipulating the data appearance despite constant underlying fitness can unveil inherent variability in distinct algorithms. In aggregate, these investigations underscore the potential for enhanced reproducibility through improved method documentation and the use of open-source software.
  • Thumbnail Image
    Item
    Data-Driven Support Tools for Transit Data Analysis, Scheduling and Planning
    (Intelligent Transportation Systems Institute Center for Transportation Studies, 2011-07) Liao, Chen-Fu
    Many transit agencies in the U.S. have instrumented their fleet with Automatic Data Collection Systems (ADCS) to monitor the performance of transit vehicles, support schedule planning and improve quality of services. The objective of this study is to use an urban local route (Metro Transit Route 10 in Twin Cities) as a case study and develop a route-based trip time model to support scheduling and planning while applying different transit strategies. Usually, timepoints (TP) are virtually placed on a transit route to monitor its schedule adherence and system performance. Empirical TP time and inter-TP link travel time models are developed. The TP-based models consider key parameters such as number of passengers boarding and alighting, fare payment type, bus type, bus load (seat availability), stop location (nearside or far side), traffic signal and volume that affect bus travel time. TP time and inter-TP link travel time of bus route 10 along Central Avenue between downtown Minneapolis and Northtown were analyzed to describe the relationship between trip travel time and primary independent variables. Regression models were calibrated and validated by comparing the simulation results with existing schedule using adjusted travel time derived from data analyses. The route-based transit simulation model can support Metro Transit in evaluating different schedule plans, stop consolidations, and other strategies. The transit model provides an opportunity to predict and evaluate potential impact of different transit strategies prior to deployment.
  • Thumbnail Image
    Item
    The E and B EXperiment: implementation and analysis of the 2009 engineering flight.
    (2011-06) Milligan, Michael Bryce
    The E and B EXperiment (EBEX) is a balloon-borne telescope designed to map the polarization of the cosmic microwave background (CMB) and emission from galactic dust at millimeter wavelengths from 150 to 410 GHz. The primary science objectives of EBEX are to: detect or constrain the primordial B-mode polarization of the CMB predicted by in ationary cosmology; measure the CMB B-mode signal induced by gravitational lensing; and characterize the polarized thermal emission from interstellar dust. EBEX will observe a 420 square degree patch of the sky at high galactic latitude with a telescope and camera that provide an 80 beam at three observing bands (150, 250, and 410 GHz) and a 6:2#14; diffraction limited field of view to two large-format bolometer array focal planes. Polarimetry is achieved via a continuously rotating half-wave plate (HWP), and the optical system is designed from the ground up for control of sidelobe response and polarization systematic errors. EBEX is intended to execute y or more Antarctic long duration balloon campaigns. In June 2009 EBEX completed a North American engineering flight launched from NASA's Columbia Scientific Ballooning Facility (CSBF) in Ft. Sumner, NM and operated in the stratosphere above 30 km altitude for #24; 10 hours. During flight EBEX must be largely autonomous as it conducts pointed, scheduled observations; tunes and operates 1432 TES bolometers via 28 embedded Digital frequency-domain multiplexing (DfMux) computers; logs over 3 GiB/hour of science and housekeeping data to onboard redundant disk storage arrays; manages and dispatches jobs over a fault-tolerant onboard Ethernet network; and feeds a complex real-time data processing infrastructure on the ground via satellite and line-of-sight (LOS) downlinks. In this thesis we review the EBEX instrument, present the optical design and the computational architecture for in-flight control and data handling, and the quick-look software stack. Finally we describe the 2009 North American test flight and present analysis of data collected at the end of that flight that characterizes scan-synchronous signals and the expected response to emission from thermal dust in our galaxy.
  • Thumbnail Image
    Item
    Models for Predicting RWIS Sensor Misalignments and Their Causes
    (University of Minnesota Center for Transportation Studies, 2010-01) Bhalekar, Prafulla; Crouch, Carolyn J.; Crouch, Donald B.; Maclin, Richard M.
    The Minnesota Department of Transportation uses the Road Weather Information System (RWIS) for monitoring the current weather and surface conditions of its highways. The real-time data received from these sensors reduce the need for road patrolling in specific locations by providing information to those responsible for directing winter maintenance operations. Since most road maintenance decisions and weather forecasts are explicitly dependent on the reliability and accuracy of the RWIS sensor data, it is important for one to be able to determine the reliability of the sensor data, that is, to determine whether a sensor is malfunctioning. In a previous project we investigated the use of machine learning techniques to predict sensor malfunctions and thereby improve accuracy in forecasting weather-related conditions. In this project, we used our findings to automate the process of identifying malfunctioning weather sensors in real time. We analyze the weather data reported by various sensors to detect possible anomalies. Our interface system allows users to define decision- making rules based on their real-world experience in identifying malfunctions. Since decision rule parameters set by the user may result in a false indication of a sensor malfunction, the system analyzes all proposed rules based on historical data and recommends optimal rule parameters. If the user follows these automated suggestions, the accuracy of the software to detect a malfunctioning sensor increases significantly. This report provides an overview of the software tool developed to support detection of sensor malfunctions.
  • Thumbnail Image
    Item
    Using Archived Truck GPS Data for Freight Performance Analysis on I-94/I-90 from the Twin Cities to Chicago
    (University of Minnesota Center for Transportation Studies, 2009-11) Liao, Chen-Fu
    Interstate 94 is a key freight corridor for goods transportation between Minneapolis and Chicago. This project proposes to utilize the FPM data and information from ATRI to study the I-94/I-90 freight corridor. Freight performance will be evaluated and analyzed to compare truck travel time with respect to duration, reliability, and seasonal variation. This data analysis process can be used for freight transportation planning and decision-making and potentially will be scalable for nationwide deployment and implementation on the country’s significant freight corridors.