Browsing by Subject "Robotics"

Now showing 1 - 20 of 32

3D Printed Functional Materials and Devices and Applications in AI-powered 3D Printing on Moving Freeform Surfaces
(2020-08) Zhu, Zhijie
The capability of 3D printing a diverse palette of functional inks will enable the mass democratization of manufactured patient-specific wearable devices and smart biomedical implants for applications such as health monitoring and regenerative biomedicines. These personalized wearables could be fabricated via in situ printing --- direct printing of 3D constructs on the target surfaces --- at ease of the conventional fabricate-then-transfer procedure. This new 3D printing technology requires functional (e.g., conductive and viscoelastic) inks and devices (e.g., wearable and implantable sensors) that are compatible with in situ printing, as well as the assistance of artificial intelligence (AI) to sense, adapt, and predict the state of the printing environment, such as a moving hand and a dynamically morphing organ. To advance this in situ printing technology, this thesis work is focused on (1) the development of functional materials and devices for 3D printing, and (2) the AI-assisted 3D printing system. To extend the palette of 3D printable materials and devices, on-skin printable silver conductive inks, hydrogel-based deformable sensors, and transparent electrocorticography sensors were developed. As with the AI for in situ 3D printing, solutions for four types of scenarios were studied (with complexity from low to high): (1) printing on static, planar substrates without AI intervention, with a demonstration of fully printed electrocorticography sensors for implantation in mice; (2) printing on static, non-planar parts with open-loop AI, with a demonstration of printing viscoelastic dampers on hard drives to eliminate specific modes of vibration; (3) printing on moving targets with closed-loop and predictive AI, with demonstrations of printing wearable electronics on a human hand and depositing cell-laden bio-inks on live mice; (4) printing on deformable targets with closed-loop and predictive AI, with demonstrations of printing a hydrogel sensor on a breathing lung and multi-material printing on a phantom face. We anticipate that this convergence of AI, 3D printing, functional materials, and personalized biomedical devices will lead to a compelling future for on-the-scene autonomous medical care and smart manufacturing.
Advanced Control Strategies for the Robotic Hand
(2017-12) Baz Khallouf, Ibrahim
The research in this master’s thesis presents a new state-space representation of the nonlinear dynamics of two-link (thumb) and three-link (index) fingers of a robotic hand and an effective online solution of finite-time, nonlinear, closed-loop optimal control regulator and tracking problems using the state-dependent Riccati equations (SDRE). The technique involves the use of the solution of the algebraic Riccati equation for the in finite-time case (hence the technique is approximate) and the change of variables that converts a state-dependent, nonlinear, differential Riccati equation (SD-DRE) to a linear differential Lyapunov equation (DLE) which can be solved in closed form. The approximate technique is demonstrated by software simulation and hardware experimentation for the two-link and three-link fingers of the robotic hand.
Adversarial and Stochastic Search for Mobile Targets in Complex Environments
(2016-02) Noori, Narges
A new era of robotics has begun. In this era, robots are coming out of simple, structured environments (such as factory floors) into the real world. They are no longer performing simple, repetitive tasks. Instead, they will soon be operating autonomously in complex environments filled with uncertainties and dynamic interactions. Many applications have already emerged as a result of these potential advances. A few examples are precision agriculture, space exploration, and search-and-rescue operations. Most of the robotics applications involve a ``search'' component. In a search mission, the searcher is looking for a mobile target while the target is avoiding capture intentionally or obliviously. Some examples are environmental monitoring for population control and behavioral study of animal species, and searching for victims of a catastrophic event such as an earthquake. In order to design search strategies with provable performance guarantees, researchers have been focusing on two common motion models. The first one is the adversarial target model in which the target uses best possible strategy to avoid capture. The problem is then mathematically formulated as a pursuit-evasion game where the searcher is called the ``pursuer'' and the target is referred to as the ``evader''. In pursuit-evasion games, when a pursuit strategy exists, it guarantees capture against any possible target strategy and, for this reason, can be seen as the worst-case scenario. Considering the worst-case behavior can be too conservative in many practical situations where the target may not be an adversary. The second approach deals with non-adversarial targets by modeling the target's motion as a stochastic process. In this case, the problem is referred to as one-sided probabilistic search for a mobile target, where the target cannot observe the searcher and does not actively evade detection. In this dissertation, we study both adversarial and probabilistic search problems. In this regard, the dissertation is divided into two main parts. HASH(0x7f7fa33ea740) HASH(0x7f7fa33dadd8) In the first part, we focus on pursuit-evasion games, i.e., when the target is adversarial. We provide capture strategies that guarantee capture in finite time against any possible escape strategy. Our contributions are mainly in two areas whether the players have full knowledge of each other's location or not. First, we show that when the pursuer has line-of-sight vision, i.e., when the pursuer sees the evader only when there are no obstacles in the between them, it can guarantee capture in monotone polygons. Here, the pursuer must first ensure that it ``finds'' the evader when it is invisible by establishing line-of-sight visibility, and then it must guarantee capture by getting close to the evader within its capture distance. In our second set of results, we focus on pursuit-evasion games on the surface of polyhedrons assuming that the pursuers are aware of the location of the evader at all times and their goal is to get within the capture distance of the evader. HASH(0x7f7fa33f6a00) In the second part, we study search strategies for finding a random walking target. We investigate the search problem on linear graphs and also 2-D grids. Our goal here is to design strategies that maximize the detection probability subject to constraints on the time and energy, which is available to the searcher. We then provide field experiments to demonstrate the applicability of our proposed strategies in an environmental monitoring project where the goal is to find invasive common carp in Minnesota lakes using autonomous surface/ground vehicles.
Characterization of Lightweight, Low-Force Cable and Hydraulic Transmission Systems
(2022-08) Kivi, Andrew
In the field of rehabilitation robotics and wearable exoskeletons, a common challenge forsystem designers is how to transmit force from the actuators to the joints. In small-scale applications, for the working range of 50-500 N, cables and hydraulics are the two most common ways to transmit force. This study characterized wire rope, braided synthetic line, Bowden cable, and hydraulic transmission types based on their size, weight, efficiency, and controllability. Analytical and experimental methods were used to evaluate individual aspects of each transmission. Analysis was performed to compare the transmission types. The rate at which cables increase in size and weight is approximately linearly with rated load; however, cable construction had the largest influence on the rate of increase. It was observed that cable stiffness can be fit to a 1/L model in the approximate range of 20 to 50 cm, but not for much longer lengths. Hydraulic stiffness was modeled, and it was shown for small diameter actuators the stiffness is comparable to the cables studied. Cable efficiency was studied using the capstan equation and found to be Coulomb friction dependent decreasing as wrap angle or coefficient of friction increased. Bowden cable efficiency is also friction dependent, however Bowden cables do not follow the capstan equation. Over-constrained Bowden cable paths led to more surface contact and decreased efficiency. Hydraulic transmission efficiency is dependent on hose diameter and flowrate. Optimal designs operate at high working pressures and low flowrates. It was shown in a case study that the optimal transmission type is often application dependent.
Computational Digital Inline Holography for In Situ Particle Tracking and Characterization
(2020-05) Mallery, Kevin
Digital inline holography (DIH) is a powerful single-camera 3D microscopic imaging tool that is able to digitally refocus a recorded image to reconstruct the 3D field of view. Compared to other single-camera techniques, DIH has a much larger depth of field in which objects can be seen, leading to drastically increased sampling volumes. Many particle features can be measured with DIH including size, shape, refractive index, identity, and motion. However, DIH has traditionally been limited by challenges related to the difficulty of accurately and quickly processing holographic images. In this thesis, I present technical developments focused on the digital processing of holographic images that are intended to alleviate these challenges and enable the application of DIH to new measurements. Specifically, a new approach for hologram reconstruction -- regularized holographic volume reconstruction (RIHVR) -- is introduced. This method is able to produce substantially noise-free reconstructions of particle fields. A data-driven approach to predictive particle tracking is also introduced in order to enable increased particle concentrations for particle tracking velocimetry applications. Each of these developments is validated using synthetic data and experimental demonstrations. Three applications of holographic imaging are presented to demonstrate the broad applicability of the method. The effect of temperature on the density of colonial cyanobacteria is identified by measuring the buoyant velocity and size of individual colonies. This could lead to better modelling of toxic algal blooms. Another type of algae, \emph{Dunaliella primolecta}, is useful and can be farmed for materials used in nutritional supplements, pharmaceuticals, and biodiesel. DIH is used to identify behavior signatures that could be used as indicators of optimal lipid production. This could enable optimal harvest timing leading to improved biodiesel yield. Finally, a low-cost miniature underwater holographic microscope was developed for \emph{in situ} field applications. This microscope is paired with a robotic platform to enable autonomous exploration of lakes or other aquatic environments. Despite its handheld size, the sensor is able to perform real-time particle concentration measurements using a deep neural network. The recorded images can also be used to identify the type of microorganisms found in the water.
Consistency Analysis and Improvement for Vision-aided Inertial Navigation
(2016-03) Hesch, Joel
Navigation systems capable of estimating the six-degrees-of-freedom (d.o.f.) position and orientation (pose) of an object while in motion have been actively developed within the research community for several decades. Numerous potential applications include human-navigation aids for the visually impaired, first responders, and firefighters, as well as localization systems for autonomous vehicles such as submarines, ground robots, unmanned aerial vehicles, and spacecraft. The mobile industry has also recently become interested in six-dof localization for enabling interesting new applications on smart phones and tablets, such as games that are aware of motions in 3D space. The Global Positioning System (GPS) satellite network has been relied on extensively in pose-estimation applications; however, both humans and vehicles often need to operate in a wide variety of environments that preclude the use of GPS (e.g., underwater, inside buildings, in the urban canyon, and on other planets). In order to estimate the 3D motion of person or robot in GPS-denied areas, it is requisite to employ sensors to determine the platform's displacement over time. To this end, inertial measurement units (IMUs) that sense the three-d.o.f rotational velocity as well as three-d.o.f. linear acceleration have been extensively used. IMU measurements, however, are corrupted by both sensor noise and bias, causing the resulting pose estimates to quickly become unreliable for navigation purposes. Although high-accuracy IMUs exist, they remain prohibitively expensive for widespread use. For this reason, it is common to aid an inertial navigation system (INS) with an alternative sensor such as a laser scanner, sonar, radar, or camera whose measurements can be employed to determine the platform's pose (or motion) with respect to the surrounding environment. Of these possible aiding sources, cameras have received significant attention due to their small size and weight, and the rich information that they supply. State-of-the-art vision-aided inertial navigation systems (VINS) are able to provide highly-accurate pose estimates over short periods of time, however, they continue to exhibit limitations that prevent them from being used in critical applications for long-term deployment. Most notably, current approaches produce inconsistent state estimates, i.e., the errors are biased and the corresponding uncertainty in the estimate is unduely small. In this thesis, we examine two key sources of estimator inconsistency for VINS, and propose solutions to mitigate these issues.
De-noising Motion Predictions of Scuba Divers for Aquatic Robots
(2021)
Current diver predictors output a sequence of bounding boxes, with two corners randomly sampled from two bivariate gaussians. This introduces noise and uncertainty into the prediction outputs and makes the conversion of this sequence of 2D boxes into a 3D motion vector challenging. This poster describes an approach to de-noise this output and convert the predictions into a format that can be used by aquatic robots to plan their motion and follow scuba divers robustly.
Design, Development, and Evaluation of Wearable Length Fastening Devices for Use with Twisted Coiled Actuators
(2023-04) Dorn, Timothy
Artificial muscles and compliant, large stroke linear actuators have enabled new classes of wearable robotics. However, these actuators are inefficient, needing constant power to maintain force and displacement, decreasing their utility in wearable systems. Variable length latching mechanisms alleviate this problem, matching actuator displacement, and holding force and displacement constant when the actuator is powered off. However, most existing latching designs are either not wearable, or must be disengaged manually, limiting their robotic applications. In this research, three wearable and remotely releasable latching mechanisms were designed for use in wearable robotic systems: a stepper motor with a belt and pulley; a linear ratchet; and a cam cleat. The designs were manufactured and tested, with all three designs maintaining force and displacement values up to 15N of cable tension and releasable up to 5N of cable tension. These results demonstrate the viability of integrating latches into soft wearable robotic systems.
Efficient Propulsion for Versatile Unmanned Aerial Vehicles: Studies in Mechanics and Control
(2021-02) Henderson, Travis
This thesis presents a control algorithm for significantly enhancing the available thrust and minimizing the required electrical power consumption of a Variable-Pitch Propulsion (VPP) system, where the VPP system is made up of a brushless DC motor and a variable-collective-pitch propeller with its own servo motor. The variable-collective-pitch propeller mechanism has received recent attention because of the mechanism’s capability to enhance thrust response bandwidth and propulsive efficiency compared to conventional Unmanned Aerial Vehicle (UAV) propulsion systems with rigid-geometry propellers; the mechanism has this capability due to a second mechanical degree of freedom in the propeller geometry, allowing the collective pitch angle of the propeller blades to vary according to actuation from a servo motor. When paired with a properly designed control algorithm, the motor speed and pitch angle can be tuned in real time to track prescribed thrust trajectories while satisfying some optimality condition. Motivation for research into highly efficient VPP propulsion systems is encouraged by the intense interest from private and public sectors in UAVs that are capable of Vertical TakeOff and Landing (VTOL); while generally capable of both fixed-wing and hovering flight, VTOL UAVs with rigid-geometry propellers often exhibit short flight time due to non-optimal propulsion system efficiency across-the-board. Prior research into power-minimizing control strategies for small VPP systems has been targeted at multi-rotor platforms and has thus made assumptions that limit variation in the speed of propeller inflow and in the magnitude of thrust, thus limiting the technology’s applicability to VTOL platforms. The control algorithm presented in this thesis is designed to accommodate for the wide range of air inflow speeds and thrust magnitudes through the following algorithm components: a linear feedback thrust controller with a nonlinear, adaptive feedforward thrust model derived from Blade Element Momentum propeller theory; an estimator to tune the thrust feedforward model parameters in real-time; and an Extremum Seeking algorithm for tracking the minimum-power control input configuration. Analysis of controller performance is discussed with reference to simulated and physical validation experiments.
Efficient Robotic Manipulation with Scene Knowledge
(2023-05) Lou, Xibai
In recent years, robots have transformed manufacturing, logistics, and transportation. However, extending the success to unstructured real-world environments (e.g., domestic kitchens, warehouses, grocery stores, etc.) remains difficult due to three key challenges: (1) assumption of structured environments (such as organized bottles in the factories); (2) hand-engineered solutions that are difficult to generalize to novel scenarios; (3) limited flexibility of action primitives, which prevents the robot from reaching target objects. In this thesis, we address these challenges by learning scene knowledge that improves the efficiency of robotic manipulation systems. Grasping is a fundamental manipulation skill that is constrained by the scene arrangement (i.e., the locations of the robot, the objects, and the environmental structures). Understanding scene knowledge, such as the robot's reachability to objects, is crucial to improve the robot's capability. We developed a reachability-aware grasp pose generator that predicts feasible 6-degree-of-freedom (6-DoF) grasp poses (i.e., approaching with an arbitrary direction and wrist orientation). Then, we extended to target-driven grasping in constrained environments and added collision awareness to our scene knowledge. When objects are densely cluttered, we improved the robot's efficiency by employing graph neural networks (GNN) to exploit the underlying relationships in the scene. To accomplish complex manipulation tasks in constrained environments, such as rearranging adversarial objects, we hierarchically integrated a heterogeneous graph neural network (HetGNN)-based coordinator and the 3D CNN-based actors. The system reasons about the relational knowledge between scene components and coordinates multiple robotic skills (e.g., grasping, pushing) to minimize the planning cost. As we anticipate an increase in the number of domestic robots, the robotics community necessitates a framework that not only commands the robot accurately, but also reasons about the unstructured scene to improve robots' efficiency. This thesis contributes to the goal by equipping robotics manipulation with learned scene knowledge. We present 6-DoF robotic systems that can grasp novel objects in dense clutter with reachability awareness, retrieve target objects within arbitrary structures, and rearrange multiple objects into goal configurations in constrained environments.
Energy-Aware Robotics For Environmental Monitoring
(2020-04) Plonski, Patrick
Outdoor autonomous robots carrying sensors have the potential to provide transformative data for science, industry, and safety. However, limited battery life is a factor that restricts their wide deployment. To fully realize the potential of these robots, this restriction must be loosened. We improve battery life by designing algorithms that allow outdoor sensors to intelligently plan their movements to minimize power consumption while taking the necessary measurements for their tasks. This dissertation focuses on three main problems: The first problem we address is the problem of planning energy-minimal trajectories for solar-powered robots in a cluttered environment that contains shadows. We make contributions to this problem in both planning and estimation. We present a novel planning algorithm that efficiently computes an optimal trajectory on a discretized grid, given a solar map and a target position. We also present three methods of estimating the solar map that relates position and time with the amount of solar energy that can be collected if the robot is in that position at that time, using only previous measurements of solar power tagged with positions and times. The first method uses Gaussian Process regression to directly predict the solar power at a position based on neighboring measurements. The second method uses spatiotemporal Gaussian Process classification to estimate the likelihood that a particular position will be in the sun. The third method uses the measurements of sun and shade to infer shadow-casting objects, and performs raytracing through the uncertain height map to estimate the likely solar map for any given sun angle. These methods are justified through field experiments and in simulation. The second problem we address is the problem of exploring unknown environments in a manner that guarantees bounded motion cost. We consider two variants of this problem. First we consider a boat, equipped with a sonar sensor, that encounters an obstacle while it is attempting to reach a destination. Second we consider a solar-powered vehicle attempting to map the shadows in an environment. For both variants, we present online algorithms and examine their performance using competitive analysis. In competitive analysis, the performance of an online algorithm is compared against the optimal offline algorithm. For obstacle avoidance, the offline algorithm knows the shape of the obstacle. For solar exploration, the offline algorithm knows the geometry of the shadow casting objects. We obtain an O(1) competitive ratio for obstacle avoidance, and an O(log n) competitive ratio for solar exploration, where n is the number of critical points to observe. The strategies for obstacle avoidance are validated through extensive field experiments, and the strategies for exploration are validated with simulations. The third problem we address is a novel coverage problem that arises in aerial surveying applications. The goal is to compute a shortest path that visits a given set of cones--this is the path that maximizes the coverage ability for a UAV with a limited battery size. The apex of each cone is restricted to lie on the ground plane. The common angle alpha of the cones represent the field of view of the onboard camera. The cone heights, which can be varying, correspond with the desired observation quality (e.g. resolution). This problem is a novel variant of the Traveling Salesman Problem with Neighborhoods (TSPN), and we call it Cone-TSPN. Our main contribution is a polynomial time approximation algorithm for Cone-TPSN. We analyze its theoretical performance and show that it returns a solution whose length is at most O(1 + log(hmax/hmin)) times the length of the optimal solution where hmax and hmin are the heights of the tallest and shortest input cones, respectively. We demonstrate the use of our algorithm in a representative precision agriculture application. We further study its performance in simulation using randomly generated cone sets. Our results indicate that the performance of our algorithm is superior to standard solutions. The results in this dissertation have advanced the state of the art in planning energy-minimizing trajectories for outdoor vehicles, by presenting algorithms with strong theoretical guarantees, justified in field experiments and simulations.
Enhancing Visual Perception in Noisy Environments using Generative Adversarial Networks
(2018-08) Fabbri, Cameron
Autonomous robots rely on a variety of sensors – acoustic, inertial, and visual – for intelligent decision making. Due to its non-intrusive, passive nature, and high information content, vision is an attractive sensing modality. However, many environments contain natural sources of visual noise such as snow, rain, dust, and other forms of distortion. This work focuses on the underwater environment, in which visual noise is a prominent component. Factors such as light refraction and absorption, suspended particles in the water, and color distortion affect the quality of visual data, resulting in noisy and distorted images. Autonomous Underwater Vehicles (AUVs) that rely on visual sensing thus face difficult challenges, and consequently exhibit poor performance on vision driven tasks. This thesis proposes a method to improve the quality of visual underwater scenes using Generative Adversarial Networks (GANs), with the goal of improving input to vision-driven behaviors further down the autonomy pipeline. Furthermore, we show how recently proposed methods are able to generate a dataset for the purpose of such underwater image restoration. For any visually-guided underwater robots, this improvement can result in increased safety and reliability through robust visual perception. To that effect, we present quantitative and qualitative data which demonstrates that images corrected through the proposed approach generate more visually appealing images, and also provide increased accuracy for a diver tracking algorithm.
Environmental Monitoring with Unmanned Aerial Vehicles
(2020-04) Stefas, Nikolaos
Recent advances in miniaturization of processing units, storage capacity, battery power and sensory equipment have allowed Unmanned Aerial Vehicles (UAVs) to perform environmental monitoring tasks with unprecedented speed and accuracy. Data collection is important for algorithms and systems that try to learn how the physical world works or try to interact with it. The number, variety and quality of the data directly affects the performance of these algorithms. In order to fully realize this vision we need to compliment it with efficient systems that can collect the required data. In this dissertation we develop new robotic solutions for fully automating monitoring and data collection in natural, outdoor environments. First, we study the design of an Unmanned Aerial Vehicle (UAV) for safe tree surface inspection flying at low altitude inside orchard type fields. The objective of this study is threefold. The system needs to collect complete sets of data for different types of data collection sensors. Furthermore, it has to be able to operate successfully under the effects of wind disturbances. Finally, the integrity of the field has to be guaranteed. To achieve this goal, we modify and integrate several methods and technologies including a non-standard distance-velocity Proportional-Integral-Derivative (PID) based controller and real time obstacle map navigation based on occupancy voxels. The resulting system demonstrates successful operation and data collection inside a honeycrisp apple orchard. The demonstration includes multiple tests across several days, under various weather conditions (e.g. sunlight, wind) to ensure consistency and was shown to be fully functional even during GPS signal loss. Second, we study the problem of high altitude optimal trajectory generation for capturing aerial image footage of known but difficult to see areas (e.g. under trees or structures, reflective surfaces). In this problem we consider the relation between the camera resolution and UAV altitude. We associate each camera image with an inverted cone apexed at the location of the interest. The height of each cone is associated with the desired resolution and the apex angle corresponds to camera field of view. In other words, each cone encodes the set of view points from which a target can be imaged at a desired location. We provide a polynomial time approximation algorithm that produces a close to optimal solution and was evaluated in existing applications. We analyze the performance of our strategy and demonstrate through simulations and field experiments that by exploiting the special structure of the cones we can achieve shorter flight times than previously available solutions. The strategy can be used with any number of cones and split coverage into multiple flights in order to account for limited battery power or storage capacity. Third, we describe a method that can localize and approach a radio signal source at an unknown location with UAVs. We start by fitting a multi-rotor UAV system with a small on-board computer and a directional antenna that can detect the signal source. We then model the area around the signal source based on the antenna radiation field and classify the locations in which we can or cannot obtain reliable directionality measurements (i.e. bearing measurements). The results of this modeling resemble a cone-like region above the signal source inside of which bearing measurements are unreliable. In order to verify that our modeling is realistic, we also collect data with a real UAV system. Using this modeling, we develop a “home-in” strategy that takes advantage of a UAV’s ability to change altitude and exploits the special structure of the modeled conic-like region in order to approach the signal source from above. We analyze the performance of our strategy and demonstrate through simulations and field experiments that by exploiting this structure we can achieve short flight times. In this dissertation we make progress towards the creation of robotic sensing solutions that satisfy two important criteria. The first criterion is to provide theoretical guarantees about the performance of the proposed solutions. This is achieved by mathematically proving what the worst case scenario is and using it as an upper bound. The second criterion is to demonstrate the feasibility of the proposed solutions in real world applications. This is achieved by providing practical implementations tested in both simulations and with robotic systems operating in realistic settings.
Extrinsic and intrinsic sensor calibration
(2013-12) Mirzaei, Faraz M.
Sensor Calibration is the process of determining the intrinsic (e.g., focal length) and extrinsic (i.e., position and orientation (pose) with respect to the world, or to another sensor) parameters of a sensor. This task is an essential prerequisite for many applications in robotics, computer vision, and augmented reality. For example, in the field of robotics, in order to fuse measurements from different sensors (e.g., camera, LIDAR, gyroscope, accelerometer, odometer, etc. for the purpose of Simultaneous Localization and Mapping or SLAM), all the sensors' measurements must be expressed with respect to a common frame of reference, which requires knowing the relative pose of the sensors. In augmented reality the pose of a sensor (camera in this case) with respect to the surrounding world along with its internal parameters (focal length, principal point, and distortion coefficients) have to be known in order to superimpose an object into the scene. When designing calibration procedures and before selecting a particular estimation algorithm, there exist two main issues of concern than one needs to consider: Whether the system is observable, meaning that the sensor's measurements contain sufficient information for estimating all degrees of freedom (d.o.f.) of the unknown calibration parameters; Given an observable system, whether it is possible to find the globally optimal solution.Addressing these issues is particularly challenging due to the nonlinearity of the sensors' measurement models. Specifically, classical methods for analyzing the observability of linear systems (e.g., the observability Gramian) are not directly applicable to nonlinear systems. Therefore, more advanced tools, such as Lie derivatives, must be employed to investigate these systems' observability. Furthermore, providing a guarantee of optimality for estimators applied to nonlinear systems is very difficult, if not impossible. This is due to the fact that commonly used (iterative) linearized estimators require initialization and may only converge to a local optimum. Even with accurate initialization, no guarantee can be made regarding the optimality of the solution computed by linearized estimators. In this dissertation, we address some of these challenges for several common sensors, including cameras, 3D LIDARs, gyroscopes, Inertial Measurement Units (IMUs), and odometers. Specifically, in the first part of this dissertation we employ Lie-algebra techniques to study the observability of gyroscope-odometer and IMU-camera calibration systems. In addition, we prove the observability of the 3D LIDAR-camera calibration system by demonstrating that only a finite number of values for the calibration parameters produce a given set of measurements. Moreover, we provide the conditions on the control inputs and measurements under which these systems become observable. In the second part of this dissertation, we present a novel method for mitigating the initialization requirements of iterative estimators for the 3D LIDAR-camera and monocular camera calibration systems. Specifically, for each problem we formulate a nonlinear Least-Squares (LS) cost function whose optimality conditions comprise a system of polynomial equations. We subsequently exploit recent advances in algebraic geometry to analytically solve these multivariate polynomial systems and compute the LS critical points. Finally, the guaranteed LS-optimal solutions are directly found by evaluating the cost function at the critical points without requiring any initialization or iteration.Together, our observability analysis and analytical LS methods provide a framework for accurate and reliable calibration of common sensors in robotics and computer vision.
Humanoid Robot - Human Interaction: Towards Compliance and Reciprocity with a Social Robot Through Completion of a Pregiving Favor
(2023-09) Moberg, Reilly
Understanding the social and natural relationships that humans have with ad-vanced technology is an extremely important consideration in the design and develop- ment of humanoid social robots. By perceiving the social rules within human-human interaction and applying them to human-robot interaction, social influence can lead to participants being more willing and eager to interact with a robot, resulting in the robot being used to its full potential. By combining the work done by Reeves and Nass, 2006 studying the media equa- tion with the social rule of reciprocity (Cialdini, 2008), we suggest that when a robot completes a pregiving favor for a human participant, then the human participant will be influenced by the social rule of reciprocation to comply by the robot’s later request. A phasic, between-subjects experiment (N = 72) using facial electromyography (zygomatic and corrugator) was conducted to learn more about how the natural, hu- man behavior of reciprocation can be applied to human-robot interaction. Measured in this study is the user’s valence of emotions, the user’s willingness to reciprocate a favor, and the measure of compliance based on the number of raffle tickets purchased by the user at the robot’s request. The results suggest that the social rule of recipro- cation exists within human-robot interaction and that when a robot offers a pregiving favor to a person, then that person is more likely to comply with the robot’s later request. In concluding, we discuss theoretical contributions, limitations, and avenues for future research.
Implementation of Robotics in Costumes and Theatre
(2015) Bockbrader, Hannah; Bias, Kelsie
Improved Computer Vision Algorithms for High-Throughput Targeting of Single Cells in Intact Tissue for Automated Microinjections
(2021-10) O'Brien, Jacob
Microinjection is a technique for organism-level and cellular-level manipulation of biological systems. The precise nature of microinjection permits the ability to target single cells in intact tissue which has enabled the study of cell-type related phenomena in development and disease progression. We envisioned the use of single-cellular microinjection as a tool for tagging cells with unique oligonucleotide barcodes that can be used during post-injection transcriptomic analysis to relate the transcriptomic reads with originally injected cells. For this process to be viable, we needed a system that was capable of precisely identifying the locations of cells in 3D tissue, assessing their feasibility for injection, and conducting rapid and large-scale microinjection into the identified cells. In this thesis, we report the development of such system. Our automated system uses computer vision algorithms to identify the 3D position of epifluorescent cells in intact tissue slices and assign them a quality metric to prioritize injections. The system guides a robotic micromanipulator to these cells and attempts injections while another computer vision algorithm and Kalman filter are used to improve the robot’s positioning accuracy. Additionally, cell impalement and cell filling detection algorithms were developed to evaluate injection success. We discovered, through a microinjection parameter sweep, an optimum combination of parameters to enable successful microinjection into a variety of cell types and tissue types. We used the optimized parameters to demonstrate automated tagging of single cells with a fluorescently labeled antibody targeting the nuclear pore complex proteins as a precursor step to fluorescence-based nuclei sorting and later transcriptomic analysis.
Improving the Safety and Efficiency of Roadway Maintenance Phase I: Developing a Robotic Roadway Message Painter Prototype
(Intelligent Transportation Systems Institute, Center for Transportation Studies, University of Minnesota, 2012-05) Rosandich, Ryan G.
A large-scale prototype for a robotic roadway message painter was developed, built, and tested. The system is a gantry-style robot capable of painting a four-by-eight-foot area and is based on off-the-shelf linear motion components, readily available motion control hardware, and commercial operator interface software. The system is mounted on a modified trailer that can be manually rolled around for positioning or towed behind a vehicle. The system is equipped with a standard automatic paint head and airless paint pump. Software was developed for the system that enables it to paint a variety of characters and symbols on the roadway. An operator interface was also developed that allows an operator to easily select the painting operation to be conducted and to monitor and control the actual painting process. The software resides in a laptop computer that communicates with the robotic painting system in real-time using a dedicated Ethernet connection. The system was used to determine the feasibility of painting with or without stencils and to determine many design parameters for the eventual development of a commercially viable system for painting symbols and messages on roadways. It is expected that the system will eventually enable states, counties, and municipalities to improve the safety, productivity, and flexibility of their pavement marking operations.
Improving the Safety and Efficiency of Roadway Maintenance Phase II: Developing a Vision Guidance System for the Robotic Roadway Message Painter
(Intelligent Transportation Systems Institute, Center for Transportation Studies, University of Minnesota, 2013-02) Rosandich, Ryan G.
Repainting existing roadway markings (turn arrows, STOP messages, railroad crossings, etc.) is an important task for transportation maintenance organizations. MnDOT estimates that over 75% of symbol and message painting is the repainting of existing markings. It would be extremely valuable for an automated painting system to have a vision guidance capability whereby an existing mark could be repainted accurately with little operator input. In this project a vision system was developed that is capable of identifying existing painted pavement markings and determining their dimensions, location, and orientation. Techniques were also developed whereby this information could be used to determine the location of the marking in the workspace of a painting robot to enable it to accurately repaint the marking. The vehicle-mounted robotic painter is still being built and tested, so final test results will not be available until the vision system can be completely integrated with the painter, and the two can be tested together. The accuracy of the projection produced using the techniques developed in this project would suggest that the final system will be capable of repainting pavement markings almost exactly where they appear on the roadway. Expected benefits of the deployment of a vision-guided robotic painting device include improved operator safety, improved productivity, and improved flexibility in roadway marking and repainting operations. Eventual users of a device using this technology could be city, county, state, and federal government agencies and private companies or contractors.
Learning from Pixels: Image-Centric State Representation Reinforcement Learning for Goal Conditioned Surgical Task Automation
(2023-11) Gowdru Lingaraju, Srujan
Over the past few years, significant exploration has occurred in the field of automating surgical tasks through off-policy Reinforcement Learning (RL) methods. These methods have witnessed notable advancements in enhancing sample efficiency (such as with the use of Hindsight Experience Replay - HER) and addressing the challenge of exploration (as seen in Imitation Learning approaches). While these advancements have boosted RL model performance, they all share a common reliance on accurate ground truth state observations. This reliance poses a substantial hurdle, particularly in real-world scenarios where capturing an accurate state representation becomes notably challenging.This study addresses the aforementioned challenge by exploiting an Asymmetric Actor-Critic framework while addressing the issues of sample efficiency and exploration burden by using HER and behavior cloning. Within this framework, the Critic component is trained on the complete state information, whereas the Actor component is trained on partial state observations, thus diminishing the necessity for pre-trained state representation models. The proposed methodology is evaluated within the context of SurRoL, a surgical task simulation platform. The experimental results showcased that the RL model, operating with this configuration, achieves task performance akin to models trained with complete ground truth state representations. Additionally, we delve into the necessity for Sim-to-Real transfer methods and elucidate some of the formidable challenges inherent in this process and present a comprehensive pipeline that addresses the intricacies of domain adaptation. This research thus presents a promising avenue to mitigate the reliance on pre-trained models for state representation in the pursuit of effective surgical task automation.

University Digital Conservancy

Browse by Subject

Browsing by Subject "Robotics"