Efficient Robotic Manipulation with Scene Knowledge
2023-05
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
Efficient Robotic Manipulation with Scene Knowledge
Authors
Published Date
2023-05
Publisher
Type
Thesis or Dissertation
Abstract
In recent years, robots have transformed manufacturing, logistics, and transportation. However, extending the success to unstructured real-world environments (e.g., domestic kitchens, warehouses, grocery stores, etc.) remains difficult due to three key challenges: (1) assumption of structured environments (such as organized bottles in the factories); (2) hand-engineered solutions that are difficult to generalize to novel scenarios; (3) limited flexibility of action primitives, which prevents the robot from reaching target objects. In this thesis, we address these challenges by learning scene knowledge that improves the efficiency of robotic manipulation systems. Grasping is a fundamental manipulation skill that is constrained by the scene arrangement (i.e., the locations of the robot, the objects, and the environmental structures). Understanding scene knowledge, such as the robot's reachability to objects, is crucial to improve the robot's capability. We developed a reachability-aware grasp pose generator that predicts feasible 6-degree-of-freedom (6-DoF) grasp poses (i.e., approaching with an arbitrary direction and wrist orientation). Then, we extended to target-driven grasping in constrained environments and added collision awareness to our scene knowledge. When objects are densely cluttered, we improved the robot's efficiency by employing graph neural networks (GNN) to exploit the underlying relationships in the scene. To accomplish complex manipulation tasks in constrained environments, such as rearranging adversarial objects, we hierarchically integrated a heterogeneous graph neural network (HetGNN)-based coordinator and the 3D CNN-based actors. The system reasons about the relational knowledge between scene components and coordinates multiple robotic skills (e.g., grasping, pushing) to minimize the planning cost. As we anticipate an increase in the number of domestic robots, the robotics community necessitates a framework that not only commands the robot accurately, but also reasons about the unstructured scene to improve robots' efficiency. This thesis contributes to the goal by equipping robotics manipulation with learned scene knowledge. We present 6-DoF robotic systems that can grasp novel objects in dense clutter with reachability awareness, retrieve target objects within arbitrary structures, and rearrange multiple objects into goal configurations in constrained environments.
Keywords
Description
University of Minnesota Ph.D. dissertation. May 2023. Major: Electrical/Computer Engineering. Advisor: Changhyun Choi. 1 computer file (PDF); xiv, 93 pages.
Related to
Replaces
License
Collections
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Lou, Xibai. (2023). Efficient Robotic Manipulation with Scene Knowledge. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/259766.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.