Computer Vision Algorithms for Yield Mapping in Specialty Farms
2019-11
Loading...
View/Download File
Persistent link to this item
Statistics
View StatisticsJournal Title
Journal ISSN
Volume Title
Title
Computer Vision Algorithms for Yield Mapping in Specialty Farms
Authors
Published Date
2019-11
Publisher
Type
Thesis or Dissertation
Abstract
Precision farming and phenotyping technologies have the potential to drastically transform the agricultural landscape. For commodity crops such as maize, wheat and soy recurring farming tasks such as seeding, weeding, irrigation, fertilization, application of pesticides, harvesting, and storage are in the process of being completely automated. Specialty crops (tree fruit, flowers, vegetables, and nuts) are excellent candidates for similar automation as they have high monetary value, high management cost and high variability in growth. An important capability for both precision agriculture and phenotyping is yield mapping. Yield mapping for tree fruit is challenging because it involves solving multiple computer vision (fruit detection, counting, recovering underlying 3D geometry for tracking fruit across different frames in continuously changing illumination) as well as planning problems (path planning for covering all fruit, picking fruit). The main goal of this dissertation is to develop computer vision and deep learning algorithms for yield mapping in specialty farms. The dissertation is divided into three parts. The first part is dedicated to developing practical solutions for yield mapping in specialty farms. We present solutions for fruit detection, counting, recovering the underlying scene geometry and fruit tracking. We integrate these individual solutions in a modular manner and create a flexible framework for complete yield estimation. Additionally, we perform an extensive experimental evaluation of the developed system and sub-components. Our algorithms successfully predict 97% of the ground truth yield and outperform all existing state-of-the-art methods. Some of these efforts are now in the process of being commercialized. In the second part of the dissertation, we study a problem where a manipulator equipped with a camera, mounted on a ground robot is charged with accurately counting fruit by taking a minimum number of views. We present a method for efficiently enumerating combinatorially distinct world models most likely to generate the captured views. These are incorporated into single and multi-step planners for accurate fruit counting. We evaluate these planners in simulation as well as with experiments on a real robot. In the third part, we study the problem of realistic synthetic data generation for training deep neural networks. We present a method that jointly translates the synthetic images and their underlying semantics to the domain of the real data so that an adversarial discriminator (a deep neural network) cannot distinguish between the real and synthetic data. This method enables us to stylize the synthetic data to any fruit, lighting condition and environment. It can be applied to a wide variety of domain transfer tasks beyond fruit detection and counting (e.g from Grand Theft Auto (GTA) to Cityscapes for autonomous driving). Additionally, it enables us to perform image to image translation with significant changes in underlying geometry (e.g circles to triangles, sheep to giraffe, etc). These results in this dissertation together present a complete yield monitoring system for specialty crops, view planning strategies for accurate fruit counting and a framework for generating realistic synthetic data. These methods together push the state-of-the-art and take us one step closer toward building a sustainable infrastructure for intelligent integrated farm management.
Description
University of Minnesota Ph.D. dissertation. November 2019. Major: Computer Science. Advisor: Volkan Isler. 1 computer file (PDF); xii, 175 pages.
Related to
Replaces
License
Collections
Series/Report Number
Funding information
Isbn identifier
Doi identifier
Previously Published Citation
Suggested citation
Roy, Pravakar. (2019). Computer Vision Algorithms for Yield Mapping in Specialty Farms. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/211321.
Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.