Towards Natural Underwater Human-Robot Interaction: Pointing Gesture Recognition for Autonomous Underwater Vehicles

Abstract

Underwater robotics is a motivating field of research with a wide variety of both industrial and scientific applications. In particular, the development of autonomous underwater vehicles to assist divers in performing difficult, dangerous, or undesirable tasks has the potential to expand our abilities in the aquatic domain while reducing the risks presented to divers. For a diver and an autonomous underwater vehicle to work in collaboration, there must be an established interaction protocol; the study of such protocols is central to the field of human-robot interaction. In the underwater domain, the attenuation of both electromagnetic signals and sound limits these traditional communication protocols, leaving machine vision as the primary perception methodology. Thus gestures become a natural choice for diver-robot communication. Since pointing gestures are represented and recognized in cultures around the world, they serve as a foundational, natural gesture for divers in a demanding aquatic environment. Thus, in this work we lay the groundwork for implementing a pointing gesture recognition algorithm for use onboard autonomous underwater vehicles. Specifically, we contribute a human study of individuals performing four classes of pointing gestures, three datasets developed to study pointing gestures, and an analysis of four state-of-the-art object detection frameworks for recognizing pointing gestures in the aquatic domain.