User-Directed Autonomous Image Acquisition for 3D Object Reconstruction Using Eye-in-Hand Camera on Quadruoped Robot

Abstract

This research presents a system to automate the process of capturing images of an object for use in image-based 3D reconstruction by use of a quadruped robot equipped with an eye-in-hand camera. While image-based reconstruction techniques have substantially lowered the barrier to entry for high-quality 3D asset creation, the process of capturing the images necessary for a quality reconstruction either requires a great amount of time or an expensive multi-camera array. The presented system requires only a single 2D bounding box on a camera image to indicate the target object. The workflow begins with object segmentation prompted by the provided bounding box and calculated by Meta’s SAM2 segmentation model. The system uses the produced segmentation mask to calculate a segmented point cloud of the object. This data, combined with information from the robot’s occupancy grid is used to generate a ring of camera poses surrounding the object from which images are taken. Evaluation compared the output of the system with images manually taken with an iPhone 16 Pro. While the system’s output were able to reconstruct the object, the quality was substantially lower than that of the manually taken photos.