Annotation of 3D Object Pose from Egocentric Video

Abstract

Current datasets for human egocentric vision have been focused on 2D tasks such as object segmentation, object detection, and activity recognition. For extended tasks such as AR and robotics, 3D understanding is needed to learn human skills. A new dataset called Indoor 3D Egocentric Object (IDEO) was created to address the shortcomings of existing datasets. Part of this dataset includes annotated 9-DOF pose of objects being manipulated by hand, which introduces nontrivial characteristic occlusion and uncommon object poses. A novel annotation tool was created to streamline the annotation process allowing for fast and accurate annotation by untrained crowd workers for the 9-DOF pose (translation, rotation, scale) of objects being manipulated by hand in egocentric images.