Sharing the Load - Offloading Processing and Improving Emotion Classification for the SoftBank Robot Pepper""

Abstract

Pepper is a humanoid robot created by SoftBank Robotics that was designed and built with the purpose of being used for robot-human interaction. There is an application interface that allows development of custom interactive programs as well as a number of built-in applications that can be extended and used when creating other custom programs for the robot. Among the pre-installed applications are applications that will classify a person's emotion and mood using data from several data points including facial characteristics and vocal pitch and tone. Due to the Covid-19 pandemic many people have been wearing face masks in both public and private areas. Detecting emotions based on facial recognition and voice tone analysis may not be as accurate when a person is wearing a mask. An alternative method that can be used to classify emotion is to analyze the actual words that are spoken by a person. However, this feature is not currently available on Pepper. In this study we describe a software solution that will allow Pepper to perform sentiment classification based on spoken words using a neural network. We will describe the testing procedure that was used to interview participants by Pepper and compare the F1 score of each classification method with each other. Pepper was able to be programmed to use a neural network for emotion classification. A total of 32 participants were interviewed, with the NLP spoken-word analysis classification achieving an averaged F1 score of .2860 as compared to the built-in software average F1 scores of .2362 from the mood application, .1986 from the vocal tone and pitch application, and .0811 from the facial characteristics application.