What is your other hand doing, robot? A model of behavior for shopkeeper robot's idle hand

In retail settings, a robot's one-handed manipulation of objects can come across as thoughtless and impolite, thus creating a negative customer experience. To solve this problem, we first observed how human shopkeepers interact with customers, specifically focusing on their hand movements during object manipulation. From the observation and analysis of shopkeepers' hand movements, we identified an essential element of their idle hand movements: "support" provided by the idle hand as the primary hand manipulates an object. Based on this observation, we proposed a model that coordinates the movements of a robot's idle hand with its primary task-engaged hand, emphasizing its supportive behaviors. In a within-subjects study, 20 participants interacted with robot shopkeepers under different conditions to assess the impact of incorporating support behavior with the idle hand. The results show that the proposed model significantly outperforms a baseline in terms of politeness and competence, suggesting enhanced object-based interactions between the robot shopkeepers and customers.