Robot-assisted Inside-mouth Bite Transfer using Robust Mouth Perception and Physical Interaction-Aware Control

Robot-assisted feeding can greatly enhance the lives of those with mobility limitations. Modern feeding systems can pick up and position food in front of a care-recipient's mouth for a bite. However, many with severe mobility constraints cannot lean forward and need direct inside-mouth food placement. Inside-mouth bite transfer demands precision, especially for those with restricted mouth openings, and adeptly managing various physical interactions — incidental contacts as the utensil moves inside, impulsive contacts due to sudden muscle spasms, deliberate maneuvers by the person being fed to guide the utensil, and intentional bites. In this paper, we propose an inside-mouth bite transfer method that addresses these challenges with two key components: a multi-view mouth perception pipeline robust to tool occlusion, and a control mechanism that employs multimodal time-series classification to discern and aptly react to different physical interactions. We demonstrate the individual efficacy of these components through two ablation studies. In a comprehensive user study, our system successfully fed 13 care-recipients with diverse mobility challenges. Participants consistently emphasized the comfort and safety of our inside-mouth bite transfer approach, and gave it high technology acceptance ratings, underscoring the transformative potential of this method in real-world scenarios.