Understanding Large-Language Model (LLM)-powered Human-Robot Interaction

Generative AI, particularly large-language models (LLMs), hold significant promise in improving human-robot interaction. LLM-powered robots can not only maintain greater conversational capabilities, but they can also handle open-ended user requests across a wide range of tasks and domains. Despite the potential to transform human-robot interaction, very little is known about the distinctive design requirements for utilizing LLMs in robots, which may differ from other interaction modalities such as text and voice, and how these requirements might change across tasks and contexts. To better understand these requirements, we conducted a user study (n=32) that compared an LLM-powered social robot against two other agents---a text-based agent and a voice-based agent. To understand how these requirements differed across tasks, participants completed one of four conversational tasks: choose, generate, execute, and negotiate. Our findings show that LLM-powered robots elevate expectations for sophisticated non-verbal cues. While they excel in connection-building and deliberation tasks, they are less preferred for challenges in logical communication and anxiety-inducing situations. We provide design implications both for robots integrating LLMs and for fine-tuning LLMs for use with robots.