Toward Enabling Natural Conversation with Older Adults via the Design of LLM-Powered Voice Agents that Support Interruptions and Backchannels

Research Background and Issues

• What problems or challenges did the authors identify?
  Current voice assistants for older adults primarily follow a strict turn-taking dialogue model ("speak-then-wait" mode), lacking interruption and response features commonly found in natural conversations. This rigid dialogue model fails to match the experience of natural human interaction, making it difficult for older adults to feel a deep connection and emotional engagement with voice assistants.

• Why is this issue important?
  Social isolation and loneliness are major risks faced by older adults, impacting their physical and mental health. Studies have shown that natural and emotionally engaging conversations can provide emotional companionship to older adults, alleviate loneliness, and enhance their willingness to interact. Moreover, traditional voice assistants struggle to accommodate the linguistic habits of older adults (e.g., short phrases, hesitations), failing to effectively support their needs.

• Research Motivation and Related Work
  Recent advancements in voice assistant technologies, such as large language models (LLMs) and multimodal interaction models, offer opportunities to improve natural dialogue. While LLMs have significantly enhanced context awareness and the naturalness of dialogue generation, enabling interruption and response functionalities in voice interactions to meet the specific needs of older adults remains an unresolved challenge. Existing research primarily focuses on task-oriented dialogues, lacking support for social conversations tailored to older adults.

Solution

• What methods or solutions did the authors propose?
  The authors proposed an LLM-driven "Barge-in" voice agent that supports interruptions (including "cooperative" and "competitive" interruptions) and responses (e.g., "mm-hmm," "yes") commonly found in natural conversations. They optimized the interaction experience for older adults through a series of design considerations (DCs). These include prioritizing cooperative interruptions, simplifying language structures, providing personalized settings, recognizing user intentions, and proactively detecting and addressing conversational silences.

• What are the innovative aspects of this solution?

  1. Introducing interruption and response functionalities to simulate natural dialogue processes, enhancing conversational flexibility by allowing bidirectional interruptions (both user and agent can interrupt).
  2. Optimizing the agent's interaction strategies based on five types of interruptions (sentence completion, clarification/questioning, turn-taking, disagreement, topic switching), balancing naturalness and user experience.
  3. Using preset filler words to mask response delays under system latency, reducing users' perceived waiting time.
  4. Proactively interrupting silences to ensure conversational continuity and prevent "conversation termination" scenarios.

• What are the implementation steps and key technologies used?

  1. Interruption Multi-Agent Module: Utilizes STT (Speech-to-Text) components (iFlytek), LLM (Alibaba Doubao), and TTS (Text-to-Speech) modules to achieve real-time speech recognition, dialogue generation, and voice output; supports context-based detection of interruption opportunities and corresponding actions.
  2. User Interruption Module: When users interrupt the agent, keyword recognition and voice detection are used to quickly stop the agent's output and respond to the user's content.
  3. Proactive Response Module: For conversational silences, the agent reinitiates dialogue based on the context of previous interactions.
  4. Design Considerations (DCs): Implements five optimization measures, such as adjusting interruption frequency and wait times, simplifying language structures, and ensuring compatibility with older adults' cognitive and auditory processing capabilities.

Research Outcomes

• What specific outcomes were achieved?

  1. Compared to traditional voice agents (without interruption functionality), the "Barge-in" agent significantly enhanced the naturalness and interactivity of dialogues with older adults.
  2. The interruption functionality increased interaction frequency, while the response functionality reduced users' perceived waiting time. The proactive interruption of silences ensured conversational fluidity.
  3. Cooperative interruptions (e.g., sentence completion and clarification/questioning) were well-received by most participants, aiding the coherence of older adults' thoughts and memories.
  4. Experimental data showed that the Barge-in agent significantly increased user dialogue turns (32.31 vs. 21.94) and the number of topics discussed (average 3.88 vs. 2.50).

• What advantages does it have compared to existing solutions?

  • Supports bidirectional interruption mechanisms and diverse interruption types, making dialogues more human-like.
  • Proactively detects and optimizes conversational silences, increasing engagement and avoiding awkward pauses.
  • Adjusts speech speed, language complexity, and interaction frequency to meet the specific needs of older adults (e.g., memory decline, auditory impairment).

• What were the experimental or evaluation results?
  A user comparison experiment involving 16 older participants demonstrated that the Barge-in agent improved three key metrics: Focused Attention, Aesthetic Appeal, and Interaction Reward. Participants also noted that the agent with interruption mechanisms felt more like an "old friend" rather than a mechanical command-response system.

• Limitations and Future Directions

  1. Task-Oriented Capabilities: The current system primarily focuses on emotional companionship, with performance in specific tasks (e.g., information retrieval, scheduling) yet to be validated.
  2. Participant Scope: The experiment mainly involved close relationships (spouses/friends); future research could expand to intergenerational dialogues (e.g., interactions with grandchildren).
  3. Latency Management Optimization: The processing speed of LLMs affects dialogue fluidity, necessitating the use of more efficient models or optimized architectures.
  4. Multimodal Interaction: Incorporating visual cues such as facial expressions and gestures could further enhance the agent's emotional responsiveness and naturalness.

Overall, this study successfully demonstrates a new paradigm for designing human-like voice agent systems, providing older adults with more meaningful and interactive conversational experiences. It also identifies key areas for future research, such as task-oriented capabilities, multimodal interaction, and dynamic adaptation features.