Investigating Effect of Altered Auditory Feedback on Self-Representation, Subjective Operator Experience, and Task Performance in Teleoperation of a Social Robot

Title of the Paper

Investigating Effect of Altered Auditory Feedback on Self-Representation, Subjective Operator Experience, and Task Performance in Teleoperation of a Social Robot

Paper Information

• Subject Area: Human-Computer Interaction, Social Robots, Auditory Feedback
• Keywords: Social Robots, Teleoperation, Altered Auditory Feedback, Self-Representation, User Experience, Voice Transformation, Virtual Avatar, Human-Computer Interaction

Research Background and Problem

• Problem Description: In teleoperating social robots, operators need to "act as" the robot, meaning their voice must align with the robot's appearance. However, achieving natural real-time voice transformation remains technically challenging, which may affect operators' self-representation, task usability, and service quality.
• Significance: A mismatch between appearance and voice can lead to inconsistent or even uncomfortable impressions of the robot, reducing user acceptance in service scenarios. Optimizing teleoperation interfaces to lower operators' mental burden and improve user experience is a crucial step in advancing the deployment of social robots for service delivery.
• Motivation and Related Work: Previous studies have primarily focused on customers' perceptions and behaviors toward robots (e.g., acceptance and satisfaction) while paying less attention to the operators' experiences. This study aims to explore whether altered auditory feedback (AAF) can help operators strengthen their identity as the robot, thereby enhancing work experience and service quality.

Proposed Solution

• Method Overview: The study proposes combining voice transformation (VT) with altered auditory feedback (AAF) to match the robot's appearance and personality settings. Experiments are conducted to investigate the impact of this design on operators' self-representation, subjective experience, and task performance.
• Innovations:
  1. Introducing AAF into social robot teleoperation to help operators more easily "become" the robot and improve task experience.
  2. Developing a low-latency real-time voice transformation system to avoid the suppressive effects of auditory feedback delay on speech.
• Implementation Steps and Key Technologies:
  • Experimental Design: Participants remotely operate a robot to complete service tasks under three voice transformation conditions: no voice transformation (No-VT), voice transformation only (VT-only), and voice transformation with auditory feedback (VT-AAF).
  • Measurement Metrics: Changes in self-representation (via questionnaires, implicit association tests, and voice pitch changes), subjective task evaluations (NASA-TLX scale and questionnaires), and task performance (conversation duration and number of utterances).
  • Voice Transformation System: A hardware-based voice effects processor is used to achieve real-time voice transformation with an average delay of approximately 5 milliseconds.

Research Findings

• Specific Results:
  • Changes in Self-Representation: The VT-AAF condition significantly enhanced operators' sense of self-identity, including their perception and portrayal of the child-like robot role (e.g., feeling more extroverted, robotic, and child-like).
  • Improved Subjective Experience: Compared to VT-only and No-VT, VT-AAF significantly increased task enjoyment, motivation, and role immersion. Task-related stress was significantly reduced under both VT-only and VT-AAF conditions.
  • Task Performance: Task performance (conversation duration and number of utterances) showed no significant differences across the voice conditions.
• Comparison with Existing Solutions:
  • Compared to solutions relying solely on voice transformation, VT-AAF demonstrated clear advantages in subjective experience (e.g., role immersion and enjoyment), although it did not significantly outperform traditional methods in objective task performance.
  • AAF places greater emphasis on operators' "role-playing" ability rather than solely focusing on task efficiency.
• Limitations and Future Directions:
  • Limitations: The experiment fixed the robot's appearance and specific role attributes, requiring further validation of AAF's applicability to other types of robots. Additionally, the effects of long-term use and multi-task scenarios remain unexplored.
  • Future Research Directions:
    • Investigating the effects of AAF in applications involving robots with different appearances and personality settings.
    • Exploring the potential for operators to autonomously select voice targets or customize voice characteristics.
    • Extending the application to virtual avatars or broader virtual interaction scenarios such as VR shopping.
    • Optimizing voice transformation and AAF technologies to reduce discomfort during use while improving the naturalness of transitions.

Output Format

The study provides an innovative perspective on voice-assisted design for the teleoperation of social robots and proposes a method to enhance self-identity through real-time auditory feedback, laying a foundation for future research and applications in related fields.