ParaGlassMenu: Towards Social-Friendly Subtle Interactions in Conversations

Document Title

ParaGlassMenu: Towards Social-Friendly Subtle Interactions in Conversations

Document Information

• Research Domain: Human-Computer Interaction, Smart Device Interaction, Social Interaction Acceptability
• Keywords: IoT Device Control, Circular Menu, Ring Interaction, Social Interaction, Conversation, HMD, Smart Glasses, Internet of Things

Research Background and Issues

• What problems or challenges did the authors identify? Using digital devices during social interactions reduces social engagement and disrupts conversations. Existing interaction methods struggle to maintain users' attention on conversation partners in social settings, often interfering with eye contact and increasing cognitive load during operation.

• Why is this problem important? Minimizing disruptions to conversations when switching between single-tasking and social interactions is crucial. As IoT devices become more prevalent, there is a growing need for convenient and unobtrusive ways to operate digital devices in social scenarios.

• Research Motivation and Related Work

  • Many studies have proposed subtle interaction techniques (e.g., Jaw-Teeth interaction, hidden vibration cues), but these approaches face limitations in flexibility, scalability, and adaptation to social contexts.
  • IoT device control involves complex tasks such as information retrieval, discrete and continuous operations, yet lacks a unified design that supports all interaction needs while fitting social scenarios.

Solution

• What methods or solutions did the authors propose? The authors designed a technology called ParaGlassMenu, which displays a semi-transparent circular menu on optical see-through head-mounted displays (OHMD) and enables discreet operation through a ring mouse. This menu is presented around the conversation partner's face to support socially friendly IoT device control.

• What are the innovative aspects of this solution?

  1. Utilizing OHMD to display a circular menu, allowing users to operate within their peripheral vision without disrupting the conversation.
  2. The ring mouse enables discreet input methods, reducing interference with others.
  3. Supporting both discrete and continuous operations, including quick task-switching functionality.
  4. Menu design accommodates hierarchical scalability, adapting to a large number of functional commands.

• What are the implementation steps and key technologies used?

  1. The menu surrounds the conversation partner, with optimized icon colors and transparency to avoid obstructing dialogue.
  2. Ring mouse operations include button clicks for selection and touchpad scrolling for continuous options.
  3. Menu activation and deactivation are controlled via the ring mouse, minimizing unnecessary visual distractions.
  4. System development is based on Unity and OpenCV, enabling facial tracking and interaction tasks.

Research Outcomes

• What specific results were achieved?

  • In simulation experiments, ParaGlassMenu achieved the best conversation quality (highest facial attention, politeness, and naturalness scores) and IoT device control performance (shortest task time, highest accuracy).
  • Its usability was further validated in real social scenarios, where users found the system efficient for task completion while maintaining conversational flow.

• What advantages does it have compared to existing solutions?

  • Compared to smartphones and voice assistants, ParaGlassMenu significantly reduces the impact of tasks on conversation quality.
  • Particularly in complex tasks (e.g., scrolling through song selections), it demonstrated lower cognitive load than linear menu layouts.

• What were the experimental or evaluation results?

  • Experimental results showed that ParaGlassMenu supports IoT control across discrete, continuous, and list selection tasks.
  • User ratings indicated it as the most intuitive and efficient interaction method.
  • In real conversations, hosts were able to flexibly manage IoT tasks through discreet or transparent interaction methods.

• Limitations and Future Directions

  • Current OHMD hardware design (e.g., weight and lenses) does not fully accommodate long-term wearing comfort.
  • The ring mouse's scrolling precision requires further optimization.
  • The sample of experimental participants primarily consisted of university groups; generalizability across different cultural backgrounds and age groups needs further validation.
  • Future research could explore more complex real-time functionalities, such as voice-guided transparent selection, and expand application scenarios to drone control, remote learning, etc.