Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented Reality

Title of the Paper

Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented Reality

Paper Information

• Subject Area: Design and optimization of mid-air gesture interactions in Augmented Reality (AR)
• Keywords: Augmented Reality, mid-air gestures, rotational interaction, ergonomics, user interface design

Research Background and Problem

• Identified Issues or Challenges:

  • Mid-air gestures are widely used in AR but may lead to the "gorilla arm effect," causing shoulder and arm fatigue and discomfort.
  • Current research focuses primarily on simple one-dimensional interactions (e.g., clicking or pushing buttons) while neglecting complex tasks (e.g., rotating knobs).
  • There is a lack of in-depth understanding of how rotational interactions in mid-air gestures affect user comfort, accuracy, and behavior.

• Significance:

  • Rotational interactions are widely applied in AR (e.g., manipulating virtual objects or controlling interfaces), but existing designs have not fully considered ergonomic factors.

• Motivation and Related Work:

  • Enhancing the comfort and usability of mid-air gesture design is critical for the development of AR.
  • Previous studies have proposed metrics for quantifying arm fatigue (e.g., Consumed Endurance and RULA), but most have focused only on translational interactions of the hand, without delving into rotational interactions.

Solution

• Proposed Solution:

  • Design and conduct experiments to systematically evaluate the effects of rotational interactions under different postures, task axes, and spatial positions on users.
  • Propose an ergonomics-centered design space for rotational interactions and provide recommendations for future AR interface design.

• Innovations:

  • Focus on complex rotational interactions (involving coordinated movements of the hand, arm, and chest) rather than simple translational tasks.
  • Propose specific design methods to optimize the comfort of rotational interactions, such as positioning interactions closer to the user's body.

• Implementation Steps and Techniques:

  • Experimental Design:
    • Participants perform tasks involving rotating virtual knobs using mid-air gestures, with real-time recording of task accuracy, interaction counts, and comfort ratings.
    • Controlled variables include posture (standing/sitting), interaction distance (near/far), and knob rotation axis (X/Y/Z).
  • Equipment Used:
    • AR head-mounted display (Microsoft HoloLens 2) for task rendering and gesture tracking.
    • OptiTrack system to measure participants' posture and motion parameters.

Research Findings

• Specific Findings:

  1. Comfort:
     • Interactions closer to the body significantly improved user comfort, especially in a seated posture (comfort increased by 22.12%).
     • The rotation axis had a significant impact on comfort (Y-axis rotation was more suitable than X/Z axes) and interaction count.
  2. Accuracy:
     • Interaction positions closer to the body also improved task accuracy, with an 8% increase in accuracy in the seated posture.
     • Rotational interactions along the vertical axis (Y-axis) required the fewest interaction counts, making them more efficient than other axes.
  3. Task Design Recommendations:
     • Whenever possible, interaction positions should be designed closer to the body to optimize comfort, while tasks requiring standing and free movement can be placed farther away.
     • Rotational designs along the upper axis (Y-axis) are particularly important for reducing ineffective interactions and improving efficiency.

• Advantages over Existing Solutions:

  • Integrates ergonomics into the design of complex rotational interactions, providing concrete principles to enhance the comfort of AR user interface design.

• Experimental and Evaluation Results:

  • Through experiments involving 19 participants under 72 conditions (2 postures × 27 task positions × 3 axes), the core effects of interaction distance and rotation axis on comfort and efficiency were validated.
  • The results support specific optimization recommendations for the spatial distribution of interaction tasks and rotational axes.

• Limitations and Future Directions:

  • Limitations:
    • The study sample size (19 participants) only included young adults, and the generalizability of the results to different age groups and physiological characteristics remains unverified.
    • Technical limitations: The accuracy of gesture tracking by AR devices may have influenced the experimental results.
  • Future Directions:
    • Expand the study sample to include diverse populations (e.g., children, older adults).
    • Explore different rotational directions and more complex interaction tasks to further enhance user experience.
    • Introduce real-time muscle fatigue measurement techniques (e.g., electromyography) to more precisely quantify user comfort and fatigue.

Conclusion

This study systematically explores the ergonomic issues of complex rotational interactions in augmented reality and proposes specific optimization solutions and design recommendations. The findings provide robust support for future AR user interface design, with the potential to significantly enhance user comfort and interaction efficiency while reducing discomfort and fatigue during prolonged use.