Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor Impairments

Title of the Paper

Understanding Gesture Input Articulation with Upper-Body Wearables for Users with Upper-Body Motor Impairments

Paper Information

• Research Area: Human-Computer Interaction, Wearable Devices, Accessibility Technology
• Keywords: Upper-body motor impairments, smartwatches, smart glasses, smart rings, gesture input, accessible input, wearable devices

Research Background and Problem

• Background: With the increasing prevalence of smart wearable devices (e.g., smartwatches, smart rings, smart glasses), gesture input has become an increasingly important interaction method. However, there is still limited research focusing on users with upper-body motor impairments. The design of such devices often assumes a certain level of motor ability, which may pose additional challenges for users with motor impairments.
• Problems and Challenges:
  • Most current device interaction methods assume users have good motor control of their fingers, wrists, and arms to perform precise swipes, taps, and other gestures. This creates challenges for users with motor impairments.
  • Existing research primarily focuses on general consumers and lacks in-depth quantitative analysis of how users with motor impairments perform gesture inputs on wearable devices.
• Significance: Understanding the gesture usage patterns of users with motor impairments on wearable devices is crucial for improving the accessibility and universal design of these devices.
• Motivation: There is a lack of comprehensive performance studies on gesture input for smartwatches, rings, and glasses, particularly in terms of the detailed quantitative analysis of how users with upper-body motor impairments perform on devices worn in different positions.

Solution

• Methods and Design:

  • The authors conducted experiments to collect 7,290 touchscreen gestures (stroke gestures) and 3,809 motion gestures from 28 participants (14 with upper-body motor impairments and 14 without impairments).
  • They compared and analyzed the gesture input performance of both groups on smartwatches (worn on the wrist), smart rings (worn on the finger), and smart glasses (worn on the head).
  • Data analysis focused on production time, articulation consistency, and kinematic features (e.g., acceleration, trajectory).

• Innovations:

  • This study is the first to systematically and deeply investigate the performance of touchscreen and motion gestures by users with motor impairments across multiple wearable device positions.
  • The authors proposed accessibility design principles based on user capabilities and provided ten practical design recommendations.
  • The dataset was made publicly available to support future research in this field.

• Experimental Procedures and Techniques:

  • A Samsung Gear Fit 2 smartwatch was modified to function as a ring and glasses device for data collection.
  • A test set was designed, including 12 types of touchscreen gestures and 18 types of motion gestures.
  • Gesture actions were recorded with high-precision timestamps to analyze and quantify execution efficiency and consistency.

Research Findings

• Key Findings:

  • Touchscreen Gestures:
    • Users with motor impairments took twice as long as non-impaired users to complete gestures (3.16s vs. 1.56s) and exhibited 17.9% lower consistency.
    • Their gestures were longer, more "wavy" in shape, and involved more strokes.
    • Device position significantly impacted performance, with wrist-worn devices performing best, while ring and head-worn devices posed greater challenges.
  • Motion Gestures:
    • For motion gestures, users with and without motor impairments exhibited similar performance in terms of gesture production time, consistency, acceleration, and dynamic characteristics.
    • However, finger and wrist motion gestures were faster than head gestures, though they were less consistent.

• Experimental Data:

  • Overall task completion rates: 90.4% for touchscreen gestures and 94.5% for motion gestures.
  • The dataset includes detailed spatiotemporal features, providing valuable resources for future research on accessible gesture recognition.

• Design Recommendations:

  • Prioritize user capabilities (design gestures that accommodate users' motor abilities).
  • Support personalized correction and configuration, including gesture simplification and prediction.
  • Create reusable gesture sets across platforms to adapt to different devices.
  • Provide clear feedback during gesture execution to enhance user interaction transparency.

• Limitations and Future Directions:

  • Limitations:
    • Experimental devices did not fully meet specific contextual needs (e.g., ring devices may be too large).
    • Potential usage strategies in all scenarios (e.g., alternating hands) were not covered.
  • Future Directions:
    • Extend research on gesture recognition accuracy and optimize algorithms using user-generated data.
    • Explore the interaction potential of more device forms and devices worn on other body parts.
    • Further investigate the relationship between gesture performance and user satisfaction.

This research provides a new perspective for designing more inclusive and universally accessible gesture interaction interfaces, while the released dataset encourages further innovation in related fields.