"I Don't Want People to Look At Me Differently'': Designing User-Defined Above-the-Neck Gestures for People with Upper Body Motor Impairments

Title of the Paper

“I Don’t Want People to Look At Me Differently” - Designing User-Defined Above-the-Neck Gestures for People with Upper Body Motor Impairments

Paper Information

• Subject Area: Human-Computer Interaction; Touchless Interaction Technologies for Individuals with Upper Body Motor Impairments
• Keywords: Upper Body Motor Impairments, Above-the-Neck Gestures, User-Defined Gestures, Gesture Guidance, Human-Computer Interaction

Research Background and Problem Statement

• Identified Problems or Challenges:
  • Individuals with upper body motor impairments face difficulties in accurately interacting with touchscreens, while common alternatives like voice interaction pose challenges related to input speed and privacy.
  • Existing eyelid gesture designs fail to adequately consider user preferences and physical conditions, limiting their acceptability.
• Significance of the Research: Providing patients with natural, acceptable, and accessible mobile interaction methods enhances user experience while ensuring social acceptability.
• Motivation and Related Work:
  • Current gesture interaction designs are predominantly technology-driven, lacking user involvement in the design process.
  • Drawing from research on user-defined gestures, exploring user-designed head, eye, and mouth gestures holds greater significance.

Proposed Solution

• Proposed Method:
  1. User Study: Engaged 17 individuals with motor impairments to design gestures for 26 common mobile interaction commands.
  2. Gesture Classification and Selection: Analyzed 442 collected gestures based on users' design logic and preferences.
  3. Validation Testing: Conducted a survey to gather feedback from 24 participants (including individuals without impairments) on the designed gestures.
• Innovative Contributions:
  • For the first time, incorporated multiple above-the-neck areas (eyes, mouth, head) in designing user-defined gestures.
  • Systematically involved users in the design process to maximize gesture preference and user acceptance.
• Implementation Steps:
  1. Define experimental tasks: Selected and categorized 26 common mobile commands based on the complexity principles of HCI.
  2. Guide users to create gestures suitable for their physical conditions while recording their evaluations.
  3. Resolve conflicts: Prioritized user preferences to address conflicts in gesture design.
  4. Evaluation survey: Collected feedback from participants without motor impairments to validate the generalizability of the gestures.

Research Outcomes

• Specific Results:
  • Compiled a set of 30 user-defined touchless gestures suitable for individuals with motor impairments.
  • Most participants preferred gestures that were visually unobtrusive, small-scale, and natural.
• Comparison with Existing Solutions:
  • User-defined gestures better aligned with user needs compared to researcher-designed gestures.
  • Enriched interaction methods based on above-the-neck gestures, improving feasibility and coverage of practical interaction commands.
• Experimental and Evaluation Results:
  • Above-the-neck gestures demonstrated strong social acceptability, with eye-based gestures (44.4%) being the most common, followed by combined eye and head gestures (30.8%).
  • Users’ physical conditions did not hinder their ability to design gestures suited to their capabilities.
  • Consistency in gesture preferences between individuals with and without impairments reached 62%.
• Limitations and Future Directions:
  • Memory Load: Increased gesture diversity may pose recognition and usage challenges for users.
  • Design Adaptation: Gestures require further personalization support to address health concerns (e.g., repetitive motion).
  • Cross-Cultural Adaptation: Further research is needed on the consensus of gestures across regions and cultural contexts.
  • Other Limitations: The impact of age, background, and task categorization on users’ gesture preferences remains to be explored.

Conclusion and Value

This study successfully demonstrated the feasibility of enabling individuals with disabilities to autonomously define interaction methods. By establishing a standardized and customizable set of interaction gestures, it provides an important paradigm and design reference for creating inclusive and accessible platforms in diverse social contexts.