The People's Gaze: Co-Designing and Refining Gaze Gestures with Users and Experts

Paper Title

The People's Gaze: Co-Designing and Refining Gaze Gestures with Users and Experts

Publication Info

• Topic area: Gaze gesture design for hands-free interaction in mobile devices.
• Keywords: Gaze gestures, eye-tracking, co-design, mobile interaction, participatory design, expert evaluation, gesture vocabulary, user-centered design, Midas Touch problem, hands-free interfaces.

Background and Problem

• Problem / challenge: Existing gaze gesture vocabularies are predominantly expert-designed, often misaligned with natural user behaviors, and lack ecological validity for mobile contexts. They also face challenges like the Midas Touch problem and limited ergonomic feasibility.
• Significance: With the increasing ubiquity of eye-tracking in mobile devices, intuitive and reliable gaze gestures are critical for hands-free interaction, especially for users with motor impairments and in mobile or multitasking scenarios.
• Motivation and related work: Prior research has focused on technical advancements in gaze tracking and gesture recognition but has largely overlooked participatory approaches to gesture design. Existing vocabularies often fail to reflect user mental models or address real-world constraints like mobility and social acceptability.

Solution

• Proposed approach: A two-phase methodology combining participatory co-design workshops with non-expert users and structured expert peer review to develop a user-grounded, expert-validated gaze gesture vocabulary.
• Novelty:
  1. Introduction of a participatory co-design process for gaze gestures, engaging non-expert users to generate intuitive designs.
  2. Development of a compositional gaze grammar (activate-then-confirm) to mitigate the Midas Touch problem.
  3. Expert validation of user-generated gestures, ensuring ergonomic, semantic, and technical feasibility.
  4. Actionable design principles for creating intuitive and robust gaze gestures.
• Procedure and key techniques:
  1. Phase 1: Four co-design workshops with 20 non-expert participants using a 9-point spatial grid to generate 102 initial gesture concepts, refined to 59 unique gestures.
  2. Phase 2: Structured peer review by four gaze interaction experts, who evaluated and refined the 59 gestures into a final set of 32 robust designs.
  3. Thematic analysis of user and expert feedback to identify design patterns, ergonomic constraints, and technical requirements.

Results

• Concrete findings:
  • Final set of 32 expert-validated gaze gestures, including designs optimized for mobile use.
  • Identification of a compositional gaze grammar (dwell-to-arm, blink-to-commit) as a foundational syntax for gaze interaction.
  • Quantitative insights: Experts rated usability, novelty, and learnability of gestures, with inter-rater reliability ranging from 0.042 to 0.300.
• Advantage over baselines:
  • User-defined gestures grounded in familiar metaphors and mobile constraints, improving intuitiveness and learnability.
  • Expert filtering ensured ergonomic and technical feasibility, addressing issues like false positives and fatigue.
• Experiments / evaluation:
  • Phase 1: Workshops with 20 participants, generating and evaluating gestures using a 9-point grid.
  • Phase 2: Expert review of 59 gestures, consolidated into 32 designs based on usability, distinctiveness, and robustness.
  • Metrics: Likert-scale ratings for memorability, ease of execution, robustness, and overall impression.
• Limitations and future work:
  • Limited participant diversity (primarily university-affiliated individuals).
  • Expert panel consisted only of academic researchers, lacking practitioner perspectives.
  • No real-time system testing or performance metrics (e.g., error rates, throughput).
  • Future work should include diverse user groups, real-world deployment, and evaluation of long-term learnability and social acceptability.

Summary

This study introduces a two-phase methodology for designing intuitive and robust gaze gestures for mobile devices. Phase 1 engaged 20 non-expert users in co-design workshops, generating 59 unique gestures, while Phase 2 involved expert evaluation to refine these into a final set of 32 gestures. Key contributions include the development of a compositional gaze grammar (dwell-to-arm, blink-to-commit) and actionable design principles prioritizing ergonomic and cognitive feasibility. The findings bridge the gap between user intuition and technical implementation, offering a practical framework for creating gaze-enabled interfaces. Future work should focus on real-world testing and broader user inclusion.