Iterative Design of Gestures During Elicitation: Understanding the Role of Increased Production

Title of the Paper

Iterative Design of Gestures During Elicitation: Understanding the Role of Increased Production

Paper Information

• Research Area: Human-Computer Interaction (HCI), Gesture Interaction Design
• Keywords: Gesture elicitation, user mental model, full-body gestures, free-space interaction, public displays, fatigue

Research Background and Problem

• Identified Problems:

  1. Users are influenced by "legacy bias" when proposing gesture designs, which limits gesture innovation.
  2. Experimental methods that increase user gesture production can alleviate legacy bias but may lead to reduced gesture diversity.
  3. Current research on changes in gesture diversity and quality presents conflicting conclusions, and the potential limitations of high-production gesture methods remain underexplored.

• Research Significance: The proliferation of technologies such as touch interfaces and virtual/augmented reality has driven the demand for gesture-based interaction. However, existing gesture systems often suffer from design flaws that cause fatigue (e.g., arm fatigue). Designing intuitive, low-fatigue gestures is crucial for user experience, especially in scenarios involving public displays and free-space interaction.

• Research Motivation and Related Work: The authors were inspired by previous gesture elicitation studies, particularly the three strategies proposed by Morris et al. to overcome legacy bias (priming, production, partners). While initial results have been achieved by shifting from single-output to high-production gesture studies, the underlying mechanisms of their impact remain unclear. This paper aims to fill these theoretical gaps.

Proposed Solution

• Proposed Solution: The authors designed a gesture elicitation experiment based on the "unlimited production" method and conducted retrospective interviews in subsequent stages to explore the details of gesture design. This approach allows users to rapidly iterate and refine gestures, indirectly revealing their mental models of gesture meaning.

• Methodological Innovations:

  1. Introduction of the "unlimited production" strategy, allowing participants to freely plan and repeatedly test gestures.
  2. Integration of post-experiment retrospective sessions to deeply explore the logic behind users' iterative gesture modifications.
  3. Adoption of a detailed gesture coding process, breaking gestures down into specific body movement units (gesture primitives).

• Implementation Steps and Key Techniques:

  1. Experiment Design and Participation:
     • 22 participants completed gesture elicitation tasks in a simulated public display scenario.
     • Tasks included designing gestures for 10 screen interaction actions (e.g., selection, zooming, scrolling).
  2. Phased Experiment:
     • Phase 1: Gesture elicitation experiment, where participants proposed as many gestures as possible for each action.
     • Phase 2: Retrospective interviews, where participants evaluated the quality of their designed gestures (guessability, applicability, ease of use).
  3. Data Analysis:
     • Gestures in the video recordings were segmented and coded, including parameters such as trajectory, speed, body parts used, and joint rotation.
     • Quantified gesture diversity (consistency scores, consensus distribution metrics) and corresponding user evaluations.

Research Findings

• Specific Findings:

  1. "Unlimited production" facilitated iterative optimization of gestures. Approximately 10% of gestures were refined versions of earlier ones.
  2. Users tended to prefer later-stage gestures (after the 3rd iteration), with 34% of participants selecting gestures from the 3rd iteration or later as their favorites.
  3. Among refined gestures, the most frequently modified features were hand orientation (73.7%) and gesture configuration (59.7%), while movements of other body parts changed less, with variations mainly concentrated in the arms and hands.
  4. Data supported the hypothesis that users prioritize fatigue factors when refining gestures, showing a trend toward using smaller motion ranges and less fatigue-prone joints.

• Advantages:

  1. Provides a dynamic and flexible method for user gesture generation and optimization.
  2. Reveals individual differences in "iteration frequency," offering insights for designing more personalized systems.
  3. Introduces new methodologies and perspectives for understanding user mental models and decision-making.

• Experimental or Evaluation Results:

  1. High-consensus actions (e.g., a 90% consensus rate for the "drag" action) indicate that legacy bias still exists.
  2. Gesture scores during the refinement process did not significantly decrease compared to the original gestures, suggesting that the logic behind refinement may be natural improvement.

• Limitations and Future Directions:

  1. The specific impact of fatigue on gesture design requires further exploration, including quantitative fatigue assessments.
  2. The study's sample size was relatively small (10 participants for coded data), limiting its generalizability.
  3. Future research could validate the findings in more interaction scenarios and expand the exploration of "non-arm gestures."

Conclusion

This study employs innovative methods, including "unlimited production" and "retrospective interviews," to deeply analyze users' psychological processes and decision-making during gesture iteration and optimization in public display interactions. The findings provide constructive insights into overcoming legacy bias and enhancing natural interaction potential, while also offering significant implications for data construction and algorithm design in gesture recognition systems.