Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across Domains

Title of the Paper

"Towards a Consensus Gesture Set: A Survey of Mid-Air Gestures in HCI for Maximized Agreement Across Domains"

Paper Information

• Subject Area: Human-Computer Interaction (HCI), focusing on mid-air gesture-based interaction
• Keywords: mid-air gestures, systematic literature review, agreement analysis, application domains, user interaction

Research Background and Problems

• Problems or Challenges:

  • Mid-air gestures are widely adopted in various HCI applications due to their intuitiveness and natural execution. However, current gesture designs are fragmented across specific devices or domains, lacking cross-domain consistency.
  • There is a lack of systematic evaluation of the transferability of existing gestures across different domains, hindering designers from selecting and creating gestures with broad applicability.

• Significance:

  • A unified gesture set can enhance the transferability of user experience across different devices and domains, reducing the learning and memory burden.
  • Cross-domain consistency is particularly crucial in ubiquitous computing environments, such as smart homes and virtual reality, which span multiple devices and contexts.

• Research Motivation and Related Work:

  • Existing literature predominantly focuses on gesture design and evaluation within specific environments, with limited in-depth exploration of cross-domain consistency.
  • While user-defined gestures have been widely encouraged, there is little discussion on their consistency and potential application across different contexts and domains.

Solution

• Methods or Solutions: A systematic literature review (SLR) was conducted, analyzing 172 relevant research papers to establish a gesture feature classification system based on existing taxonomic dimensions and calculate the agreement rate of various gestures across different domains.

• Innovations:

  • Proposed a cross-domain "consensus gesture set" comprising 22 gestures, demonstrating high transferability across domains.
  • Systematically analyzed the distribution and characteristics of gestures in different domains and the factors influencing consistency.

• Implementation Steps and Key Techniques:

  1. Systematic generation of the literature set: Collecting target papers from multiple databases (2,078 initial papers, narrowed down to 172 after screening).
  2. Gesture coding and classification: Classifying gestures in each paper based on five dimensions, including gesture form, binding method, naturalness, fluidity, and body parts.
  3. Agreement rate calculation: Analyzing overall gesture agreement rates to quantify cross-domain consistency.
  4. Building the "consensus gesture set": Selecting high-agreement gestures to form a representative set of 22 gestures and their task mappings.

Research Outcomes

• Specific Results:

  • Identified the distribution and consistency differences of common gesture characteristics across 12 major application domains (e.g., media and entertainment, large displays, medical technology, smart homes, automotive systems).
  • Extracted a cross-domain consensus gesture set, encompassing 22 high-agreement gestures suitable for various tasks (e.g., zooming, selection, rotation, confirmation).

• Advantages and Comparisons:

  • Compared to existing gesture studies limited to single-domain design or evaluation, this research is the first to systematically explore the issue of cross-domain consistency.
  • The gesture set derived from the literature can assist designers in creating more consistent and transferable interaction systems in new domains.

• Experimental or Evaluation Results:

  • Significant differences in agreement rates were observed across domains, but certain tasks (e.g., rotation, zooming in/out) exhibited high consistency across multiple domains.
  • Gesture agreement rates improved after categorizing gestures using "similar themes," indicating that users tend to prefer similar gesture paradigms conceptually.

• Limitations and Future Directions:

  1. The current study is limited by the scope of the retrieval period (up to April 2021) and the sources of literature, excluding subsequent updates.
  2. The proposed "consensus gesture set" requires further validation in large-scale real-world environments to assess its social, spatial, and cognitive applicability.
  3. Literature on specific domains (e.g., gaming, assistive technologies) remains insufficient, necessitating more empirical studies to support comprehensive data.