Intermanual Deictics: Uncovering Users' Gesture Preferences for Opposite-Arm Referential Input, from Fingers to Shoulder

Research Background and Problem

• Identified Challenges: The authors explored gesture input related to "Intermanual Deictics," a form of referential gesture input based on the contralateral limb (the other arm). While the body as an input surface for human-computer interaction has received widespread attention, users' preferences for intuitive gesture operations on this surface remain underexplored, particularly regarding interaction input on the upper arm and forearm.
• Significance: This study aims to fill the knowledge gap regarding user preferences to better design intuitive interaction methods, which are crucial for enhancing the usability and user experience of "on-body" interaction systems.
• Research Motivation and Related Work: Previous studies have primarily focused on technical implementation and applications for specific body areas (e.g., palms or forearms), with limited exploration of users' preferences for intuitive gestures. Specifically, gestures based on contralateral body reference points lack systematic investigation in interaction design.

Solution

• Proposed Solution: Through a large-scale user gesture preference study involving 75 participants, the authors systematically examined touch-based and non-contact "Intermanual Deictics" gesture input, focusing on interactions with the palm, forearm, and upper arm. Based on this study, they proposed a set of 60 consensus gestures and outlined principles and practical design insights for these gestures.
• Innovations:
  1. Established "Intermanual Deictics" as a distinct gesture category (different from conventional two-handed operations and skin-contact gestures).
  2. Systematically collected and analyzed user preferences for gestures across three body areas (palm, forearm, upper arm), emphasizing a novel "supporter-executor" interaction model.
  3. Extracted statistical patterns of user preferences and developed corresponding design guidelines.
• Implementation Steps:
  1. User-defined gesture study: Participants proposed intuitive gestures for common commands (e.g., "next," "cancel") relative to contralateral body areas (palm, forearm, or upper arm).
  2. Data collection and analysis: Gestures were categorized into contact-based (touch, swipe) and non-contact (hover) gestures, with consensus levels, complexity, and applicability recorded.
  3. Identification of user consensus gesture set: A set of 60 consensus gestures based on statistical consistency analysis was established from user suggestions.

Research Outcomes

• Specific Results:
  1. Users strongly preferred physical contact gestures (91.2%), with swipe (62.4%) and touch (28.8%) being the most popular; hover gestures accounted for only 5.2%. Swipe gestures were primarily used on the middle regions of the forearm and upper arm, while touch gestures were concentrated on the palm center and finger areas.
  2. Simple gestures (88.9%) were far more prevalent than compound gestures (11.1%); users overwhelmingly preferred using the index finger to perform gestures (94.3%).
  3. The "60 consensus gesture set" extracted from gesture consensus analysis was detailed for the three body areas, providing references for future system design.
  4. Similar user tendencies were observed across the palm, forearm, and upper arm, with gesture characteristics of the forearm and upper arm being particularly close.
• Advantages Compared to Existing Solutions:
  1. More comprehensively explored user gesture preferences, extending the scope of traditional research to include the entire contralateral arm region.
  2. The proposed gesture set benefits from a large sample size (75 participants), making it more representative.
  3. Provided practical guidance (e.g., cross-region gestures, center-area-based designs) to optimize body-based interaction design.
• Experimental and Evaluation Results:
  1. The average gesture consensus rate was 26.3%, with slight fluctuations across specific commands in different body areas (20.2%-34.1%).
  2. Gesture consensus between the forearm and upper arm (39.2%) was significantly higher than that between the palm and other areas.
  3. Consistency analysis revealed that swipe and touch gestures exhibited high functional and significance similarity.
• Limitations and Future Directions:
  1. Limitations: The study focused on generalized ten functional commands, lacking exploration of gestures for specific application scenarios; did not directly validate the recognition accuracy of existing technologies for these gestures; sample gender distribution was uneven (90% male).
  2. Future Directions: Include developing wearable technologies that support more complex gesture recognition, creating novel virtual reality applications based on "Intermanual Deictics," exploring the applicability of these gestures in more contexts (e.g., dynamic movement or wearing thick clothing), and studying the impact of gender and ability differences on gesture preferences.