Transferable Microgestures Across Hand Posture and Location Constraints: Leveraging the Middle, Ring, and Pinky Fingers

Title of the Paper

Transferable Microgestures Across Hand Posture and Location Constraints: Leveraging the Middle, Ring, and Pinky Fingers

Paper Information

• Domain: Human-Computer Interaction (HCI), Gesture Interaction, Microgesture Design
• Keywords: Gesture Input, Microgestures, Middle Finger, Ring Finger, Pinky Finger, Hand Location Constraints, Hand Posture Constraints, HCI Design, Auxiliary Task Handling, Multitasking Interaction

Research Background and Problem

• Problems and Challenges:

  • Microgesture technology primarily focuses on the thumb and index finger, which are highly flexible but may be unavailable when grasping objects.
  • Many current studies on microgestures fail to address interaction issues when hand position and posture are constrained during tasks.
  • There is a lack of systematic research on how the middle, ring, and pinky fingers can be used as gesture inputs to support multitasking.

• Significance:

  • In tasks such as writing, soldering, and photography, using other fingers for gesture operations can free up the thumb and index finger, helping to balance task demands with auxiliary task operations.
  • Gestures involving the middle, ring, and pinky fingers have potential applications, providing stable input operations in scenarios where hand position and posture are constrained.

• Motivation and Related Work:

  • This research continues the tradition of exploring microgestures and hand-based input, investigating an extended gesture design space.
  • The authors draw on significant findings in hand anatomy, biomechanics, and flexible control to design gestures that transcend hand posture and location constraints, proposing innovative methods to address current application bottlenecks.

Solution

• Primary Methods and Solutions:

  • A design space model was proposed, illustrating the impact of hand posture constraints (free, posture-constrained, grasp-constrained) and hand location constraints (arbitrary location, near the body, relatively fixed position) on microgestures.
  • Five proof-of-concept applications were developed to demonstrate how the middle, ring, and pinky fingers can enable gesture operations across constraints.
  • Comfort, operability, and user preferences for the designed gestures were evaluated through large-scale online user studies and laboratory experiments.

• Innovations:

  • Conducted a systematic analysis of microgesture interaction design using the middle, ring, and pinky fingers, along with the first large-scale quantitative ergonomic evaluation.
  • Combined theory and practice to propose a usable gesture operation design framework, validating its transferability and comfort.
  • Extended microgestures to scenarios with traditionally unmanageable position and posture constraints, addressing auxiliary input issues when hands are occupied.

• Implementation Steps:

  1. Proposed a gesture design space, defining hand posture and location constraints.
  2. Selected and identified feasible microgestures, including sliding, tapping, and finger extension operations.
  3. Implemented five gesture-based applications to demonstrate transferability, including:
     • Switching songs via middle finger sliding (across location constraints).
     • Adjusting focus and filters via middle finger sliding (across posture constraints).
     • Controlling music with the pinky finger while grasping different objects (across grasp constraints).
     • Transferring gesture operations between different fingers.
  4. Designed online and laboratory experiments to test the ergonomics and preferences of various gestures and operational scenarios.

Research Outcomes

• Specific Results:

  • Developed a design space describing how the middle, ring, and pinky fingers can be used for microgesture operations under different hand location and posture constraints.
  • Conducted the first large-scale evaluation of this design space, involving 210 participants in an online study covering 43 types of microgestures. Additionally, 10 laboratory participants tested performance during actual tasks.

• Advantages Over Existing Solutions:

  • The comfort level of middle finger microgestures was only slightly lower than that of index finger gestures and outperformed most existing solutions for constrained hand input.
  • Microgesture operations were preferred over baseline methods, particularly when maintaining specific hand postures.
  • The designed gestures demonstrated strong transferability across posture and grasp conditions.

• Experiment and Evaluation Results:

  • Large-Scale Online Experiment:
    • The overall comfort score for middle finger gestures was significantly higher than for ring and pinky finger gestures.
    • Tapping operations were generally more comfortable than sliding and extension operations.
    • Under two major constraint conditions, approximately 98% of microgestures were rated as practically operable.
  • Laboratory Experiment:
    • Tasks such as photo capturing and drawing using middle finger microgestures showed efficiency and comfort levels comparable to baseline methods.
    • Users preferred performing middle finger microgestures with postures like hovering rather than grasping.
  • Feedback:
    • Several participants noted that microgestures reduced overall task interruptions and made interactions more intuitive.

• Limitations and Future Directions:

  • Microgesture design was primarily focused on conventional postures and grasps; future research should explore a broader range of objects and gestures.
  • The study did not deeply analyze long-term comfort, error rates, or social acceptability.
  • Better integration with actual hardware is needed for electronic sensors or visual tracking methods.
  • Broader testing is required for diverse user groups, especially those with different occupational backgrounds and hand dexterity.

Through a systematic study of microgesture technology using the middle, ring, and pinky fingers, this research not only provides new ideas for interaction design under constrained hand conditions but also holds significant application value for implementing multitasking scenarios in future human-computer interaction systems.