Over the Mouse: Navigating across the GUI with Finger-Lifting Operation Mouse

Research Background and Problem

• What problems or challenges did the authors identify?
  Modern graphical user interfaces (GUIs) often feature hierarchical structures (commonly referred to as the z-axis), yet traditional mice do not support efficient z-axis navigation. This results in increased physical movement and cognitive load for users when interacting with such structures. Traditional solutions to this issue, such as adding more mouse buttons or using touch/pressure-based input methods, suffer from problems like operational complexity, memory burden, and limited scalability of the structure.

• Why is this problem important?
  Hierarchical structures in GUIs are a core feature of modern user interfaces, and efficient navigation is critical to the user experience. This includes quickly switching between task windows, conveniently browsing content, and reducing the risk of errors. The lack of z-axis interaction capabilities in traditional mice significantly reduces user efficiency in complex desktop environments.

• Research Motivation and Related Work
  The authors analyzed existing research on touch-based, pressure-based, and extended mouse button solutions. While these methods provide partial solutions, they still have limitations such as operational interference, accuracy issues, and steep learning curves. Based on these shortcomings, the authors proposed a new interaction method: using finger-lifting gestures to control the z-axis hierarchy in GUIs, enabling more intuitive and efficient interface navigation.

Solution

• What methods or solutions did the authors propose?
  The authors introduced OtMouse, a novel mouse that supports finger-lifting gestures. OtMouse uses distance sensors embedded beneath the mouse buttons to detect finger height and is paired with the Over the Mouse (OtM) interface, designed to facilitate z-axis interactions.

• What are the innovative aspects of this solution?

  • Finger-lifting detection: Utilizes Time-of-Flight (ToF) sensors to capture finger height, combined with touch sensors to minimize false triggers.
  • Operation-to-function mapping: Maps finger height gestures to GUI hierarchical structures (e.g., content, tabs, windows) and integrates them with planar mouse actions for functionality triggers.
  • Multi-level interaction support: Includes both continuous and discrete input modes, suitable for tasks like slider adjustments or z-axis hierarchy selection.

• What are the implementation steps and key technologies used?

  1. Hardware Design: Modified a commercially available mouse by installing ToF sensors for height detection and adding touch sensors to calibrate lifting gestures.
  2. Interaction Design: Defined different height thresholds for finger-lifting gestures and developed task-specific interaction scenarios, such as window switching, file preview, and map zooming.
  3. Experimental Evaluation: Conducted user experiments to validate the speed and accuracy of finger-lifting gestures, as well as the usability advantages of the OtM interface.

Research Outcomes

• What specific outcomes were achieved?

  • Achieved average task completion times of 0.47 seconds and 0.63 seconds for two-level and three-level finger-lifting gestures, respectively, with error rates below 2.5%.
  • The OtM interface significantly reduced mouse movement and improved user efficiency in dynamic window switching and file navigation tasks.
  • Compared to traditional mice, OtM showed significant improvements in SUS (System Usability Scale) scores across two tasks, providing a more intuitive interaction experience.

• What advantages does it have over existing solutions?

  • More intuitive interaction: Finger-lifting gestures directly correspond to hierarchical structures (z-axis), eliminating the need for memorizing side buttons or complex gestures.
  • Efficiency improvement: Reduces operation time by minimizing physical movement and clicks.
  • Interaction scalability: Dynamically adjusts lifting levels to accommodate complex task requirements, rather than being limited to fixed buttons or pressure levels.

• What were the experimental or evaluation results?

  • In Task 2 (file navigation), OtM reduced task completion time by an average of 22% compared to traditional methods.
  • Subjective user feedback indicated that OtM reduced the likelihood of errors, particularly in multi-level menu or window-switching scenarios.

• Limitations and Future Directions

  • Limitations:
    • Finger-lifting gestures remain challenging for precise control in certain task scenarios (e.g., map zooming).
    • Prolonged use may cause wrist fatigue, necessitating hardware design improvements for better ergonomic adaptation.
    • New users face a steep learning curve with the OtM interface.
  • Future Directions:
    • Improve operational mechanisms, such as adding a "lock" feature to address fatigue from sustained finger-lifting gestures.
    • Develop more levels or smoother control methods for continuous zooming or precise adjustment tasks.
    • Design more ergonomically optimized mouse shapes to reduce wrist strain.

In summary, the authors innovatively addressed the limitations of traditional mice in z-axis interactions through OtMouse and the OtM interface, opening a new dimension for efficient navigation of complex GUIs. At the same time, the study highlights potential areas for further improvement, particularly in terms of operational precision and ergonomic design.