GazeSwipe: Enhancing Mobile Touchscreen Reachability through Seamless Gaze and Finger-Swipe Integration

Research Background and Issues

• Identified Problems or Challenges:

  • As the screen sizes of smartphones and tablets continue to increase, it becomes difficult for the thumb to reach certain areas of the screen when operating the device with one hand. This issue of "touch inaccessibility" leads to challenges such as reduced operational efficiency and accidental device drops.
  • Current solutions (e.g., screen zooming techniques and cursor-based methods) have alleviated some touch inaccessibility issues but at the cost of usable screen space, increased user fatigue, or lack of intuitiveness.
  • Traditional gaze interaction solutions require expensive eye-tracking equipment and cumbersome calibration steps, making them impractical for everyday devices.

• Significance:

  • Large-screen devices are widely used for watching videos, reading, and multitasking. Addressing the touch inaccessibility issue can significantly enhance the user experience.
  • Convenient, low-friction solutions that do not require additional hardware are critical for improving the usability of large-screen devices, especially in entertainment and productivity scenarios.

• Research Motivation and Related Work:

  • Investigating how gaze estimation can be achieved using the built-in camera of user devices and designing an intuitive solution that combines gaze estimation with touchscreen interaction.
  • Related fields have proposed screen transformation techniques, cursor-based methods, and gaze + touch interactions, but these solutions have limitations in terms of accuracy, portability, or user-friendliness.

Solution

• Proposed Method or Solution:

  • A multimodal interaction technique called GazeSwipe is proposed, which combines gaze estimation with swipe gestures to extend the thumb's reachable range on large-screen devices.
  • A real-time gaze estimation method based on the device's front-facing camera is designed, eliminating the need for additional hardware and explicit calibration.
  • An "unperceived" automatic calibration strategy is introduced, dynamically optimizing gaze accuracy by recording swipe release points.
  • A "gaze pointing + finger swipe" interaction model is provided to support intuitive selection and manipulation of screen elements.

• Innovative Aspects of the Solution:

  • Utilizes the device's built-in camera, removing the need for external eye-tracking hardware and enhancing system usability.
  • The automatic calibration method eliminates cumbersome explicit calibration steps while dynamically adapting to user head movements.
  • Seamlessly integrates gaze interaction with finger swiping, addressing the "Midas touch" problem (accidental activation) in gaze interaction.

• Implementation Steps:

  1. Real-time Gaze Estimation: Using a lightweight neural network to estimate gaze points on the screen from images captured by the front-facing camera.
  2. Automatic Calibration: Recording swipe release points during user interactions to optimize gaze estimation deviations.
  3. Swipe Interaction: After locking onto a target with gaze, the user adjusts the cursor via swiping to complete target selection.

• Key Technologies:

  • Lightweight deep learning networks for rapid gaze point estimation.
  • Distance inverse weighting strategies and head orientation information for dynamic calibration of gaze estimation results.
  • Interaction model design combining touchscreen gestures and gaze.

Research Outcomes

• Specific Results:

  • The automatic calibration strategy significantly reduces gaze estimation errors; compared to traditional explicit calibration, it offers better target selection accuracy and user experience.
  • GazeSwipe performs exceptionally well in addressing touchscreen inaccessibility issues on smartphones and tablets, particularly surpassing other methods on tablet devices.

• Advantages Over Existing Solutions:

  • Does not require additional hardware, reducing deployment costs.
  • Dynamic automatic calibration eliminates explicit setup, reducing user learning costs.
  • Provides a low-friction, intuitive interaction method suitable for everyday application scenarios.

• Experimental or Evaluation Results:

  • Multiple user studies demonstrate that GazeSwipe outperforms other techniques in terms of success rate, thumb movement distance, and user preference rankings.
  • On tablet devices, GazeSwipe achieves the best performance across all evaluation metrics, with completion time slightly longer than direct touch methods.

• Limitations and Future Directions:

  • Limitations:
    • Initial gaze estimation accuracy is relatively low, requiring multiple interactions for calibration.
    • Performance in low-light environments is limited, which could be optimized by adjusting screen brightness.
    • Randomness in the initial cursor position may affect user response time.
  • Future Directions:
    • Further optimize gaze estimation accuracy to adapt to complex mobile environments.
    • Explore more application scenarios based on multimodal interactions (e.g., gaze, voice, touch).
    • Integrate user identity recognition and historical calibration data to improve system practicality.

The analysis above demonstrates that the GazeSwipe technology effectively addresses the touchscreen inaccessibility issue on large-screen devices while showcasing significant potential in terms of user-friendliness, usability, and cost-effectiveness through its innovative design. Future research can delve deeper into optimizing technical details and expanding application scenarios to promote its widespread practical adoption.