An Adaptive Model of Gaze-based Selection

Title of the Paper

An Adaptive Model of Gaze-based Selection

Paper Information

• Field of Study: Human-Computer Interaction and Eye-Tracking Technology
• Keywords: Reinforcement Learning, Gaze-based Selection, Adaptive Model, Computational Rationality, Eye Movement Control, Partially Observable Markov Decision Process (POMDP)

Research Background and Problem

• Problem or Challenge:
  • Research on the performance of gaze-based selection (an input method based on eye movement control) has shown inconsistent results, particularly when compared to other input methods such as mouse and head pointing.
  • Existing models focus on motor constraints while neglecting the impact of visual constraints on gaze-based selection.
  • There is a lack of clear mechanisms and theoretical models to understand the complex eye movement sequences and selection strategies involved in gaze-based selection.
• Significance:
  • Gaze-based selection enables faster, more intuitive, and hands-free human-computer interaction. Understanding its mechanisms can help optimize its application in virtual/augmented reality.
  • Developing more accurate cognitive and computational models is crucial for improving gaze-based interaction technologies and their design.
• Motivation and Related Work:
  • By modeling eye movement sequences and gaze-based selection mechanisms, this study explores how visual and motor system constraints influence strategy decisions.
  • This research aims to use a reinforcement learning-based model to better match human experimental results, surpassing traditional models that rely solely on empirical data fitting.

Solution

• Proposed Method or Solution:
  • A gaze-based selection model based on a Partially Observable Markov Decision Process (POMDP) is proposed.
  • Reinforcement learning algorithms are used to optimize policies, enabling the model to achieve "bounded optimality" under the constraints of visual and motor systems.
  • Key assumption: Gaze-based selection is a strategy to optimize utility functions (e.g., selection time and accuracy) constrained by the properties of the visual and motor systems.
• Innovations:
  • Provides a mathematical framework capable of predicting gaze-based selection behavior, effectively simulating human gaze control strategies through reinforcement learning.
  • The model predicts not only the total task completion time but also simulates each step of eye movement control.
  • Unlike traditional models, it does not rely on extensive human data for training but adapts through trial-and-error learning in an artificial environment.
• Implementation Steps and Key Techniques:
  1. Model the gaze-based selection task as a POMDP problem, defining states, actions, observations, reward functions, and transition functions.
  2. Simulate the visual and motor noise characteristics of eye movements: define signal-dependent noise in eye movements, uncertainty in visual position, and gaze drift.
  3. Use a deep reinforcement learning algorithm (Proximal Policy Optimization, PPO) to solve the POMDP and train the model.
  4. Compare model results with experimental data (e.g., human gaze-based selection experiments from Schuetz 2019 and Zhang 2010).

Research Outcomes

• Specific Outcomes:
  • The model successfully predicts five key behavioral phenomena consistent with experimental results, including the relationships between target size and selection time, dwell time, and the number of eye movements.
  • The model demonstrates that smaller targets require more corrective saccades to complete the selection task and accurately calculates eye movement strategies under different target conditions.
• Advantages Compared to Existing Solutions:
  • Provides a deeper cognitive understanding of gaze-based selection mechanisms, capable of explaining complex eye movement behaviors that existing models cannot describe.
  • Allows for model parameter calibration at a single experimental point without fitting the entire dataset.
• Experimental or Evaluation Results:
  • The model's predictions for target selection time align with experimental results, with a root mean square error (RMSE) within a reasonable range: for instance, achieving an overall accuracy of 54.5 milliseconds for Zhang 2010.
  • Experimental validation shows the model's adaptability, indicating that as target distance and size increase, the dominant factor for selection time is the number of corrective saccades.
• Limitations and Future Directions:
  • The model slightly overestimates gaze behavior for smaller targets (<1°), likely due to the precision limitations of eye-tracking devices not fully accounted for.
  • The current model assumes a fixed range of gaze drift; future work could consider making the drift range adaptive to target size.
  • Potential directions include extending the model to adapt to visual biases (e.g., target estimation biases based on the current gaze point) and incorporating more dynamic scene experiments for validation.