Online-EYE: Multimodal Implicit Eye Tracking Calibration for XRUnlike other inputs for extended reality (XR) that work out of the box, eye tracking typically requires custom calibration per user or session. We present a multimodal inputs approach for implicit calibration of eye tracker in VR, leveraging UI interaction for continuous, background calibration. Our method analyzes gaze data alongside controller interaction with UI elements, and employing ML techniques it continuously refines the calibration matrix without interrupting users from their current tasks. Potentially eliminating the need for explicit calibration. We demonstrate the accuracy and effectiveness of this implicit approach across various tasks and real time applications achieving comparable eye tracking accuracy to native, explicit calibration. While our evaluation focuses on VR and controller-based interactions, we anticipate the broader applicability of this approach to various XR devices and input modalities.2025BHBaosheng James HOU et al.Google; Lancaster University , Computing and CommunicationsEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
On-body Icons: Designing a 3D Interface for Launching Apps in Augmented RealityOn-body tapping provides a quick way to launch augmented reality (AR) apps using virtual shortcuts placed on the user’s skin, clothes, and jewelry. While prior work has focused on tapping performance, social acceptance, and sensing techniques, users’ behaviour in placing shortcuts on their body has been underexplored. In this work, we propose On-body Icons — a novel interface for launching apps via touching virtual icons placed across the user’s entire body, and use it to investigate locations, reasons for chosen icon placement, and users’ attitudes towards the feature. Results of the qualitative study conducted with 24 participants demonstrated that people employ a wide variety of placement strategies that balance memorability of the locations with accuracy and comfort of reaching the icons. We discuss these findings in regard to current understanding of memorability of icon placement, placement appropriateness, and privacy, and offer design implications for similar features in spatial applications.2025UTUliana Tsimbalistaia et al.HSE UniversityAR Navigation & Context AwarenessOn-Skin Display & On-Skin InputCHI
It’s Not Always the Same Eye That Dominates: Effects of Viewing Angle, Handedness and Eye Movement in 3DUnderstanding eye dominance, the subconscious preference for one eye, has significant implications for 3D user interfaces in VR and AR, particularly in interface design and rendering. Although HCI recognizes eye dominance, little is known about what causes it to switch from one eye to another. To explore this, we studied eye dominance in VR, where 28 participants manually aligned a cursor with a distant target across three tasks. We manipulated the horizontal viewing angle, the hand used for alignment, and eye movement induced by target behaviour. Our results confirm the dynamic nature of eye dominance, though with fewer switches than expected and varying influences across tasks. This highlights the need for adaptive HCI techniques, which account for shifts in eye dominance in system design, such as gaze-based interaction, visual design, or rendering, and can improve accuracy, usability, and experience.2025FPFranziska Prummer et al.Lancaster University, School of Computing and CommunicationsEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
Hands-on, Hands-off: Gaze-Assisted Bimanual 3D InteractionExtended Reality (XR) systems with hand-tracking support direct manipulation of objects with both hands. A common interaction in this context is for the non-dominant hand (NDH) to orient an object for input by the dominant hand (DH). We explore bimanual interaction with gaze through three new modes of interaction where the input of the NDH, DH, or both hands is indirect based on Gaze+Pinch. These modes enable a new dynamic interplay between our hands, allowing flexible alternation between and pairing of complementary operations. Through applications, we demonstrate several use cases in the context of 3D modelling, where users exploit occlusion-free, low-effort, and fluid two-handed manipulation. To gain a deeper understanding of each mode, we present a user study on an asymmetric rotate-translate task. Most participants preferred indirect input with both hands for lower physical effort, without a penalty on user performance. Otherwise, they preferred modes where the NDH oriented the object directly, supporting preshaping of the hand, which is more challenging with indirect gestures. The insights gained are of relevance for the design of XR interfaces that aim to leverage eye and hand input in tandem.2024MLMathias N. Lystbæk et al.Hand Gesture RecognitionEye Tracking & Gaze InteractionMixed Reality WorkspacesUIST
Eye-Hand Movement of Objects in Near Space Extended RealityHand-tracking in Extended Reality (XR) enables moving objects in near space with direct hand gestures, to pick, drag and drop objects in 3D. In this work, we investigate the use of eye-tracking to reduce the effort involved in this interaction. As the eyes naturally look ahead to the target for a drag operation, the principal idea is to map the translation of the object in the image plane to gaze, such that the hand only needs to control the depth component of the operation. We have implemented four techniques that explore two factors: the use of gaze only to move objects in X-Y vs.\ extra refinement by hand, and the use of hand input in the Z axis to directly move objects vs.\ indirectly via a transfer function. We compared all four techniques in a user study (N=24) against baselines of direct and indirect hand input. We detail user performance, effort and experience trade-offs and show that all eye-hand techniques significantly reduce physical effort over direct gestures, pointing toward effortless drag-and-drop for XR environments.2024UWUta Wagner et al.Hand Gesture RecognitionEye Tracking & Gaze InteractionUIST
Spatial Gaze Markers: Supporting Effective Task Switching in Augmented RealityTask switching can occur frequently in daily routines with physical activity. In this paper, we introduce Spatial Gaze Markers, an augmented reality tool to support users in immediately returning to the last point of interest after an attention shift. The tool is task-agnostic, using only eye-tracking information to infer distinct points of visual attention and to mark the corresponding area in the physical environment. We present a user study that evaluates the effectiveness of Spatial Gaze Markers in simulated physical repair and inspection tasks against a no-marker baseline. The results give insights into how Spatial Gaze Markers affect user performance, task load, and experience of users with varying levels of task type and distractions. Our work is relevant to assist physical workers with simple AR techniques and render task switching faster with less effort.2024MLMathias N. Lystbæk et al.Aarhus UniversityEye Tracking & Gaze InteractionAR Navigation & Context AwarenessCHI
Snap, Pursuit and Gain: Virtual Reality Viewport Control by GazeHead-mounted displays let users explore virtual environments through a viewport that is coupled with head movement. In this work, we investigate gaze as an alternative modality for viewport control, enabling exploration of virtual worlds with less head movement. We designed three techniques that leverage gaze based on different eye movements: Dwell Snap for viewport rotation in discrete steps, Gaze Gain for amplified viewport rotation based on gaze angle, and Gaze Pursuit for central viewport alignment of gaze targets. All three techniques enable 360-degree viewport control through naturally coordinated eye and head movement. We evaluated the techniques in comparison with controller snap and head amplification baselines, for both coarse and precise viewport control, and found them to be as fast and accurate. We observed a high variance in performance which may be attributable to the different degrees to which humans tend to support gaze shifts with head movement.2024HLHock Siang Lee et al.LancasterEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
Blended Whiteboard: Physicality and Reconfigurability in Remote Mixed Reality CollaborationThe whiteboard is essential for collaborative work. To preserve its physicality in remote collaboration, Mixed Reality (MR) can blend real whiteboards across distributed spaces. Going beyond reality, MR can further enable interactions like panning and zooming in a virtually reconfigurable infinite whiteboard. However, this reconfigurability conflicts with the sense of physicality. To address this tension, we introduce Blended Whiteboard, a remote collaborative MR system enabling reconfigurable surface blending across distributed physical whiteboards. Blended Whiteboard supports a unique collaboration style, where users can sketch on their local whiteboards but also reconfigure the blended space to facilitate transitions between loosely and tightly coupled work. We describe design principles inspired by proxemics; supporting users in changing between facing each other and being side-by-side, and switching between navigating the whiteboard synchronously and independently. Our work shows exciting benefits and challenges of combining physicality and reconfigurability in the design of distributed MR whiteboards.2024JGJens Emil Grønbæk et al.Aarhus University, University of MelbourneMixed Reality WorkspacesDistributed Team CollaborationCHI
Gaze on the Go: Effect of Spatial Reference Frame on Visual Target Acquisition During Physical Locomotion in Extended RealitySpatial interaction relies on fast and accurate visual acquisition. In this work, we analyse how visual acquisition and tracking of targets presented in a head-mounted display is affected by the user moving linearly at walking and jogging paces. We study four reference frames in which targets can be presented: Head and World where targets are affixed relative to the head and environment, respectively; HeadDelay where targets are presented in the head coordinate system but follow head movement with a delay, and novel Path where targets remain at fixed distance in front of the user, in the direction of their movement. Results of our study in virtual reality demonstrate that the more stable the target is relative to the environment, the faster and more precise it can be fixated. The results have practical significance as head-mounted displays enable interaction during mobility, and in particular when eye tracking is considered as input.2024PMPavel Manakhov et al.Aarhus UniversityFull-Body Interaction & Embodied InputEye Tracking & Gaze InteractionAR Navigation & Context AwarenessCHI
Classifying Head Movements to Separate Head-Gaze and Head Gestures as Distinct Modes of InputHead movement is widely used as a uniform type of input for human-computer interaction. However, there are fundamental differences between head movements coupled with gaze in support of our visual system, and head movements performed as gestural expression. Both Head-Gaze and Head Gestures are of utility for interaction but differ in their affordances. To facilitate the treatment of Head-Gaze and Head Gestures as separate types of input, we developed HeadBoost as a novel classifier, achieving high accuracy in classifying gaze-driven versus gestural head movement (F1-Score: 0.89). We demonstrate the utility of the classifier with three applications: gestural input while avoiding unintentional input by Head-Gaze; target selection with Head-Gaze while avoiding Midas Touch by head gestures; and switching of cursor control between Head-Gaze for fast positioning and Head Gesture for refinement. The classification of Head-Gaze and Head Gesture allows for seamless head-based interaction while avoiding false activation.2023BHBaosheng James HOU et al.Lancaster UniversityEye Tracking & Gaze InteractionHuman Pose & Activity RecognitionCHI
A Fitts' Law Study of Gaze-Hand Alignment for Selection in 3D User InterfacesGaze-Hand Alignment has recently been proposed for multimodal selection in 3D. The technique takes advantage of gaze for target pre-selection, as it naturally precedes manual input. Selection is then completed when manual input aligns with gaze on the target, without need for an additional click method. In this work we evaluate two alignment techniques, Gaze&Finger and Gaze&Handray, combining gaze with image plane pointing versus raycasting, in comparison with hands-only baselines and Gaze&Pinch as established multimodal technique. We used Fitts' Law study design with targets presented at different depths in the visual scene, to assess effect of parallax on performance. The alignment techniques outperformed their respective hands-only baselines. Gaze&Finger is efficient when targets are close to the image plane but less performant with increasing target depth due to parallax.2023UWUta Wagner et al.Aarhus UniversityHand Gesture RecognitionEye Tracking & Gaze InteractionCHI
Partially Blended Realities: Aligning Dissimilar Spaces for Distributed Mixed Reality MeetingsMixed Reality allows for distributed meetings where people's local physical spaces are virtually aligned into blended interaction spaces. In many cases, people's physical rooms are dissimilar, making it challenging to design a coherent blended space. We introduce the concept of Partially Blended Realities (PBR) --- using Mixed Reality to support remote collaborators in partially aligning their physical spaces. As physical surfaces are central in collaborative work, PBR supports users in transitioning between different configurations of tables and whiteboard surfaces. In this paper, we 1) describe the design space of PBR, 2) present RealityBlender to explore interaction techniques for how users may configure and transition between blended spaces, and 3) provide insights from a study on how users experience transitions in a remote collaboration task. With this work, we demonstrate new potential for using partial solutions to tackle the alignment problem of dissimilar spaces in distributed Mixed Reality meetings.2023JGJens Emil Sloth Grønbæk et al.Aarhus UniversityMixed Reality WorkspacesDistributed Team CollaborationCHI
Vergence Matching: Inferring Attention to Objects in 3D Environments for Gaze-Assisted SelectionGaze pointing is the de facto standard to infer attention and interact in 3D environments but is limited by motor and sensor limitations. To circumvent these limitations, we propose a vergence-based motion correlation method to detect visual attention toward very small targets. Smooth depth movements relative to the user are induced on 3D objects, which cause slow vergence eye movements when looked upon. Using the principle of motion correlation, the depth movements of the object and vergence eye movements are matched to determine which object the user is focussing on. In two user studies, we demonstrate how the technique can reliably infer gaze attention on very small targets, systematically explore how different stimulus motions affect attention detection, and show how the technique can be extended to multi-target selection. Finally, we provide example applications using the concept and design guidelines for small target and accuracy-independent attention detection in 3D environments.2023LSLudwig Sidenmark et al.Lancaster UniversityEye Tracking & Gaze InteractionHuman Pose & Activity RecognitionCHI
Resolving Target Ambiguity in 3D Gaze Interaction through VOR Depth EstimationTarget disambiguation is a common problem in gaze interfaces, as eye tracking has accuracy and precision limitations. In 3D environments this is compounded by objects overlapping in the field of view, as a result of their positioning at different depth with partial occlusion. We introduce VOR depth estimation, a method based on the Vestibulo-ocular reflex of the eyes in compensation of head movement, and explore its application to resolve target ambiguity. The method estimates gaze depth by comparing the rotations of the eye and the head when the users look at a target and deliberately rotate their head. We show that VOR eye movement presents an alternative to vergence for gaze depth estimation, that is feasible also with monocular tracking. In an evaluation of its use for target disambiguation, our method outperforms vergence for targets presented at greater depth.2019DMDiako Mardanbegi et al.Lancaster UniversityEye Tracking & Gaze InteractionCHI
SuperVision: Playing with Gaze Aversion and Peripheral VisionIn this work, we challenge the Gaze interaction paradigm "What you see is what you get" to introduce "playing with peripheral vision". We developed the conceptual framework to introduce this novel gaze-aware game dynamic. We illustrated the concept with SuperVision, a collection of three games that play with peripheral vision. We propose perceptual and interaction challenges that require players not to look and rely on their periphery. To validate the game dynamic and experience, we conducted a user study with twenty-four participants. Results show how the game concept created an engaging and playful experience playing with peripheral vision. Participants showed proficiency in overcoming the game challenges, developing clear strategies to succeed. Moreover, we found evidence that playing the game can affect our visual skills, with greater peripheral awareness.2019AGArgenis Ramirez Gomez et al.Lancaster UniversityEye Tracking & Gaze InteractionGame UX & Player BehaviorCHI
Eye&Head: Synergetic Eye and Head Movement for Pointing and SelectionEye gaze involves the coordination of eye and head movement to acquire gaze targets, but existing approaches to gaze pointing are based on eye-tracking in abstraction from head motion. We propose to leverage the synergetic movement of eye and head, and identify design principles for Eye&Head gaze interaction. We introduce three novel techniques that build on the distinction of head-supported versus eyes-only gaze, to enable dynamic coupling of gaze and pointer, hover interaction, visual exploration around pre-selections, and iterative and fast confirmation of targets. We demonstrate Eye&Head interaction on applications in virtual reality, and evaluate our techniques against baselines in pointing and confirmation studies. Our results show that Eye&Head techniques enable novel gaze behaviours that provide users with more control and flexibility in fast gaze pointing and selection.2019LSLudwig Sidenmark et al.Full-Body Interaction & Embodied InputEye Tracking & Gaze InteractionUIST
Motion Correlation: Selecting Objects by Matching Their MovementSelection is a canonical task in user interfaces, commonly supported by presenting objects for acquisition by pointing. In this article, we consider motion correlation as an alternative for selection. The principle is to represent available objects by motion in the interface, have users identify a target by mimicking its specific motion, and use the correlation between the system’s output with the user’s input to determine the selection. The resulting interaction has compelling properties, as users are guided by motion feedback, and only need to copy a presented motion. Motion correlation has been explored in earlier work but only recently begun to feature in holistic interface designs. We provide a first comprehensive review of the principle, and present an analysis of five previously published works, in which motion correlation underpinned the design of novel gaze and gesture interfaces for diverse application contexts. We derive guidelines for motion correlation algorithms, motion feedback, choice of modalities, overall design of motion correlation interfaces, and identify opportunities and challenges identified for future research and design.2018EVEduardo Velloso et al.The University of MelbourneHand Gesture RecognitionFull-Body Interaction & Embodied InputCHI