StylusPort: Investigating Teleportation using Stylus in VRWith a stylus, users can both sweep sketches across models and pinpoint locations with precision. Building on this dual capability, we explore how teleportation can be integrated into stylus interaction without disrupting the flow of common stylus usage. We introduce two key ideas: flipping the stylus as an intuitive mode switch between drawing and teleportation, and using gaze to set orientation while the stylus handles positioning. In a user study that features a teleport-and-orient task, we evaluate six teleportation techniques, covering two mode-switching methods (Button and Flip) and three orientation approaches (StylusRoll, StylusPoint, and GazePoint). The results offer new insights into the relative merits and limitations of each technique. Our work contributes to knowledge about teleportation in VR and fills the gap in seamlessly integrating teleportation with stylus use in 3D.2026YLYang Liu et al.Aarhus UniversitySocial & Collaborative VRHand Gesture RecognitionEye Tracking & Gaze InteractionCHI
Gaze and Speech in Multimodal Human-Computer Interaction: A Scoping ReviewMultimodal interaction has long promised to make interfaces more intuitive and effective by combining complementary inputs. Among these, gaze and speech form a compelling pairing: gaze provides rapid spatial grounding, while speech conveys rich semantic information. Together, they offer rich cues for understanding user behaviour and intent. Yet despite decades of exploration, the research remains fragmented, making this synthesis timely as these inputs mature and are integrated into consumer-ready devices. This scoping review examined 103 studies published between 1991 and 2025, organised into \emph{explicit}, where users intentionally provide gaze and speech, and \emph{implicit}, where systems leverage users' natural behaviours to support interaction. Across both, we identified recurring ways for combining gaze and speech to resolve ambiguity, ground references, and support adaptivity. We contribute a synthesis of research on their combined use while highlighting challenges of temporal alignment, fusion and privacy, offering guidance for future research toward richer multimodal human-computer interaction.2026AKAnam Ahmad Khan et al.KAISTEye Tracking & Gaze InteractionVoice User Interface (VUI) DesignAffective Human-Computer DialogueCHI
Preshaping Hand Behaviour for Direct and Indirect Manipulation of 3D ObjectsEffortless manipulation informs and relies on preshaping: the subconscious posing of the hand before grasping. Virtual environments and the design of interaction techniques alters interaction requirements like contact and reach, forcing behavioural adaptation. We present a comparative study investigating preshaping behaviour across direct versus indirect (gaze-assisted) and bare-hand versus controller techniques on a docking task. Results reveal that response patterns scale with anticipated task difficulty, and that direct techniques elicit effective posing of the hand. Indirect techniques shortcut hand transport and, in turn lacks the sensory feedback to guide planning, inducing efficient but attenuated responses that necessitate compensatory manipulation and clutching. Notably, controllers that afford in-hand rotation allow users to extend their range of motion. These findings can inform interaction design to better afford preshaping and optimise 3D manipulation tasks.2026TMThorbjørn Mikkelsen et al.Aarhus UniversityHand Gesture RecognitionFull-Body Interaction & Embodied Input3D Modeling & AnimationCHI
Boundary Switching and Cursor Warping: A Comparative Study of Performance and Comfort in Multi-Display XR EnvironmentsExtended Reality (XR) headsets enable large, reconfigurable multi-display workspaces and support view manipulation, allowing the workspace to reposition itself around the user. Cursor warping similarly reduces traversal distance and pointer search by reinitialising the cursor at defined locations. Yet when both mechanisms operate together, the spatial relationship between user, displays, and cursor becomes dynamic, and it remains unclear how cursor repositioning behaves when the workspace itself moves. In a study (N=20) of five cursor-warping strategies with two view manipulations, we show that the benefits of both do not automatically combine: workspace motion can disrupt spatial consistency and alter both performance and movement costs. We show that continuous cursor movement in world space is limited compared to alternative warping techniques, and cursor behaviour and view control are tightly coupled. Hence, cursor initialisation and view manipulation must be co-designed to support efficient and comfortable interaction in XR multi-display environments.2026YCYuzheng Chen et al.Lancaster UniversityMixed Reality WorkspacesImmersion & Presence ResearchPrototyping & User TestingCHI
The Eye–Head Mover Spectrum: Modelling Individual and Population Head Movement Tendencies in Virtual RealityPeople differ in how much they move their head versus their eyes when shifting gaze, yet such tendencies remain largely unexplored in HCI. We introduce head movement tendencies as a fundamental dimension of individual difference in VR and provide a quantitative account of their population-level distribution. Using a 360° video free-viewing dataset (N=87), we model head contributions to gaze shifts with a hinge-based parametric function, revealing a spectrum of strategies from eye-movers to head-movers. We then conduct a user study (N=28) combining 360° video viewing with a short controlled task using gaze targets. While parameter values differ across tasks, individuals show partial alignment in their relative positions within the population, indicating that tendencies are meaningful but shaped by context. Our findings establish head movement tendencies as an important concept for VR and highlight implications for adaptive systems such as foveated rendering, viewport alignment, and multi-user experience design.2026JHJinghui Hu et al.Lancaster UniversityImmersion & Presence ResearchEye Tracking & Gaze InteractionSocial & Collaborative VRCHI
On-body Icons: Designing a 3D Interface for Launching Apps in Augmented RealityOn-body tapping provides a quick way to launch augmented reality (AR) apps using virtual shortcuts placed on the user’s skin, clothes, and jewelry. While prior work has focused on tapping performance, social acceptance, and sensing techniques, users’ behaviour in placing shortcuts on their body has been underexplored. In this work, we propose On-body Icons — a novel interface for launching apps via touching virtual icons placed across the user’s entire body, and use it to investigate locations, reasons for chosen icon placement, and users’ attitudes towards the feature. Results of the qualitative study conducted with 24 participants demonstrated that people employ a wide variety of placement strategies that balance memorability of the locations with accuracy and comfort of reaching the icons. We discuss these findings in regard to current understanding of memorability of icon placement, placement appropriateness, and privacy, and offer design implications for similar features in spatial applications.2025UTUliana Tsimbalistaia et al.HSE UniversityAR Navigation & Context AwarenessOn-Skin Display & On-Skin InputCHI
It’s Not Always the Same Eye That Dominates: Effects of Viewing Angle, Handedness and Eye Movement in 3DUnderstanding eye dominance, the subconscious preference for one eye, has significant implications for 3D user interfaces in VR and AR, particularly in interface design and rendering. Although HCI recognizes eye dominance, little is known about what causes it to switch from one eye to another. To explore this, we studied eye dominance in VR, where 28 participants manually aligned a cursor with a distant target across three tasks. We manipulated the horizontal viewing angle, the hand used for alignment, and eye movement induced by target behaviour. Our results confirm the dynamic nature of eye dominance, though with fewer switches than expected and varying influences across tasks. This highlights the need for adaptive HCI techniques, which account for shifts in eye dominance in system design, such as gaze-based interaction, visual design, or rendering, and can improve accuracy, usability, and experience.2025FPHans Gellersen et al.Lancaster University, School of Computing and CommunicationsEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
Hands-on, Hands-off: Gaze-Assisted Bimanual 3D InteractionExtended Reality (XR) systems with hand-tracking support direct manipulation of objects with both hands. A common interaction in this context is for the non-dominant hand (NDH) to orient an object for input by the dominant hand (DH). We explore bimanual interaction with gaze through three new modes of interaction where the input of the NDH, DH, or both hands is indirect based on Gaze+Pinch. These modes enable a new dynamic interplay between our hands, allowing flexible alternation between and pairing of complementary operations. Through applications, we demonstrate several use cases in the context of 3D modelling, where users exploit occlusion-free, low-effort, and fluid two-handed manipulation. To gain a deeper understanding of each mode, we present a user study on an asymmetric rotate-translate task. Most participants preferred indirect input with both hands for lower physical effort, without a penalty on user performance. Otherwise, they preferred modes where the NDH oriented the object directly, supporting preshaping of the hand, which is more challenging with indirect gestures. The insights gained are of relevance for the design of XR interfaces that aim to leverage eye and hand input in tandem.2024MLMathias N. Lystbæk et al.Hand Gesture RecognitionEye Tracking & Gaze InteractionMixed Reality WorkspacesUIST
Eye-Hand Movement of Objects in Near Space Extended RealityHand-tracking in Extended Reality (XR) enables moving objects in near space with direct hand gestures, to pick, drag and drop objects in 3D. In this work, we investigate the use of eye-tracking to reduce the effort involved in this interaction. As the eyes naturally look ahead to the target for a drag operation, the principal idea is to map the translation of the object in the image plane to gaze, such that the hand only needs to control the depth component of the operation. We have implemented four techniques that explore two factors: the use of gaze only to move objects in X-Y vs.\ extra refinement by hand, and the use of hand input in the Z axis to directly move objects vs.\ indirectly via a transfer function. We compared all four techniques in a user study (N=24) against baselines of direct and indirect hand input. We detail user performance, effort and experience trade-offs and show that all eye-hand techniques significantly reduce physical effort over direct gestures, pointing toward effortless drag-and-drop for XR environments.2024UWUta Wagner et al.Hand Gesture RecognitionEye Tracking & Gaze InteractionUIST
Spatial Gaze Markers: Supporting Effective Task Switching in Augmented RealityTask switching can occur frequently in daily routines with physical activity. In this paper, we introduce Spatial Gaze Markers, an augmented reality tool to support users in immediately returning to the last point of interest after an attention shift. The tool is task-agnostic, using only eye-tracking information to infer distinct points of visual attention and to mark the corresponding area in the physical environment. We present a user study that evaluates the effectiveness of Spatial Gaze Markers in simulated physical repair and inspection tasks against a no-marker baseline. The results give insights into how Spatial Gaze Markers affect user performance, task load, and experience of users with varying levels of task type and distractions. Our work is relevant to assist physical workers with simple AR techniques and render task switching faster with less effort.2024MLMathias N. Lystbæk et al.Aarhus UniversityEye Tracking & Gaze InteractionAR Navigation & Context AwarenessCHI
Snap, Pursuit and Gain: Virtual Reality Viewport Control by GazeHead-mounted displays let users explore virtual environments through a viewport that is coupled with head movement. In this work, we investigate gaze as an alternative modality for viewport control, enabling exploration of virtual worlds with less head movement. We designed three techniques that leverage gaze based on different eye movements: Dwell Snap for viewport rotation in discrete steps, Gaze Gain for amplified viewport rotation based on gaze angle, and Gaze Pursuit for central viewport alignment of gaze targets. All three techniques enable 360-degree viewport control through naturally coordinated eye and head movement. We evaluated the techniques in comparison with controller snap and head amplification baselines, for both coarse and precise viewport control, and found them to be as fast and accurate. We observed a high variance in performance which may be attributable to the different degrees to which humans tend to support gaze shifts with head movement.2024HLHock Siang Lee et al.LancasterEye Tracking & Gaze InteractionImmersion & Presence ResearchCHI
Blended Whiteboard: Physicality and Reconfigurability in Remote Mixed Reality CollaborationThe whiteboard is essential for collaborative work. To preserve its physicality in remote collaboration, Mixed Reality (MR) can blend real whiteboards across distributed spaces. Going beyond reality, MR can further enable interactions like panning and zooming in a virtually reconfigurable infinite whiteboard. However, this reconfigurability conflicts with the sense of physicality. To address this tension, we introduce Blended Whiteboard, a remote collaborative MR system enabling reconfigurable surface blending across distributed physical whiteboards. Blended Whiteboard supports a unique collaboration style, where users can sketch on their local whiteboards but also reconfigure the blended space to facilitate transitions between loosely and tightly coupled work. We describe design principles inspired by proxemics; supporting users in changing between facing each other and being side-by-side, and switching between navigating the whiteboard synchronously and independently. Our work shows exciting benefits and challenges of combining physicality and reconfigurability in the design of distributed MR whiteboards.2024JGJens Emil Sloth Grønbæk et al.Aarhus University, University of MelbourneMixed Reality WorkspacesDistributed Team CollaborationCHI
Gaze on the Go: Effect of Spatial Reference Frame on Visual Target Acquisition During Physical Locomotion in Extended RealitySpatial interaction relies on fast and accurate visual acquisition. In this work, we analyse how visual acquisition and tracking of targets presented in a head-mounted display is affected by the user moving linearly at walking and jogging paces. We study four reference frames in which targets can be presented: Head and World where targets are affixed relative to the head and environment, respectively; HeadDelay where targets are presented in the head coordinate system but follow head movement with a delay, and novel Path where targets remain at fixed distance in front of the user, in the direction of their movement. Results of our study in virtual reality demonstrate that the more stable the target is relative to the environment, the faster and more precise it can be fixated. The results have practical significance as head-mounted displays enable interaction during mobility, and in particular when eye tracking is considered as input.2024PMPavel Manakhov et al.Aarhus UniversityFull-Body Interaction & Embodied InputEye Tracking & Gaze InteractionAR Navigation & Context AwarenessCHI
Classifying Head Movements to Separate Head-Gaze and Head Gestures as Distinct Modes of InputHead movement is widely used as a uniform type of input for human-computer interaction. However, there are fundamental differences between head movements coupled with gaze in support of our visual system, and head movements performed as gestural expression. Both Head-Gaze and Head Gestures are of utility for interaction but differ in their affordances. To facilitate the treatment of Head-Gaze and Head Gestures as separate types of input, we developed HeadBoost as a novel classifier, achieving high accuracy in classifying gaze-driven versus gestural head movement (F1-Score: 0.89). We demonstrate the utility of the classifier with three applications: gestural input while avoiding unintentional input by Head-Gaze; target selection with Head-Gaze while avoiding Midas Touch by head gestures; and switching of cursor control between Head-Gaze for fast positioning and Head Gesture for refinement. The classification of Head-Gaze and Head Gesture allows for seamless head-based interaction while avoiding false activation.2023BHBaosheng James HOU et al.Lancaster UniversityEye Tracking & Gaze InteractionHuman Pose & Activity RecognitionCHI
A Fitts' Law Study of Gaze-Hand Alignment for Selection in 3D User InterfacesGaze-Hand Alignment has recently been proposed for multimodal selection in 3D. The technique takes advantage of gaze for target pre-selection, as it naturally precedes manual input. Selection is then completed when manual input aligns with gaze on the target, without need for an additional click method. In this work we evaluate two alignment techniques, Gaze&Finger and Gaze&Handray, combining gaze with image plane pointing versus raycasting, in comparison with hands-only baselines and Gaze&Pinch as established multimodal technique. We used Fitts' Law study design with targets presented at different depths in the visual scene, to assess effect of parallax on performance. The alignment techniques outperformed their respective hands-only baselines. Gaze&Finger is efficient when targets are close to the image plane but less performant with increasing target depth due to parallax.2023UWUta Wagner et al.Aarhus UniversityHand Gesture RecognitionEye Tracking & Gaze InteractionCHI
Partially Blended Realities: Aligning Dissimilar Spaces for Distributed Mixed Reality MeetingsMixed Reality allows for distributed meetings where people's local physical spaces are virtually aligned into blended interaction spaces. In many cases, people's physical rooms are dissimilar, making it challenging to design a coherent blended space. We introduce the concept of Partially Blended Realities (PBR) --- using Mixed Reality to support remote collaborators in partially aligning their physical spaces. As physical surfaces are central in collaborative work, PBR supports users in transitioning between different configurations of tables and whiteboard surfaces. In this paper, we 1) describe the design space of PBR, 2) present RealityBlender to explore interaction techniques for how users may configure and transition between blended spaces, and 3) provide insights from a study on how users experience transitions in a remote collaboration task. With this work, we demonstrate new potential for using partial solutions to tackle the alignment problem of dissimilar spaces in distributed Mixed Reality meetings.2023JGJens Emil Sloth Grønbæk et al.Aarhus UniversityMixed Reality WorkspacesDistributed Team CollaborationCHI
Vergence Matching: Inferring Attention to Objects in 3D Environments for Gaze-Assisted SelectionGaze pointing is the de facto standard to infer attention and interact in 3D environments but is limited by motor and sensor limitations. To circumvent these limitations, we propose a vergence-based motion correlation method to detect visual attention toward very small targets. Smooth depth movements relative to the user are induced on 3D objects, which cause slow vergence eye movements when looked upon. Using the principle of motion correlation, the depth movements of the object and vergence eye movements are matched to determine which object the user is focussing on. In two user studies, we demonstrate how the technique can reliably infer gaze attention on very small targets, systematically explore how different stimulus motions affect attention detection, and show how the technique can be extended to multi-target selection. Finally, we provide example applications using the concept and design guidelines for small target and accuracy-independent attention detection in 3D environments.2023LSLudwig Sidenmark et al.Lancaster UniversityEye Tracking & Gaze InteractionHuman Pose & Activity RecognitionCHI
Resolving Target Ambiguity in 3D Gaze Interaction through VOR Depth EstimationTarget disambiguation is a common problem in gaze interfaces, as eye tracking has accuracy and precision limitations. In 3D environments this is compounded by objects overlapping in the field of view, as a result of their positioning at different depth with partial occlusion. We introduce VOR depth estimation, a method based on the Vestibulo-ocular reflex of the eyes in compensation of head movement, and explore its application to resolve target ambiguity. The method estimates gaze depth by comparing the rotations of the eye and the head when the users look at a target and deliberately rotate their head. We show that VOR eye movement presents an alternative to vergence for gaze depth estimation, that is feasible also with monocular tracking. In an evaluation of its use for target disambiguation, our method outperforms vergence for targets presented at greater depth.2019DMDiako Mardanbegi et al.Lancaster UniversityEye Tracking & Gaze InteractionCHI
SuperVision: Playing with Gaze Aversion and Peripheral VisionIn this work, we challenge the Gaze interaction paradigm "What you see is what you get" to introduce "playing with peripheral vision". We developed the conceptual framework to introduce this novel gaze-aware game dynamic. We illustrated the concept with SuperVision, a collection of three games that play with peripheral vision. We propose perceptual and interaction challenges that require players not to look and rely on their periphery. To validate the game dynamic and experience, we conducted a user study with twenty-four participants. Results show how the game concept created an engaging and playful experience playing with peripheral vision. Participants showed proficiency in overcoming the game challenges, developing clear strategies to succeed. Moreover, we found evidence that playing the game can affect our visual skills, with greater peripheral awareness.2019AGArgenis Ramirez Gomez et al.Lancaster UniversityEye Tracking & Gaze InteractionGame UX & Player BehaviorCHI
Eye&Head: Synergetic Eye and Head Movement for Pointing and SelectionEye gaze involves the coordination of eye and head movement to acquire gaze targets, but existing approaches to gaze pointing are based on eye-tracking in abstraction from head motion. We propose to leverage the synergetic movement of eye and head, and identify design principles for Eye&Head gaze interaction. We introduce three novel techniques that build on the distinction of head-supported versus eyes-only gaze, to enable dynamic coupling of gaze and pointer, hover interaction, visual exploration around pre-selections, and iterative and fast confirmation of targets. We demonstrate Eye&Head interaction on applications in virtual reality, and evaluate our techniques against baselines in pointing and confirmation studies. Our results show that Eye&Head techniques enable novel gaze behaviours that provide users with more control and flexibility in fast gaze pointing and selection.2019LSLudwig Sidenmark et al.Full-Body Interaction & Embodied InputEye Tracking & Gaze InteractionUIST