All

Papers

Browse and search HCI research papers from All

Active Filters

Author: 36556

11 results

Gaze and Speech in Multimodal Human-Computer Interaction: A Scoping Review

Multimodal interaction has long promised to make interfaces more intuitive and effective by combining complementary inputs. Among these, gaze and speech form a compelling pairing: gaze provides rapid spatial grounding, while speech conveys rich semantic information. Together, they offer rich cues for understanding use…

Anam Ahmad Khan et al.University of Melbourne

Understanding the Impact of the Reality-Virtuality Continuum on Visual Search using Physiological Measures

While Mixed Reality allows the seamless blending of digital content in their surroundings, it is not clear if such a fusion of digital and physical information impacts users' perceptual and cognitive resources differently. While the fusion of real and virtual objects provides numerous opportunities to present addition…

Francesco Chiossi et al.Ludwig Maximilian University of Munich

Classifying Head Movements to Separate Head-Gaze and Head Gestures as Distinct Modes of Input

Head movement is widely used as a uniform type of input for human-computer interaction. However, there are fundamental differences between head movements coupled with gaze in support of our visual system, and head movements performed as gestural expression. Both Head-Gaze and Head Gestures are of utility for interacti…

Baosheng James HOU et al.Lancaster University

Speech-Augmented Cone-of-Vision for Exploratory Data Analysis

Mutual awareness of visual attention is crucial for successful collaboration. Previous research has explored various ways to represent visual attention, such as field-of-view visualizations and cursor visualizations based on eye-tracking, but these methods have limitations. Verbal communication is often utilized as a…

Riccardo Bovo et al.Imperial College London

AdRecommended

Learn AI Coding at CodeNow

Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work.

Explore Now

Vergence Matching: Inferring Attention to Objects in 3D Environments for Gaze-Assisted Selection

Gaze pointing is the de facto standard to infer attention and interact in 3D environments but is limited by motor and sensor limitations. To circumvent these limitations, we propose a vergence-based motion correlation method to detect visual attention toward very small targets. Smooth depth movements relative to the u…

Ludwig Sidenmark et al.Lancaster University

Integrating Gaze and Speech for Enabling Implicit Interactions

Gaze and speech are rich contextual sources of information that, when combined, can result in effective and rich multimodal interactions. This paper proposes a machine learning-based pipeline that leverages and combines users’ natural gaze activity, the semantic knowledge from their vocal utterances and the synchronic…

Anam Ahmad Khan et al.University of Melbourne

To Type or To Speak? The Effect of Input Modality on Text Understanding During Note-taking

Though recent technological advances have enabled note-taking through different modalities (e.g., keyboard, digital ink, voice), there is still a lack of understanding of the effect of the modality choice on learning. In this paper, we compared two note-taking input modalities—keyboard and voice—to study their effects…

Anam Ahmad Khan et al.University of Melbourne

Faces of Focus: A Study on the Facial Cues of Attentional States

Automatically detecting attentional states is a prerequisite for designing interventions to manage attention — knowledge workers' most critical resource. As a first step towards this goal, it is necessary to understand how different attentional states are made discernible through visible cues in knowledge workers. In…

Ebrahim Babaei et al.University of Melbourne

Frame Analysis of Voice Interaction Gameplay

Voice control is an increasingly common feature of digital games, but the experience of playing with voice control is often hampered by feelings of embarrassment and dissonance. Past research has recognised these tensions, but has not offered a general model of how they arise and how players respond to them. In this s…

Fraser Allison et al.University of Melbourne

Looks Can Be Deceiving: Using Gaze Visualisation to Predict and Mislead Opponents in Strategic Gameplay

In competitive co-located gameplay, players use their opponents' gaze to make predictions about their plans while simultaneously managing their own gaze to avoid giving away their plans. This socially competitive dimension is lacking in most online games, where players are out of sight of each other. We conducted a la…

Joshua Newn et al.RMIT University

Motion Correlation: Selecting Objects by Matching Their Movement

Selection is a canonical task in user interfaces, commonly supported by presenting objects for acquisition by pointing. In this article, we consider motion correlation as an alternative for selection. The principle is to represent available objects by motion in the interface, have users identify a target by mimicking…

Eduardo Velloso et al.University of Melbourne

Paper Title	Authors	Research Topics	Paper Database	Year
Gaze and Speech in Multimodal Human-Computer Interaction: A Scoping Review Multimodal interaction has long promised to make interfaces more intuitive and effective by combining complementary inputs. Among these, gaze and speech form a compelling pairing: gaze provides rapid spatial grounding, while speech conveys rich semantic information. Together, they offer rich cues for understanding use…	AK Anam Ahmad Khan et al.University of Melbourne
Understanding the Impact of the Reality-Virtuality Continuum on Visual Search using Physiological Measures While Mixed Reality allows the seamless blending of digital content in their surroundings, it is not clear if such a fusion of digital and physical information impacts users' perceptual and cognitive resources differently. While the fusion of real and virtual objects provides numerous opportunities to present addition…	FC Francesco Chiossi et al.Ludwig Maximilian University of Munich
Classifying Head Movements to Separate Head-Gaze and Head Gestures as Distinct Modes of Input Head movement is widely used as a uniform type of input for human-computer interaction. However, there are fundamental differences between head movements coupled with gaze in support of our visual system, and head movements performed as gestural expression. Both Head-Gaze and Head Gestures are of utility for interacti…	BH Baosheng James HOU et al.Lancaster University
Speech-Augmented Cone-of-Vision for Exploratory Data Analysis Mutual awareness of visual attention is crucial for successful collaboration. Previous research has explored various ways to represent visual attention, such as field-of-view visualizations and cursor visualizations based on eye-tracking, but these methods have limitations. Verbal communication is often utilized as a…	RB Riccardo Bovo et al.Imperial College London
AdRecommended Learn AI Coding at CodeNow Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work. Explore Now
Vergence Matching: Inferring Attention to Objects in 3D Environments for Gaze-Assisted Selection Gaze pointing is the de facto standard to infer attention and interact in 3D environments but is limited by motor and sensor limitations. To circumvent these limitations, we propose a vergence-based motion correlation method to detect visual attention toward very small targets. Smooth depth movements relative to the u…	LS Ludwig Sidenmark et al.Lancaster University
Integrating Gaze and Speech for Enabling Implicit Interactions Gaze and speech are rich contextual sources of information that, when combined, can result in effective and rich multimodal interactions. This paper proposes a machine learning-based pipeline that leverages and combines users’ natural gaze activity, the semantic knowledge from their vocal utterances and the synchronic…	AK Anam Ahmad Khan et al.University of Melbourne
To Type or To Speak? The Effect of Input Modality on Text Understanding During Note-taking Though recent technological advances have enabled note-taking through different modalities (e.g., keyboard, digital ink, voice), there is still a lack of understanding of the effect of the modality choice on learning. In this paper, we compared two note-taking input modalities—keyboard and voice—to study their effects…	AK Anam Ahmad Khan et al.University of Melbourne
Faces of Focus: A Study on the Facial Cues of Attentional States Automatically detecting attentional states is a prerequisite for designing interventions to manage attention — knowledge workers' most critical resource. As a first step towards this goal, it is necessary to understand how different attentional states are made discernible through visible cues in knowledge workers. In…	EB Ebrahim Babaei et al.University of Melbourne
Frame Analysis of Voice Interaction Gameplay Voice control is an increasingly common feature of digital games, but the experience of playing with voice control is often hampered by feelings of embarrassment and dissonance. Past research has recognised these tensions, but has not offered a general model of how they arise and how players respond to them. In this s…	FA Fraser Allison et al.University of Melbourne
Looks Can Be Deceiving: Using Gaze Visualisation to Predict and Mislead Opponents in Strategic Gameplay In competitive co-located gameplay, players use their opponents' gaze to make predictions about their plans while simultaneously managing their own gaze to avoid giving away their plans. This socially competitive dimension is lacking in most online games, where players are out of sight of each other. We conducted a la…	JN Joshua Newn et al.RMIT University
Motion Correlation: Selecting Objects by Matching Their Movement Selection is a canonical task in user interfaces, commonly supported by presenting objects for acquisition by pointing. In this article, we consider motion correlation as an alternative for selection. The principle is to represent available objects by motion in the interface, have users identify a target by mimicking…	EV Eduardo Velloso et al.University of Melbourne