All

Papers

Browse and search HCI research papers from All

Active Filters

Author: 2389

12 results

Towards AI-driven Sign Language Generation with Non-manual Markers

Sign languages are essential for the Deaf and Hard-of-Hearing (DHH) community. Sign language generation systems have the potential to support communication by translating from written languages, such as English, into signed videos. However, current systems often fail to meet user needs due to poor translation of gramm…

Han Zhang et al.University Of Washington

Vision-Based Hand Gesture Customization from a Single Demonstration

Hand gesture recognition is becoming a more prevalent mode of human-computer interaction, especially as cameras proliferate across everyday devices. Despite continued progress in this field, gesture customization is often underexplored. Customization is crucial since it enables users to define and demonstrate gestures…

Soroush Shahi et al.Apple

Enabling Hand Gesture Customization on Wrist-Worn Devices

We present a framework for gesture customization requiring minimal examples from users, all without degrading the performance of existing gesture sets. To achieve this, we first deployed a large-scale study (N=500+) to collect data and train an accelerometer-gyroscope recognition model with a cross-user accuracy of 95…

Xuhai Xu et al.University Of Washington

Death of a Robot: Social Media Reactions and Language Usage in Response to Robot Retirement

People take to social media to share their thoughts, joys, and sorrows. A recent popular trend has been to support and mourn people and pets that have died as well as other objects that have suffered catastrophic damage. As several popular robots have been discontinued, including the Opportunity Rover, Jibo, and Kuri,…

Elizabeth Jeanne Carter et al.Carnegie Mellon University

AdRecommended

Learn AI Coding at CodeNow

Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work.

Explore Now

Enhancing Mobile Voice Assistants with WorldGaze

Contemporary voice assistants require that objects of inter-est be specified in spoken commands. Of course, users are often looking directly at the object or place of interest – fine-grained, contextual information that is currently unused. We present WorldGaze, a software-only method for smartphones that provides the…

Sven Mayer et al.University of Stuttgart

Automated Class Discovery and One-Shot Interactions for Acoustic Activity Recognition

Acoustic activity recognition has emerged as a foundational element for imbuing devices with context-driven capabilities, enabling richer, more assistive, and more accommodating computational experiences. Traditional approaches rely either on custom models trained in situ, or general models pre-trained on preexisting…

Jason Wu et al.Carnegie Mellon University

Sensing Fine-Grained Hand Activity with Smartwatches

Capturing fine-grained hand activity could make computational experiences more powerful and contextually aware. Indeed, philosopher Immanuel Kant argued, "the hand is the visible part of the brain." However, most prior work has focused on detecting whole-body activities, such as walking, running and bicycling. In this…

Gierad Laput et al.Carnegie Mellon University

SurfaceSight: A New Spin on Touch, User, and Object Sensing for IoT Experiences

IoT appliances are gaining consumer traction, from smart thermostats to smart speakers. These devices generally have limited user interfaces, most often small buttons and touchscreens, or rely on voice control. Further, these devices know little about their surroundings – unaware of objects, people and activities happ…

Gierad Laput et al.Carnegie Mellon University

Sensing Posture-Aware Pen+Touch Interaction on Tablets

Many status-quo interfaces for tablets with pen + touch input capabilities force users to reach for device-centric UI widgets at fixed locations, rather than sensing and adapting to the user-centric posture. To address this problem, we propose sensing techniques that transition between various nuances of mobile and st…

Yang Zhang et al.University of California

ElectroDermis: Fully Untethered, Stretchable, and Highly-Customizable Electronic Bandages

Wearables have emerged as an increasingly promising interactive platform, imbuing the human body with always-available computational capabilities. This unlocks a wide range of applications, including discreet information access, health monitoring, fitness, and fashion. However, unlike previous platforms, wearable elec…

Eric Markvicka et al.Carnegie Mellon University

Ubicoustics: Plug-and-Play Acoustic Activity Recognition

Despite sound being a rich source of information, computing devices with microphones do not leverage audio to glean useful insights about their physical and social context. For example, a smart speaker sitting on a kitchen countertop cannot figure out if it is in a kitchen, let alone know what a user is doing in a kit…

Gierad Laput et al.Carnegie Mellon University

Vibrosight: Long-Range Vibrometry for Smart Environment Sensing

Smart and responsive environments rely on the ability to detect physical events, such as appliance use and human activities. Currently, to sense these types of events, one must either upgrade to “smart” appliances, or attach aftermarket sensors to existing objects. These approaches can be expensive, intrusive and infl…

Yang Zhang et al.University of California

Paper Title	Authors	Research Topics	Paper Database	Year
emoji_events Towards AI-driven Sign Language Generation with Non-manual Markers Sign languages are essential for the Deaf and Hard-of-Hearing (DHH) community. Sign language generation systems have the potential to support communication by translating from written languages, such as English, into signed videos. However, current systems often fail to meet user needs due to poor translation of gramm…	HZ Han Zhang et al.University Of Washington
Vision-Based Hand Gesture Customization from a Single Demonstration Hand gesture recognition is becoming a more prevalent mode of human-computer interaction, especially as cameras proliferate across everyday devices. Despite continued progress in this field, gesture customization is often underexplored. Customization is crucial since it enables users to define and demonstrate gestures…	SS Soroush Shahi et al.Apple
emoji_events Enabling Hand Gesture Customization on Wrist-Worn Devices We present a framework for gesture customization requiring minimal examples from users, all without degrading the performance of existing gesture sets. To achieve this, we first deployed a large-scale study (N=500+) to collect data and train an accelerometer-gyroscope recognition model with a cross-user accuracy of 95…	XX Xuhai Xu et al.University Of Washington
Death of a Robot: Social Media Reactions and Language Usage in Response to Robot Retirement People take to social media to share their thoughts, joys, and sorrows. A recent popular trend has been to support and mourn people and pets that have died as well as other objects that have suffered catastrophic damage. As several popular robots have been discontinued, including the Opportunity Rover, Jibo, and Kuri,…	EC Elizabeth Jeanne Carter et al.Carnegie Mellon University
AdRecommended Learn AI Coding at CodeNow Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work. Explore Now
Enhancing Mobile Voice Assistants with WorldGaze Contemporary voice assistants require that objects of inter-est be specified in spoken commands. Of course, users are often looking directly at the object or place of interest – fine-grained, contextual information that is currently unused. We present WorldGaze, a software-only method for smartphones that provides the…	SM Sven Mayer et al.University of Stuttgart
emoji_events Automated Class Discovery and One-Shot Interactions for Acoustic Activity Recognition Acoustic activity recognition has emerged as a foundational element for imbuing devices with context-driven capabilities, enabling richer, more assistive, and more accommodating computational experiences. Traditional approaches rely either on custom models trained in situ, or general models pre-trained on preexisting…	JW Jason Wu et al.Carnegie Mellon University
Sensing Fine-Grained Hand Activity with Smartwatches Capturing fine-grained hand activity could make computational experiences more powerful and contextually aware. Indeed, philosopher Immanuel Kant argued, "the hand is the visible part of the brain." However, most prior work has focused on detecting whole-body activities, such as walking, running and bicycling. In this…	GL Gierad Laput et al.Carnegie Mellon University
SurfaceSight: A New Spin on Touch, User, and Object Sensing for IoT Experiences IoT appliances are gaining consumer traction, from smart thermostats to smart speakers. These devices generally have limited user interfaces, most often small buttons and touchscreens, or rely on voice control. Further, these devices know little about their surroundings – unaware of objects, people and activities happ…	GL Gierad Laput et al.Carnegie Mellon University
emoji_events Sensing Posture-Aware Pen+Touch Interaction on Tablets Many status-quo interfaces for tablets with pen + touch input capabilities force users to reach for device-centric UI widgets at fixed locations, rather than sensing and adapting to the user-centric posture. To address this problem, we propose sensing techniques that transition between various nuances of mobile and st…	YZ Yang Zhang et al.University of California
ElectroDermis: Fully Untethered, Stretchable, and Highly-Customizable Electronic Bandages Wearables have emerged as an increasingly promising interactive platform, imbuing the human body with always-available computational capabilities. This unlocks a wide range of applications, including discreet information access, health monitoring, fitness, and fashion. However, unlike previous platforms, wearable elec…	EM Eric Markvicka et al.Carnegie Mellon University
Ubicoustics: Plug-and-Play Acoustic Activity Recognition Despite sound being a rich source of information, computing devices with microphones do not leverage audio to glean useful insights about their physical and social context. For example, a smart speaker sitting on a kitchen countertop cannot figure out if it is in a kitchen, let alone know what a user is doing in a kit…	GL Gierad Laput et al.Carnegie Mellon University
Vibrosight: Long-Range Vibrometry for Smart Environment Sensing Smart and responsive environments rely on the ability to detect physical events, such as appliance use and human activities. Currently, to sense these types of events, one must either upgrade to “smart” appliances, or attach aftermarket sensors to existing objects. These approaches can be expensive, intrusive and infl…	YZ Yang Zhang et al.University of California