All

Papers

Browse and search HCI research papers from All

Active Filters

Author: 8283

12 results

ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators

Audio Description (AD) provides essential access to visual media for blind and low vision (BLV) audiences. Yet current AD production tools remain largely inaccessible to BLV video creators, who possess valuable expertise but face barriers due to visually-driven interfaces. We present ADCanvas, a multimodal authoring s…

Franklin Mingzhe Li et al.Carnegie Mellon University

Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning

Multimodal large language models (MLLMs), with their expansive world knowledge and reasoning capabilities, present a unique opportunity for end-users to create personalized AI sensors capable of reasoning about complex situations. A user could describe a desired sensing task in natural language (e.g., "let me know if…

Michael Xieyang Liu et al.Carnegie Mellon University

Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models

Sensemaking in unfamiliar domains can be challenging, demanding considerable user effort to compare different options with respect to various criteria. Prior research and our formative study found that people would benefit from reading an overview of an information space upfront, including the criteria others previous…

Michael Xieyang Liu et al.Carnegie Mellon University

A Contextual Inquiry of People with Vision Impairments in Cooking

Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understan…

Franklin Mingzhe Li et al.Carnegie Mellon University

AdRecommended

Learn AI Coding at CodeNow

Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work.

Explore Now

Facilitating Experiential Training for Counselors using a Real-time Annotation Tool

Experiential training, where mental health professionals practice their learned skills, remains the most costly component of therapeutic training. We introduce Pin-MI, a video-call-based tool that supports experiential learning of counseling skills used in motivational interviewing (MI) through interactive role-play a…

Tianying Chen et al.Carnegie Mellon University

"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models

Code-generating large language models map natural language to code. However, only a small portion of the infinite space of naturalistic utterances is effective at guiding code generation. For non-expert end-user programmers, learning this is the challenge of abstraction matching. We examine this challenge in the speci…

Michael Xieyang Liu et al.Carnegie Mellon University

Wigglite: Low-cost Information Collection and Triage

Consumers conducting comparison shopping, researchers making sense of competitive space, and developers looking for code snippets online all face the challenge of capturing the information they find for later use without interrupting their current flow. In addition, during many learning and exploration tasks, people n…

Michael Xieyang Liu et al.Carnegie Mellon University

Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making

Developers perform online sensemaking on a daily basis, such as researching and choosing libraries and APIs. Prior research has introduced tools that help developers capture information from various sources and organize it into structures useful for subsequent decision-making. However, it remains a laborious process f…

Michael Xieyang Liu et al.Carnegie Mellon University

Understanding How Programmers Can Use Annotations on Documentation

Modern software development requires developers to find and effectively utilize new APIs and their documentation, but documentation has many well-known issues. Despite this, developers eventually overcome these issues but have no way of sharing what they learned. We investigate sharing this documentation-specific info…

Amber Horvath et al.Carnegie Mellon University

To Reuse or Not To Reuse? A Framework and System for Evaluating Summarized Knowledge

As the amount of information online continues to grow, a correspondingly important opportunity is for individuals to reuse knowledge which has been summarized by others rather than starting from scratch. However, appropriate reuse requires judging the relevance, trustworthiness, and thoroughness of others' knowledge i…

Michael Xieyang Liu et al.Carnegie Mellon University

Algorithms and Decision Making

Unakite: Scaffolding Developers’ Decision-Making Using the Web

Developers spend a significant portion of their time searching for solutions and methods online. While numerous tools have been developed to support this exploratory process, in many cases the answers to developers’ questions involve trade-offs among multiple valid options and not just a single solution. Through inter…

Michael Xieyang Liu et al.Carnegie Mellon University

Popup: Reconstructing 3D Video Using Particle Filtering to Aggregate Crowd Responses

Collecting a sufficient amount of 3D training data for autonomous vehicles to handle rare, but critical, traffic events (e.g., collisions) may take decades of deployment. Abundant video data of such events from municipal traffic cameras and video sharing sites (e.g., YouTube) could provide a potential alternative, but…

Jean Y Song et al.University of Michigan

Paper Title	Authors	Research Topics
ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators Audio Description (AD) provides essential access to visual media for blind and low vision (BLV) audiences. Yet current AD production tools remain largely inaccessible to BLV video creators, who possess valuable expertise but face barriers due to visually-driven interfaces. We present ADCanvas, a multimodal authoring s…	FL Franklin Mingzhe Li et al.Carnegie Mellon University
Gensors: Authoring Personalized Visual Sensors with Multimodal Foundation Models and Reasoning Multimodal large language models (MLLMs), with their expansive world knowledge and reasoning capabilities, present a unique opportunity for end-users to create personalized AI sensors capable of reasoning about complex situations. A user could describe a desired sensing task in natural language (e.g., "let me know if…	ML Michael Xieyang Liu et al.Carnegie Mellon University
Selenite: Scaffolding Online Sensemaking with Comprehensive Overviews Elicited from Large Language Models Sensemaking in unfamiliar domains can be challenging, demanding considerable user effort to compare different options with respect to various criteria. Prior research and our formative study found that people would benefit from reading an overview of an information space upfront, including the criteria others previous…	ML Michael Xieyang Liu et al.Carnegie Mellon University
A Contextual Inquiry of People with Vision Impairments in Cooking Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understan…	FL Franklin Mingzhe Li et al.Carnegie Mellon University
AdRecommended Learn AI Coding at CodeNow Structured lessons, hands-on projects, and continuous updates for people bringing AI into real development work. Explore Now
Facilitating Experiential Training for Counselors using a Real-time Annotation Tool Experiential training, where mental health professionals practice their learned skills, remains the most costly component of therapeutic training. We introduce Pin-MI, a video-call-based tool that supports experiential learning of counseling skills used in motivational interviewing (MI) through interactive role-play a…	TC Tianying Chen et al.Carnegie Mellon University
emoji_events "What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models Code-generating large language models map natural language to code. However, only a small portion of the infinite space of naturalistic utterances is effective at guiding code generation. For non-expert end-user programmers, learning this is the challenge of abstraction matching. We examine this challenge in the speci…	ML Michael Xieyang Liu et al.Carnegie Mellon University
Wigglite: Low-cost Information Collection and Triage Consumers conducting comparison shopping, researchers making sense of competitive space, and developers looking for code snippets online all face the challenge of capturing the information they find for later use without interrupting their current flow. In addition, during many learning and exploration tasks, people n…	ML Michael Xieyang Liu et al.Carnegie Mellon University
Crystalline: Lowering the Cost for Developers to Collect and Organize Information for Decision Making Developers perform online sensemaking on a daily basis, such as researching and choosing libraries and APIs. Prior research has introduced tools that help developers capture information from various sources and organize it into structures useful for subsequent decision-making. However, it remains a laborious process f…	ML Michael Xieyang Liu et al.Carnegie Mellon University
Understanding How Programmers Can Use Annotations on Documentation Modern software development requires developers to find and effectively utilize new APIs and their documentation, but documentation has many well-known issues. Despite this, developers eventually overcome these issues but have no way of sharing what they learned. We investigate sharing this documentation-specific info…	AH Amber Horvath et al.Carnegie Mellon University
emoji_events To Reuse or Not To Reuse? A Framework and System for Evaluating Summarized Knowledge As the amount of information online continues to grow, a correspondingly important opportunity is for individuals to reuse knowledge which has been summarized by others rather than starting from scratch. However, appropriate reuse requires judging the relevance, trustworthiness, and thoroughness of others' knowledge i…	ML Michael Xieyang Liu et al.Carnegie Mellon University	Algorithms and Decision Making
Unakite: Scaffolding Developers’ Decision-Making Using the Web Developers spend a significant portion of their time searching for solutions and methods online. While numerous tools have been developed to support this exploratory process, in many cases the answers to developers’ questions involve trade-offs among multiple valid options and not just a single solution. Through inter…	ML Michael Xieyang Liu et al.Carnegie Mellon University
Popup: Reconstructing 3D Video Using Particle Filtering to Aggregate Crowd Responses Collecting a sufficient amount of 3D training data for autonomous vehicles to handle rare, but critical, traffic events (e.g., collisions) may take decades of deployment. Abundant video data of such events from municipal traffic cameras and video sharing sites (e.g., YouTube) could provide a potential alternative, but…	JS Jean Y Song et al.University of Michigan