Understanding the Challenges of Maker EntrepreneurshipThe maker movement embodies a resurgence in DIY creation, merging physical craftsmanship and arts with digital technology support. However, mere technological skills and creativity are insufficient for economically and psychologically sustainable practice. By illuminating and smoothing the path from "maker" to "maker entrepreneur," we can help broaden the viability of making as a livelihood. Our research centers on makers who design, produce, and sell physical goods. In this work, we explore the transition to entrepreneurship for these makers and how technology can facilitate this transition online and offline. We present results from interviews with 20 USA-based maker entrepreneurs {(i.e., lamps, stickers)}, six creative service entrepreneurs {(i.e., photographers, fabrication)}, and seven support personnel (i.e., art curator, incubator director). Our findings reveal that many maker entrepreneurs 1) are makers first and entrepreneurs second; 2) struggle with business logistics and learn business skills as they go; and 3) are motivated by non-monetary values. We discuss training and technology-based design implications and opportunities for addressing challenges in developing economically sustainable businesses around making.2025NFNatalie Friedman et al.Content Creation & CreatorsCSCW
Punctuated and Prolonged: A Workers’ Inquiry into Infrastructural Failures in Bus TransitIn North America, bus operators are essential but undervalued public servants — the “human infrastructure” of public transit. For decades, the working conditions they face have garnered too little attention or public concern. This is changing. Hazardous working conditions have disrupted transit services, and drivers themselves have called for change with increasing urgency. In a moment of increasing technological disruption in the transit sector, we argue that CSCW can contribute to further contesting such conditions through the framework of “infrastructural inversion.” In this paper, we lay the groundwork for this kind of intervention through a diary study of bus operators’ working conditions. We detail how punctuated moments of workplace violence, inhumane scheduling, and unsafe operational conditions become prolonged infrastructural failure. We outline how CSCW researchers and practitioners can contribute to the design of transit systems that enhance worker dignity and contribute to ongoing efforts to address urgent health and safety concerns.2025HAHunter Akridge et al.Infrastructure StudiesCSCW
Non-Emergency Notification Timing for Drivers Doing Non-Driving-Related Tasks in Autonomous Vehicles: An Interruptibility StudyFuture high-level autonomous vehicles (AVs) will enable drivers to engage in non-driving-related tasks (NDRTs) during autopilot. Occasionally, an in-vehicle agent may need to notify drivers of important, yet not urgent, information. Through a four-session interruptibility study on a desktop autonomous driving simulator, we investigated how drivers assess their availability to receive notifications by rating moments as good or bad for interruption. Our results suggest drivers fall into four notification availability groups: always available, prioritizing NDRTs, task-content dependent, and mental-state dependent. Using multimodal behavioral data of the participants and vehicle data from the simulation, we trained a proof-of-concept classification model to determine the appropriate timing to send non-emergency notifications to drivers doing NDRTs. Head pose and gaze direction data from the eye tracker were crucial in the predictions. Based on our quantitative modeling and qualitative observation, we discuss the feasibility of notification timing prediction in the real world and design considerations from individual, task, and context perspectives.2025HWHongyu Howie Wang et al.In-Vehicle Haptic, Audio & Multimodal FeedbackEye Tracking & Gaze InteractionNotification & Interruption ManagementAutoUI
Exploring the Potential of Metacognitive Support Agents for Human-AI Co-CreationDespite the potential of generative AI (GenAI) design tools to enhance design processes, professionals often struggle to integrate AI into their workflows. Fundamental cognitive challenges include the need to specify all design criteria as distinct parameters upfront (intent formulation) and designers’ reduced cognitive involvement in the design process due to cognitive offloading, which can lead to insufficient problem exploration, underspecification, and limited ability to evaluate outcomes. Motivated by these challenges, we envision novel metacognitive support agents that assist designers in working more reflectively with GenAI. To explore this vision, we conducted exploratory prototyping through a Wizard of Oz elicitation study with 20 mechanical designers probing multiple metacognitive support strategies. We found that agent-supported users created more feasible designs than non-supported users, with differing impacts between support strategies. Based on these findings, we discuss opportunities and tradeoffs of metacognitive support agents and considerations for future AI-based design tools.2025FGFrederic Gmeiner et al.Generative AI (Text, Image, Music, Video)Human-LLM CollaborationCreative Collaboration & Feedback SystemsDIS
At the Breaking Point: How Bus Operators Cope with Transit Technology Failures and What That Can Tell Us About the Integration of Future InnovationsThis paper examines the cascading effects of technical failures in transit, focusing on the challenges faced by bus operators when communication, passenger-facing, and mechanical technologies fail. Through a diary study, we gather operator accounts of critical tools like radios, mobile data terminals (MDTs), payment systems, and ramps, alongside their failures. Issues like radio outages, GPS malfunctions, and broken fare systems lead to operational delays, safety risks, and increased stress. Triaging breakdowns becomes crucial to operations and drivers adapt by using personal phones in emergencies, highlighting gaps in system integration. As transit electrifies its fleets and considers a wider range of innovations, these failures offer key insights into the challenges ahead, emphasizing the need for robust, adaptable systems that ensure operational continuity and protect worker well-being amid rapid technological change.2025ATAlice Xiaodi Tang et al.Head-Up Display (HUD) & Advanced Driver Assistance Systems (ADAS)Context-Aware ComputingUbiquitous ComputingDIS
Intent Tagging: Exploring Micro-Prompting Interactions for Supporting Granular Human-GenAI Co-Creation WorkflowsDespite Generative AI (GenAI) systems' potential for enhancing content creation, users often struggle to effectively integrate GenAI into their creative workflows. Core challenges include misalignment of AI-generated content with user intentions (intent elicitation and alignment), user uncertainty around how to best communicate their intents to the AI system (prompt formulation), and insufficient flexibility of AI systems to support diverse creative workflows (workflow flexibility). Motivated by these challenges, we created IntentTagger: a system for slide creation based on the notion of Intent Tags—small, atomic conceptual units that encapsulate user intent—for exploring granular and non-linear micro-prompting interactions for Human-GenAI co-creation workflows. Our user study with 12 participants provides insights into the value of flexibly expressing intent across varying levels of ambiguity, meta-intent elicitation, and the benefits and challenges of intent tag-driven workflows. We conclude by discussing the broader implications of our findings and design considerations for GenAI-supported content creation workflows.2025FGFrederic Gmeiner et al.Carnegie Mellon University, Human-Computer Interaction Institute; Microsoft ResearchGenerative AI (Text, Image, Music, Video)AI-Assisted Creative WritingCreative Collaboration & Feedback SystemsCHI
BioSpark: Beyond Analogical Inspiration to LLM-augmented TransferWe present BioSpark, a system for analogical innovation designed to act as a creativity partner in reducing the cognitive effort in finding, mapping, and creatively adapting diverse inspirations. While prior approaches have focused on initial stages of finding inspirations, BioSpark uses LLMs embedded in a familiar, visual, Pinterest-like interface to go beyond inspiration to supporting users in identifying the key solution mechanisms, transferring them to the problem domain, considering tradeoffs, and elaborating on details and characteristics. To accomplish this BioSpark introduces several novel contributions, including a tree-of-life enabled approach for generating relevant and diverse inspirations, as well as AI-powered cards including 'Sparks' for analogical transfer; 'Trade-offs' for considering pros and cons; and 'Q&A' for deeper elaboration. We evaluated BioSpark through workshops with professional designers and a controlled user study, finding that using BioSpark led to a greater number of generated ideas; those ideas being rated higher in creative quality; and more diversity in terms of biological inspirations used than a control condition. Our results suggest new avenues for creativity support tools embedding AI in familiar interaction paradigms for designer workflows.2025HKHyeonsu B Kang et al.Carnegie Mellon University, Human-Computer Interaction InstituteHuman-LLM CollaborationCreative Collaboration & Feedback SystemsCHI
CompAct: Designing Interconnected Compliant Mechanisms with Targeted Actuation TransmissionsCompliant mechanisms enable the creation of compact and easy-to-fabricate devices for tangible interaction. This work explores interconnected compliant mechanisms consisting of multiple joints and rigid bodies to transmit and process displacements as signals that result from physical interactions. As these devices are difficult to design due to their vast and complex design space, we developed a graph-based design algorithm and computational tool to help users program and customize such computational functions and procedurally model physical designs. When combined with active materials with actuation and sensing capabilities, these devices can also render and detect haptic interaction. Our design examples demonstrate the tool’s capability to respond to relevant HCI concepts, including building modular physical interface toolkits, encrypting tangible interactions, and customizing user augmentation for accessibility. We believe the tool will facilitate the generation of new interfaces with enriched affordance.2025HYHumphrey Yang et al.Carnegie Mellon University, Human-Computer Interaction InstituteShape-Changing Interfaces & Soft Robotic MaterialsCustomizable & Personalized ObjectsCHI
Inkspire: Supporting Design Exploration with Generative AI through Analogical SketchingWith recent advancements in the capabilities of Text-to-Image (T2I) AI models, product designers have begun experimenting with them in their work. However, T2I models struggle to interpret abstract language and the current user experience of T2I tools can induce design fixation rather than a more iterative, exploratory process. To address these challenges, we developed Inkspire, a sketch-driven tool that supports designers in prototyping product design concepts with analogical inspirations and a complete sketch-to-design-to-sketch feedback loop. To inform the design of Inkspire, we conducted an exchange session with designers and distilled design goals for improving T2I interactions. In a within-subjects study comparing Inkspire to ControlNet, we found that Inkspire supported designers with more inspiration and exploration of design ideas, and improved aspects of the co-creative process by allowing designers to effectively grasp the current state of the AI to guide it towards novel design intentions.2025DLDavid Chuan-En Lin et al.Carnegie Mellon University, Human-Computer Interaction InstituteGenerative AI (Text, Image, Music, Video)Creative Collaboration & Feedback SystemsCustomizable & Personalized ObjectsCHI
Power and Play: Investigating “License to Critique” in Teams’ AI Ethics DiscussionsPast work has sought to design AI ethics interventions–such as checklists or toolkits–to help practitioners design more ethical AI systems. However, other work demonstrates how these interventions may instead serve to limit critique to that addressed within the intervention, while rendering broader concerns illegitimate. In this paper, drawing on work examining how standards enact discursive closure and how power relations affect whether and how people raise critique, we recruit three corporate teams, and one activist team, each with prior context working with one another, to play a game designed to trigger broad discussion around AI ethics. We use this as a point of contrast to trigger reflection on their teams’ past discussions, examining factors which may affect their “license to critique” in AI ethics discussions. We then report on how particular affordances of this game may influence discussion, and find that the hypothetical context created in the game is unlikely to be a viable mechanism for real world change. We discuss how power dynamics within a group and notions of “scope” affect whether people may be willing to raise critique in AI ethics discussions, and discuss our finding that games are unlikely to enable direct changes to products or practice, but may be more likely to allow members to find critically-aligned allies for future collective action.2024DWDavid Gray Widder et al.Session 4e: Navigating AI Ethical ChallengesCSCW
Videogenic: Identifying Highlight Moments in Videos with Professional Photographs as a PriorThis paper investigates the challenge of extracting highlight moments from videos. To perform this task, we need to understand what constitutes a highlight for arbitrary video domains while at the same time being able to scale across different domains. Our key insight is that photographs taken by photographers tend to capture the most remarkable or photogenic moments of an activity. Drawing on this insight, we present Videogenic, a technique capable of creating domain-specific highlight videos for a diverse range of domains. In a human evaluation study (N=50), we show that a high-quality photograph collection combined with CLIP-based retrieval (which uses a neural network with semantic knowledge of images) can serve as an excellent prior for finding video highlights. In a within-subjects expert study (N=12), we demonstrate the usefulness of Videogenic in helping video editors create highlight videos with lighter workload, shorter task completion time, and better usability.2024DLDavid Chuan-En Lin et al.Generative AI (Text, Image, Music, Video)Video Production & EditingC&C
VideoMap: Supporting Video Exploration, Brainstorming, and Prototyping in the Latent SpaceVideo editing is a creative and complex endeavor and we believe that there is potential for reimagining a new video editing interface to better support the creative and exploratory nature of video editing. We take inspiration from latent space exploration tools that help users find patterns and connections within complex datasets. We present VideoMap, a proof-of-concept video editing interface that operates on video frames projected onto a latent space. We support intuitive navigation through map-inspired navigational elements and facilitate transitioning between different latent spaces through swappable lenses. We built three VideoMap components to support editors in three common video tasks. In a user study with both professionals and non-professionals, editors found that VideoMap helps reduce grunt work, offers a user-friendly experience, provides an inspirational way of editing, and effectively supports the exploratory nature of video editing. We further demonstrate the versatility of VideoMap by implementing three extended applications.2024DLDavid Chuan-En Lin et al.Interactive Data VisualizationVideo Production & EditingC&C
“The bus is nothing without us”: Making Visible the Labor of Bus Operators amid the Ongoing Push Towards Transit AutomationThis paper describes how the complexity of circumstances bus operators manage presents unique challenges to the feasibility of high-level automation in public transit. Avoiding an overly rationalized view of bus operators' labor is critical to ensure the introduction of automation technologies does not compromise public wellbeing, the dignity of transit workers, or the integrity of critical public infrastructure. Our findings from a group interview study show that bus operators take on work — undervalued by those advancing automation technologies — to ensure the well-being of passengers and community members. Notably, bus operators are positioned to function as shock absorbers during social crises in their communities and in moments of technological breakdown as new systems come on board. These roles present a critical argument against the rapid push toward driverless automation in public transit. We conclude by identifying opportunities for participatory design and collaborative human-machine teaming for a more just future of transit.2024HAHunter Akridge et al.Carnegie Mellon UniversityActivism & Political ParticipationImpact of Automation on WorkParticipatory DesignCHI
Jigsaw: Supporting Designers to Prototype Multimodal Applications by Chaining AI Foundation ModelsRecent advancements in AI foundation models have made it possible for them to be utilized off-the-shelf for creative tasks, including ideating design concepts or generating visual prototypes. However, integrating these models into the creative process can be challenging as they often exist as standalone applications tailored to specific tasks. To address this challenge, we introduce Jigsaw, a prototype system that employs puzzle pieces as metaphors to represent foundation models. Jigsaw allows designers to combine different foundation model capabilities across various modalities by assembling compatible puzzle pieces. To inform the design of Jigsaw, we interviewed ten designers and distilled design goals. In a user study, we showed that Jigsaw enhanced designers' understanding of available foundation model capabilities, provided guidance on combining capabilities across different modalities and tasks, and served as a canvas to support design exploration, prototyping, and documentation.2024DLDavid Chuan-En Lin et al.Carnegie Mellon UniversityGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationPrototyping & User TestingCHI
Co-design Accessible Public Robots: Insights from People with Mobility Disability, Robotic Practitioners and Their CollaborationsSidewalk robots are increasingly common across the globe. Yet, their operation on public paths poses challenges for people with mobility disabilities (PwMD) who face barriers to accessibility, such as insufficient curb cuts. We interviewed 15 PwMD to understand how they perceive sidewalk robots. Findings indicated that PwMD feel they have to compete for space on the sidewalk when robots are introduced. We next interviewed eight robotics practitioners to learn about their attitudes towards accessibility. Practitioners described how issues often stem from robotic companies addressing accessibility only after problems arise. Both interview groups underscored the importance of integrating accessibility from the outset. Building on this finding, we held four co-design workshops with PwMD and practitioners in pairs. These convenings brought to bear accessibility needs around robots operating in public spaces and in the public interest. Our study aims to set the stage for a more inclusive future around public service robots.2024HHJoel Chan et al.Carnegie Mellon UniversityInclusive DesignEmpowerment of Marginalized GroupsCHI
Soundify: Matching Sound Effects to VideoIn the art of video editing, sound helps add character to an object and immerse the viewer within a space. Through formative interviews with professional editors (N=10), we found that the task of adding sounds to video can be challenging. This paper presents Soundify, a system that assists editors in matching sounds to video. Given a video, Soundify identifies matching sounds, synchronizes the sounds to the video, and dynamically adjusts panning and volume to create spatial audio. In a human evaluation study (N=889), we show that Soundify is capable of matching sounds to video out-of-the-box for a diverse range of audio categories. In a within-subjects expert study (N=12), we demonstrate the usefulness of Soundify in helping video editors match sounds to video with lighter workload, reduced task completion time, and improved usability.2023DLJessica Lin et al.Music Composition & Sound Design ToolsVideo Production & EditingCreative Collaboration & Feedback SystemsUIST
Exploring Challenges and Opportunities to Support Designers in Learning to Co-create with AI-based Manufacturing Design ToolsAI-based design tools are proliferating in professional software to assist engineering and industrial designers in complex manufacturing and design tasks. These tools take on more agentic roles than traditional computer-aided design tools and are often portrayed as “co-creators.” Yet, working effectively with such systems requires different skills than working with complex CAD tools alone. To date, we know little about how engineering designers learn to work with AI-based design tools. In this study, we observed trained designers as they learned to work with two AI-based tools on a realistic design task. We find that designers face many challenges in learning to effectively co-create with current systems, including challenges in understanding and adjusting AI outputs and in communicating their design goals. Based on our findings, we highlight several design opportunities to better support designer-AI co-creation.2023FGFrederic Gmeiner et al.Carnegie Mellon UniversityGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationCHI
Leveraging the Twitch Platform and Gamification to Generate Home Audio DatasetsTraining AI systems requires large datasets. While there are a range of existing methods for collecting such data, such as paid work on crowdsourcing platforms, the strengths and weaknesses of each method leads us to believe that new, complementary methods are needed. The Polyphonic project contributes a novel method for collecting real-world data by piggybacking on game streaming communities such as Twitch, which capture over a trillion minutes of viewer attention a year. By embedding activities within the sociotechnical context of the stream, we can leverage some of this attention for data collection and processing. In this paper, we describe the design and implementation of a proof-of-concept system for collecting home audio data. We conducted a field study in four live streams and found that our proof-of-concept effectively supports data capture. We also contribute further design insights about stream-based data collection systems.2021NMNikolas Martelaro et al.Live Streaming & Spectating ExperienceOpen-Source Collaboration & Code ReviewCitizen Science & Crowdsourced DataDIS
Learning Personal Style from Few ExamplesA key task in design work is grasping the client's implicit tastes. Designers often do this based on a set of examples from the client. However, recognizing a common pattern among many intertwining variables such as color, texture, and layout and synthesizing them into a composite preference can be challenging. In this paper, we leverage the pattern recognition capability of computational models to aid in this task. We offer a set of principles for computationally learning personal style. The principles are manifested in PseudoClient, a deep learning framework that learns a computational model for personal graphic design style from only a handful of examples. In several experiments, we found that PseudoClient achieves a 79.40% accuracy with only five positive and negative examples, outperforming several alternative methods. Finally, we discuss how PseudoClient can be utilized as a building block to support the development of future design applications.2021DLDavid Chuan-En Lin et al.Generative AI (Text, Image, Music, Video)Graphic Design & Typography ToolsDIS
Using Remote Controlled Speech Agents to Explore Music Experience in ContextIt can be difficult for user researchers to explore how people might interact with interactive systems in everyday contexts; time and space limitations make it hard to be present everywhere that technology is used. Digital music services are one domain where designing for context is important given the myriad places people listen to music. One novel method to help design researchers embed themselves in everyday contexts is through remote-controlled speech agents. This paper describes a practitioner-centered case study of music service interaction researchers using a remote-controlled speech agent, called DJ Bot, to explore people’s music interaction in the car and the home. DJ Bot allowed the team to conduct remote user research and contextual inquiry and to quickly explore new interactions. However, challenges using a remote speech-agent arose when adapting DJ Bot from the constrained environment of the car to the unconstrained home environment.2020NMNikolas Martelaro et al.Voice User Interface (VUI) DesignIntelligent Voice Assistants (Alexa, Siri, etc.)Context-Aware ComputingDIS