MorphingSkin: A Skin-like Platform that Integrates Multimodal Hydraulic Actuators Based on Flexible Electroosmotic PumpsInteractive surfaces have garnered significant attention in Human-Computer Interaction, with fluid-driven actuators being a promising actuation technology due to their flexible form factors and multimodal interactivities. However, traditional fluid-driven systems typically rely on bulky and noisy electromechanical hardware, limiting their portability and practicality. While recent work has introduced compact hydraulic actuators like electroosmotic pumps (EOPs) in haptic devices, their potential for building multifunctional interactive surfaces remains largely unexplored. In this work, we present MorphingSkin, a skin-like platform that integrates multiple, multimodal hydraulic actuators using flexible EOPs as lightweight and self-contained fluidic actuators. We introduce the architecture of MorphingSkin and its versatile design space for multimodal actuation in force, shape, optical, and weight domains. We demonstrate interactive and robotic applications that integrate multiple actuators working collectively within a single MorphingSkin device. Through this work, we envision the future of using MorphingSkin technology for interactive surfaces that integrate flexible form factors and multimodal actuation capabilities.2025TYTianyu Yu et al.Haptic WearablesShape-Changing Interfaces & Soft Robotic MaterialsUIST
Outline and Detail: A Semantic-Driven Framework for Layered 2D Character Generation2D cartoon-style digital characters represent an important art form in games, animation, and virtual live streaming. However, traditional 2D character creation workflows involve tedious manual layering, complex skeleton rigging, and professional animation skills, posing challenges for independent studios and non-professional users. While existing AI generation technologies can quickly create visual content, they typically produce non-layered, difficult-to-edit composite images that cannot be integrated into current workflows. This paper presents Spiritus, a semantic-driven 2D character generation framework. Unlike existing text-based AI animation workflows, Spiritus integrates mixed text and sketch inputs, achieving character image generation and automatic component segmentation through an open mask library and semantic matching. We validated the system's effectiveness in character generation freedom, character animation quality, and technical barrier reduction through comparative evaluation of user experiment results. Finally, we explored the possibilities of applying generated characters to various workflows and scenarios, including game development, animation production, and interactive illustrations.2025QSQirui Sun et al.Music Composition & Sound Design ToolsVideo Production & Editing3D Modeling & AnimationUIST
A Card-based Co-Design Toolkit for Exploring Smart Material Applications with Multiple Stakeholders: A Case Study on Automotive Interior DesignSmart materials have garnered significant attention in both academia and industry, yet identifying pragmatically impactful applications still requires contributions from multiple stakeholders, including researchers, designers, and industry professionals. Although previous research has explored novel technical approaches or user-centered applications of smart materials, this study focuses on how to stimulate effective dialogue among stakeholders to explore impactful smart material applications. To this end, we provided a card-based co-design toolkit, tailored to automotive interior design as a case study. We conducted a multi-stakeholder co-design workshop to examine the performance and reveal the benefits of employing the toolkit in the co-design process. The workshops resulted in 16 concept designs. Qualitative interviews further revealed that, using our toolkit, the co-design process effectively fostered mutual understanding among stakeholders, enhanced both creativity and depth throughout the design process, and provided practical insights for each stakeholder's future work.2025TYTianyu Yu et al.Shape-Changing Interfaces & Soft Robotic MaterialsDIS
Exploring the Design of LLM-based Agent in Enhancing Self-disclosure Among the Older AdultsSocial difficulties have become an increasingly serious issue among older adults. For older adults, regular self-disclosure is essential for maintaining mental health and building close relationships. Leveraging conversational agents to encourage self-disclosure in older adults has shown increasing potential. Understanding how LLM-based agents can influence and stimulate self-disclosure across different topics is crucial for designing future agents tailored to older users. This study introduces Disclosure-Agent, an LLM-based conversational agent, and examines its impact on self-disclosure in older adults through a user study involving 20 participants, 8 topics, and two interactive interfaces equipped with Disclosure-Agent. The findings provide valuable insights into how LLM-based agents can promote self-disclosure in older adults and offer design recommendations for future elderly-oriented conversational agents.2025YGYijie Guo et al.Tsinghua University, Academy of Arts and Design; Tsinghua University, The Future LaboratoryAgent Personality & AnthropomorphismHuman-LLM CollaborationCHI
Mul-O: Encouraging Olfactory Innovation in Various Scenarios Through a Task-Oriented Development PlatformOlfactory interfaces are pivotal in HCI, yet their development is hindered by limited application scenarios, stifling the discovery of new research opportunities. This challenge primarily stems from existing design tools focusing predominantly on odor display devices and the creation of standalone olfactory experiences, rather than enabling rapid adaptation to various contexts and tasks. Addressing this, we introduce Mul-O, a novel task-oriented development platform crafted to aid semi-professionals in navigating the diverse requirements of potential application scenarios and effectively prototyping ideas. Mul-O facilitates the swift association and integration of olfactory experiences into functional designs, system integrations, and concept validations. Comprising a web UI for task-oriented development, an API server for seamless third-party integration, and wireless olfactory display hardware, Mul-O significantly enhances the ideation and prototyping process in multisensory tasks. This was verified by a 15-day workshop attended by 30 participants. The workshop produced seven innovative projects, underscoring Mul-O's efficacy in fostering olfactory innovation.2024PGPeizhong Gao et al.Haptic WearablesShape-Changing Interfaces & Soft Robotic MaterialsContext-Aware ComputingUIST
Lumina: A Software Tool for Fostering Creativity in Designing Chinese Shadow PuppetsShadow puppetry, a culturally rich storytelling art, faces challenges transitioning to the digital realm. Creators in the early design phase struggle with crafting intricate patterns, textures, and basic animations while adhering to stylistic conventions - hindering creativity, especially for novices. This paper presents Lumina, a tool to facilitate the early Chinese shadow puppet design stage. Lumina provides contour templates, animations, scene editing tools, and machine-generated traditional puppet patterns. These features liberate creators from tedious tasks, allowing focus on the creative process. Developed based on a formative study with puppet creators, the web-based Lumina enables wide dissemination. An evaluation with 18 participants demonstrated Lumina's effectiveness and ease of use, with participants successfully creating designs spanning traditional themes to contemporary and science-fiction concepts.2024ZYZhihao Yao et al.Museum & Cultural Heritage DigitizationInteractive Narrative & Immersive StorytellingUIST
InkBrush: A Sketching Tool for 3D Ink PaintingInkBrush is a new sketch-based 3D drawing tool for creating 3D ink paintings using free-form 3D ink strokes. It offers a digital calligraphy brush and various editing tools to generate realistic ink-like brush strokes with attributes like hairy edges, ink drips, and scattered dots. Users can adjust parameters such as moisture, color, darkness, dryness, and stroke style to customize the appearance of the brush strokes. The development of InkBrush was guided by a design study involving artists and designers. It was developed as a plugin for Blender, a popular 3D modeling tool, and its effectiveness and usability were evaluated through a user study involving 75 participants. Preliminary feedback from the participants was overwhelmingly positive, indicating that InkBrush was intuitive and easy to use. Following this, we also sought in-depth assessments from experts in ink painting and 3D design. Their evaluations further demonstrated the effectiveness of InkBrush.2024ZYZhihao Yao et al.Tsinghua University3D Modeling & AnimationCustomizable & Personalized ObjectsCHI
Co-designing the Collaborative Digital Musical Instruments for Group Music TherapyDigital Musical Instruments (DMIs) have been integrated into group music therapy, providing therapists with alternative ways to engage in musical dialogues with their clients. However, existing DMIs used in group settings are primarily designed for individual use and often overlook the social dynamics inherent in group therapy. Recognizing the crucial role of social interaction in the effectiveness of group therapy, we argue that Collaborative Digital Musical Instruments (CDMIs), seamlessly integrating social interaction with musical expression, hold significant potential to enhance group music therapy. To better tailor CDMIs for group music therapy, we engaged in a co-design process with music therapists, designing and practicing group therapy sessions involving the prototype ComString. In the end, we reflected on the co-design case to suggest future directions for designing CDMIs in group music therapy.2024YFYuan-Ling Feng et al.Tsinghua UniversityVibrotactile Feedback & Skin StimulationConversational ChatbotsMusic Composition & Sound Design ToolsCHI
Sustainflatable: Harvesting, Storing and Utilizing Ambient Energy for Pneumatic Morphing InterfacesWhile the majority of pneumatic interfaces are powered and controlled by traditional electric pumps and valves, alternative sustainable energy-harnessing technology has been attracting attention. This paper presents a novel solution to this challenge with the development of the Sustainflatable system, a self-sustaining pneumatic system that can harvest renewable energy sources such as wind, water flow, moisture, and sunlight, convert the energy into compressed air, and store it for later use in a programmable and intelligent way. The system is completely electronic-free, incorporating customized energy harvesting pumps, storage units with variable volume-pressure characteristics, and tailored valves that operate autonomously. Additionally, the paper provides a design tool to guide the development of the system and includes several environmental applications to showcase its capabilities.2023QLQiuyu Lu et al.Shape-Changing Interfaces & Soft Robotic MaterialsEcological Design & Green ComputingUIST
Thermotion: Design and fabrication of thermofluidic composites for animation effects on object surfacesWe introduce Thermotion, a novel method using thermofluidic composites to design and display thermochromic animation effects on object surfaces. With fluidic channels embedded under the object surfaces, the composites utilize thermofluidic flows to dynamically control the surface temperature as an actuator for thermochromic paints, which enables researchers and designers for the first time to create animations not only on two and three-dimensional surfaces but also on the surface made of a few flexible everyday materials. We report the design space with six animation primitives and two modification effects, and we demonstrate the design and fabrication workflow with a customized software platform for design and simulation. A range of applications is shown leveraging the objects' dynamic displays both visually and thermally, including dynamic artifacts, teaching aids, and ambient displays. We envision an opportunity to extend thermofluidic composites to other heat-related practices for further dynamic and programmable interactions with temperature.2023TYTianyu Yu et al.Tsinghua UniversityShape-Changing Interfaces & Soft Robotic MaterialsShape-Changing Materials & 4D PrintingCHI
Digital Making for Inheritance and Enlivening Intangible Cultural Heritage: A Case of Hairy Monkey HandicraftsDigital technologies can conduct an important role in preserving intangible cultural heritage (ICH). Nonetheless, existing work tends to be limited to digital storage, presentation, dissemination, and education, with comparatively little concentrating on production and reproduction of the craft, the key to revitalizing ICH. In this paper, we explore digital making as an approach for both the inheritance and innovation of ICH handicrafts. Taking Hairy Monkey craftsmanship as an instance, we conducted a workshop with 15 groups of makers, teaching them the traditional Hairy Monkey craft and subsequently enabling them to create innovative works with digital technologies in their own time. As revealed by our interviews with these participants, ICH brings cultural inspiration to digital making, and digital making rejuvenates ICH through innovative art forms and its positive influence on participants. As demonstrated in this paper, digital technologies can be deeply integrated with ICH through making to revitalize ICH from the core through living transmission.2023GLGuanhong Liu et al.Tsinghua UniversityDesktop 3D Printing & Personal FabricationMuseum & Cultural Heritage DigitizationCHI
LaserShoes: Low-Cost Ground Surface Detection Using Laser Speckle ImagingGround surfaces are often carefully designed and engineered with various textures to fit the functionalities of human environments and thus could contain rich context information for smart wearables. Ground surface detection could power a wide array of applications including activity recognition, mobile health, and context-aware computing, and potentially provide an additional channel of information for many existing kinesiology approaches such as gait analysis. To facilitate the detection of ground surfaces, we present LaserShoes, a texture-sensing-enabled system using laser speckle imaging that can be retrofitted to shoes. Our system captures videos of speckle patterns induced on ground surfaces and uses pre-processing to identify ideal images with clear speckle patterns collected when users' feet are in contact with ground surfaces. We demonstrated our technique with a ResNet-18 model and achieved real-time inference. We conducted an evaluation in different conditions and demonstrated results that verified the feasibility.2023ZYZihan Yan et al.MIT Media LabFull-Body Interaction & Embodied InputBiosensors & Physiological MonitoringContext-Aware ComputingCHI
Reviewing and Reflecting Smart Home Research from the Human-Centered PerspectiveWhile there has been rapid growth in smart home research from a technical perspective – focusing on home automation, devices, software, and protocols – few review papers examine the human-centered perspective. A human-centered focus is crucial for achieving the goals of providing natural, convenient, comfortable, friendly, and safe user experiences in the smart home. To understand key innovations in human-centered smart home research, we analyzed keyword changes over time via 19,091 papers from 2000 to 2022, then selected 55 papers from high-impact venues in the last five years, and summarized them through a combination of qualitative and quantitative methods. Our analysis revealed five research trends with unique characteristics and interdependence. Drawing on this review, we elaborate on the future of smart home design research with respect to multidisciplinary development, stakeholder involvement, and the shift of design implications.2023YYYuan Yao et al.School of Architecture and Design, Academy of Arts & DesignUniversal & Inclusive DesignSmart Home Interaction DesignCHI
Establishing Design Consensus toward Next-Generation Retail: Data-Enabled Design Exploration and Participatory AnalysisIntegration of online and offline retail faces challenges in technology adoption, interaction style evolution and customer behavior shifts, while also being complicated by diverse perspectives from different stakeholders of consumers, retail staff, and retail business unit people. To explore how we can tackle the aforementioned challenges, this work applied the data-enabled design method and participatory data analysis to a case study, where 400 student consumers' shopping behavior data was collected, cross-analyzed, and visualized in a campus chain store. We then invited 13 stakeholders to join a co-creation workshop for a further participatory data analysis. In the workshop, the different stakeholders came to a design consensuses which we summarized into a series of practical design recommendations for improving the current store. Finally, we generalized the case study process as a contextual-informed-aware model, which can contribute to professional design practice for the retail industry.2022YYYuan Yao et al.Tsinghua UniversityInteractive Data VisualizationParticipatory DesignPrototyping & User TestingCHI
"It Puts Life into My Creations": Understanding Fluid Fiber as a Media for Expressive DisplayFluid fiber, with fluid flowing in a tube, is an attractive material to flexibly and dynamically display digital information. While it has recently attracted attention from the HCI field, there is currently little knowledge about this material, limited to controlling the position of the droplets to present representational information like letters and numbers. To develop a broader and deepened understanding of this material and its potential for display design, we conducted a study based on a design workshop where art and design practitioners engaged in creation practice with a toolkit we designed and developed. The toolkit includes hardware components for controlling bubbles and droplets and a GUI design tool for arranging the fluid layout. Our research reveals the structural and expressive affordance of such a fluid fiber for displaying information, highlighting the unique value of fluidity as an intuitive form to express life, emotion, movement and changes.2022GLGuanhong Liu et al.Tsinghua UniversityShape-Changing Interfaces & Soft Robotic MaterialsDigital Art Installations & Interactive PerformanceCHI
Tactile Compass: Enabling Visually Impaired People to Follow a Path with Continuous Directional FeedbackAccurate and effective directional feedback is crucial for an electronic traveling aid device that guides visually impaired people in walking through paths. This paper presents Tactile Compass, a hand-held device that provides continuous directional feedback with a rotatable needle pointing toward the planned direction. We conducted two lab studies to evaluate the effectiveness of the feedback solution. Results showed that using Tactile Compass, participants could reach the target direction in place with a mean deviation of 3.03° and could smoothly navigate along paths of 60cm width, with a mean deviation from the centerline of 12.1cm. Subjective feedback showed that Tactile Compass was easy to learn and use.2021GLGuanhong Liu et al.Tsinghua UniversityVibrotactile Feedback & Skin StimulationHaptic WearablesVisual Impairment Technologies (Screen Readers, Tactile Graphics, Braille)CHI
Exploring Potential Scenarios and Design Implications Through A Camera-like Physical Odor Capture PrototypeRecently, researchers have become increasingly interested in finding new input methods for olfactory interfaces. Physical odor capture is a potential solution to this issue and, in order to make it more accessible for users, we designed a portable and fast smell capture prototype based on headspace technology and inspired by point-and-shoot cameras. We conducted a two-week diary study with 13 participants, in which they were allowed to freely use the prototype for odor capture activities. Through diary and interview feedback, we summarized factors such as the motivations of capturing, the collected odor types, and perceptual effects of odor replay. We found that the capture activities can positively affect user emotions, memory, or perception. User preferences on device parameters were also gathered to guide further design iterations. Physical odor capture has many potential applications in daily-life and other implications resulting from the study have been proposed for further research.2020QLQi Lu et al.Shape-Changing Interfaces & Soft Robotic MaterialsUbiquitous ComputingDIS
Chinese CHI 2018 WorkshopThe extended abstract describes the background, goals and organization of the sixth International Workshop of Chinese CHI (Chinese CHI 2018).2018ZFZhiyong Fu et al.Tsinghua UniversityParticipatory DesignUser Research Methods (Interviews, Surveys, Observation)CHI