Füpop: "Real Food" Flavor Delivery via Focused UltrasoundFood and flavors are integral to our existence in the world. Nonetheless, taste remains an under-explored sense in interaction design. We present Füpop, a technical platform for delivering in-mouth flavors that leverages advances in electronics and molecular gastronomy. Füpop comprises a fully edible pouch placed inside the mouth against a cheek that programmatically releases different flavors when wirelessly triggered by a focused ultrasound transducer from outside the cheek. Füpop does not interfere with activities such as chewing and drinking, and its electronics may be integrated into devices already used near the cheek, such as mobile phones, audio headphones, and head-mounted displays. Füpop's flavors are from "real foods," not ones imitated with synthetic reagents, providing authentic, nutritive flavors. We envision that with Füpop, flavors may be synced to music, a phone call, or events in virtual reality to enhance a user's experience of their food and the world.2024KSKatherine W Song et al.UC BerkeleyHaptic WearablesFood Culture & Food InteractionCHI
Vɪᴍ: Customizable, Decomposable Electrical Energy StorageProviding electrical power is essential for nearly all interactive technologies, yet it often remains an afterthought. Some designs handwave power altogether as an "exercise for later." Others hastily string together batteries to meet the system's electrical requirements, enclosing them in whatever box fits. Vɪᴍ is a new approach -- it elevates power as a first-class design element; it frees power from being a series of discrete elements, instead catering to exact requirements; it enables power to take on new, flexible forms; it is fabricated using low-cost, accessible materials and technologies; finally, it advances sustainability by being rechargeable, non-toxic, edible, and compostable. Vɪᴍs are decomposable battery alternatives that rapidly charge and can power small applications for hours. We present Vɪᴍs, detail their characteristics, offer design guidelines for their fabrication, and explore their use in applications spanning prototyping, fashion, and food, including novel systems that are entirely decomposable and edible.2023KSKatherine W Song et al.UC BerkeleyShape-Changing Materials & 4D PrintingSustainable HCIEcological Design & Green ComputingCHI
Lotio: Lotion-Mediated Interaction with an Electronic Skin-Worn DisplaySkin-based electronics are an emerging genre of interactive technologies. In this paper, we leverage the natural uses of lotions and propose them as mediators for driving novel, low-power, quasi-bistable, and bio-degradable electrochromic displays on the skin and other surfaces. We detail the design, fabrication, and evaluation of one such "Lotion Interface," including how it can be customized using low-cost everyday materials and technologies to trigger various visual and temporal effects – some lasting up to fifteen minutes when unpowered. We characterize different fabrication techniques and lotions to demonstrate various visual effects on a variety of skin types and tones. We highlight the safety of our design for humans and the environment. Finally, we report findings from an exploratory user study and present a range of compelling applications for Lotion Interfaces that expand the on-skin and surface interaction landscapes to include the familiar and often habitual practice of applying lotion.2023KSKatherine W Song et al.UC BerkeleyOn-Skin Display & On-Skin InputCHI
Kaleidoscope: A Reflective Documentation Tool for a User Interface Design CourseDocumentation can support design work and create opportunities for learning and reflection. We explore how a novel documentation tool for a remote interaction design course provides insight into design process and integrates strategies from expert practice to support studio-style collaboration and reflection. Using Research through Design, we develop and deploy Kaleidoscope, an online tool for documenting design process, in an upper-level HCI class during the COVID-19 pandemic, iteratively developing it in response to student feedback and needs. We discuss key themes from the real-world deployment of Kaleidoscope, including: tensions between documentation and creation; effects of centralizing discussion; privacy and visibility in shared spaces; balancing evidence of achievement with feelings of overwhelm; and the effects of initial perceptions and incentives on tool usage. These successes and challenges provide insights to guide future tools for design documentation and HCI education that scaffold learning process as an equal partner to execution.2023SSSarah Sterman et al.UC BerkeleyUser Research Methods (Interviews, Surveys, Observation)Prototyping & User TestingCHI
Understanding Version Control as Material Interaction with QuickposeWhether a programmer with code or a potter with clay, practitioners engage in an ongoing process of working and reasoning with materials. Existing discussions in HCI have provided rich accounts of these practices and processes, which we synthesize into three themes: (1) reciprocal discovery of goals and materials, (2) local knowledge of materials, and (3) annotation for holistic interpretation. We then apply these design principles generatively to the domain of version control to present Quickpose: a version control system for creative coding. In an in-situ, longitudinal study of Quickpose guided by our themes, we collected usage data, version history, and interviews. Our study explored our participants’ material interaction behaviors and the initial promise of our proposed measures for recognizing these behaviors. Quickpose is an exploration of version control as material interaction, using existing discussions to inform domain-specific concepts, measures, and designs for version control systems.2023EREric Rawn et al.UC BerkeleyCognitive Impairment & Neurodiversity (Autism, ADHD, Dyslexia)Knowledge Worker Tools & WorkflowsPrototyping & User TestingCHI
Towards Creative Version ControlVersion control systems are powerful tools for managing history information and shaping personal and collaborative processes. While many complex tools exist for software engineering, and basic functionality for capturing versions is often found in collaborative applications such as text editors and design layout tools, these systems are not attuned to the needs and behaviors of creative practitioners within those domains, and fail to support creative practitioners in many others. Through 18 semi-structured interviews across diverse domains of creativity, we investigate how creative practitioners use version histories in their process. With the familiar paradigms and features of software version control as an organizing structure, we discuss how these creative practitioners embrace, challenge, and complicate uses of version histories in four ways: using versions as a palette of materials, providing confidence and freedom to explore, leveraging low-fidelity version capture, and reflecting on and reusing versions across long time scales. We discuss how the themes present across this wide range of mediums and domains can provide insight into future designs and uses of version control systems to support creative process.2022SSSarah Sterman et al.Software Development; Software DevelopmentCSCW
Creative and Motivational Strategies Used by Expert Creative PractitionersCreative practice often requires persevering through moments of ambiguity, where the outcome of a process is unclear. Creative practitioners intentionally manage this process, for example by developing strategies to break out of creative ruts, or stay motivated through uncertainty. Understanding the way experts engage with and manage these creativity-relevant processes represents a rich source of foundational knowledge for designers of Creativity Support Tools. These strategies represent an opportunity for CST research: to create CSTs that embody emotional and process-focused strategies and techniques. Through interviews with expert practitioners in diverse domains including performance, craft, engineering, and design, we identify four strategies for managing process: Strategic Forgetting, Mode Switching, Embodying Process, and Aestheticizing. Understanding tool- and domain-agnostic creative strategies used by experts to manage their own creative process can inform the design of future CSTs that amplify the benefits of successful strategies and scaffold new techniques.2022MNMolly Jane Nicholas et al.Graphic Design & Typography ToolsCreative Collaboration & Feedback SystemsC&C
Towards Decomposable Interactive Systems: Design of a Backyard-Degradable Wireless Heating InterfaceSustainability is critical to our planet and thus our designs. Within HCI, there is a tension between the desire to create interactive electronic systems and sustainability. In this paper, we present the design of an interactive system comprising components that are entirely decomposable. We leverage the inherent material properties of natural materials, such as paper, leaf skeletons, and chitosan, along with silver nanowires to create a new system capable of being electrically controlled as a portable heater. This new decomposable system, capable of wirelessly heating to >70°C, is flexible, lightweight, low-cost, and reusable, and it maintains its functionality over long periods of heating and multiple power cycles. We detail its design and present a series of use cases, from enabling a novel resealable packaging system to acting as a catalyst for shape-changing designs and beyond. Finally, we highlight the important decomposable property of the interactive system when it meets end-of-life.2022KSKatherine W Song et al.Accenture Labs, UC BerkeleyShape-Changing Materials & 4D PrintingEcological Design & Green ComputingCHI
Expanding the Design Space for Technology-Mediated Theatre ExperiencesWork combining live performance and technology often involves incorporating technology directly into the performance as it occurs onstage, including interactive costumes, or performer-controlled sets, lighting or sound. We invert this common approach, developing technology-mediated experiences outside the temporal and spatial confines of a live theatre production. We describe the 4-month co-design process with expert theatre practitioners, and detail how the process 1) shaped our design guidelines and 2) expands the discussion around existing best practices for cross-disciplinary collaboration. In the style of research through design, we present three annotated prototypes: the Augmented Playbill, the Prayer Wheel, and Tarot Cards as well as accompanying AR applications to convey the decisions we made and the philosophy we iteratively developed throughout the project. These artifacts also embody our six design guidelines: resonant affordances, extended narrative, reflective interaction, selective reveal, personalized experience, and privileged access.2021MNMolly Jane Nicholas et al.Digital Art Installations & Interactive PerformanceInteractive Narrative & Immersive StorytellingDIS
Unmaking: Enabling and Celebrating the Creative Material of Failure, Destruction, Decay, and DeformationThe access and growing ubiquity of digital fabrication has ushered in a celebration of creativity and ``making.'' However, the focus is often on the resulting static artifact or the creative process and tools to design it. We envision a post-making process that extends past these final static objects --- not just in their making but in their ``unmaking.'' By drawing from artistic movements such as Auto-Destructive Art, intentionally inverting well-established engineering principles of structurally sound designs, and safely misusing unstable materials, we demonstrate an important extension to making --- unmaking. In this paper, we provide designers with a new vocabulary of unmaking operations within standard 3D modeling tools. We demonstrate how such designs can be realized using a novel multi-material 3D printing process. Finally, we detail how unmaking allows designs to change over time, is an ally to sustainability and re-usability, and captures themes of ``aura,'' emotionality, and personalization.2021KSKatherine W Song et al.UC BerkeleyShape-Changing Materials & 4D PrintingDigital Art Installations & Interactive PerformanceCHI
Interacting with Literary Style through Computational ToolsStyle is an important aspect of writing, shaping how audiences interpret and engage with literary works. However, for most people style is difficult to articulate precisely. While users frequently interact with computational word processing tools with well-defined metrics, such as spelling and grammar checkers, style is a significantly more nuanced concept. In this paper, we present a computational technique to help surface style in written text. We collect a dataset of crowdsourced human judgments of style, derive a model of style by training a neural net on this data, and present novel applications for visualizing and browsing style across broad bodies of literature, as well as an interactive text editor with real-time style feedback. We study these interactive style applications with users and discuss implications for enabling this novel approach to style.2020SSSarah Sterman et al.University of California, BerkeleyAI-Assisted Creative WritingCreative Collaboration & Feedback SystemsCHI
Turn-by-Wire: Computationally Mediated Physical FabricationCurrent digital fabrication tools allow users to exert precise computer control over the making process. However, the `push-to-make' usage model of these tools limits not only their use, but also stifles our own mental models for how we should be able to directly, fluidly, and creatively interact with these tools. In this paper, we investigate how a new class of hybrid-controlled machines can collaborate with novice and expert users alike to yield a more lucid making experience. We demonstrate these ideas through our system, Turn-by-Wire. By combining the capabilities of a traditional lathe with haptic input controllers that modulate both position and force, we detail a series of novel interaction metaphors that invite an adaptable making process spanning digital, model-centric, computer control, and embodied, adaptive, human control. We evaluate our system through a user study and discuss how these concepts generalize to other fabrication tools.2019RTRundong Tian et al.Force Feedback & Pseudo-Haptic WeightCircuit Making & Hardware PrototypingUIST
MatchSticks: Woodworking through Improvisational Digital FabricationDigital fabrication tools have broadened participation in making and enabled new methods of rapid physical prototyping across diverse materials. We present a novel smart tool designed to complement one of the first materials employed by humans - wood - and celebrate the fabrication practice of joinery. Our tool, MatchSticks, is a digital fabrication system tailored for joinery. Combining a portable CNC machine, touchscreen user interface, and parametric joint library, MatchSticks enables makers of varying skill to rapidly explore and create artifacts from wood. Our system embodies tacit woodworking knowledge and distills the distributed workflow of CNC tools into a hand tool; it operates on materials existing machines find difficult, produces assemblies much larger than its workspace, and supports the parallel creation of geometries. We describe the workflow and technical details of our system, present example artifacts produced by our tool, and report results from our user study.2018RTRundong Tian et al.University of California, BerkeleyDesktop 3D Printing & Personal FabricationCircuit Making & Hardware PrototypingMakerspace CultureCHI
AlterWear: Battery-Free Wearable Displays for Opportunistic InteractionsAs the landscape of wearable devices continues to expand, power remains a major issue for adoption, usability, and miniaturization. Users are faced with an increasing number of personal devices to manage, charge, and care for. In this work, we argue that power constraints limit the design space of wearable devices. We present AlterWear: an architecture for new wearable devices that implement a batteryless design using electromagnetic induction via NFC and bistable e-ink displays. Although these displays are active only when in proximity to an NFC-enabled device, this unique combination of hardware enables both quick, dynamic and long-term interactions with persistent visual displays. We demonstrate new wearables enabled through AlterWear with dynamic, fashion-forward, and expressive displays across several form factors, and evaluate them in a user study. By forgoing the need for onboard power, AlterWear expands the ecosystem of functional and fashionable wearable technologies.2018CDChristine Dierk et al.University of CaliforniaHaptic WearablesOn-Skin Display & On-Skin InputCHI