Nabokov's Cards: An AI Assisted Prewriting Systems to Support Bottom-Up Creative WritingWe introduce Nabokov's Cards, a creativity support tool that uses Large Language Models (LLMs) to support prewriting. Inspired by the writing process of Vladimir Nabokov, Nabokov's Cards enables prewriting ideation by providing users with an interface to write idea fragments on notecards and combine them into new sentences or concepts using an LLM. We conducted a one-week user study of Nabokov's Cards with professional creative writers (n=13) to explore writers' prewriting processes and learn about their usage of Nabokov's cards. We found from our interviews that writers characterized prewriting as a long, amorphous process that involved observations of the real world and the accumulation of idea fragments and that Nabokov's Cards facilitated prewriting through nonlinear interactions, divergent thinking, play, improvisation, and reflection. Nabokov's Cards also encouraged innovative approaches among writers that surpassed the cliches and redundancy often found within AI generated text today. We also note how future AI co-writing systems may benefit from designs that facilitate prompt engineering and modular thinking.2025DCDashiel Carrera et al.Generative AI (Text, Image, Music, Video)Human-LLM CollaborationAI-Assisted Creative WritingC&C
VibraForge: A Scalable Prototyping Toolkit For Creating Spatialized Vibrotactile Feedback SystemsSpatialized vibrotactile feedback systems deliver tactile information by placing multiple vibrotactile actuators on the body. As increasing numbers of actuators are required to adequately convey information in complicated applications, haptic designers find it difficult to create such systems due to limited scalability of existing toolkits. We propose VibraForge, an open-source vibrotactile toolkit that supports up to 128 vibrotactile actuators. Each actuator is encapsulated within a self-contained vibration unit and driven by its own microcontroller. By leveraging a chain-connection method, each unit receives independent vibration commands from a control unit, with fine-grained control over intensity and frequency. We also designed a GUI Editor to expedite the authoring of spatial vibrotactile patterns. Technical evaluation showed that vibration units reliably reproduced audio waveforms with low-latency and high-bandwidth data communication. Case studies of a phonemic tactile display, virtual reality fitness training, and drone teleoperation demonstrated the potential usage of VibraForge within different domains. A usability study with non-expert users highlighted the low technical barrier and customizability of the toolkit.2025BHBingjian Huang et al.University of Toronto, Dynamic Graphics Project LabVibrotactile Feedback & Skin StimulationForce Feedback & Pseudo-Haptic WeightCHI
Creepy Assistant: Development and Validation of a Scale to Measure the Perceived Creepiness of Voice AssistantsVoice assistants have afforded users rich interaction opportunities to access information and issue commands in a variety of contexts. However, some users feel uneasy or creeped out by voice assistants, leading to a decreased desire to use them. As there has yet to be a comprehensive understanding of the factors that cause users to perceive voice assistants as being creepy, this research developed an empirical scale to measure the creepiness inherent in various voice assistants. Utilizing prior scale creation methodologies, a 7-item Perceived Creepiness of Voice Assistants Scale (PCAS) was created and validated. The scale measures how creepy a new voice assistant would be for users of voice assistants. The scale was developed to ensure that researchers and designers can evaluate the next generation of voice assistants before such voice assistants are released to the wider public.2023RPRachel Phinnemore et al.University of TorontoVoice User Interface (VUI) DesignAgent Personality & AnthropomorphismExplainable AI (XAI)CHI
CrossCode: Multi-level Visual Representations of Computer Program ExecutionProgram visualizations help to form useful mental models of how programs work, and to reason and debug code. But these visualizations exist at a fixed level of abstraction, e.g., line-by-line. In contrast, programmers switch between many levels of abstraction when inspecting program behavior. Based on results from a formative study of hand-designed program visualizations, we designed CrossCode, a web-based program visualization system for JavaScript that leverages structural cues in syntax, control flow, and data flow to aggregate and navigate program execution across multiple levels of abstraction. In an exploratory qualitative study with experts, we found that CrossCode enabled participants to maintain a strong sense of place in program execution, was conducive to explaining program behavior, and helped track changes and updates to the program state.2023DHDevamardeep Hayatpur et al.University of California, San DiegoInteractive Data VisualizationProgramming Education & Computational ThinkingCHI
“Positive Energy”: Perceptions and Attitudes Towards COVID-19 Information on Social Media in ChinaThe outbreak of COVID-19 has resulted in a worldwide public health crisis. In such times of crisis, access to relevant and accurate information is critical. For many people in China, domestic social media platforms such as WeChat and Weibo have become dominant sources of COVID-19-related information and news. On these platforms, government censorship policies, astroturfers, and other government interventions have contributed to an increasingly complex social media environment. People have to evaluate the trustworthiness of COVID-19-related information and make sharing decisions in such a complex environment. We conducted interviews with 33 Chinese WeChat users to understand how individuals were seeking COVID-19-related information and how they identified and evaluated specific COVID-19-related misinformation. This work exposes that COVID-19-related content with ‘positive energy’ is prevalent on social media in China. A significant number of interviewees exhibited a willingness to prioritize information valence over veracity when evaluating and sharing content with others. Further, the work revealed that Chinese citizens’ understanding of information ecosystems plays an important role in their attitudes towards censorship and official media, and their evaluation of both domestic and international information during the global crisis.2021ZLZhicong Lu et al.Technology/Information Use in Times of PandemicCSCW
False Positives vs. False Negatives: The Effects of Recovery Time and Cognitive Costs on Input Error PreferenceExisting approaches to trading off false positive versus false negative errors in input recognition are based on imprecise ideas of how these errors affect user experience that are unlikely to hold for all situations. To inform dynamic approaches to setting such a tradeoff, two user studies were conducted on how relative preference for false positive versus false negative errors is influenced by differences in the temporal cost of error recovery, and high-level task factors (time pressure, multi-tasking). Participants completed a tile selection task in which false positive and false negative errors were injected at a fixed rate, and the temporal cost to recover from each of the two types of error was varied, and then indicated a preference for one error type or the other, and a frustration rating for the task. Responses indicate that the temporal costs of error recovery can drive both frustration and relative error type preference, and that participants exhibit a bias against false positive errors, equivalent to ~1.5 seconds or more of added temporal recovery time. Several explanations for this bias were revealed, including that false positive errors impose a greater attentional demand on the user, and that recovering from false positive errors imposes a task switching cost.2021BLBen Lafreniere et al.Visualization Perception & CognitionUser Research Methods (Interviews, Surveys, Observation)UIST
PatchProv: Supporting Improvisational Design Practices for Modern QuiltingThe craft of improvisational quilting involves working without the use of a predefined pattern. Design decisions are made "in the fabric," with design experimentation tightly interleaved with the creation of the final artifact. To investigate how this type of design process can be supported, and to address challenges faced by practitioners, this paper presents PatchProv, a system for supporting improvisational quilt design. Based on a review of popular books on improvisational quilting, a set of design principles and key challenges to improvisational quilt design were identified, and PatchProv was developed to support the unique aspects of this process. An evaluation with a small group of quilters showed enthusiasm for the approach and revealed further possibilities for how computational tools can support improvisational quilting and improvisational design practices more broadly.2021MLMackenzie Leake et al.Chatham Labs, Stanford UniversityTextile Art & Craft DigitizationCHI
More Kawaii than a Real-Person Streamer: Understanding How the Otaku Community Engages with and Perceives Virtual YouTubersLive streaming has become increasingly popular, with most streamers presenting their real-life appearance. However, Virtual YouTubers (VTubers), virtual 2D or 3D avatars that are voiced by humans, are emerging as live streamers and attracting a growing viewership in East Asia. Although prior research has found that many viewers seek real-life interpersonal interactions with real-person streamers, it is currently unknown what makes VTuber live streams engaging or how they are perceived differently than real-person streamers. We conducted an interview study to understand how viewers engage with VTubers and perceive the identities of the voice actors behind the avatars (i.e., Nakanohito). The data revealed that Virtual avatars bring unique performative opportunities which result in different viewer expectations and interpretations of VTuber behavior. Viewers intentionally upheld the disembodiment of VTuber avatars from their voice actors. We uncover the nuances in viewer perceptions and attitudes and further discuss the implications of VTuber practices to the understanding of live streaming in general.2021ZLZhicong Lu et al.City University of Hong Kong, University of TorontoAgent Personality & AnthropomorphismLive Streaming & Content CreatorsCHI
Armstrong: An Empirical Examination of Pointing at Non-Dominant Arm-Anchored UIs in Virtual RealityIn virtual reality (VR) environments, asymmetric bimanual interaction techniques can increase users' input bandwidth by complementing their perceptual and motor systems (e.g., using the dominant hand to select 3D UI controls anchored around the non-dominant arm). However, it is unclear how to optimize the layout of such 3D UI controls for near-body and mid-air interactions. We evaluate the performance and limitations of non-dominant arm-anchored 3D UIs in VR environments through a bimanual pointing study. Results demonstrated that targets appearing closer to the skin, located around the wrist, or placed on the medial side of the forearm could be selected more quickly than targets farther away from the skin, located around the elbow, or on the lateral side of the forearm. Based on these results, we developed Armstrong guidelines, demonstrated through a Unity plugin to enable designers to create performance-optimized arm-anchored 3D UI layouts.2021ZLZhen Li et al.University of Toronto, Chatham LabsMid-Air Haptics (Ultrasonic)Haptic WearablesFull-Body Interaction & Embodied InputCHI
The Labor of Fun: Understanding the Social Relationships between Gamers and Paid Gaming Teammates in ChinaOnline video games support the development of social relationships through gameplay. However, gamers often cannot cultivate and maintain relationships based on social factors such as personality when using in-game matchmaking services. To address this, teammate matching sites external to games have emerged and enable gamers to offer to play games with others in exchange for payment. The affordances of these services are different from other existing gamer social sites, e.g., live streaming. Interviews were conducted with 16 dedicated users on Bixin, one of China’s largest paid teammate matching sites, to examine user motivations, practices, and perceptions. The interviews found that gamers selected paid teammates on Bixin using different criteria compared to in-game matchmaking services and emphasized the importance of real-life characteristics such as voice. To maintain connections, paid teammates often also extended communication to external communication services such as WeChat. Although most gamers expected to communicate with paid teammates as if they were friends, very few reported building real friendships with their matched counterparts.2021CSChenxinran Shen et al.University of TorontoBrain-Computer Interface (BCI) & NeurofeedbackMultiplayer & Social GamesCHI
StickyPie: A Gaze-Based, Scale-Invariant Marking Menu Optimized for AR/VRThis work explores the design of marking menus for gaze-based AR/VR menu selection by expert and novice users. It first identifies and explains the challenges inherent in ocular motor control and current eye tracking hardware, including overshooting, incorrect selections, and false activations. Through three empirical studies, we optimized and validated design parameters to mitigate these errors while reducing completion time, task load, and eye fatigue. Based on the findings from these studies, we derived a set of design guidelines to support gaze-based marking menus in AR/VR. To overcome the overshoot errors found with eye-based expert marking menu behaviour, we developed StickyPie, a marking menu technique that enables scale-independent marking input by estimating saccade landing positions. An evaluation of StickyPie revealed that StickyPie was easier to learn than the traditional technique (i.e., RegularPie) and was 10% more efficient after 3 sessions.2021SASunggeun Ahn et al.Chatham Labs, KAISTEye Tracking & Gaze InteractionMixed Reality WorkspacesCHI
DataHop: Spatial Data Exploration in Virtual RealityVirtual reality has recently been adopted for use within the domain of visual analytics because it can provide users with an endless workspace within which they can be actively engaged and use their spatial reasoning skills for data analysis. However, virtual worlds need to utilize layouts and organizational schemes that are meaningful to the user and beneficial for data analysis. This paper presents DataHop, a novel visualization system that enables users to lay out their data analysis steps in a virtual environment. With a Filter, a user can specify the modification they wish to perform on one or more input data panels (i.e., containers of points), along with where output data panels should be placed in the virtual environment. Using this simple tool, highly intricate and useful visualizations may be generated and traversed by harnessing a user’s spatial abilities. An exploratory study conducted with six virtual reality users evaluated the usability, affordances, and performance of DataHop for data analysis tasks, and found that spatially mapping one’s workflow can be beneficial when exploring multidimensional datasets.2020DHDevamardeep Hayatpur et al.Mixed Reality WorkspacesInteractive Data VisualizationUIST
The Government's Dividend: Complex Perceptions of Social Media Misinformation in ChinaThe social media environment in China has become the dominant source of information and news over the past decade. This news environment has naturally suffered from challenges related to mis- and dis-information, encumbered by an increasingly complex landscape of factors and players including social media services, fact-checkers, censorship policies, and astroturfing. Interviews with 44 Chinese WeChat users were conducted to understand how individuals perceive misinformation and how it impacts their news consumption practices. Overall, this work exposes the diverse attitudes and coping strategies that Chinese users employ in complex social media environments. Due to the complex nature of censorship in China and participants' lack of understanding of censor-ship, they expressed varied opinions about its influence on the credibility of online information sources. Further, although most participants claimed that their opinions would not be easily swayed by astroturfers, many admitted that they could not effectively distinguish astroturfers from ordinary Internet users. Participants' inability to make sense of comments found online lead many participants to hold pro-censorship attitudes: the Government's Dividend.2020ZLZhicong Lu et al.University of TorontoAI Ethics, Fairness & AccountabilityContent Moderation & Platform GovernanceMisinformation & Fact-CheckingCHI
Head-Coupled Kinematic Template Matching: A Prediction Model for Ray Pointing in VRThis paper presents a new technique to predict the ray pointer landing position for selection movements in virtual reality (VR) environments. The technique adapts and extends a prior 2D kinematic template matching method to VR environments where ray pointers are used for selection. It builds on the insight that the kinematics of a controller and Head-Mounted Display (HMD) can be used to predict the ray's final landing position and angle. An initial study provides evidence that the motion of the head is a key input channel for improving prediction models. A second study validates this technique across a continuous range of distances, angles, and target sizes. On average, the technique's predictions were within 7.3° of the true landing position when 50% of the way through the movement and within 3.4° when 90%. Furthermore, compared to a direct extension of Kinematic Template Matching, which only uses controller movement, this head-coupled approach increases prediction accuracy by a factor of 1.8x when 40% of the way through the movement.2020RHRorik Henrikson et al.Chatham LabsEye Tracking & Gaze InteractionHuman Pose & Activity RecognitionImmersion & Presence ResearchCHI
PseudoBend: Producing Haptic Illusions of Stretching, Bending, and Twisting Using Grain VibrationsWe present a technique for haptic feedback that creates the illusion that a rigid device is being stretched, bent, or twisted. The method uses a single 6-DOF force sensor and a vibrotactile actuator to render grain vibrations to simulate the vibrations produced during object deformation based on the changes in force or torque exerted on a device. Because this method does not require any moving parts aside from the vibrotactile actuator, devices designed using this method can be small and lightweight. Psychophysical studies conducted using a prototype that implements this method confirmed that the method could be used to successfully create the illusion of deformation and could also change users’ perception of stiffness by changing the virtual stiffness parameters.2019SHSeongkook Heo et al.Force Feedback & Pseudo-Haptic WeightUIST
A Lie Reveals the Truth: Quasimodes for Task-Aligned Data PresentationDesigners are often discouraged from creating data visualizations that omit or distort information, because they can easily be misleading. However, the same representations that could be used to deceive can provide benefits when chosen to appropriately align with user tasks. We present an interaction technique, Perceptual Glimpses, which allows for the transparent presentation of so-called 'deceptive' views of information that are made temporary using quasimodes. When presented using Perceptual Glimpses, message-level exaggeration caused by a truncated axis on a bar chart was reduced under some conditions, but users require guidance to avoid errors, and view presentation order may affect trust. When Perceptual Glimpses was extended to display a range of views that might otherwise be deceptive or difficult to understand if shown out of context, users were able to understand and leverage these transformations to perform a range of low-level tasks. Design recommendations and examples suggest extensions of the technique.2019JRJacob Ritchie et al.University of TorontoUncertainty VisualizationVisualization Perception & CognitionCHI
Learning Cooperative Personalized Policies from Gaze DataAn ideal Mixed Reality (MR) system would only present virtual information (e.g., a label) when it is useful to the person. However, figuring out when a label is useful is challenging; it depends on a variety of factors, including the current task, previous knowledge, context, etc. In this paper, we propose a Reinforcement Learning (RL) method to learn when to show or hide an object’s label given eye movement data. We demonstrate the capabilities of this approach by showing that an intelligent agent can learn cooperative policies that better support users in a visual search task than design heuristics. Furthermore, we show the applicability of our approach in realistic environments and use cases (e.g., grocery shopping). By posing MR object labeling as an RL control problem we can learn policies implicitly by observing users’ behavior without requiring experience sampling or any other form of supervision.2019CGChristoph Gebhardt et al.Eye Tracking & Gaze InteractionMixed Reality WorkspacesAI-Assisted Decision-Making & AutomationUIST
HoloDoc: Enabling Mixed Reality Workspaces that Harness Physical and Digital ContentPrior research identified that physical paper documents have many positive attributes, for example natural tangibility and inherent physical flexibility. When documents are presented on digital devices, however, they can provide unique functionality to users, such as the ability to search, view dynamic multimedia content, and make use of indexing. This work explores the fusion of physical and digital paper documents. It first presents the results of a study that probed how users perform document-intensive analytical tasks when both physical and digital versions of documents were available. The study findings then informed the design of HoloDoc, a mixed reality system that augments physical artifacts with rich interaction and dynamic virtual content. Finally, we present the interaction techniques that HoloDoc affords, and the results of a second study that assessed HoloDoc's utility when working with digital and physical copies of academic articles.2019ZLZhen Li et al.University of TorontoMixed Reality WorkspacesInteractive Data VisualizationCHI
Vicariously Experiencing it all without Going outside: A Study of Outdoor Live Streaming in ChinaThe livestreaming industry in China is gaining greater traction than its European and North American counterparts and has a profound impact on the stakeholders’ online and offline lives. An emerging genre of livestreaming that has become increasingly popular in China is outdoor livestreaming. With outdoor livestreams, streamers broadcast outdoor activities, travel, or socialize with passersby in outdoor settings, often for 6 or more hours, and viewers watch such streams for hours each day. However, given that professionally produced content about travel and outdoor activities are not very popular, it is currently unknown what makes this category of livestreams so engaging and how these techniques can be applied to other content or genres. Thus, we conducted a mixed methods study consisting of a survey (N=287) and interviews (N = 20) to understand how viewers watch and engage with outdoor livestreams in China. The data revealed that outdoor livestreams encompass many categories of content, environments and passersby behaviors create challenges and uncertainty for viewers and streamers, and viewers watch livestreams for surprising lengths of time (e.g., sometimes more than 5 continuous hours). We also gained insights into how live commenting and virtual gifting encourage engagement. Lastly, we detail how the behaviors of dedicated fans and casual viewers differ and provide implications for the design of livestreaming services that support outdoor activities.2019ZLZhicong Lu et al.Videos and Live StreamingCSCW
Plane, Ray, and Point: Enabling Precise Spatial Manipulations with Shape ConstraintsModern virtual reality controllers offer direct, quick, and easy manipulation of virtual objects. However, precisely placing objects using direct manipulation is still challenging in 3D environments. This paper presents Plane, Ray, & Point, a set of novel interaction techniques that enable quick object alignment and manipulation in virtual reality. The interaction techniques use hand gestures to create constraints that separate manipulation degrees of freedom. The user can create a Ray by outstretching the index finger or the thumb and use it to limit the rotation or translation of an object to a single axis. By opening both the index finger and the thumb, the user can create a Plane, and use it to limit the object’s movement along the 2D plane. Such gesture-invoked constraints help users quickly align and place virtual objects. We evaluate the applicability and use-cases of our technique in an expert user study; as well as evaluate its learnability in an informal user study with novice users.2019DHDevamardeep Hayatpur et al.Shape-Changing Interfaces & Soft Robotic MaterialsHand Gesture RecognitionFull-Body Interaction & Embodied InputUIST