Visual Story-Writing: Writing by Manipulating Visual Representations of StoriesWe define "visual story-writing" as using visual representations of story elements to support writing and revising narrative texts. To demonstrate this approach, we developed a text editor that automatically visualizes a graph of entity interactions, movement between locations, and a timeline of story events. Interacting with these visualizations results in suggested text edits: for example, connecting two characters in the graph creates an interaction between them, moving an entity updates their described location, and rearranging events on the timeline reorganizes the narrative sequence. Through two user studies on narrative text editing and writing, we found that visuals supported participants in planning high-level revisions, tracking story elements, and exploring story variations in ways that encourage creativity. Broadly, our work lays the foundation for writing support, not just through words, but also visuals.2025DMDamien Masson et al.Data StorytellingGame AccessibilityAI-Assisted Creative WritingUIST
Making the Write Connections: Linking Writing Support Tools with Writer NeedsThis work sheds light on whether and how creative writers' needs are met by existing research and commercial writing support tools (WST). We conducted a need finding study to gain insight into the writers' process during creative writing through a qualitative analysis of the response from an online questionnaire and Reddit discussions on \textit{r/Writing}. Using a systematic analysis of 115 tools and 67 research papers, we map out the landscape of how digital tools facilitate the writing process. Our triangulation of data reveals that research predominantly focuses on the writing activity and overlooks pre-writing activities and the importance of visualization. We distill 10 key takeaways to inform future research on WST and point to opportunities surrounding underexplored areas. Our work offers a holistic and up-to-date account of how tools have transformed the writing process, guiding the design of future tools that address writers' evolving and unmet needs.2025ZZZixin Zhao et al.University of Toronto, Department of Computer ScienceAI-Assisted Creative WritingCHI
Does Adding Visual Signifiers in Animated Transitions Improve Interaction Discoverability?Smartphones support diverse inputs, however, the multitude of devices and platforms makes it challenging for people to discover when and where interactions are meaningful. Motivated by the effectiveness of visual signifiers in communicating interactivity, we explore the viability of integrating temporary visual signifiers in animated transitions between UI screens to promote the discoverability of swipe-revealed widgets. We implemented two transition types (Container Transform, Panels), and compared them to a baseline. We found that transitions with a standard duration did not impact the discovery of swipe-related widgets (N=33). We ran a follow-up study (N=22) with extremely slow 5000ms transitions to guarantee noticeability, but similarly found no impact on discovery of swipe-revealed widgets, diverging from previous findings for visual signifiers. This raises interesting questions about the perception and understanding of interaction signifiers, and indicates a disconnect between noticeability and discoverability, while highlighting difficulties with adapting established interface elements beyond their entrenched functionality.2025EMEva Mackamul et al.Université Grenoble-Alpes, CNRS, LIGVisualization Perception & CognitionPrototyping & User TestingCHI
Textoshop: Interactions Inspired by Drawing Software to Facilitate Text EditingWe explore how interactions inspired by drawing software can help edit text. Making an analogy between visual and text editing, we consider words as pixels, sentences as regions, and tones as colours. For instance, direct manipulations move, shorten, expand, and reorder text; tools change number, tense, and grammar; colours map to tones explored along three dimensions in a tone picker; and layers help organize and version text. This analogy also leads to new workflows, such as boolean operations on text fragments to construct more elaborated text. A study shows participants were more successful at editing text and preferred using the proposed interface over existing solutions. Broadly, our work highlights the potential of interaction analogies to rethink existing workflows, while capitalizing on familiar features.2025DMDamien Masson et al.University of Toronto, Department of Computer ScienceHuman-LLM CollaborationAI-Assisted Creative WritingPrototyping & User TestingCHI
Leveraging Idle Games to Incentivize Intermittent and Frequent Practice of Deep BreathingThe need for frequent and brief practice in deep breathing presents challenges in maintaining motivation and consistency. While persuasive technologies have been shown to improve engagement in therapeutic exercises, there is a lack of insight into specific motivational strategies for such intermittent activities. We investigate how idle games can incentivize behaviors like deep breathing and identify specific mechanics for fostering an optimal practice cycle. We illustrate this approach in a game called \textit{BreathPurr-suade}. After validating the physiological efficacy of the embedded breathing guide, our four-week study revealed idle games are more effective in maintaining deep breathing adherence than a standard breathing guide. Our work highlights the capacity of idle games to foster deep breathing, revealing their efficacy in subtle persuasive game designs that encourage intermittent therapeutic practices.2024BSBook Sadprasid et al.University of TorontoSerious & Functional GamesGamification DesignMental Health Apps & Online Support CommunitiesCHI
Functional Design Requirements to Facilitate Menstrual Health Data ExplorationMenstrual trackers currently lack the affordances required to help individuals achieve their goals beyond menstrual event predictions and symptom logging. Taking an initial step towards this aspiration, we propose, validate, and refine five functional design requirements for future interface designs that facilitate menstrual data exploration. We interviewed 30 individuals who menstruate and collected their feedback on the practical application of these requirements. To elicit ideas and impressions, we designed two proof-of-concept interfaces to use as design probes with similar core functionalities but different presentations of phase timing predictions and signal arrangement. Our analysis revealed participants' feedback regarding the presentation of predictions for menstrual-related events, the visualization of future signal patterns, personalization abilities for viewing signals relevant to their menstrual experience, the availability of resources to understand the underlying biological connections between signals, and the ability to compare multiple cycles side-by-side with context.2024GLGeorgianna Lin et al.University of TorontoReproductive & Women's HealthDiet Tracking & Nutrition ManagementCHI
The HaLLMark Effect: Supporting Provenance and Transparent Use of Large Language Models in Writing with Interactive VisualizationThe use of Large Language Models (LLMs) for writing has sparked controversy both among readers and writers. On one hand, writers are concerned that LLMs will deprive them of agency and ownership, and readers are concerned about spending their time on text generated by soulless machines. On the other hand, AI-assistance can improve writing as long as writers can conform to publisher policies, and as long as readers can be assured that a text has been verified by a human. We argue that a system that captures the provenance of interaction with an LLM can help writers retain their agency, conform to policies, and communicate their use of AI to publishers and readers transparently. Thus we propose HaLLMark, a tool for visualizing the writer's interaction with the LLM. We evaluated HaLLMark with 13 creative writers, and found that it helped them retain a sense of control and ownership of the text.2024MHMd Naimul Hoque et al.University of MarylandHuman-LLM CollaborationExplainable AI (XAI)AI-Assisted Creative WritingCHI
multiverse: Multiplexing Alternative Data Analyses in R NotebooksThere are myriad ways to analyse a dataset. But which one to trust? In the face of such uncertainty, analysts may adopt multiverse analysis: running all reasonable analyses on the dataset. Yet this is cognitively and technically difficult with existing tools—how does one specify and execute all combinations of reasonable analyses of a dataset?—and often requires discarding existing workflows. We present multiverse, a tool for implementing multiverse analyses in R with expressive syntax supporting existing computational notebook workflows. multiverse supports building up a multiverse through local changes to a single analysis and optimises execution by pruning redundant computations. We evaluate how multiverse supports programming multiverse analyses using (a) principles of cognitive ergonomics to compare with two existing multiverse tools; and (b) case studies based on semi-structured interviews with researchers who have successfully implemented an end-to-end analysis using multiverse. We identify design tradeoffs (e.g. increased flexibility versus learnability), and suggest future directions for multiverse tool design.2023ASAbhraneel Sarma et al.Northwestern UniversityInteractive Data VisualizationComputational Methods in HCICHI
Designing Resource Allocation Tools to Promote Fair Allocation: Do Visualization and Information Framing Matter?Studies on human decision-making focused on humanitarian aid have found that cognitive biases can hinder the fair allocation of resources. However, few HCI and Information Visualization studies have explored ways to overcome those cognitive biases. This work investigates whether the design of interactive resource allocation tools can help to promote allocation fairness. We specifically study the effect of presentation format (using text or visualization) and a specific framing strategy (showing resources allocated to groups or individuals). In our three crowdsourced experiments, we provided different tool designs to split money between two fictional programs that benefit two distinct communities. Our main finding indicates that individual-framed visualizations and text may be able to curb unfair allocations caused by group-framed designs. This work opens new perspectives that can motivate research on how interactive tools and visualizations can be engineered to combat cognitive biases that lead to inequitable decisions.2023AVArnav Verma et al.University of TorontoExplainable AI (XAI)Uncertainty VisualizationAlgorithmic Fairness & BiasCHI
Get To The Point! Problem-Based Curated Data Views To Augment Care For Critically Ill PatientsElectronic health records in critical care medicine offer unprecedented opportunities for clinical reasoning and decision making. Paradoxically, these data-rich environments have also resulted in clinical decision support systems (CDSSs) that fit poorly into clinical contexts, and increase health workers cognitive load. In this paper, we introduce a novel approach to designing CDSSs that are embedded in clinical workflows, by presenting problem-based curated data views tailored for problem-driven discovery, team communication, and situational awareness. We describe the design and evaluation of one such CDSS, In-Sight, that embodies our approach and addresses the clinical problem of monitoring critically ill pediatric patients. Our work is the result of a co-design process, further informed by empirical data collected through formal usability testing, focus groups, and a simulation study with domain experts. We discuss the potential and limitations of our approach, and share lessons learned in our iterative co-design process.2022MZMinfan Zhang et al.University of TorontoAI-Assisted Decision-Making & AutomationMental Health Apps & Online Support CommunitiesCHI
Space, Time, and Choice: A unified approach to flexible personal scheduling In the context of increasingly busy lives and mobility constraints, we present a unified space-time approach to support flexible personal scheduling. We distill an analysis of the design requirements of interactive space-time scheduling into a single coherent workflow where users can manipulate a rich vocabulary of spatio-temporal parameters, and plan/explore itineraries that satisfy or optimize the resulting space-time constraints. We demonstrate our approach using a proof-of-concept mobile application that enables exploration of the inter-connected continuum between task scheduling (temporal), and multi-destination route mapping (spatial). We evaluate the application with a user study involving an itinerary reproduction task and a free-form planning task. We also provide usage scenarios illustrating the potential of our approach in various contexts and tasks. Results suggest that our approach fills an important gap between route mapping and calendar scheduling, suggesting a new research direction in personal planning interface design.2021VBVicky Bilbily et al.Context-Aware ComputingPublic Transit & Trip PlanningUIST
Sketchnote Components, Design Space Dimensions, and Strategies for Effective Visual Note TakingSketchnoting is a form of visual note taking where people listen to, synthesize, and visualize ideas from a talk or other event using a combination of pictures, diagrams, and text. Little is known about the design space of this kind of visual note taking. With an eye towards informing the implementation of digital equivalents of sketchnoting, inking, and note taking, we introduce a classification of sketchnote styles and techniques, with a qualitative analysis of 103 sketchnotes, and situated in context with six semi-structured follow up interviews. Our findings distill core sketchnote components (content, layout, structuring elements, and visual styling) and dimensions of the sketchnote design space, classifying levels of conciseness, illustration, structure, personification, cohesion, and craftsmanship. We unpack strategies to address particular note taking challenges, for example dealing with constraints of live drawings, and discuss relevance for future digital inking tools, such as recomposition, styling, and design suggestions.2021RZRebecca Zheng et al.University College London, MumbliInteractive Data VisualizationData StorytellingUser Research Methods (Interviews, Surveys, Observation)CHI
Leveraging Text-Chart Links to Support Authoring of Data-Driven Articles with VizFlowData-driven articles --- i.e., articles featuring text and supporting charts --- play a key role in communicating information to the public. New storytelling formats like scrollytelling apply compelling dynamics to these articles to help walk readers through complex insights, but are challenging to craft. In this work, we investigate ways to support authors of data-driven articles using such storytelling forms via a text-chart linking strategy. From formative interviews with 6 authors and an assessment of 43 scrollytelling stories, we built VizFlow, a prototype system that uses text-chart links to support a range of dynamic layouts. We validate our text-chart linking approach via an authoring study with 12 participants using VizFlow, and a reading study with 24 participants comparing versions of the same article with different VizFlow intervention levels. Assessments showed our approach enabled a rapid and expressive authoring experience, and informed key design recommendations for future efforts in the space.2021NSNicole Sultanum et al.University of TorontoInteractive Data VisualizationData StorytellingCHI
Interactive Exploration and Refinement of Facial Expression using Manifold LearningPosing expressive 3D faces is extremely challenging. Typical facial rigs have upwards of 30 controllable parameters, that while anatomically meaningful, are hard to use due to redundancy of expression, unrealistic configurations, and many semantic and stylistic correlations between the parameters. We propose a novel interface for rapid exploration and refinement of static facial expressions, based on a data-driven face manifold of “natural” expressions. Rapidly explored face configurations are interactively projected onto this manifold of meaningful expressions. These expressions can then be refined using a 2D embedding of nearby faces, both on and off the manifold. Our validation is fourfold: we show expressive face creation using various devices; we verify that our learnt manifold transcends its training face, to expressively control very different faces; we perform a crowd-sourced study to evaluate the quality of manifold face expressions; and we report on a usability study that shows our approach is an effective interactive tool to author facial expression.2020RARinat Abdrashitov et al.3D Modeling & AnimationCreative Collaboration & Feedback SystemsUIST
DataQuilt: Extracting Visual Elements from Images to Craft Pictorial VisualizationsRecent years have seen an increasing interest in the authoring and crafting of personal visualizations. Mainstream data analysis and authoring tools lack the flexibility for customization and personalization, whereas tools from the research community either require creativity and drawing skills, or are limited to simple vector graphics. We present DataQuilt, a novel system that enables visualization authors to iteratively design pictorial visualizations as collages. Real images (e.g., paintings, photographs, sketches) act as both inspiration and as a resource of visual elements that can be mapped to data. The creative pipeline involves the semi-guided extraction of relevant elements of an image (arbitrary regions, regular shapes, color palettes, textures) aided by computer vision techniques; the binding of these graphical elements and their features to data in order to create meaningful visualizations; and the iterative refinement of both features and visualizations through direct manipulation. We demonstrate the usability of DataQuilt in a controlled study and its expressiveness through a collection of authored visualizations from a second open-ended study.2020JZJiayi Eris Zhang et al.University of TorontoInteractive Data VisualizationData PhysicalizationCHI
Increasing the Transparency of Research Papers with Explorable Multiverse AnalysesWe present explorable multiverse analysis reports, a new approach to statistical reporting where readers of research papers can explore alternative analysis options by interacting with the paper itself. This approach draws from two recent ideas: i) multiverse analysis, a philosophy of statistical reporting where paper authors report the outcomes of many different statistical analyses in order to show how fragile or robust their findings are; and ii) explorable explanations, narratives that can be read as normal explanations but where the reader can also become active by dynamically changing some elements of the explanation. Based on five examples and a design space analysis, we show how combining those two ideas can complement existing reporting approaches and constitute a step towards more transparent research papers.2019PDPierre Dragicevic et al.InriaAlgorithmic Transparency & AuditabilityComputational Methods in HCICHI
A Lie Reveals the Truth: Quasimodes for Task-Aligned Data PresentationDesigners are often discouraged from creating data visualizations that omit or distort information, because they can easily be misleading. However, the same representations that could be used to deceive can provide benefits when chosen to appropriately align with user tasks. We present an interaction technique, Perceptual Glimpses, which allows for the transparent presentation of so-called 'deceptive' views of information that are made temporary using quasimodes. When presented using Perceptual Glimpses, message-level exaggeration caused by a truncated axis on a bar chart was reduced under some conditions, but users require guidance to avoid errors, and view presentation order may affect trust. When Perceptual Glimpses was extended to display a range of views that might otherwise be deceptive or difficult to understand if shown out of context, users were able to understand and leverage these transformations to perform a range of low-level tasks. Design recommendations and examples suggest extensions of the technique.2019JRJacob Ritchie et al.University of TorontoUncertainty VisualizationVisualization Perception & CognitionCHI
Color Builder: A Direct Manipulation Interface for Versatile Color Theme AuthoringColor themes or palettes are popular for sharing color combinations across many visual domains. We present a novel interface for creating color themes through direct manipulation of color swatches. Users can create and rearrange swatches, and combine them into smooth and step-based gradients and three-color blends -- all using a seamless touch or mouse input. Analysis of existing solutions reveals a fragmented color design workflow, where separate software is used for swatches, smooth and discrete gradients and for in-context color visualization. Our design unifies these tasks, while encouraging playful creative exploration. Adjusting a color using standard color pickers can break this interaction flow with mechanical slider manipulation. To keep interaction seamless, we additionally design an in situ color tweaking interface for freeform exploration of an entire color neighborhood. We evaluate our interface with a group of professional designers and students majoring in this field.2019MSMaria Shugrina et al.University of Toronto360° Video & Panoramic ContentInteractive Data VisualizationGraphic Design & Typography ToolsCHI
More Text Please! Understanding and Supporting the Use of Visualization for Clinical Text OverviewClinical practice is heavily reliant on the use of unstructured text to document patient stories due to its expressive and flexible nature. However, a physician's capacity to recover information from text for clinical overview is severely affected when records get longer and time pressure increases. Data visualization strategies have been explored to aid in information retrieval by replacing text with graphical summaries, though often at the cost of omitting important text features. This causes physician mistrust and limits real-world adoption. This work presents our investigation into the role and use of text in clinical practice, and reports on efforts to assess the best of both worlds---text and visualization---to facilitate clinical overview. We report on insights garnered from a field study, and the lessons learned from an iterative design process and evaluation of a text-visualization prototype, MedStory, with 14 medical professionals. The results led to a number of grounded design recommendations to guide visualization design to support clinical text overview.2018NSNicole Sultanum et al.University of Toronto, The Hospital for Sick ChildrenInteractive Data VisualizationMedical & Scientific Data VisualizationCHI
DataInk: Direct and Creative Data-Oriented DrawingCreating whimsical, personal data visualizations remains a challenge due to a lack of tools that enable for creative visual expression while providing support to bind graphical content to data. Many data analysis and visualization creation tools target the quick generation of visual representations, but lack the functionality necessary for graphics design. Toolkits and charting libraries offer more expressive power, but require expert programming skills to achieve custom designs. In contrast, sketching affords fluid experimentation with visual shapes and layouts in a free-form manner, but requires one to manually draw every single data point. We aim to bridge the gap between these extremes. We propose DataInk, a system supports the creation of expressive data visualizations with rigorous direct manipulation via direct pen and touch input. Leveraging our commonly held skills, coupled with a novel graphical user interface, DataInk enables direct, fluid, and flexible authoring of creative data visualizations.2018HXHaijun Xia et al.University of TorontoInteractive Data VisualizationCHI