From Sports Videos to Immersive Training: Augmenting Human Motion to Enrich Basketball Training ExperienceVideo plays a crucial role in sports training, enabling participants to analyze their movements and identify opponents' weaknesses. Despite the easy access to sports videos, the rich motion data within them remains underutilized due to the lack of clear performance indicators and discrepancies from real-game conditions. To address this, we employed advanced computer vision algorithms to reconstruct human motions in an immersive environment, where users can freely observe and interact with the movements. Basketball shooting was chosen as a representative scenario to validate this framework, given its fast pace and extensive physical contact. Collaborating with experts, we iteratively designed motion-related visualizations to improve the understanding of complex movements. A one-on-one matchup simulating real games was also provided, allowing users to compete directly with the reconstructed motions. Our user studies demonstrate that this method enhances participants' movement comprehension and engagement, while insights derived from interviews inform future immersive training designs.2025YWYihong Wu et al.Full-Body Interaction & Embodied InputHuman Pose & Activity RecognitionUIST
ViseGPT: Towards Better Alignment of LLM-generated Data Wrangling Scripts and User PromptsLarge language models (LLMs) enable the rapid generation of data wrangling scripts based on natural language instructions, but these scripts may not fully adhere to user-specified requirements, necessitating careful inspection and iterative refinement. Existing approaches primarily assist users in understanding script logic and spotting potential issues themselves, rather than providing direct validation of correctness. To enhance debugging efficiency and optimize the user experience, we develop ViseGPT, a tool that automatically extracts constraints from user prompts to generate comprehensive test cases for verifying script reliability. The test results are then transformed into a tailored Gantt chart, allowing users to intuitively assess alignment with semantic requirements and iteratively refine their scripts. Our design decisions are informed by a formative study (N=8) that explores user practices and challenges. We further evaluate the effectiveness and usability of ViseGPT through a user study (N=18). Results indicate that ViseGPT significantly improves debugging efficiency for LLM-generated data-wrangling scripts, enhances users’ ability to detect and correct issues, and streamlines the workflow experience.2025JZJiajun Zhu et al.Human-LLM CollaborationExplainable AI (XAI)Interactive Data VisualizationUIST
CAnnotator: Photo-Guided Color Annotation for Degraded Ancient PaintingsAncient paintings suffer irreversible color degradation due to aging and improper conservation. Labeling degraded paintings with authentic colors becomes vital to protect these valuable cultural heritages, which is challenging due to missing color information. Users typically need to investigate relevant photos to infer authentic colors and then validate these colors by mixing traditional pigments. However, such a task could be exhausting. To ease the difficulty, we propose an interactive visualization tool, namely CAnnotator, that streamlines efficient human-AI collaboration for the color annotation of degraded ancient paintings. CAnnotator consists of three views: a paint-annotation view, a photo-reference view, and a pigment-mixing view. Given an ancient painting, the paint-annotation view is developed to help users extract its color-degraded object textures that would be propagated to the relevant photos using a texture tracking model. Based on the tracking results, the photo-reference view provides texture-color and object-posture filters to explore the photos that include the given texture colors and object postures. We train a deep learning model to simulate the mixing of physical pigments and employ the chain rule to support progressive pigment mixture using a novel flow-based color visualization. We demonstrate the usage of CAnnotator through a use case and evaluate its effectiveness through model experiments and an in-lab user study. Compared to the baseline, CAnnotator could improve user confidence of labeled colors and foster user engagement at the cost of additional time.2025TTTan Tang et al.Museum & Cultural Heritage DigitizationInteractive Narrative & Immersive StorytellingUIST
ReSpark: Leveraging Previous Data Reports as References to Generate New Reports with LLMsCreating data reports is a labor-intensive task involving iterative data exploration, insight extraction, and narrative construction. A key challenge lies in composing the analysis logic-from defining objectives and transforming data to identifying and communicating insights. Manually crafting this logic can be cognitively demanding. While experienced analysts often reuse scripts from past projects, finding a perfect match for a new dataset is rare. Even when similar analyses are available online, they usually share only results or visualizations, not the underlying code, making reuse difficult. To address this, we present ReSpark, a system that leverages large language models (LLMs) to reverse-engineer analysis logic from existing reports and adapt it to new datasets. By generating draft analysis steps, ReSpark provides a warm start for users. It also supports interactive refinement, allowing users to inspect intermediate outputs, insert objectives, and revise content. We evaluate ReSpark through comparative and user studies, demonstrating its effectiveness in lowering the barrier to generating data reports without relying on existing analysis code.2025YTYuan Tian et al.Human-LLM CollaborationInteractive Data VisualizationData StorytellingUIST
VisMimic: Integrating Motion Chain in Feedback Video Generation for Motor CoachingAugmented video is a common medium for remote sports coaching, facilitating communication between trainees and coaches. Existing video augmentation techniques struggle to simultaneously convey both the overall motion dynamics and static key poses. This limitation hinders feedback comprehension in motor learning, making it difficult to understand where errors occur and how to correct them. To address this, we first reviewed popular video augmentation solutions. In collaboration with professional coaches, we integrated motion chain into feedback videos to combine key poses with motion trajectories. It supports multi-view observation and feedback explanation from overview to detail. To assist coaches in creating feedback videos, we present VisMimic, a human-AI interaction system that automatically analyzes trainee videos against reference movements, generates animated feedback, and enables customization. User studies show VisMimic's usability and effectiveness in enhancing motion analysis and communication for motor coaching.2025LCLiqi Cheng et al.Full-Body Interaction & Embodied InputHuman Pose & Activity RecognitionUIST
TableCanoniser: Interactive Grammar-Powered Transformation of Messy, Non-Relational Tables to Canonical TablesTableCanoniser is a declarative grammar and interactive system for constructing relational tables from messy tabular inputs such as spreadsheets. We propose the concept of axis alignment to categorise input types and characterise the expanded scope of our system relative to existing tools. The declarative grammar consists of match conditions, which specify repeating patterns of input cells, and extract operations, which specify how matched values map to the output table. In the interactive interface, users can specify match and extract patterns by interacting with an input table, or author more advanced specifications in the coding panel. To refine and verify specifications, users interact with grammar-based provenance visualisations such as linked highlighting of input and output values, tree-based visualisation of matching patterns, and a mini-map overview of matched instances of patterns with annotations showing where cells are extracted to. We motivate and illustrate our work with real-world usage scenarios and workflows.2025KXKai Xiong et al.Zhejiang University, State Key Lab of CAD&CGInteractive Data VisualizationPrototyping & User TestingCHI
RidgeBuilder: Interactive Authoring of Expressive Ridgeline PlotsRidgeline plots are frequently employed to visualize the evolution or distributions of multiple series with a pile of overlapping line, area, or bar charts, highlighting the peak patterns. While traditionally viewed as small multiple visualizations, their ridge-like patterns have increasingly attracted graphic designers to create appealing customized ridgeline plots. However, many tools only support creating basic ridgeline plots and overlook their diverse layouts and styles. This paper introduces a comprehensive design space for ridgeline plots, focusing on their varied layouts and expressive styles. We present RidgeBuilder, an intuitive tool for creating expressive ridgeline plots with customizable layouts and styles. In particular, we summarize three goals for refining the layout of ridgeline plots and propose an optimization method. We assess RidgeBuilder's usability and usefulness through a reproduction study and evaluate the layout optimization algorithm through anonymized questionnaires. The effectiveness is demonstrated with a gallery of ridgeline plots created by RidgeBuilder.2025SLShuhan Liu et al.State Key Lab of CAD & CG, Zhejiang UniversityInteractive Data VisualizationData StorytellingCHI
ProTAL: A Drag-and-Link Video Programming Framework for Temporal Action LocalizationTemporal Action Localization (TAL) aims to detect the start and end timestamps of actions in a video. However, the training of TAL models requires a substantial amount of manually annotated data. Data programming is an efficient method to create training labels with a series of human-defined labeling functions. However, its application in TAL faces difficulties of defining complex actions in the context of temporal video frames. In this paper, we propose ProTAL, a drag-and-link video programming framework for TAL. ProTAL enables users to define \textbf{key events} by dragging nodes representing body parts and objects and linking them to constrain the relations (direction, distance, etc.). These definitions are used to generate action labels for large-scale unlabelled videos. A semi-supervised method is then employed to train TAL models with such labels. We demonstrate the effectiveness of ProTAL through a usage scenario and a user study, providing insights into designing video programming framework.2025YHYuchen He et al.Zhejiang University, State Key Lab of CAD&CGInteractive Data VisualizationComputational Methods in HCICHI
StructVizor: Interactive Profiling of Semi-Structured Textual DataData profiling plays a critical role in understanding the structure of complex datasets and supporting numerous downstream tasks, such as social media analytics and financial fraud detection. While existing research predominantly focuses on structured data formats, a substantial portion of semi-structured textual data still requires ad-hoc and arduous manual profiling to extract and comprehend its internal structures. In this work, we propose StructVizor, an interactive profiling system that facilitates sensemaking and transformation of semi-structured textual data. Our tool mainly addresses two challenges: a) extracting and visualizing the diverse structural patterns within data, such as how information is organized or related, and b) enabling users to efficiently perform various wrangling operations on textual data. Through automatic data parsing and structure mining, StructVizor enables visual analytics of structural patterns, while incorporating novel interactions to enable profile-based data wrangling. A comparative user study involving 12 participants demonstrates the system's usability and its effectiveness in supporting exploratory data analysis and transformation tasks.2025YHYanwei Huang et al.Zhejiang University, State Key Lab of CAD&CGInteractive Data VisualizationTime-Series & Network Graph VisualizationVisualization Perception & CognitionCHI
Xavier: Toward Better Coding Assistance in Authoring Tabular Data Wrangling ScriptsData analysts frequently employ code completion tools in writing custom scripts to tackle complex tabular data wrangling tasks. However, existing tools do not sufficiently link the data contexts such as schemas and values with the code being edited. This not only leads to poor code suggestions, but also frequent interruptions in coding processes as users need additional code to locate and understand relevant data. We introduce Xavier, a tool designed to enhance data wrangling script authoring in computational notebooks. Xavier maintains users' awareness of data contexts while providing data-aware code suggestions. It automatically highlights the most relevant data based on the user's code, integrates both code and data contexts for more accurate suggestions, and instantly previews data transformation results for easy verification. To evaluate the effectiveness and usability of Xavier, we conducted a user study with 16 data analysts, showing its potential to streamline data wrangling scripts authoring.2025YZYunfan Zhou et al.Zhejiang University, State Key Lab of CAD&CGInteractive Data VisualizationComputational Methods in HCICHI
VisCourt: In-Situ Guidance for Interactive Tactic Training in Mixed RealityIn team sports like basketball, understanding and executing tactics---coordinated plans of movements among players---are crucial yet complex, requiring extensive practice. These tactics require players to develop a keen sense of spatial and situational awareness. Traditional coaching methods, which mainly rely on basketball tactic boards and video instruction, often fail to bridge the gap between theoretical learning and the real-world application of tactics, due to shifts in view perspectives and a lack of direct experience with tactical scenarios. To address this challenge, we introduce VisCourt, a Mixed Reality (MR) tactic training system, in collaboration with a professional basketball team. To set up the MR training environment, we employed semi-automatic methods to simulate realistic 3D tactical scenarios and iteratively designed visual in-situ guidance. This approach enables full-body engagement in interactive training sessions on an actual basketball court and provides immediate feedback, significantly enhancing the learning experience. A user study with athletes and enthusiasts shows the effectiveness and satisfaction with VisCourt in basketball training and offers insights for the design of future SportsXR training systems.2024LCLiqi Cheng et al.Full-Body Interaction & Embodied InputMixed Reality WorkspacesImmersion & Presence ResearchUIST
Understanding Nonlinear Collaboration between Human and AI Agents: A Co-design Framework for Creative DesignCreative design is a nonlinear process where designers generate diverse ideas in the pursuit of an open-ended goal and converge towards consensus through iterative remixing. In contrast, AI-powered design tools often employ a linear sequence of incremental and precise instructions to approximate design objectives. Such operations violate customary creative design practices and thus hinder AI agents' ability to complete creative design tasks. To explore better human-AI co-design tools, we first summarize human designers’ practices through a formative study with 12 design experts. Taking graphic design as a representative scenario, we formulate a nonlinear human-AI co-design framework and develop a proof-of-concept prototype, OptiMuse. We evaluate OptiMuse and validate the nonlinear framework through a comparative study. We notice a subconscious change in people's attitudes towards AI agents, shifting from perceiving them as mere executors to regarding them as opinionated colleagues. This shift effectively fostered the exploration and reflection processes of individual designers.2024JZJiayi Zhou et al.Zhejiang UniversityGenerative AI (Text, Image, Music, Video)Human-LLM CollaborationCreative Collaboration & Feedback SystemsCHI
VAID: Indexing View Designs in Visual Analytics SystemVisual analytics (VA) systems have been widely used in various application domains. However, VA systems are complex in design, which imposes a serious problem: although the academic community constantly designs and implements new designs, the designs are difficult to query, understand, and refer to by subsequent designers. To mark a major step forward in tackling this problem, we index VA designs in an expressive and accessible way, transforming the designs into a structured format. We first conducted a workshop study with VA designers to learn user requirements for understanding and retrieving professional designs in VA systems. Thereafter, we came up with an index structure VAID to describe advanced and composited visualization designs with comprehensive labels about their analytical tasks and visual designs. The usefulness of VAID was validated through user studies. Our work opens new perspectives for enhancing the accessibility and reusability of professional visualization designs.2024LYLu Ying et al.Zhejiang UniversityInteractive Data VisualizationVisualization Perception & CognitionCHI
Table Illustrator: Puzzle-based interactive authoring of plain tablesPlain tables excel at displaying data details and are widely used in data presentation, often polished to an elaborate appearance for readability in many scenarios. However, existing authoring tools fail to provide both flexible and efficient support for altering the table layout and styles, motivating us to develop an intuitive and swift tool for table prototyping. To this end, we contribute Table Illustrator, a table authoring system taking a novel visual metaphor, puzzle, as the primary interaction unit. Through combinations and configurations on puzzles, the system enables rapid table construction and supports a diverse range of table layouts and styles. The tool design is informed by practical challenges and requirements from interviews with 10 table practitioners and a structured design space based on an analysis of over 2,500 real-world tables. User studies showed that Table Illustrator achieved comparable performance to Microsoft Excel while reducing users' completion time and perceived workload.2024YHYanwei Huang et al.Zhejiang UniversityInteractive Data VisualizationData StorytellingCHI
PColorizor: Re-coloring Ancient Chinese Paintings with Ideorealm-congruent PoemsColor restoration of ancient Chinese paintings plays a significant role in Chinese culture protection and inheritance. However, traditional color restoration is challenging and time-consuming because it requires professional restorers to conduct detailed literature reviews on numerous paintings for reference colors. After that, they have to fill in the inferred colors on the painting manually. In this paper, we present PColorizor, an interactive system that integrates advanced deep-learning models and novel visualizations to ease the difficulties of color restoration. PColorizor is established on the principle of poem-painting congruence. Given a color-fading painting, we employ both explicit and implicit color guidance implied by ideorealm-congruent poems to associate reference paintings. To enable quick navigation of color schemes extracted from the reference paintings, we introduce a novel visualization based on a mountain metaphor that shows color distribution overtime at the ideorealm and imagery levels. Moreover, we demonstrate the ideorealm understood by deep learning models through intuitive visualizations to bridge the communication gap between human restorers and deep learning models. We also adopt intelligent color-filling techniques to accelerate manual color restoration further. To evaluate PColorizor, we collaborate with domain experts to conduct two case studies to collect their feedback. The results suggest that PColorizor could be beneficial in enabling the effective restoration of color-fading paintings.2023TTTan Tang et al.Generative AI (Text, Image, Music, Video)Data StorytellingMuseum & Cultural Heritage DigitizationUIST
AR-Enhanced Workouts: Exploring Visual Cues for At-Home Workout Videos in AR EnvironmentIn recent years, with growing health consciousness, at-home workout has become increasingly popular for its convenience and safety. Most people choose to follow video guidance during exercising. However, our preliminary study revealed that fitness-minded people face challenges when watching exercise videos on handheld devices or fixed monitors, such as limited movement comprehension due to static camera angles and insufficient feedback. To address these issues, we reviewed popular workout videos, identified user requirements, and came up with an augmented reality (AR) solution. Following a user-centered iterative design process, we proposed a design space of AR visual cues for workouts and implemented an AR-based application. Specifically, we captured users’ exercise performance with pose-tracking technology and provided feedback via AR visual cues. Two user experiments showed that incorporating AR visual cues could improve movement comprehension and enable users to adjust their movements based on real-time feedback. Finally, we presented several suggestions to inspire future design and apply AR visual cues to sports training.2023YWYingcai Wu et al.Fitness Tracking & Physical Activity MonitoringContext-Aware ComputingUIST
NaCanva: Exploring and Enabling the Nature-Inspired Creativity for ChildrenNature has been a bountiful source of materials, replenishment, inspiration, and creativity. Nature collage, as a crafting technique, offers children a fun and educational way to explore nature and express their creativity. However, the collection of raw material has been limited to static objects like leaves, ignoring inspiration from nature’s sounds and dynamic elements such as babbling creeks. To address this limitation, we have developed a mobile application with the aim of encouraging children’s creativity through renewed material collection and careful observation in nature. To explore the possibility of this approach, we conducted a formative study with children (N=20) and a design workshop with experts (N=6). With the results of these studies, we formulate NaCanva, an AI-assisted multi-modal collage creation system for children. Drawing upon the interactive relationship between children and nature, NaCanva facillitates a multi-modal material collection, including images, sound, and videos, which differs our system from traditional collages. We validated this system with a between-subject user study (N =30), and the results indicated that NaCanva enhances children’s multidimensional observation and engagement with nature, thereby unleashing their creativity in the creation of nature collages.2023ZYZihan Yan et al.Generative AI (Text, Image, Music, Video)Digital Art Installations & Interactive PerformanceFood Culture & Food InteractionMobileHCI
Notable: On-the-fly Assistant for Data Storytelling in Computational NotebooksComputational notebooks are widely used for data analysis. Their interleaved displays of code and execution results (e.g., visualizations) are welcomed since they enable iterative analysis and preserve the exploration process. However, the communication of data findings remains challenging in computational notebooks. Users have to carefully identify useful findings from useless ones, document them with texts and visual embellishments, and then organize them in different tools. Such workflow greatly increases their workload, according to our interviews with practitioners. To address the challenge, we designed Notable to offer on-the-fly assistance for data storytelling in computational notebooks. It provides intelligent support to minimize the work of documenting and organizing data findings and diminishes the cost of switching between data exploration and storytelling. To evaluate Notable, we conducted a user study with 12 data workers. The feedback from user study participants verifies its effectiveness and usability.2023HLHaotian Li et al.The Hong Kong University of Science and Technology, Microsoft Research AsiaHuman-LLM CollaborationData StorytellingCHI
EventAnchor: Reducing Human Interactions in Event Annotation of Racket Sports VideosThe popularity of racket sports (e.g., tennis and table tennis) leads to high demands for data analysis, such as notational analysis, on player performance. While sports videos offer many benefits for such analysis, retrieving accurate information from sports videos could be challenging. In this paper, we propose EventAnchor, a data analysis framework to facilitate interactive annotation of racket sports video with the support of computer vision algorithms. Our approach uses machine learning models in computer vision to help users acquire essential events from videos (e.g., serve, the ball bouncing on the court) and offers users a set of interactive tools for data annotation. An evaluation study on a table tennis annotation system built on this framework shows significant improvement of user performances in simple annotation tasks on objects of interest and complex annotation tasks requiring domain knowledge.2021DDDazhen Deng et al.Zhejiang University, Zhejiang UniversityHuman Pose & Activity RecognitionInteractive Data VisualizationCHI
HomeFinder Revisited: Finding Ideal Homes with Reachability-Centric Multi-Criteria Decision MakingFinding an ideal home is a difficult and laborious process. One of the most crucial factors in this process is the reachability between the home location and the concerned points of interest, such as places of work and recreational facilities. However, such importance is unrecognized in the extant real estate systems. By characterizing user requirements and analytical tasks in the context of finding ideal homes, we designed ReACH, a novel visual analytics system that assists people in finding, evaluating, and choosing a home based on multiple criteria, including reachability. In addition, we developed an improved data-driven model for approximating reachability with massive taxi trajectories. This model enables users to interactively integrate their knowledge and preferences to make judicious and informed decisions. We show the improvements in our model by comparing the theoretical complexities with the prior study and demonstrate the usability and effectiveness of the proposed system with task-based evaluation.2018DWDi Weng et al.Zhejiang UniversityGeospatial & Map VisualizationOnline Learning & MOOC PlatformsSmart Cities & Urban SensingCHI