DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing

Title of the Paper

DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing

Paper Information

• Subject Area: Data Visualization, User Interface Design, Human-Computer Interaction
• Keywords: Visualization Editing, Dynamic User Interface, Natural Language Interface, Large Language Models, Interactive Tools, User Experience Design, Dynamic Widgets, Visualization Evaluation, LLM Applications

Research Background and Problem

• Problems and Challenges:

  • Modern interactive visualization tools (e.g., Tableau, PowerBI) simplify the creation of high-level data visualizations, but in-depth editing requires users to navigate complex GUIs or master low-level visualization specifications, which is highly challenging for non-expert users.
  • Natural Language Interfaces (NLIs) can significantly reduce the burden of using complex GUIs, but they overlook the advantages of traditional GUIs, such as instant visual feedback, fine-tuning controls, and repeatable operations.
  • Users may feel frustrated when using tools due to overly complex GUIs or limited customization options.

• Significance:

  • Data visualization is a core component of data analysis and presentation. Providing user-friendly visualization editing tools can significantly lower the knowledge threshold, enabling users to complete tasks more efficiently and accurately.

• Research Motivation and Related Work:

  • This study builds on the composability and declarative visualization syntax of modern tools (e.g., Vega-Lite) and combines the advantages of natural language and dynamic interaction to develop a hybrid tool that offers precise editing capabilities along with point-and-click interaction effects.
  • Related research has extensively explored natural language visualization (V-NLI) and dynamic interactive interfaces, such as SUPPLE and EVIZA. However, these approaches often generate static or overly NLI-dependent solutions. This study aims to strike a balance between the two.

Solution

• Method or Solution:

  • A novel interaction method is proposed: Dynamic Widgets. This method integrates Natural Language Interfaces (NLIs) with dynamically generated GUI widgets, providing users with a simple yet powerful visualization editing experience.
  • A tool, DynaVis, was developed to allow users to describe editing tasks or directly command the generation of widgets for dynamic visualization editing.

• Innovations:

  1. Interaction Design: Innovatively combines natural language and dynamically generated widgets to provide users with a multimodal, highly flexible interaction mode.
  2. Backend Support: Utilizes Large Language Models (LLMs) to dynamically generate UI from user input while leveraging LLMs' capabilities in natural language processing and dynamic interface generation.
  3. System Performance Optimization: Dynamic widgets are modular, can be persisted, and reused in subsequent editing tasks.
  4. Task Efficiency Enhancement: Dynamically synthesized components support coordinated multi-attribute editing, providing instant visual feedback and fine-tuning capabilities.

• Implementation Steps and Key Technologies:

  • Widget Design: Each widget consists of an HTML interface and JavaScript callback functions, responsible for user interaction and dynamic modification of data visualization graphics, respectively.
  • Data Context Summarization: Provides highly abstracted data and visualization context to avoid overloading the context window.
  • LLM Synthesis Architecture:
    • Uses LLMs (GPT-3.5 or GPT-4) to translate natural language input into JSON-formatted Vega-Lite specifications or dynamic UI component code.
    • Performs backend error checking and retries to ensure the correctness of generated code.
  • Dynamic Interface Management: Widgets are arranged in reverse order of generation to avoid information overload while allowing persistence, modification, and deletion of components.

Research Outcomes

• Specific Outcomes:

  • Successfully developed the DynaVis system, which supports dual-modal interaction through natural language and GUI widgets for chart editing and customization.
  • Dynamically synthesized widgets accurately capture user needs, reducing task barriers and lowering the probability of user retries and errors.
  • The system's superiority was validated through user studies, particularly in complex exploratory tasks and high-frequency editing scenarios.

• Comparative Advantages Over Existing Solutions:

  • Compared to chart editing tools solely based on NLI, DynaVis offers stronger interactivity and instant feedback.
  • Dynamic widgets significantly enhance the usability of repetitive tasks, enabling users to quickly adjust multiple parameter combinations.
  • The system is also compatible with multimodal user needs, flexibly addressing complex or uncertain user requirements.

• Experimental or Evaluation Results:

  • User Study Methodology: In the experiment, DynaVis was compared with baseline tools (NLI-based static UIs). Twenty-four participants completed two sets of visualization editing tasks, and their preferences, efficiency, and cognitive load were surveyed.
  • Quantitative Analysis:
    • Participants' task failure rate was significantly reduced when using DynaVis compared to the baseline tools.
    • The frequency of editing operations completed using widgets increased substantially, while the reliance on natural language commands decreased correspondingly.
  • Qualitative Feedback: Users generally agreed that the persistence, instant feedback, and repeatability of widgets significantly improved the system's usability and practicality.

• Limitations and Future Directions:

  1. Continuously changing dynamic interfaces may increase cognitive load during long-term editing.
  2. Further expansion is needed in supporting low-level visualization editing, domain-specific constraints, and accessibility design.
  3. Explore more multimodal interaction possibilities, such as integrating touch, gestures, or voice commands.
  4. For complex editing tasks, further optimization is required to refine the JSON specifications generated by LLMs to avoid redundancy or errors.
  5. The applicability of the approach in other domains, such as video editing and document editing, warrants further exploration.