Do You See What I See? A Qualitative Study Eliciting High-Level Visualization Comprehension

Document Title

Do You See What I See? A Qualitative Study Eliciting High-Level Visualization Comprehension

Document Information

• Subject Area: Information Visualization and Graphical Understanding
• Keywords: Visualization, High-Level Comprehension, Communication Goals, Insight, Qualitative Study, Design Guidelines

Research Background and Problem

• Problems and Challenges:

  1. Existing studies on visualization effectiveness primarily focus on user performance in completing specific, narrowly defined tasks (e.g., estimating statistics). This paradigm struggles to capture users' holistic understanding naturally occurring in real-world contexts.
  2. Designers' communication goals often do not align with users' understanding of the visualization.
  3. There is a lack of systematic research on how high-level comprehension influences users' ability to construct insights.

• Importance:
  Clear and effective information visualization design can help users quickly extract meaningful patterns from complex data. However, whether these visualizations truly achieve their intended design goals remains an unresolved issue.

• Research Motivation and Related Work:
  The motivation lies in exploring how users naturally understand visualizations and uncovering potential barriers in the comprehension process. The study draws on rich theoretical and practical experiences ranging from graphical perception experiments to task-driven visualization evaluation.

Solution

• Methods and Solutions:
  The study proposes a qualitative experimental approach by asking participants to describe three major types of charts (scatter plots, bar charts, line charts) and recording their responses. The research focuses on two core questions:

  1. Does people's understanding align with the design intent?
  2. How do users naturally extract patterns and statistical information from charts in real-world settings?

• Research Innovations:

  1. Placing high-level visual comprehension at the core of evaluating visualization effectiveness, surpassing traditional task accuracy-driven research frameworks.
  2. Introducing a bottom-up to top-down connection perspective, emphasizing the exploration of users' natural understanding without task-driven guidance.

• Implementation Steps and Techniques:

  1. Extracting 60 chart samples from mainstream media and real-world data, reconstructing the charts to remove titles, descriptions, and other elements that might guide users.
  2. Using axial coding and thematic analysis to qualitatively analyze user responses, quantifying patterns and statistical tasks in their comprehension.

Research Results

• Key Findings:

  1. Theme 1: Misalignment between design intent and understanding—only about 41% of participants described content completely matching the design goals.
  2. Theme 2: Predicting users' natural understanding of visualizations through guided low-level tasks has limitations.
  3. Theme 3: Chart type alone cannot fully predict the information users extract from visualizations; data type, complexity, and chart structure also play significant roles.

• Specific Outcomes:

  1. Highlighted the importance of high-level visual comprehension as a critical consideration in developing universal design guidelines.
  2. Provided substantial insights for generating heuristic recommendations, such as the intuitive presentation of design intent in single-category non-comparative charts.

• Experiments and Evaluation:

  1. Collected 288 valid user response data points and conducted layered analysis on their alignment with target tasks.
  2. Data revealed that chart complexity (e.g., multi-category, multi-subplot comparisons) significantly impacts comprehension consistency.

• Advantages:
  Compared to traditional task completion-oriented studies, this research offers a multidimensional perspective on the comprehension process, particularly regarding the impact of design complexity and user background on understanding.

• Limitations and Future Directions:

  1. The study only selected three major types of visualizations; future research should expand to include maps, pie charts, and other types.
  2. Future studies should further refine sample populations (e.g., novice users) and explore the influence of educational and professional backgrounds on comprehension.
  3. Suggest incorporating more precise measurement tools, such as eye-tracking technology, to investigate the data features users focus on.

Conclusion and Significance

This study systematically reveals potential discrepancies between designers and users in visualization comprehension and provides evidence to guide designers in optimizing their visualization designs for more efficient information communication. Additionally, by combining high-level comprehension with low-level tasks in a bidirectional framework, the study introduces a new evaluation paradigm for the field of visualization research.