"Explain What a Treemap is": Exploratory Investigation of Strategies for Explaining Unfamiliar Chart to the Blind and Low Vision Users

Title of the Paper

“Explain What a Treemap is”: Exploratory Investigation of Strategies for Explaining Unfamiliar Chart to Blind and Low Vision Users

Paper Information

• Research Area: Accessibility in data visualization, specifically strategies for explaining data visualizations to visually impaired users
• Keywords: Visualization accessibility, visually impaired users, explanation strategies, data visualization, treemaps, educational theory, assistive technology, user studies

Research Background and Problem

• Main Problem or Challenge: Data visualizations heavily rely on visual perception, making it difficult for visually impaired users to access the information effectively. Additionally, many assistive technologies and educational approaches focus on basic chart types (e.g., bar charts, line charts), neglecting strategies for explaining more complex visualizations. As data visualization types become increasingly diverse, visually impaired users face limitations in understanding complex visualizations such as treemaps or violin plots.
• Significance: The inability to interpret these visualizations restricts visually impaired users' equitable participation in the digital information landscape. Creating effective textual explanations can help bridge this gap and is more universally applicable compared to tactile or audio representations.
• Motivation and Related Work:
  • The study draws on educational theory, combining comparative prior knowledge, chart construction instructions, and concrete examples to design various explanation strategies.
  • Previous research has largely focused on converting visual charts into other modalities (e.g., tactile or audio), but these approaches have limitations and fail to fully support visually impaired users.
  • Current assistive technologies and education (e.g., K-12 curricula) primarily address basic chart types (e.g., bar and pie charts) and lack support for complex visualizations.

Proposed Solution

• Proposed Method or Solution:

  • Introduced textual explanations as a universal method to help visually impaired users understand complex visualizations.
  • Identified a three-dimensional framework for chart explanation strategies:
    1. Comparison with prior knowledge (with/without comparison).
    2. Type of knowledge (declarative/procedural).
    3. Level of abstraction (abstract/concrete).
  • Developed a prototype system capable of generating multiple textual explanations from given chart specifications, supporting over 50 chart types.
  • Conducted user studies to analyze the effectiveness of explanation strategies and user perceptions.

• Innovations:

  • Proposed a three-dimensional framework for chart explanations based on educational theory.
  • Automated the generation of chart explanations tailored to the needs of blind and low vision (BLV) users.
  • Introduced a similarity metric for chart types to aid in selecting familiar charts for comparative explanations.
  • Provided principles for designing effective explanation systems for visually impaired users.

• Implementation Steps and Key Technologies:

  • Defined chart type specifications, including marks, data types, encodings, and coordinate systems.
  • Built an algorithm to quantify the similarity between any two chart types, used as an input for generating explanations.
  • Designed textual explanation templates incorporating three dimensions and eight strategy combinations.
  • Collected feedback from BLV users through interviews and experiments, observing their drawing performance and question-answering accuracy.

Research Outcomes

• Specific Findings:

  • Declarative descriptions of chart appearances were generally more effective than procedural instructions for drawing charts.
  • Comparative explanations had varying effectiveness across users and chart types; for instance, they were less effective for highly complex charts (e.g., colored bubble charts) compared to non-comparative strategies.
  • The effectiveness of abstract versus concrete explanations varied by individual, with most users preferring concrete descriptions.
  • Individual characteristics (e.g., spatial ability and locus of control) significantly influenced comprehension performance.

• Advantages Over Existing Solutions:

  • Demonstrated the feasibility of automatically generating textual explanations.
  • Provided a systematic approach to applying educational theory to accessibility for visually impaired users.
  • User studies revealed pain points and improvement directions for existing assistive technologies, such as the need for more concrete examples, clearer terminology, and effective explanations for stacked marks.

• Experimental or Evaluation Results:

  • Users' performance in question-answering and drawing tasks was quantitatively evaluated, showing that declarative explanations outperformed procedural ones (with significantly higher mean question-answering scores).
  • Non-comparative strategies outperformed comparative ones for highly complex charts.
  • Visually impaired users felt more confident using "color as texture" to represent chart colors.
  • Certain chart-related terms (e.g., polar coordinate system terminology) were found to lack intuitive comprehensibility.

• Limitations and Future Directions:

  • The definition and measurement of chart complexity need refinement to better align with users' actual cognitive load.
  • Terminology selection requires further optimization to balance precision and ease of understanding.
  • The study did not cover complex types such as network graphs or 3D visualizations.
  • Future work should incorporate natural language processing techniques to enhance text generation quality and design more adaptive, user-friendly interfaces for selecting explanation strategies.