A Human Information Processing Theory of the Interpretation of Visualizations: Demonstrating Its Utility

Title of the Paper

A Human Information Processing Theory of the Interpretation of Visualizations: Demonstrating Its Utility

Paper Information

• Research Domain: Intersection of information visualization and cognitive science, focusing on constructing cognitive models of visualization users.
• Keywords: Graphical visualization interpretation, human memory structure, cognitive load, cognitive processes, information processing, RISN modeling, information retrieval, visualization effect analysis

Research Background and Issues

• Problems and Challenges:
  • Current research in information visualization lacks in-depth analysis of how users interpret graphical representations, with limited studies on user cognitive models.
  • The ability to describe complex cognitive processes underlying visual presentations in a linear manner needs improvement.
  • There is a lack of theoretical frameworks integrating human cognition into the evaluation of information visualization.
• Significance:
  • Understanding how users process and construct cognitive models of visualization information can improve interface design, educational programs, and visualization practices.
  • Analyzing the memory structures users construct during interpretation provides a theoretical basis for designing more efficient visualization tools.
• Research Motivation and Related Work:
  • The study adopts the "Representation Interpretive Structure Theory" (RIST) proposed by the authors, along with its derived graphical annotation language (RIS-Notation, RISN) and modeling tool (RIS-Editor, RISE), to systematically model how users construct cognitive interpretations of visualizations.
  • Compared to traditional visual design guidelines and cognitive models, RIST is designed as a more flexible and comprehensive cognitive analysis tool.

Solution

• Methods or Solutions:
  • RIST theory explains how users construct cognitive models through multi-layer associations between external visualization objects and internal memory representations.
  • RISN is developed as a graphical language to describe the interpretive structure of visualizations.
  • A browser-based modeling tool, RISE, is provided to create graphical models aligned with RIST theory.
• Innovations:
  • Treats external graphical objects and internal cognitive representations equally, constructing hierarchical and richly structured interpretive models.
  • RISN maps multi-level associations between visual elements and cognitive concepts, differing from traditional single-symbol matching methods.
• Implementation Steps and Techniques:
  • RISN is used to construct cognitive models for four visualization scenarios, including comparisons between Sankey diagrams and Chord diagrams, and graphical versus sentence-based representations of pulley system problems.
  • Cognitive demands of different visualization designs are analyzed and quantified by summarizing comparative standards such as model depth and structural complexity.

Research Outcomes

• Specific Results:
  • All four cases successfully demonstrated the utility of the RIST model: RISN model complexity indicators accurately predicted user task performance (e.g., efficiency and error rate).
  • Comparative analysis of Sankey diagrams and Chord diagrams revealed differences in cognitive load stemming from model structures (e.g., hierarchy and association chains).
  • For the pulley problem in graphical versus sentence formats, RISN showed that graphical information storage structures are more compact, leading to higher indexing efficiency.
• Advantages Comparison:
  • Compared to traditional visual analysis methods, RIST not only considers graphical features but also deeply analyzes how users understand information at the cognitive level.
  • RISN reveals the impact of different visualizations on human cognitive processes, providing deeper insights into the cognitive mechanisms behind task performance.
• Experimental or Evaluation Results:
  • In all cases, visualization designs corresponding to RISN models with low complexity, high homogeneity, and shallow depth showed significantly better task performance.
  • Simulations of human information search paths during tasks further validated RISN's potential for predicting cognitive behaviors.
• Limitations and Future Directions:
  • Current evaluation metrics cannot directly derive absolute estimates of specific task performance (e.g., time cost or error rate); future work should develop more detailed efficiency evaluation methods for RISN models.
  • Modeling the diversity of visualization interpretations requires broader participation from researchers to reduce potential biases of single models.
  • Potential directions for RIST theory include its use as a tool to analyze user misunderstandings, predict cognitive factors such as visual literacy burden or memorability, and establish indicator systems.