How Can Deep Neural Networks Aid Visualization Perception Research?: Three Studies on Correlation Judgments in Scatterplots

Title of the Paper

How Can Deep Neural Networks Aid Visualization Perception Research? Three Studies on Correlation Judgments in Scatterplots

Paper Information

• Subject Area: Visualization perception research and applications of deep neural networks
• Keywords: Deep neural networks, visualization features, predictive modeling, perception, scatterplots

Research Background and Problem

• What problems or challenges did the authors identify?

  • How to improve the accuracy of predicting human perception data.
  • Limitations of current regression-based models in explaining and predicting perception.
  • Lack of extensive datasets and studies on model transferability for correlation perception.

• Why is this problem important?

  • Visualization design and data analysis increasingly rely on perception. Accurately modeling the correlation perception process is crucial for automated visualization design and optimization.
  • The powerful image processing capabilities of deep neural networks present opportunities for potential research.

• Motivation and related work:

  • Existing research on scatterplot correlation perception focuses on the accuracy and interpretability of "comparing correlations across different scatterplots."
  • Advances in deep learning suggest that neural networks can extract complex image features, prompting the authors to explore how these models can provide new insights into visualization perception.

Solutions

• What methods or solutions did the authors propose?

  • Three experimental studies to evaluate the performance of deep convolutional neural networks in perception prediction, model transferability, and feature learning:
    1. Study 1: Comparing the prediction accuracy of various deep convolutional neural network architectures with traditional regression models.
    2. Study 2: Evaluating the generalization performance of networks on two different scatterplot design datasets.
    3. Study 3: Using neural network feature visualization techniques to interpret the visualization features learned by the models.

• What is innovative about this solution?

  • Directly processing visualization images to extract human perception characteristics, avoiding the limitations of traditional models that rely on expert-defined features.
  • Providing experimental validation of deep learning models' transferability to new datasets and dynamic explanations of human perception processes.

• What are the implementation steps? What key techniques were used?

  • Experiment design and data collection: Correlation comparison tasks involving tens of thousands of participants, generating three datasets with different styles.
  • Neural network architecture evaluation: Including various CNN architectures (e.g., VGG, ResNet, EfficientNet).
  • Data splitting and model training: Sampling training, validation, and test sets, combined with standard machine learning techniques (e.g., SGD optimizer).
  • Feature analysis: Using feature visualization techniques to extract features learned by the models.

Research Outcomes

• What specific results were achieved?

  • Several deep convolutional neural networks successfully achieved high-accuracy predictions of scatterplot perception data, with some architectures (e.g., VGG-19) performing comparably to traditional regression analysis.
  • The VGG-19 model demonstrated strong generalization ability during transfer, accurately predicting correlation judgments on different datasets.
  • Extracted visual features learned by the models, including "point features" and "correlation linear features," were validated to align with features defined in the literature.

• What advantages does it have compared to existing solutions?

  • Broader feature interpretation: The models identified visualization features not previously studied.
  • Strong generalization ability: Pre-trained models can predict new datasets without re-annotation or retraining.
  • Eliminates reliance on expert assumptions: Directly extracts information from visualization images.

• What were the experimental or evaluation results?

  • Study 1: The VGG-19 architecture achieved a prediction accuracy of 76.6%, comparable to top regression models based on "predicted ellipse area."
  • Study 2: The VGG-19 model's accuracy on new datasets was only slightly lower than models trained from scratch, demonstrating transfer potential.
  • Study 3: Visualized features revealed that the model likely focused on density information, correlation patterns, and point spatial distribution.

• Limitations and future directions:

  • Limitations:
    • The study's applicability is constrained by its single-task focus and fixed scatterplot designs.
    • Current interpretability of model feature extraction is still limited.
    • Neural network training requires significant time and computational resources.
  • Future directions:
    • Explore other types of visualization tasks, such as mean estimation and category separation.
    • Investigate the transferability of models on larger-scale datasets.
    • Design neural network architectures specifically for visualization tasks to reduce parameter redundancy.

Summary and Insights

This study demonstrates the potential of deep neural networks in the field of visualization perception. Through three experiments, the authors systematically evaluated the predictive capabilities, generalization performance, and feature interpretation of different network models. This not only validates the feasibility of deep learning techniques for complex perception tasks but also provides a method for directly extracting human perception mechanisms from visual images. This has significant implications for automated visualization design and optimization and points to future research directions, such as developing specialized algorithm architectures and exploring broader visual perception tasks.