Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparison

Document Title

Learning to Automate Chart Layout Configurations Using Crowdsourced Paired Comparison

Document Information

• Subject Area: Data Visualization and Machine Learning
• Keywords: Machine Learning, Data Visualization, Crowdsourcing, Visual Design, Chart Quality Assessment

Research Background and Problem

• Identified Problems:

  1. Existing automated chart tools (e.g., Excel) typically generate layout parameters based on predefined heuristic rules, which may result in suboptimal layouts.
  2. Users often need to manually adjust multiple parameters (e.g., chart size, spacing between charts) to improve layouts, a process that is neither systematic nor time-efficient and does not guarantee improved results.
  3. Research on chart layout quality is limited, with a lack of empirical studies based on large-scale user experiments.

• Significance: Charts are essential tools for data visualization, and their visual appeal impacts usability, memorability, and users' first impressions. Systematically quantifying layout quality and automating optimization can enhance the visual design quality of charts, providing users with more efficient solutions.

• Research Motivation:

  • To explore how to systematically optimize chart layout parameters by learning from human preference data.
  • To propose a novel machine learning model to replace traditional handcrafted rules for evaluating chart layout quality.

Solution

• Method Overview:

  • Propose a machine learning-based system for quantitatively assessing chart layout quality, named "Layout Quality Quantifier (LQ2)."
  • Collect user preference data on paired chart comparisons via a crowdsourcing platform and train the LQ2 model to learn scoring rules.
  • Combine optimization techniques to recommend improved layout parameters, thereby enhancing chart quality.

• Innovations:

  1. Introduced the use of paired comparisons to obtain more accurate and consistent user preference data.
  2. Employed adaptive strategies such as gradient sampling and importance sampling to improve the representativeness of training data.
  3. Modeled the chart layout parameter optimization problem as a learning-to-rank problem and applied machine learning methods to quantify human preferences.

• Implementation Steps:

  1. Data Collection: Randomly generate paired charts for comparison and collect user preference data from MTurk.
  2. Model Design and Training: Use a Siamese neural network to learn a scoring function for human layout preferences.
  3. Parameter Optimization: Apply the model to predict the layout parameter combination with the highest score.
  4. User Studies and Validation: Conduct two rounds of user experiments to verify whether the model's recommended results are visually more appealing than default and manually optimized layouts.

Research Outcomes

• Specific Results:

  1. The LQ2 model achieved 78% classification accuracy in predicting human preference rankings, outperforming traditional machine learning ranking methods and handcrafted rule-based evaluation metrics.
  2. User experiments confirmed that chart layouts generated by LQ2 were more popular on average compared to manually adjusted and default settings.

• Advantages Over Existing Solutions:

  • The automated layout recommendations not only improved visual quality but also significantly reduced the time required for manual adjustments.
  • Compared to predefined heuristic rules and simple machine learning models, LQ2 captured nonlinear features, better reflecting human preferences.

• Experiments and Evaluation Results:

  • User study results showed that LQ2-generated chart layouts received 59%-67% of preference votes across two user experiments (compared to default layouts).
  • Users required approximately 50 seconds to manually improve chart layouts, while LQ2 achieved similar results without human intervention.

• Limitations and Future Directions:

  1. Limitations:
     • The training dataset was limited in size and did not cover the entire design space.
     • The current study focused solely on optimizing bar charts, without considering other chart types.
     • The balance between user preferences and information communication efficiency has not been fully explored.
  2. Future Research Directions:
     • Extend the approach to more chart types and more complex multi-parameter scenarios.
     • Introduce real-time adaptive sampling and online learning strategies to improve model performance.
     • Combine cognitive efficiency and aesthetic preferences in future studies for more comprehensive optimization.

Conclusion

This study proposed an automated chart layout optimization method based on learning from public preferences, demonstrating the potential of integrating machine learning with chart aesthetic design. Experimental results validated the effectiveness of the method, providing insights for intelligent design in visualization tools and highlighting opportunities for future research in data-driven design and explainable AI.