mTSeer: Interactive Visual Exploration of Models on Multivariate Time-series Forecast

Title of the Paper

mTSeer: Interactive Visual Exploration of Models on Multivariate Time-series Forecast

Paper Information

• Domain: Visualization, Machine Learning, Multivariate Time-series Forecasting
• Keywords: Visualization, Machine Learning, Multivariate Time-series, Model Evaluation, Forecasting, Feature Extraction

Research Background and Problem

• Challenges:
  • Time-series forecasting models exhibit inconsistent performance across different data spaces and application domains, making it difficult to select the appropriate model.
  • Accuracy-based error metrics fail to reveal deeper model performance characteristics, such as key feature identification and the impact of temporal factors.
  • There is a lack of instance-level analysis of model details and insufficient involvement of domain experts in the evaluation process.
• Significance:
  • Time-series forecasting provides critical information for industrial and institutional decision-making, and the evaluation and interpretation of model accuracy directly affect decision quality.
• Motivation and Related Work:
  • Existing visualization systems often focus on specific forecasting models or handle only univariate time-series, lacking cross-model comparisons for general multivariate models.
  • Inherent patterns of time-series data (e.g., periodicity, abrupt changes) are insufficiently considered, and associations with anomalous data during model evaluation are underexplored.

Solution

• Methodology and Innovations:
  • An interactive visualization system, mTSeer, is proposed to support domain experts in interpreting and evaluating multivariate time-series forecasting models at the instance level.
  • The system integrates periodicity detection, abrupt change detection algorithms, and intuitive interaction and visualization designs to enable more rational model evaluation.
• Implementation Steps and Key Techniques:
  1. Data Preprocessing:
     • Filter missing values, transform categorical values, and normalize data.
  2. Model Construction:
     • Provide various candidate forecasting models, including linear regression, nonlinear regression, and deep learning models (e.g., LSTM, CNN).
     • Use periodicity detection algorithms to determine the input time-series length.
  3. Result Interpretation and Analysis:
     • Analyze feature importance using SHAP values.
     • Identify anomalous prediction results using anomaly detection algorithms.
  4. Visualization and Interaction:
     • Offer a multi-view interface (data selection, parameter settings, model overview, instance analysis, etc.) to support fine-grained model comparison and evaluation.

Research Findings

• Specific Outcomes and Advantages:
  • The mTSeer system provides a comprehensive method for evaluating forecasting models from overall to instance-level perspectives.
  • Compared to existing solutions, it demonstrates stronger feature interpretability and user-friendly interaction design for domain expert involvement.
• Experimental and Evaluation Results:
  • Three case studies based on real-world datasets validate the system's effectiveness.
  • Quantitative experiments reveal that model forecasting performance is significantly influenced by input length and forecasting horizon, with LSTM performing best in most scenarios.
  • Two domain experts confirmed the system's effectiveness in model interpretation and evaluation, particularly in visualizing the relationship between feature importance and prediction results.
• Limitations and Future Directions:
  • The current design involves a limited number of features, making it challenging to visualize features in high-dimensional datasets.
  • The system has limited functionality for comparing results from multiple parameter tuning iterations for a single model.
  • Future work will focus on incorporating automated analysis methods to enhance interpretability and exploration efficiency, as well as investigating active learning systems to dynamically update evaluation results.

This paper highlights the exploration of time-series forecasting models through mTSeer, showcasing the powerful support that visualization design provides for complex model evaluation and feature interpretation, as well as the potential for integrating domain knowledge to achieve high-quality decision support.