Tessera: Discretizing Data Analysis Workflows on a Task Level

Document Title

Tessera: Discretizing Data Analysis Workflows on a Task Level

Document Information

• Subject Area: Data Analysis, Visual Analytics, User Interaction Behavior Modeling
• Keywords: Data Analysis, Interaction Log Analysis, Visualization, User Behavior Mining, Task Segmentation, Data Coverage, Goal-Oriented Activities

Research Background and Issues

• Identified Problems or Challenges:

  • Analyzing user behavior logs in data analysis is highly complex, with significant differences at individual and session levels, resulting in large-scale logs that are difficult to parse.
  • Traditional methods for log capture and analysis fail to accurately map user cognition, often leading to imprecise observations.
  • Existing hierarchical or graphical models have limitations in terms of granularity, interpretability, or handling large-scale data.
  • In exploratory visual analytics (EVA), analysts often struggle to track their progress and are prone to rapid iteration and cognitive biases.

• Significance:

  • Understanding user goals and task switching to construct high-level abstract information about the analysis process is crucial for improving tool design, guiding user behavior, and identifying failure points.
  • Current methods do not adequately address the need for segmenting user tasks and subtasks.

• Research Motivation and Related Work:

  • Motivation: To quantify user behavior logs and discretize them into goal-oriented task blocks, thereby uncovering the structure of the analysis process and changes in user goals.
  • Related Work: Previous studies have focused on interaction log analysis and the association between low-level and high-level goals in visualization systems, but typical methods still face challenges in decoding high-level abstractions and task segmentation.

Solution

• Proposed Method or Solution:

  • A framework called Tessera is proposed, which discretizes exploratory visual analytics (EVA) logs based on user interaction logs, data queries, and time windows, extracting goal-oriented task segments from user behavior.
  • The focus is on integrating user activities, data exposure, and data transformation through event logs to identify goal segmentation in analysis.

• Innovations:

  • Compared to traditional graphical models and hierarchical methods, this approach improves task-switching accuracy and efficiency.
  • Logical segmentation of fine-grained tasks is achieved using statistical distance calculations, data coverage comparisons, and similarity scores.
  • Recognizes iterative patterns (revisits or repetitions) in analyst behavior to support iterative improvements.

• Implementation Steps and Key Techniques:

  1. Data Transformation Representation:
     • Constructs formal SQL query representations, capturing user-selected data points, focused attributes, and analytical functions to infer user tasks and changes in data coverage.
  2. Attribute and Data Comparison:
     • Compares subsets of log events using coverage distance and statistical distribution distance (e.g., Z-test and Chi-square test).
  3. Analytical Function Similarity Evaluation:
     • Calculates symbolic similarity between functions based on mathematical derivation paths and user intentions (extracted from user behavior intention lists during research interviews).
  4. Task Segmentation within Time Windows:
     • Introduces decay-based weighting to determine whether events within the same time window pertain to the same goal based on similarity scores.
  5. Loop Behavior Recognition:
     • Detects revisit patterns to specific data states without directly marking task switches.

Research Outcomes

• Specific Results:

  • Developed the innovative concept of goal-directed behavior segments.
  • Provided comparative validation of the framework's performance against existing models.
  • Released a comprehensive dataset covering open-ended and goal-oriented analysis thought processes.

• Comparison with Existing Solutions:

  • Compared to hierarchical methods, Tessera improves task-switching detection accuracy, with an F-score increase of approximately 0.15.
  • Avoids instability issues caused by large-scale data in graphical methods and demonstrates robust results.
  • Tessera achieves the highest log compression efficiency in experiments.

• Experimental or Evaluation Results:

  • In task segmentation accuracy, experiments conducted on two datasets (comparing Tessera with Hierarchical and Graph Models):
    • Achieved significant improvements in precision and recall.
    • Demonstrated good scalability with respect to data size and complexity.
  • Sensitivity experiments on time window K show that model accuracy quickly converges to reliable results, supporting real-time or iterative applications.

• Limitations and Future Directions:

  • Limitations:
    • Tessera's complexity may limit its application at large scales; balancing time window K and efficiency is necessary.
    • Limited adaptability to different interaction tools or usage scenarios, requiring specific adjustments.
    • Dataset diversity is limited, necessitating validation in more varied contexts to assess generalizability.
  • Future Directions:
    • Explore the model's classification capabilities for specific task labels (e.g., hypothesis testing, deep analysis).
    • Integrate with current analysis tools, such as Tableau, to enhance real-time feedback and historical traceability for analysts.
    • Further apply the framework to other domains, such as crowdsourced workflow behavior modeling or edit log analysis.