Understanding Visual Investigation Patterns Through Digital "Field" Observations

Title of the Paper

Understanding Visual Investigation Patterns Through Digital “Field” Observations

Paper Information

• Subject Area: Visual Analytics and User Behavior Research
• Keywords: Visual Interaction, Visual Analytics, Information Visualization, User Behavior, Data Analysis Patterns, Data-Driven Insights, Collaborative Analytics

Research Background and Problem

• Identified Problems or Challenges: Existing research in the field of visual analytics primarily focuses on academic and scientific contexts, often conducted in simulated or isolated experimental environments. This disconnects from the user analysis needs in large enterprise settings, making it difficult to apply these findings to real-world business environments. These environments feature complex systems and user interaction ecosystems, alongside massive data volumes.
• Significance: Visual analytics is critical in enterprise environments as it helps users process large-scale data and derive meaningful business insights. However, current tools fail to effectively support users in large-scale tasks across multiple systems, and user behavior patterns remain poorly understood.
• Research Motivation and Related Work: This study aims to extend the existing literature by validating the applicability of known patterns in enterprise environments and uncovering new user behavior patterns, goals, and collaboration needs through digital “field” observations.

Solution

• Proposed Methods or Solutions:

  • Digital “field” observation method: Non-intrusive observation of users' real workflows via screen sharing to collect high-quality behavioral data.
  • Combining log analysis to extract user behavior patterns from another visual analytics tool and validate their generalizability.

• Innovations:

  • Digital “field” observation minimizes interference in user experiments, enabling more authentic observation of visual analytics behaviors in enterprise contexts.
  • Identified three previously unrecognized analysis patterns: Sidestep, Fanout, and Plug and Play.
  • Proposed five new themes related to collaboration and ecosystems, emphasizing the complexities of collaboration and user ecosystems beyond analytical tools.

• Implementation Steps and Key Techniques:

  1. Recorded 15 analysis tasks from 9 users across 8 teams and various data domains via screen sharing.
  2. Manually analyzed the data to generate sequence models and conducted open coding to extract behavioral patterns.
  3. Validated the patterns' applicability across different products using logs from another enterprise analytics tool (2530 sessions), employing the sequence pattern mining algorithm Closed FAST for data analysis.

Research Findings

• Specific Findings:

  • User Goals: Four core user analysis goals were identified:
    1. Examine anomalies or effects following known changes.
    2. Investigate data issues (why it deviates from expectations).
    3. Determine if data aligns with known patterns and take action.
    4. Prototype or optimize new metric definitions for others' use.
  • Behavioral Patterns: Six visual analysis patterns were discovered:
    • Drilldown (progressive filtering)
    • Sawtooth (repeated filtering and resetting)
    • Sidestep (removing prior filter views)
    • Fanout (opening multiple analysis branches simultaneously)
    • Plug and Play (loading existing templates for quick analysis)
    • Side-by-side (multi-instance comparison)
  • Collaboration and Ecosystem Observations: Five new themes were proposed:
    • Analytical work heavily relies on historical work.
    • Real-time collaboration is a critical component of analysis.
    • Analytical results often require review by others.
    • Post-analysis actions are necessary (e.g., code changes, documentation).
    • Fragmentation across products leads to frequent tool switching.

• Advantages: Compared to existing research, this study validated known behavioral patterns in real-world contexts and introduced three new patterns. Additionally, it revealed how users collaborate and operate within ecosystems in enterprise environments.

• Experimental and Evaluation Results:

  • Drilldown was the most common behavior (8920/9988 sequences), followed by Fanout and Sawtooth.
  • Sidestep, due to its low frequency (126/2530 sessions), could not be automatically identified through logs.

• Limitations and Future Directions:

  • Limitations: Data was sourced from a single company (Google), limiting generalizability to other organizations or industries; manually generated sequence models may be prone to technical errors; some patterns were not fully analyzed due to support thresholds.
  • Future Directions:
    • Explore how tools can better support users' actual collaboration and ecosystem needs.
    • Conduct in-depth studies of visual analytics behaviors and goals in other industries.
    • Develop more efficient log mining algorithms to identify low-frequency behavioral patterns.

Conclusion

This study used digital “field” observations and log analysis to uncover visual analytics behaviors in large enterprise environments, confirming the generalizability of some existing patterns while introducing three new patterns and five ecosystem themes. These findings provide critical insights for designing visual analytics tools that better meet user needs and offer guidance for future research directions.