"I Need to Find That One Chart": How Data Workers Navigate, Summarize and Communicate Analytical Conversations

Paper Title

"I Need to Find That One Chart": How Data Workers Navigate, Summarize and Communicate Analytical Conversations

Publication Info

• Topic area: Human-computer interaction and conversational interfaces for data analysis.
• Keywords: Analytical conversations, data workers, LLMs, navigation strategies, summarization, structured interfaces, AI collaboration, sensemaking, communication, user study.

Background and Problem

• Problem / challenge: Current conversational interfaces produce linear transcripts that fail to align with the iterative and nonlinear nature of real-world data analysis, making revisitation and summarization challenging.
• Significance: Effective navigation and communication of analytical conversations are critical for workflows involving sharing findings with stakeholders or handing off analyses, yet these tasks are poorly supported by existing tools.
• Motivation and related work: Prior research has explored steering LLMs, verifying outputs, and enhancing computational notebooks, but gaps remain in understanding how users revisit and communicate findings from analytical conversations. This paper builds on sensemaking frameworks and studies of computational notebooks to address these challenges.

Solution

• Proposed approach: SyncSense, a design probe that augments conversational interfaces with structured elements like filtering, multi-level navigation, and detail-on-demand to support revisitation and summarization.
• Novelty:
  1. Empirical observations from a user study with 10 participants on revisiting and summarizing analytical conversations.
  2. Development of an open-source design probe integrating structured navigation and summary authoring features.
  3. Design implications emphasizing structured elements for navigation and communication, and controllable AI assistance.
• Procedure and key techniques:
  • SyncSense organizes conversation elements into threads, speech acts, artifacts, and insights.
  • Provides synchronized panels for overview, detail exploration, and annotated raw conversations.
  • Includes an authoring panel for summary composition, allowing users to drag structured elements, refine summaries, and leverage AI for stylistic adjustments.

Results

• Concrete findings:
  • Analytical conversations averaged 29.5 turns and included diverse artifacts like code (22 turns/session), visualizations (9.4 turns/session), and data tables (7.2 turns/session).
  • Participants navigated conversations using strategies such as sequential navigation, abstractive navigation, visual recall, and filtering.
  • Authored summaries averaged 245 words, were structured with headings and bullet points, and prioritized insights and artifacts.
• Advantage over baselines: SyncSense enabled participants to efficiently orient, recall, and prioritize content, offering structured navigation and summary composition features absent in standard chat interfaces.
• Experiments / evaluation:
  • Two-session user study with 10 participants analyzing a dataset and revisiting their conversations after a week.
  • Tasks included summarizing for different audiences (e.g., non-technical executives vs. technical analysts) and formats (e.g., reports vs. messages).
• Limitations and future work:
  • Limited generalizability due to a single dataset and participant pool from one organization.
  • Need for validation of LLM-based extraction techniques and exploration of broader datasets and domains.
  • Future research should investigate causal links between navigation strategies and summary quality.

Summary

This study introduces SyncSense, a design probe for navigating and summarizing analytical conversations, addressing challenges in revisitation and communication. Empirical findings from a user study reveal recurring information needs (orienting, recalling, prioritizing) and strategies (sequential navigation, filtering, visual recall). Participants authored concise, structured summaries tailored to audience needs, emphasizing human control over AI-generated drafts. The research highlights opportunities for structured navigation, integrated communication workflows, and controllable AI assistance, offering implications for future conversational interfaces in data analysis.