Improving Steering and Verification in AI-Assisted Data Analysis with Interactive Task Decomposition
Authors
LLM-powered tools like ChatGPT Data Analysis, have the potential to help users tackle the challenging task of data analysis programming, which requires expertise in data processing, programming, and statistics. However, our formative study (n=15) uncovered serious challenges in verifying AI-generated results and steering the AI (i.e., guiding the AI system to produce the desired output). We developed two contrasting approaches to address these challenges. The first (Stepwise) decomposes the problem into step-by-step subgoals with pairs of editable assumptions and code until task completion, while the second (Phasewise) decomposes the entire problem into three editable, logical phases: structured input/output assumptions, execution plan, and code. A controlled, within-subjects experiment (n=18) compared these systems against a conversational baseline. Users reported significantly greater control with the Stepwise and Phasewise systems, and found intervention, correction, and verification easier, compared to the baseline. The results suggest design guidelines and trade-offs for AI-assisted data analysis tools.
Research Questions / Practical Problems
Question signals indexed for this paper.
- 100%
PleaSQLarify: Visual Pragmatic Repair for Natural Language Database Querying
CHI '26· Human-LLM Collaboration +2
- 100%
SCSimulator: An Exploratory Visual Analytics Framework for Partner Selection in Supply Chains through LLM-driven Multi-Agent Simulation
IUI '26· Human-LLM Collaboration +2
- 86%
VeriPlan: Integrating Formal Verification and LLMs into End-User Planning
CHI '25· Human-LLM Collaboration +2
- 86%
Lexara: A User-Centered Toolkit for Evaluating Large Language Models for Conversational Visual Analytics
CHI '26· Human-LLM Collaboration +2
- 71%
How Do Analysts Understand and Verify AI-Assisted Data Analyses?
CHI '24· Human-LLM Collaboration +2
- 71%
Jupybara: Operationalizing a Design Space for Actionable Data Analysis and Storytelling with LLMs
CHI '25· Human-LLM Collaboration +2
- 71%
Dango: A Mixed-Initiative Data Wrangling System using Large Language Model
CHI '25· Human-LLM Collaboration +2
- 71%
Results-Actionability Gap: Understanding How Practitioners Evaluate LLM Products in the Wild
CHI '26· Human-LLM Collaboration +2
- 71%
DataSpeck: An AI-Driven Human-in-the-Loop System for Automating Transformations in Data Conversion Workflows
CHI '26· Human-LLM Collaboration +2
- 71%
Cerebra: Aligning Implicit Knowledge in Interactive SQL Authoring
CHI '26· Human-LLM Collaboration +2
Based on Jaccard similarity of research subtopics & professions (≥60%)