PleaSQLarify: Visual Pragmatic Repair for Natural Language Database Querying

Paper Title

PleaSQLarify: Visual Pragmatic Repair for Natural Language Database Querying

Publication Info

• Topic area: Interactive systems for resolving ambiguity in natural language database querying.
• Keywords: Text-to-SQL, pragmatic repair, natural language interfaces, ambiguity resolution, decision variables, user interaction, clustering, SQL disambiguation, visual interface, human-computer interaction.

Background and Problem

• Problem / challenge: Natural language database interfaces often fail to handle input ambiguity effectively, collapsing uncertainty into a single query without robust mechanisms for clarifying mismatches between user intent and system interpretation.
• Significance: Addressing ambiguity in natural language interfaces is critical for improving user control, enabling efficient query refinement, and ensuring accurate database interactions.
• Motivation and related work: Prior systems have attempted interactive disambiguation but often lack interpretability and fail to expose the action space effectively. Ambiguity taxonomies and benchmarks like CoSQL, SPLASH, and AMBROSIA highlight the prevalence of unresolved ambiguity in text-to-SQL tasks. Pragmatic theories of language use suggest incremental clarification as a natural strategy for resolving underspecification, which this paper applies to human-computer interaction.

Solution

• Proposed approach: PleaSQLarify, a system that operationalizes pragmatic repair for text-to-SQL disambiguation through an algorithm and a visual interface.
• Novelty:
  1. Conceptual framing of pragmatic repair for natural language interfaces, extending pragmatic inference theories to HCI.
  2. Algorithm for grouping atomic features into semantically interpretable decision variables, ranked by expected information gain.
  3. Visual interface that surfaces the model’s action space, supports guided clarification, and maintains traceability across turns.
  4. Empirical insights from a user study demonstrating the effectiveness of the approach and design implications for broader natural language interfaces.
• Procedure and key techniques:
  1. Generate a set of probable SQL queries using a language model.
  2. Cluster queries based on functional similarity.
  3. Extract and group atomic decision variables using lift and co-occurrence metrics.
  4. Rank decision variables by expected information gain to prioritize clarification.
  5. Iteratively filter and recluster queries based on user feedback until the intended query is isolated.

Results

• Concrete findings: Clustering-based methods reduced semantic uncertainty faster and achieved higher functional similarity within fewer turns compared to baselines. Median entropy declined rapidly, and functional coherence was achieved within 3–5 clarification steps.
• Advantage over baselines: Clustering-based strategies outperformed random and greedy baselines in terms of disambiguation efficiency and functional similarity convergence.
• Experiments / evaluation: Quantitative evaluation on the AMBROSIA dataset and a user study with 12 participants. Metrics included entropy reduction, functional similarity, and task completion rates (84.4% overall).
• Limitations and future work: Fixed candidate pools limited dynamic adaptation to user input, and usability was constrained by SQL literacy requirements. Future work could explore adaptive resampling, integration for non-technical users, and scalability to larger databases.

Summary

PleaSQLarify introduces a pragmatic repair framework for resolving ambiguity in text-to-SQL tasks, combining an algorithm that prioritizes informative decision variables with a visual interface that surfaces the action space and supports incremental clarification. Quantitative evaluation and user studies demonstrated its ability to efficiently reduce semantic uncertainty, enhance user control, and uncover alternative interpretations. The system’s design principles and workflows highlight its applicability to broader natural language interfaces, though future work is needed to address scalability and accessibility for non-technical users.