Snowy: Recommending Utterances for Conversational Visual Analysis

Document Title

Snowy: Recommending Utterances for Conversational Visual Analysis

Document Information

• Domain: Natural Language Interfaces (NLIs) in Data Visualization, Recommender Systems, and Human-Computer Interaction Design
• Keywords: Natural Language Recommendation, Linguistic Pragmatics, Deictic Statements, Context, Data Interestingness

Research Background and Problem

• Identified Problems or Challenges:

  1. Using natural language interfaces (NLIs) for data visualization analysis still faces two major challenges:
     • Lack of analytical guidance (i.e., users are unsure which data features to start with).
     • Poor discoverability of natural language inputs (users are uncertain how to construct input statements).
  2. Even with improvements in NLIs' language understanding capabilities, users still struggle to construct appropriate query statements during analysis.
     • Users need to understand the characteristics of the data domain and identify potential analysis patterns.
     • The system's language understanding capabilities require users to adjust their input statements to specific syntaxes.

• Significance:

  1. When exploring data, the absence of analytical guidance or ineffective query construction can disrupt the user's analytical workflow.
  2. If these issues remain unresolved, they may hinder the widespread adoption of NLIs in visual analysis.

• Research Motivation and Related Work:

  1. Existing visualization recommendation tools often focus on data visual encoding or data interestingness but overlook the discoverability of natural language inputs.
  2. Current natural language interfaces (NLIs) also lack specific optimizations for helping users learn language construction and providing analytical guidance.
  3. The authors aim to address these issues by introducing a recommendation system to enhance the effectiveness of data analysis.

Solution

• Method or Solution: The authors propose a prototype system called "Snowy," which addresses the problem of analytical guidance by recommending relevant natural language expressions (referred to as recommended utterances) while also helping users understand the language structures supported by the system.

• Innovations:

  1. Combines data interestingness metrics with principles of linguistic pragmatics to generate and recommend utterances.
  2. Supports three types of recommended utterances:
     • New analytical directions (new queries).
     • In-depth analysis of the current data view (follow-up utterances).
     • Data-based interactive operations (e.g., deictic statements).
  3. Implements contextual recommendations through conversational centers, seamlessly integrating with the user's interaction state.

• Implementation Techniques and Steps:

  1. Data Analysis: Uses data interestingness metrics (e.g., correlation, standard deviation) to calculate potentially interesting attribute combinations.
  2. Utterance Generation: Generates parameterized utterances based on contextual state, historical interaction records, and analytical goals.
  3. Language Realization: Uses templates to convert analytical goals and data parameters into diverse natural language expressions.
  4. Recommendation Engine: Adjusts recommended content in real-time based on the user's current contextual state, providing targeted utterances.

Research Outcomes

• Specific Results:

  1. Snowy supports users in querying data through natural language while providing targeted analytical guidance.
  2. The system-generated recommendations help users construct meaningful analytical statements and guide data exploration.
  3. Through utterance recommendations, users gradually become familiar with the language structures supported by the system.

• Advantages and Comparisons:

  1. Compared to traditional visualization recommendation systems, Snowy not only derives insights from data patterns but also enhances language discoverability.
  2. Similar natural language systems typically do not focus on how utterance recommendations support language learning, whereas Snowy incorporates this design.

• Experimental or Evaluation Results:

  1. In preliminary user studies, participants generally found the recommended utterances helpful for guiding data exploration.
  2. Recommended utterances assisted users in learning how to interact with the system using natural language while providing analytical directions.
  3. In open-ended data exploration, recommendations were more effective than directed analysis in assisting users.

• Limitations and Future Directions:

  1. The current system supports only a limited range of visualization types and analytical goals, requiring further expansion of supported functionalities.
  2. Further research is needed to investigate the specific impact of recommended utterances on user workflows and learning efficiency, such as comparisons with baseline systems.
  3. The system needs to optimize the interpretability of recommended content to help users understand the logic behind the recommendations.
  4. Future exploration could apply utterance recommendations to new domains, such as voice interactions or chatbots.

Through structured design and preliminary validation, this paper presents a practical tool and design framework, demonstrating how natural language recommendations can enhance user experience and guide user cognition in data analysis.