AmbigChat: Interactive Hierarchical Clarification for Ambiguous Open-Domain Question Answering

DOI

When conversing with large language models, it is common that users ask an ambiguous open-domain question that could lead to multiple answers, especially when exploring new topics. For example, “Who won the US Open?” can result in different athletes according to the referred events and years. We propose AmbigChat, an automated approach that hierarchically disambiguates a factual question and guides users to navigate answers via UI widgets in a multi-turn conversational interface. Using the ambiguity taxonomy we generated from an analysis of 5,000 queries, AmbigChat identifies ambiguous facets of a question and constructs a disambiguation tree, where each level corresponds to a facet. Users can traverse the tree to explore answers via interactive disambiguation widgets and expand the conversation by referencing tree nodes through drag-and-drop. We developed our interaction design with six design professionals and demonstrate the effectiveness of the disambiguation tree generation algorithm on a variety of factual queries. Our evaluation with 16 participant showed that AmbigChat not only helped participants find answers more easily and efficiently, but also facilitated structured explorations of the topic space.