Trends, Challenges and Processes in Conversational Agent Design: Exploring Practitioners' Views through Semi-Structured Interviews
Authors
The aim of this study is to explore the challenges and experiences of conversational agent (CA) practitioners in order to highlight their practical needs and bring them into consideration within the scholarly sphere. A range of data scientists, conversational designers, executive managers and researchers shared their opinions and experiences through semi-structured interviews. They were asked about emerging trends, the challenges they face, and the design processes they follow when creating CAs. In terms of trends, findings included mixed feelings regarding no-code solutions and a desire for a separation of roles. The challenges mentioned included a lack of socio-technical tools and conversational archetypes. Finally, practitioners followed different design processes and did not use the design processes described in the academic literature. These findings were analyzed to establish links between practitioners' insights and discussions in related literature. The goal of this analysis is to highlight research-practice gaps by synthesising five practitioner needs that are not currently being met. By highlighting these research-practice gaps and foregrounding the challenges and experiences of CA practitioners, we can begin to understand the extent to which emerging literature is influencing industrial settings and where more research is needed to better support CA practitioners in their work.
Research Questions / Practical Problems
Question signals indexed for this paper.
Research Questions
3- What technical and sociotechnical challenges in conversational agent (CA) design pose primary barriers for practitioners?Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
- Which conversational agent design innovations from academic research have not been effectively adopted by practitioners?Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
- How can the research-practice gap in conversational agent design be bridged?Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
Practical Problems
1- Practitioners lack standardized processes and technical support tools when designing conversational agents.Category: Conversational Agent and Chatbot DesignSimilar questionsarrow_forward
- 67%
How do you Converse with an Analytical Chatbot? Revisiting Gricean Maxims for Designing Analytical Conversational Behavior
CHI '22· Conversational Chatbots +2
- 67%
Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text)
CHI '23· Conversational Chatbots +2
- 67%
A Human-Computer Collaborative Editing Tool for Conceptual Diagrams
CHI '23· Human-LLM Collaboration +1
- 67%
Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation
CHI '24· Human-LLM Collaboration +1
- 67%
Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference
CHI '24· Conversational Chatbots +2
- 67%
Evaluating Non-AI Experts' Interaction with AI: A Case Study In Library Context
CHI '25· Conversational Chatbots +1
- 67%
Mitigating Response Delays in Free-Form Conversations with LLM-powered Intelligent Virtual Agents
CUI '25· Social & Collaborative VR +1
- 67%
Beyond Text and Speech in Conversational Agents: Mapping the Design Space of Avatars
DIS '24· Conversational Chatbots +1
- 67%
PDFChatAnnotator: A Human-LLM Collaborative Multi-Modal Data Annotation Tool for PDF-Format Catalogs
IUI '24· Human-LLM Collaboration +1
- 67%
Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking
IUI '24· Human-LLM Collaboration +1
Based on Jaccard similarity of research subtopics & professions (≥60%)