Conversation Progress Guide : UI System for Enhancing Self-Efficacy in Conversational AI

Research Background and Issues

• What problems or challenges did the authors identify?
  Users often encounter failures when interacting with generative conversational AI, which undermines their sense of self-efficacy and reduces their willingness and confidence to continue using these systems. In tasks requiring multi-turn conversations, the limitations of generative conversational AI can increase cognitive load, making users feel that completing the task is difficult and diminishing their confidence.

• Why is this issue important?
  Self-efficacy directly impacts learning efficiency and task performance. Low self-efficacy not only reduces motivation for learning but also leads users to avoid using generative AI services. As generative conversational AI becomes increasingly widespread, addressing this issue is critical for improving user experience and enhancing technology acceptance.

• Research motivation and related work
  Many studies have explored how to enhance self-efficacy in specific learning scenarios through the interface design of digital devices. For example, applications targeting children or students enhance their confidence through rewards for task completion. However, few studies have focused on improving self-efficacy in everyday usage scenarios of generative conversational AI. This paper aims to design a system mechanism that improves self-efficacy by providing visual task feedback without modifying the underlying AI technology.

Solution

• What methods or solutions do the authors propose?
  The authors propose a visual interface system called the "Conversation Progress Guide (CPG)" for generative conversational AI. This interactive tool uses task progress bars and subtask markers to help users visualize task progress and enhance their self-efficacy.

• What are the innovative aspects of this solution?

  • Provides visual task feedback by breaking down conversational tasks into multiple subtasks and marking their completion through progress bars.
  • Emphasizes enhancing self-efficacy through mastery experiences of incremental task completion.
  • Implements the solution directly on existing conversational AI platforms without modifying the underlying generative technology.

• What are the implementation steps and key technologies used?

  1. System Architecture Design:

     • The system employs two AI agents:
       • Task Agent: Responsible for general conversation generation.
       • Progress Feedback Agent: Analyzes conversation content, determines subtask completion, and updates the progress bar.
     • GPT-3.5-turbo and GPT-4 models are used to implement the Task Agent and Progress Feedback Agent, respectively.

  2. User Interface Design:

     • Progress Bar: Displays task completion status through activated subtask markers.
     • Subtask Marker: Highlights completed subtasks to focus users on positive task completion experiences.

  3. System Functionality:

     • Based on user input and the Task Agent's output, the Progress Feedback Agent evaluates task completion status.
     • When a subtask is deemed complete, the corresponding subtask marker is activated on the progress bar.
     • The system allows users to manually engage in additional dialogues to address any missed tasks.

Research Outcomes

• What specific outcomes were achieved?

  • Participants in the experimental group using CPG showed a significant increase in self-efficacy, particularly in confidence and understanding related to task execution.
  • There were no significant differences between the experimental and control groups in terms of cognitive load, user satisfaction, task completion time, or interaction frequency, demonstrating that CPG does not increase cognitive burden or negatively impact user experience.

• What advantages does it have compared to existing solutions?

  • Compared to traditional generative conversational AI interfaces, CPG more effectively boosts user confidence and active engagement during task execution.
  • By optimizing the interaction interface without altering the underlying model technology, it enhances task performance.

• What were the experimental or evaluation results?

  • Across six self-efficacy-related questions, the experimental group consistently outperformed the control group, particularly in understanding and executing tasks step-by-step.
  • The study found no negative impact of CPG on users' cognitive load, task completion time, or interaction frequency, and user satisfaction ratings were comparable.

• Limitations and future directions

  • Limitations:

    • The experimental sample was small and limited to university students in computer science, lacking generalizability.
    • The current design of CPG is suitable for well-defined, step-by-step tasks but has limited effectiveness in open-ended or unstructured conversations.
    • The system does not provide feedback for repeated failures, which may lead to misunderstandings or abandonment of tasks by some users.

  • Future Directions:

    • Extend CPG to accommodate nonlinear or more complex scenarios, such as creative writing or open-ended conversations.
    • Introduce more sophisticated progress evaluation rules and integrate machine learning to improve evaluation and feedback accuracy.
    • Test a more diverse participant sample to validate CPG's broad applicability across different contexts.
    • Explore artistic or humorous ways to present negative feedback to encourage users to grow through failures.

Conclusion

This paper introduces the Conversation Progress Guide (CPG), a conversational AI interface that enhances user self-efficacy through visual task feedback. This innovative design significantly improves user confidence and engagement without imposing additional burdens on task efficiency or experience. The study advances the optimization of user experience in task-oriented scenarios for generative conversational AI and proposes directions for further development.