InterFlow: Designing Unobtrusive AI to Empower Interviewers in Semi-Structured Interviews

Paper Title

InterFlow: Designing Unobtrusive AI to Empower Interviewers in Semi-Structured Interviews

Publication Info

• Topic area: AI-assisted tools for qualitative research, focusing on semi-structured interviews.
• Keywords: Semi-structured interviews, AI assistance, cognitive load, real-time data sensemaking, interview flow, natural language processing, human-computer interaction, large language models, qualitative research, mixed-initiative systems.

Background and Problem

• Problem / challenge: Conducting high-quality semi-structured interviews is cognitively demanding, requiring interviewers to actively listen, formulate follow-up questions, and maintain conversational flow. Non-expert interviewers often struggle with script navigation, note-taking, and time management, leading to suboptimal interview outcomes.
• Significance: Improving interview quality is critical for obtaining rich, goal-relevant data, which directly impacts subsequent qualitative analysis and reporting. Addressing these challenges can democratize qualitative research by supporting non-expert interviewers.
• Motivation and related work: Prior work has explored AI-driven tools for automating qualitative data collection and analysis, but these systems often fail to address the dynamic and exploratory nature of semi-structured interviews. Existing solutions are either too rigid or intrusive, and no tools have been specifically designed to assist non-expert interviewers in real-time.

Solution

• Proposed approach: InterFlow, an AI-powered visual scaffold that supports interviewers by dynamically adapting the interview script, providing real-time visualizations, and offering mixed-initiative information capture.
• Novelty:
  1. A visual hierarchy for script navigation that dynamically updates based on the conversation.
  2. Real-time visualizations for time management and conversational balance.
  3. Mixed-initiative information capture with manual, AI-assisted, and proactive modes.
  4. A "co-interviewer" agent that surfaces follow-up opportunities grounded in context and interview goals.
• Procedure and key techniques:
  • Pre-interview: Upload script and contextual information; system parses and structures the script.
  • During interview: Interactive script highlights ongoing questions, visual timer tracks progress and balance, and mixed-initiative tools support note-taking and follow-ups.
  • AI components: Question detection, real-time summarization, and proactive suggestion generation using LLMs like Claude 3.5 Sonnet and GPT-4o.

Results

• Concrete findings:
  • Question detection accuracy: 58%, latency: 8.9 seconds.
  • Summary accuracy: 77%, latency: 4.2 seconds.
  • Cognitive load significantly reduced (e.g., mental demand: 2.75 vs. 4.83 in baseline, p = 0.002).
  • Usability rated higher (System Usability Scale: 82.5 vs. 67.5 in baseline, p = 0.003).
• Advantage over baselines:
  • Improved script navigation (M = 5.33 vs. 2.25, p = 0.002).
  • Enhanced situational awareness (M = 5.00 vs. 2.67, p = 0.009).
  • Better real-time data sensemaking (M = 5.67 vs. 2.42, p < 0.001).
  • Reduced distraction and preserved interview authenticity.
• Experiments / evaluation:
  • Within-subject study with 12 participants (interviewers) using both InterFlow and a baseline system.
  • Metrics: Cognitive load (NASA-TLX), usability (SUS), task performance, and expert-rated suggestion quality.
  • Data sources: Interaction logs, surveys, and thematic analysis of exit interviews.
• Limitations and future work:
  • Moderate accuracy of question detection and summarization due to dynamic interview settings.
  • Higher noise in AI-generated suggestions compared to baseline.
  • Future directions include improving robustness of AI models, exploring wearable interfaces, and extending to other domains like clinical consultations or educational tutoring.

Summary

InterFlow is an AI-assisted system designed to support semi-structured interviews by reducing cognitive load, enhancing script navigation, and facilitating real-time data sensemaking. It combines dynamic visualizations, mixed-initiative tools, and a proactive "co-interviewer" agent to assist non-expert interviewers. Evaluation results show significant improvements in usability, situational awareness, and cognitive load compared to a baseline system. While challenges remain in improving AI accuracy and reducing noisy suggestions, InterFlow demonstrates the potential of unobtrusive AI to empower interviewers in cognitively demanding tasks.