From Detectables to Inspectables: Understanding Qualitative Analysis of Audiovisual Data

Title of the Paper

From Detectables to Inspectables: Understanding Qualitative Analysis of Audiovisual Data

Paper Information

• Domain: Qualitative research in Human-Computer Interaction (HCI), particularly methods for analyzing audiovisual data
• Keywords: Audiovisual analysis, qualitative research, information behavior, video search, data annotation, automation support

Research Background and Problem

• Identified Problems or Challenges:

  • Audiovisual recordings provide rich behavioral and interactional information, but their unstructured temporal nature makes direct analysis challenging.
  • Transcribed text offers higher efficiency for search and navigation, but it loses many details inherent in audiovisual data.
  • Existing research lacks an understanding of the workflows and needs of qualitative researchers, especially regarding direct analysis of audiovisual recordings.

• Significance of the Problem:

  • Qualitative research methods are increasingly used in design and user studies, particularly for analyzing complex behaviors such as user-interface interactions.
  • Addressing the efficiency issues in audiovisual data analysis can significantly enhance data utilization and insights in qualitative research.

• Research Motivation and Related Work:

  • The goal is to explore how to optimize the analysis workflows of qualitative researchers to handle audiovisual recordings more efficiently.
  • Previous studies have focused on qualitative analysis of textual data, with limited research on direct analysis of audiovisual data.
  • The study draws on interaction analysis, qualitative coding tools (QDA tools), and video navigation techniques, as well as their limitations.

Proposed Solution

• Proposed Solution:

  • Introduce the concepts of "Inspectables" and "Detectables," where "Inspectables" refer to interesting audiovisual segments requiring in-depth analysis, and "Detectables" refer to visual or auditory cues used to locate these segments.
  • Explore automated methods to assist researchers in identifying "Detectables," thereby simplifying the process of locating "Inspectables."

• Innovative Contributions:

  • Proposed new terminology and a theoretical framework ("Inspectables," "Detectables," "Reportables") to structure qualitative analysis of audiovisual data.
  • Suggested integrating simple image recognition and sound detection technologies to support automated detection of audiovisual segments.

• Implementation Steps and Key Techniques:

  • Conduct interviews and surveys to gather information on researchers' workflows for analyzing audiovisual recordings.
  • Summarize methods for locating "Inspectables" during analysis tasks, including the use of timestamps, research structure information, and "Detectables."
  • Apply automation technologies to detect "Detectables," such as image processing and audio signal matching.

Research Outcomes

• Specific Findings:

  • Gained an understanding of the prevalence of direct audiovisual data analysis in qualitative research, as well as corresponding workflows and annotation practices.
  • Proposed leveraging simple automation methods (e.g., image and sound detection) to assist researchers in locating target information within audiovisual segments.

• Advantages Compared to Existing Solutions:

  • Compared to traditional text transcription, direct audiovisual data analysis captures more behavioral and interactional details.
  • Automation methods aim to reduce the manual effort required for researchers to locate target segments while maintaining control over analysis tasks.

• Experimental or Evaluation Results:

  • Interview Results: Most analyses are highly exploratory, requiring researchers to repeatedly review recordings to identify points of interest.
  • Survey Results: 63% of analyses aim at theory building, and 71% use full transcriptions. However, 32% of analyses cannot rely solely on transcriptions and require direct use of audiovisual recordings.

• Limitations and Future Directions:

  • Limitations:
    • Automated detection methods may not be suitable for studies involving complex behaviors or could result in a high number of false positives.
    • Researchers' trust in and reliance on automation may impact the scientific rigor of analysis results.
  • Future Directions:
    • Develop transparent and interpretable automation tools that closely integrate with researchers' workflows.
    • Explore the applicability and effectiveness of automated detection techniques across different types of behaviors and research contexts.

In summary, this study represents a significant step toward automation-supported qualitative analysis of audiovisual data while also highlighting practical challenges that require future resolution.