Designing Computational Tools for Exploring Causal Relationships in Qualitative Data

Paper Title

Designing Computational Tools for Exploring Causal Relationships in Qualitative Data

Publication Info

• Topic area: Computational tools for qualitative data analysis, focusing on causal relationships.
• Keywords: Qualitative data analysis, causal relationships, computational tools, causal networks, visualization, hypothesis generation, HCI, social science, LLMs, interactive systems.

Background and Problem

• Problem / challenge: Existing computational tools for qualitative data analysis primarily focus on categorization and description, lacking robust support for exploring causal relationships. Current systems either oversimplify context, lack credibility, or produce overly complex outputs.
• Significance: Understanding causal relationships in qualitative data is crucial for theory building, predicting behaviors, and untangling complex social phenomena. However, manual analysis is labor-intensive, and existing tools fail to adequately support this need.
• Motivation and related work: Prior work has explored computational methods like topic modeling, clustering, and causal inference using NLP. However, these methods often lack traceability to source data, produce overly complex networks, or fail to balance automation with user control. Visualization techniques for causal relationships in quantitative data provide inspiration but are not tailored for qualitative contexts.

Solution

• Proposed approach: QualCausal, a system for constructing and exploring causal networks from qualitative data through interactive workflows and multi-view visualizations.
• Novelty:
  1. Introduces a system that combines causal network visualization with user-controlled workflows for categorizing and connecting qualitative data.
  2. Balances automation with user autonomy, enabling traceability to source data.
  3. Provides multi-level visualization for both broad patterns and detailed analysis.
  4. Facilitates hypothesis generation and theory building through exploratory causal discovery.
• Procedure and key techniques:
  1. Indicator Extraction: Automatically identifies indicators from qualitative data based on user-provided research overviews.
  2. Concept Creation: Allows users to map indicators to concepts, with options for manual refinement and automated mapping.
  3. Causal Relationship Extraction: Uses LLMs to identify and classify causal relationships between indicators, forming directed edges in a causal network.
  4. Visualization: Provides coordinated views (Indicator View, Concept View, Node View, and Details Panel) for exploring causal networks at different levels of abstraction.

Results

• Concrete findings:
  • Indicator extraction achieved 86.46% precision and 94.32% recall on the MHStigmaInterview-20 dataset.
  • Concept mapping reached 66.39% precision, 76.96% recall, and 70.83% accuracy.
  • Causal relationship extraction outperformed baselines, achieving 80.60% precision, 83.08% recall, and 98.18% directionality accuracy on MHStigmaInterview-20, and 100.00% precision/directionality accuracy with 85.00% recall on SemEval 2010 Task 8.
• Advantage over baselines: Outperformed co-occurrence networks and causal-cue heuristics in precision, recall, and directionality accuracy by leveraging semantic and contextual understanding.
• Experiments / evaluation:
  • Conducted a formative study with 15 participants to identify challenges and design goals.
  • Evaluated the system quantitatively on two datasets (MHStigmaInterview-20 and SemEval 2010 Task 8).
  • Conducted a feedback study with 15 participants to assess usability, hypothesis generation, and user perceptions.
• Limitations and future work:
  • Limited to within-sentence causal relationships, missing discourse-level patterns.
  • Short evaluation sessions may not capture long-term integration into workflows.
  • Potential scalability issues with large datasets.
  • Future work includes longitudinal studies, discourse-level causal discovery, and addressing paradigm tensions in interpretive research.

Summary

This paper introduces QualCausal, a computational tool for exploring causal relationships in qualitative data through interactive causal network construction and visualization. The system leverages LLMs for indicator extraction, concept mapping, and causal relationship detection, and provides multi-level visualizations to support hypothesis generation and theory building. Quantitative evaluations demonstrate its effectiveness, and user feedback highlights its utility in reducing analytical burden while preserving interpretive autonomy. However, challenges remain in addressing discourse-level relationships, scalability, and alignment with diverse research paradigms. The study contributes to the design of computational tools that thoughtfully integrate into qualitative data analysis practices.