Reflexis: Supporting Reflexivity and Rigor in Collaborative Qualitative Analysis though Design for Deliberation

Paper Title

Reflexis: Supporting Reflexivity and Rigor in Collaborative Qualitative Analysis through Design for Deliberation

Publication Info

• Topic area: Collaborative qualitative analysis tools emphasizing reflexivity and methodological rigor.
• Keywords: Reflexive thematic analysis, qualitative data analysis, collaboration, reflexivity, code evolution, positionality, AI-assisted tools, transparency, deliberation, interpretive methods.

Background and Problem

• Problem / challenge: Current qualitative analysis tools prioritize speed and consensus, often neglecting reflexivity, transparency, and productive disagreement essential to Reflexive Thematic Analysis (RTA). Practices like documenting code evolution and integrating positionality are poorly supported.
• Significance: Addressing these gaps is critical for producing rigorous, impactful qualitative insights, especially in fields like HCI, sociology, and education, where interpretive depth is paramount.
• Motivation and related work: While tools like NVivo and ATLAS.ti focus on automation and scalability, they often suppress interpretive diversity. Recent AI-powered systems emphasize efficiency but fail to scaffold reflexive practices or principled disagreement. Reflexis builds on calls for AI systems that highlight ambiguity and support reflexivity.

Solution

• Proposed approach: Reflexis, a collaborative workspace designed to embed reflexivity, analytic history, and positionality directly into the qualitative analysis workflow.
• Novelty:
  1. Integration of in-situ reflexive prompts to connect researcher perspectives directly to coding decisions.
  2. Transparent code evolution tracking via detailed histories and proactive drift alerts.
  3. Positionality-aware collaboration tools to frame interpretive differences as productive dialogues.
• Procedure and key techniques: Reflexis operationalizes reflexivity through features like ReflexiveLens for in-situ reflection, Analysis History for tracking code evolution, and Discussion Focus with positionality-aware prompts to scaffold collaborative interpretation. AI assistance is advisory, ensuring human-led decision-making.

Results

• Concrete findings:
  • Reflexis increased in-situ reflexivity, with participants reporting more granular and deliberate reflection during coding.
  • The Analysis History feature was universally praised for enhancing transparency and rigor, while the Code Drift Alert sparked critical reflection on code definitions.
  • Discussion Focus streamlined collaborative workflows, enabling efficient and targeted discussions of disagreements.
• Advantage over baselines: Reflexis addressed gaps in existing tools by automating transparency, surfacing interpretive differences, and embedding reflexivity into the workflow, which participants found novel and impactful compared to their fragmented, manual methods.
• Experiments / evaluation: A paired-analyst study with 12 experienced qualitative researchers demonstrated Reflexis’s ability to support reflexivity, transparency, and collaboration. Participants worked on synthetic datasets in structured sessions, providing qualitative and quantitative feedback.
• Limitations and future work:
  • Limited to single-session evaluations; longitudinal studies are needed to assess long-term adoption.
  • Positionality operationalization risks flattening complex identities; future work should explore evolving or team-level representations.
  • Larger teams and power asymmetries were not studied; additional research is required to understand Reflexis’s applicability in diverse collaborative settings.

Summary

Reflexis introduces a collaborative workspace for qualitative analysis that prioritizes reflexivity, transparency, and principled disagreement. Through features like ReflexiveLens, Analysis History, and Discussion Focus, it transforms reflexivity from a summative activity into an integrated, granular practice. A paired-analyst study with 12 researchers demonstrated its ability to scaffold deeper reflection, enhance methodological transparency, and facilitate productive collaboration. By designing for deliberation rather than efficiency, Reflexis offers a principled approach to building human-AI systems that augment interpretive knowledge work while preserving researcher agency.