Exploring the Role of Paradata in Digitally Supported Qualitative Co-Research

Title of the Paper

Exploring the Role of Paradata in Digitally Supported Qualitative Co-Research

Bibliographic Information

• Subject Area: Human-Computer Interaction (HCI) and Qualitative Research Methodology
• Keywords: Paradata, Qualitative Research, Co-research, Data Transparency, Participatory Design, Social Sciences, Digital Support, Citizen Science

Research Background and Issues

• Issues and Challenges:

  • In qualitative co-research practices, there is insufficient transparency regarding participants' roles in data collection, analysis, and dissemination. This lack of transparency can pose problems for stakeholders (e.g., funding agencies) who require evidence of rigor and accountability in research decision-making.
  • Digitally supported qualitative research generates large volumes of interaction logs (i.e., paradata), but these logs are rarely utilized to review and improve research decisions.

• Significance:

  • As qualitative research increasingly impacts diverse stakeholders (e.g., community organizations, governments), the demand for transparency, data sharing, and accountability continues to grow.
  • The importance of paradata transparency is rising as it becomes a critical factor in ensuring data sharing, validation, and reuse.

• Research Motivation and Related Work:

  • In recent years, the HCI academic community has increasingly focused on enhancing the transparency and reproducibility of qualitative research. This study responds to this demand by exploring the potential applications of paradata in improving qualitative research transparency.
  • Previous studies have limited the application of paradata to fields such as online learning and survey methodologies, with little exploration in qualitative research. This study addresses this gap.

Proposed Solution

• Proposed Solution:

  • The authors conducted a four-month field study in collaboration with a community-led charity organization, using the digital platform Gabber to record and analyze paradata generated during qualitative research activities.
  • They proposed a conceptual model of paradata, discussing how it can enhance process transparency, facilitate data sharing, and create feedback loops for research participants.

• Innovations:

  • Extending the application of paradata from traditional survey and technical product analysis domains to qualitative research.
  • Designing specific paradata metrics (e.g., "listening coverage," "participation rate") to demonstrate how paradata can enhance the transparency and trustworthiness of qualitative research processes.

• Implementation Steps and Techniques:

  1. Use the Gabber platform to record all stages of qualitative research (data preparation, data collection, data analysis, data organization).
  2. Observe the types and characteristics of paradata generated during the research process, such as listening time proportions, comment counts, and thematic maps of the analysis process.
  3. Design and conduct post-deployment interviews to reflect on the potential roles and challenges of paradata from the perspective of research participants.

Research Outcomes

• Specific Outcomes:

  • Provided a conceptual description of paradata in qualitative research.
  • Designed multiple paradata metrics, such as "listening coverage" and "data discard rate," and proposed potential designs and applications for these metrics.
  • Developed a framework of design challenges to enhance the transparency of qualitative research.

• Advantages of the Existing Solution:

  • Through the Gabber platform, participants can see how their contributions are utilized by the community, thereby enhancing transparency and collaborative trust.
  • Paradata can be utilized throughout the entire qualitative research process, from data collection to analysis and dissemination, creating a more democratic research framework.

• Experimental or Evaluation Results:

  • Participants expressed positive feedback about paradata, including its potential to help reproduce research processes, enhance transparency, and demonstrate participants' influence on outcomes.
  • Virtual scenarios and traceable paradata mechanisms contribute to increasing trust in research results.

• Limitations and Future Directions:

  • Limitations:
    • The study is primarily based on a single community case, with most participants lacking prior experience in qualitative research, which may limit the generalizability of the findings.
    • Privacy and ethical issues surrounding paradata, especially automatically generated personal data, require further investigation.
  • Future Directions:
    • Explore user-friendly ways to present paradata and allow participants to easily review and contest data usage.
    • Expand the standardization and sharing mechanisms of qualitative research paradata to support transparency in other research fields.
    • Further investigate privacy, ethical, and authorization design issues related to paradata from both academic and practical perspectives.

Through this study, the authors provide new directions for the design of digital support tools for qualitative research while deepening the understanding of the potential role of paradata in transparency and data sharing.