Inform the uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot

Title of the Paper

Inform the Uninformed: Improving Online Informed Consent Reading with an AI-Powered Chatbot

Paper Information

• Subject Area: Human-Computer Interaction (HCI), AI Chatbots, Online Informed Consent
• Keywords: Informed Consent, Conversational Agents, AI Chatbots, Human-AI Interaction, Power Dynamics

Research Background and Issues

• Identified Problems and Challenges

  • Current informed consent processes fail to ensure participants' thorough understanding of the form's content, leading to the neglect of critical information and potential risks.
  • The lack of researcher guidance in online environments forces participants to read forms independently, further reducing the quality of informed consent.
  • Although prior studies have attempted to improve this process through simplified language and interactivity, the results remain limited.

• Significance

  • Participants who fail to adequately read and understand informed consent forms may face health risks, privacy threats, and limitations on their autonomy.
  • Information asymmetry and power dynamics can affect the validity of research and reduce data quality.

• Research Motivation and Related Work

  • In recent years, AI chatbots have emerged as a potential solution due to their scalability and interactivity, simulating the experience of in-person researcher guidance.
  • Existing literature has rarely explored replacing traditional consent forms with chatbots, particularly in terms of their broader impacts.

Solution

• Methods and Solutions

  • Developed an AI chatbot named "Rumi," which integrates a hybrid system combining rule-based and AI technologies. Rumi explains the consent form section by section, answers questions, and collects consent outcomes.
  • Designed an online survey experiment with three levels of risk to evaluate the chatbot's overall effectiveness in the informed consent process.

• Innovations

  • Conducted the first systematic comparison between an AI chatbot and traditional document-based consent processes, assessing their impact on participants' reading experience, power dynamics, and research outcomes.
  • Proposed a hybrid solution that balances the stability of rule-based systems with the flexibility of AI-powered Q&A modules.

• Implementation Steps and Key Technologies

  1. Experimental Design:
     • Two conditions (chatbot vs. form) and three risk levels (low, medium, high) were set.
     • A virtual study on social media usage issues served as the experimental scenario.
  2. Chatbot Development and Training:
     • Built the chatbot "Rumi" using the Juji platform.
     • Created over 200 Q&A pairs, supplemented with language variations and additional answers generated by GPT-3.
     • Conducted pretests to gather more potential question types.
  3. Data Collection and Analysis:
     • Collected data on participants' reading behaviors, form completion processes, and the quality of subsequent survey responses.
     • Used structural equation modeling and Bayesian statistical methods to analyze data and evaluate research hypotheses.

Research Findings

• Specific Results

  • Compared to traditional forms, Rumi significantly improved the quality of consent form reading:
    • Recall Ability: Participants could more accurately recall specific information from the form.
    • Comprehension: Participants better understood the content and correctly processed risk information.
  • Participants reported increased trust and collaboration with researchers, reducing power disparities.
  • In follow-up surveys, participants using Rumi provided higher-quality and more detailed responses.

• Comparison with Existing Solutions

  • Unlike static forms, Rumi offered a more human-like interactive experience, encouraging participants to invest more time and effort and actively ask questions.
  • Improved the information symmetry between researchers and participants, mitigating ethical risks associated with power imbalances.

• Experimental or Evaluation Results

  • Structural equation modeling demonstrated that the chatbot indirectly improved research data quality by optimizing the power dynamics between participants and researchers.
  • Compared to traditional forms, Rumi elicited deeper information disclosure in open-ended survey responses.

• Limitations and Future Directions

  • Limitations:
    • The study primarily focused on low-risk text-based surveys, and its applicability to complex research scenarios requires further exploration.
    • The specific design of the chatbot system (e.g., language style) may have influenced the results but was not differentiated in the study.
    • The potential bias introduced by the "novelty effect" of new technology has not been fully addressed.
  • Future Directions:
    • Explore applications in more diverse high-risk or complex research scenarios.
    • Investigate how chatbots can be personalized for different participants and integrated into ethical frameworks.
    • Extend the applicability of AI to everyday data-sharing and privacy consent scenarios.

This study proposes a forward-looking approach, demonstrating how AI chatbots can enhance the consent experience in online research and offering emerging design concepts and technical practices for creating more effective informed consent mechanisms.