Explaining Recommendations in E-Learning, Effects on Adolescents' Trust

Title of the Paper

Explaining Recommendations in E-Learning: Effects on Adolescents’ Trust

Paper Information

• Research Area: Human-Computer Interaction, Explainable Artificial Intelligence (XAI), Educational Technology
• Keywords: Adolescent trust, education, explainability (XAI), recommender systems, trust structure, multidimensional trust, randomized controlled experiment

Research Background and Issues

• Identified Problems or Challenges:

  • Most current research on the explainability of recommender systems focuses on adult users and domains like e-commerce or media, with limited attention to educational contexts, especially for adolescent users.
  • There is insufficient research on the explainability of recommendation algorithms in educational platforms, and the opacity of such algorithms may affect users' trust in the platform.
  • Many studies assume no baseline comparison or only use "no explanation" as a baseline, but research shows that "placebo explanations" or fake explanations can also influence user trust.

• Significance of the Research:

  • Trust is a critical factor in users' adoption of recommender systems.
  • In educational contexts, trust is particularly important since recommended educational content directly impacts learning outcomes.
  • Exploring how different forms of explanations affect adolescents' trust in educational platforms can provide guidance for the development of educational technologies tailored to this specific group.

• Research Motivation and Related Work:

  • The authors aim to design and evaluate different types of explanation interfaces to explore how to enhance adolescents' initial trust in educational recommender systems.
  • By integrating theories from Human-Computer Interaction and Explainable Artificial Intelligence (XAI), the study proposes explanation and recommendation optimization methods tailored to the needs of adolescents.

Proposed Solution

• Proposed Methods or Solutions:

  • Design a recommender system interface with three levels of explanation: Real explanations, Placebo explanations, and No explanations.
  • Develop an e-learning platform called Wiski, based on ELO scoring and collaborative filtering algorithms, featuring a recommendation system for math exercises.

• Innovative Aspects:

  • Investigating the effectiveness of explanations and their impact on trust across multiple dimensions by comparing real explanations with "placebo explanations."
  • Focusing specifically on adolescents as the research subject, addressing the lack of studies on this age group in the recommender system domain.
  • Proposing the hypothesis that "dynamic learning factors" may be more critical than explanations in influencing trust, exploring the complex structure of trust.

• Implementation Steps and Key Techniques:

  1. Develop a personalized math exercise recommendation algorithm based on students' actual math proficiency and completed exercises.
  2. Design three types of explanation interfaces for the recommended exercises:
     • Real explanations: Include information about the match between the student's current level and exercise difficulty, a predicted probability of successful completion, and a histogram showing similar users' attempts on the exercise.
     • Placebo explanations: Provide vague information with no specific meaning, such as "recommendation derived from algorithmic calculations."
     • No explanations: Only provide exercise recommendations without any explanations.
  3. Conduct a randomized controlled experiment involving 37 adolescents, randomly assigned to one of the three interface types.
  4. Use a multidimensional survey to assess trust components (e.g., competence, benevolence, integrity) and overall trust as a single dimension.

Research Findings

• Specific Results:

  • Real explanations significantly increased adolescents' trust in the educational platform (in multidimensional trust components such as competence, transparency, and benevolence).
  • The acceptance rate of recommendations was significantly higher in the real explanation group compared to the no explanation or placebo explanation groups.
  • Placebo explanations did not significantly improve trust but provided feedback on participants' needs for transparency and trust.

• Advantages Over Existing Solutions:

  • This experiment addresses limitations in existing research regarding user groups, application domains, and baseline design, filling a gap in studies on adolescents using recommender systems.
  • It proposes "transparency needs" as a core principle for customizing explanations.

• Experimental or Evaluation Results:

  • Comparative analysis showed that while real explanations improved multidimensional trust metrics, no significant effects were observed in single-dimensional trust surveys. This indicates that trust has a complex, multilayered structure that cannot be fully captured by a simplified survey.
  • Most adolescents expressed high overall satisfaction with the recommended exercises, validating the importance of dynamic learning factors (e.g., recommendation accuracy) in building trust.

• Limitations and Future Directions:

  • Sample Size Issues: The sample size was only 37 participants, with a wide age range (13-18 years), which may limit the generalizability of the findings.
  • Dynamic Nature of Algorithms and Data: Recommendation and explanation algorithms may evolve during the experiment.
  • Heterogeneity in Target Audience: Adolescents' diverse needs for explanations and transparency highlight the potential for quantitative customization of algorithms.
  • Future Work: Explore long-term studies to examine the evolution of trust; design more complex exercise scenarios; enrich the validation and comparison of multidimensional trust scales.