Tutor In-sight: Guiding and Visualizing Students’ Attention with Mixed Reality Avatar Presentation Tools

Document Title

Tutor In-sight: Guiding and Visualizing Students’ Attention with Mixed Reality Avatar Presentation Tools

Document Information

• Subject Area: Human-Computer Interaction and Educational Technology
• Keywords: Virtual avatars, remote presentation, augmented reality, mixed reality, educational tools, attention guidance, remote teaching, student attention monitoring, user study, student engagement

Research Background and Problem

• Identified Challenges:

  • Current video conferencing tools (e.g., Zoom) limit teachers to live-streaming through cameras, making it difficult to fully utilize body language, thereby reducing the sense of presence and engagement between teachers and students.
  • While virtual reality (VR) tools enhance students' immersion, teachers face technical barriers in implementing these tools, and VR headsets may hinder students' interaction with physical learning materials (e.g., note-taking).
  • Existing remote education systems fail to fully meet the needs of both teachers and students, such as balancing student engagement and teaching efficiency.

• Significance:

  • With the increasing prevalence of online learning and remote teaching, designing remote education tools that enhance teacher-student interaction quality has become crucial.
  • Effectively guiding students to focus on key learning materials and improving their engagement are critical factors in enhancing the quality of online teaching.

• Research Motivation:

  • To address these issues, the authors aim to find a solution that balances ease of use for teachers and immersion for students.
  • By integrating virtual and physical workspaces, the goal is to develop more usable educational tools that create an improved remote learning and teaching environment for both teachers and students.

Solution

• Proposed Approach:

  • The Tutor In-sight system consists of two components: a teacher dashboard and a mixed reality (MR) virtual avatar tool for students.
  • Students' workspaces are augmented with a virtual teacher avatar that automatically generates body language (including eye contact, gestures, and movements) and interacts with materials in real time.
  • The teacher dashboard enhances teachers' understanding of students' attention and improves teaching control by monitoring real-time eye-tracking data.

• Innovations:

  • The teacher avatar automatically generates body language based on PowerPoint content and adapts to the teaching material in real time, without requiring expensive hardware or additional training.
  • The teacher dashboard monitors students' attention distribution on a 2D screen, reducing the limitations of using head-mounted devices.
  • The MR tool is designed to balance usability and immersion, addressing the needs of both teachers and students.

• Implementation Steps:

  1. Teacher Dashboard: Develop a dashboard for real-time monitoring of students' attention, using a PowerPoint plugin to track the teacher's mouse movements and presentation logic.
  2. Student Augmented Workspace: Use MR devices like Microsoft HoloLens 2 to overlay virtual avatars with physical content, providing specific visual cues to guide students' attention.
  3. Virtual Avatar Animation Generation: Generate the teacher avatar's movements in real time based on teacher actions and students' eye-tracking data.
  4. Evaluation and Adjustment: Conduct two user studies to compare and test Tutor In-sight, and optimize functionality design based on feedback.

Research Outcomes

• Specific Results:

  • The Tutor In-sight system significantly improved user acceptance among teachers and students in two rounds of user studies, particularly excelling in attention guidance, sense of presence, and engagement compared to traditional video conferencing and VR technologies.
  • Teachers appreciated the real-time feedback functionality of the dashboard, while students were satisfied with the interactive features of the MR avatar.

• Advantages over Existing Solutions:

  • Provides a more user-friendly operation for teachers with a lower learning curve.
  • Simulates an immersive environment with teacher body language and interaction, allowing students to focus on learning materials while retaining the ability to take notes.
  • The system design integrates virtual and physical spaces, addressing the distraction issues seen in VR tools.

• Experiment and Evaluation Results:

  • Teacher Study: Most teachers preferred Tutor In-sight, finding it more effective in monitoring student attention and managing lessons.
  • Student Study: Students found the MR avatar more immersive and helpful for attention allocation, though VR headsets could interfere with note-taking.
  • Overall Preference: The majority of participants in both studies ranked Tutor In-sight as the preferred solution.

• Limitations and Future Directions:

  • Currently does not fully support dynamic or animated slides, only static slide content.
  • The study primarily tested pre-recorded presentations; future research should validate its practicality and learning outcomes in real-time interactive scenarios.
  • Further development is needed for multi-party interaction features (e.g., student collaboration) and alternative monitoring methods for handling privacy-sensitive data (e.g., head movement tracking).

In summary, the Tutor In-sight system provides a promising enhanced tool for remote education, offering valuable insights for designing more effective MR educational solutions.