Investigating the Effects of Real-time Student Monitoring Interface on Instructors’ Monitoring Practices in Online Teaching

Title of the Paper

Investigating the Effects of a Real-time Student Monitoring Interface on Instructors’ Monitoring Practices in Online Teaching

Paper Information

• Field of Study: Online Education, Human-Computer Interaction
• Keywords: Real-time student monitoring system, online teaching, student engagement, monitoring practices, teaching behavior adjustment, cognitive load

Research Background and Issues

• Issues and Challenges:

  • The COVID-19 pandemic has accelerated the adoption of online education, but remote teaching faces significant challenges in monitoring student engagement.
  • Traditional methods of gauging student engagement through facial expressions, gestures, etc., are significantly reduced in online environments.
  • Teachers struggle to observe students in real-time or actively interact with them, limiting their ability to monitor student behavior and implement teaching strategies effectively.

• Importance:

  • Monitoring student behavior is fundamental to effective teaching, enabling instructors to assess learning progress, manage classrooms, and prevent students from falling behind.
  • In online education, decreased student engagement significantly impacts learning outcomes, making continuous monitoring of engagement levels crucial.

• Motivation and Related Work:

  • Researchers have designed various real-time student monitoring interfaces (e.g., Glancee, AffectiveSpotlight) to display learning metrics or behavioral states.
  • Existing studies primarily focus on system development and usability evaluation, lacking in-depth understanding of how "monitoring cues" directly assist teachers in adjusting their teaching methods.

Solution

• Research Method:

  • The authors conducted experimental research to evaluate the effects of engagement cues (presence vs. absence) and student engagement levels (high vs. low) on teachers’ monitoring behaviors and teaching adjustments.
  • The experiment utilized simplified interface designs to eliminate complexity in monitoring metrics and visualization methods, focusing on two core elements: "individual student engagement" and "overall classroom engagement."

• Innovations:

  • Pre-recorded videos were used to control experimental variables, systematically exploring the fundamental impact of engagement cues.
  • A comprehensive analysis combining behavioral data (e.g., eye-tracking) and subjective data (e.g., monitoring quality and cognitive load) was conducted to assess interface effectiveness.

• Implementation Steps and Key Technologies:

  1. Experimental Design: A 2x2 within-subject design was employed, manipulating the presence of engagement cues and overall student engagement levels.
  2. Experimental Setup: Two interface designs were used, one without engagement cues and one with simplified engagement cues (displaying engagement status via color and bar charts).
  3. Participants: Twenty university instructors with extensive online teaching experience were recruited to complete two rounds of 10-minute lessons.

Research Findings

• Key Discoveries:

  • Monitoring Accuracy: Engagement cues significantly reduced monitoring errors, enabling teachers to more accurately capture student engagement states.
  • Monitoring Efficiency: Eye-tracking data indicated more efficient monitoring behaviors with engagement cues, despite no increase in overall monitoring time.
  • Teaching Adjustments: Teachers were more inclined to adjust their teaching methods based on engagement cues, such as changing tone, asking questions, or accelerating content delivery.
  • Cognitive Load: When engagement cues indicated low student engagement, cognitive load increased significantly, particularly for teachers with less teaching experience.

• Comparison with Existing Solutions:

  • The authors emphasized that engagement monitoring interfaces designed with a human-centered approach can significantly improve monitoring quality while reducing cognitive burden caused by the complexity of traditional interfaces.

• Limitations and Future Directions:

  • Lack of Real-time Interaction: The use of pre-recorded videos limited dynamic interactions between teachers and students; future studies should validate monitoring systems in real classroom settings.
  • Camera Usage Scenarios: The experiment assumed all students kept their cameras on, which differs from actual online classroom scenarios where students may turn cameras off.
  • Monitoring Data Precision: Current experiments relied on eye-tracking data; future research should adopt more refined behavioral monitoring technologies.
  • Participant Diversity: This study focused on young, digitally proficient instructors; future research should include a broader range of ages, technical skills, and teaching methods.

Conclusion

This study systematically explored the effectiveness of real-time student monitoring interfaces in online teaching, summarizing the impact of engagement cues on monitoring quality, teaching behavior adjustments, and cognitive load. The research advocates for the design of customizable monitoring systems to meet diverse teaching needs, optimize online education experiences, and enhance teaching outcomes. It also provides valuable insights for future studies to validate monitoring systems in real teaching scenarios.