A Visual Analytics Approach to Facilitate the Proctoring of Online Exams

Document Title

Using Visual Analytics to Enhance Online Exam Proctoring

Document Information

• Subject Area: Visual Analytics and Online Exam Proctoring Methods
• Keywords: Online Proctoring, Visual Analytics, Mouse Behavior Analysis, Head Pose Estimation, Cheating Detection, Online Exams, Data Visualization, Human-Computer Interaction, Student Behavior Analysis, E-learning

Research Background and Issues

• Problems and Challenges:

  • With the growth of online learning, online exams are increasingly used to assess students' knowledge, but the lack of face-to-face interaction makes proctoring difficult.
  • Online exams are vulnerable to various cheating behaviors, which may compromise their fairness and credibility.
  • Traditional proctoring methods (e.g., manual video review) are labor-intensive and time-consuming, while existing automated technologies face issues of accuracy and reliability.

• Significance:

  • Online exams are a crucial component of modern education, especially during the COVID-19 pandemic when many schools shifted to online teaching.
  • Ensuring the fairness of online exams is vital for maintaining trust, particularly for MOOC platforms and universities.

• Research Motivation:

  • Existing proctoring methods either heavily rely on human effort or are fully automated, but both struggle to achieve an optimal balance between efficiency and accuracy.
  • Combining machine learning with human interaction could be a promising direction for optimizing online exam proctoring.

Solution

• Methods and Approach:

  • A novel visual analytics method is proposed to assist online exam proctoring by analyzing video recordings and mouse movement data during exams.
  • The approach integrates human judgment with machine learning techniques and visualizes suspicious behaviors at three levels of detail.

• Innovations:

  • Data Selection: Utilizes head movements (e.g., head rotation or face disappearance) and mouse actions (e.g., copy-paste, leaving the exam interface) as indicators of suspicious behavior.
  • Visualization Design: Develops a three-tier interactive interface (overview of all students, question-level view, detailed behavior view) for efficient examination of potential cheating behaviors.
  • Low Hardware Requirements: Requires only a front-facing camera and a dynamic mouse data collection plugin, making it more feasible compared to solutions relying on multiple cameras or additional sensors.

• Implementation Steps and Key Technologies:

  1. Data Collection:
     • Record students' videos using a front-facing camera and design a lightweight JavaScript plugin to collect mouse movement data.
     • Data features include mouse events (e.g., "Blur," "Focus," "Copy," "Paste") and head movements (e.g., Yaw, Pitch angles).
  2. Cheating Detection Engine:
     • Face Detection: Uses the Faster R-CNN model to identify face bounding boxes.
     • Head Pose Estimation: Analyzes head rotation data in videos using deep learning models.
     • Mouse Interaction Behavior Marking: Detects abnormal events based on mouse trajectory data.
     • Risk Scoring: Automatically calculates cheating risk for each student and question based on the data.
  3. Multi-Level Visualization:
     • Student List View: Quickly filters high-risk students.
     • Question List View: Pinpoints high-risk areas within specific times and questions.
     • Behavior View: Provides detailed analysis of head and mouse dynamics, supporting validation of suspicious behaviors.

Research Outcomes

• Specific Results:

  • Visualization Technology: Designed an intuitive, multi-level visualization system for online exam proctoring.
  • Performance Validation: Conducted scenario-based case studies, user research, and expert interviews to comprehensively evaluate the method's effectiveness.
  • High Technical Adaptability: The system is low-cost and suitable for real-world online exam scenarios.
  • Development of Open-Source Tools: Provided web-based mouse behavior collection plugins and visualization system code.

• Advantages:

  • Efficiency: Significantly reduces the workload required for video review.
  • Accuracy: Enhances the reliability of cheating detection by combining human-computer interaction and multi-source data analysis.
  • Scalability: Adapts to exams of varying scales and different forms of cheating behavior analysis.

• Experiment and Evaluation Results:

  • Case studies demonstrate the system's ability to quickly locate and verify cheating behaviors, such as inconsistencies between mouse trajectories and head movements.
  • User testing shows improved accuracy (88.8%) and reduced task time (approximately 20% less time consumption) compared to traditional video review methods.
  • Experts unanimously agree that the system is innovative and has practical application value.

• Limitations and Future Directions:

  • Data Insufficiency: Simulated online exam data may not fully reflect real-world cheating methods and exam setups.
  • Real-Time Processing: The current model is primarily used for post-exam analysis and requires further optimization for real-time proctoring.
  • Privacy Concerns: Balancing academic integrity with the protection of students' personal privacy is necessary.
  • Adaptation Diversity: Future plans include extending the system to other online education scenarios, such as real-time anti-cheating or mouse behavior analysis in e-sports.