VizProg: Identifying Misunderstandings by Visualizing Students' Coding Progress

Document Title

VizProg: Identifying Misunderstandings By Visualizing Students’ Coding Progress

Document Information

• Subject Area: Programming Education and Real-Time Code Visualization
• Keywords: Programming Education, Code Visualization, Real-Time Monitoring, Student Behavior Analysis, Educational Technology, Learning Progress, Concept Misunderstanding, Teaching Performance Improvement

Research Background and Issues

• Problems/Challenges Identified by the Authors:

  1. During classroom programming exercises, teachers need to monitor students' progress in real time to identify struggling students and uncover misunderstandings. However, in large-scale classrooms, observing students' code in real time becomes challenging, especially during short exercises.
  2. Students' code often contains hidden learning obstacles that are not easily apparent from the surface of the code.
  3. Existing tools like Codeopticon and Overcode can help monitor students' submitted code, but the former struggles in large-scale classroom environments, and the latter does not support real-time monitoring.

• Significance: If issues are not identified promptly, students' misunderstandings may develop into long-term learning barriers, negatively impacting their learning outcomes. Addressing students' problems and providing feedback in real time is critical for improving overall learning quality.

• Research Motivation and Related Work: Based on interviews with programming instructors, the authors summarized the following needs and challenges:

  1. Teachers need to view students' coding progress and problem-solving processes in real time.
  2. A method is required to help teachers effectively verify students' progress in large-scale classrooms.
  3. The solution should provide personalized feedback to students while maintaining scalability.

Solution

• Proposed Method or Solution: The authors proposed a system called VizProg, which visualizes students' coding progress in real time using a two-dimensional map. This map encodes students' code positions based on:

  1. Problem-solving approach (vertical axis);
  2. Progress distance from the correct solution (horizontal axis);
  3. Temporal trajectory changes.

  Additionally, VizProg analyzes the semantic similarity and edit distance of code, helping teachers comprehensively understand the structure and progress of students' code.

• Innovative Aspects:

  1. This solution is the first system to visualize students' coding progress in real time using a two-dimensional map.
  2. It provides a human-readable, real-time encoded map that combines progress and problem-solving approaches.
  3. The system can display all students' real-time behavior and visually represent the temporal and spatial evolution of code.

• Implementation Steps and Key Technologies:

  1. Data Preprocessing: Includes code normalization and removal of unnecessary variables.
  2. Building the Two-Dimensional Map:
     • Use the CodeBERT model to obtain vectorized representations of code;
     • Apply PCA and T-SNE for dimensionality reduction to construct the solution space.
  3. Real-Time Code Visualization:
     • Compare students' code with existing solutions (correct code sets) for similarity;
     • Generate students' progress values in real time and display them on the two-dimensional map.

Research Outcomes

• Specific Results:

  1. VizProg helped teachers identify more than twice the number of misunderstandings in the classroom.
  2. Participants gained a more accurate understanding of students' problems and reduced time and interaction costs by over 50%.

• Comparison with Existing Solutions:

  1. Compared to Codeopticon, VizProg is better suited for large-scale classrooms; compared to OverCode, VizProg supports real-time code observation and considers the code editing process.
  2. VizProg provides more comprehensive visualization information, presenting each student's code evolution history through trajectories.

• Experimental or Evaluation Results:

  • The provided two-dimensional visualization significantly reduced teachers' cognitive load, enabling them to more efficiently identify struggling students and provide feedback.
  • In user studies, participants using VizProg demonstrated faster strategy formation and feedback generation.

• Limitations and Future Directions:

  1. The system currently requires a diverse and comprehensive solution set to initialize, and a lack of data may affect the quality of the two-dimensional map.
  2. Visualization may lead to information overload in large-scale classrooms. Future versions could explore updated encoding strategies to reduce cognitive stress.
  3. The system currently analyzes only code progress and cannot capture other forms of student learning behavior (e.g., pseudocode writing processes). Broader behavior capture and analysis could be a direction for future research.

Conclusion

VizProg provides a novel way to visualize students' coding progress in real time, aiding teachers in large-scale classroom environments. By transparently encoding students' progress, solutions, and processes into a two-dimensional map, the system effectively enhances teaching quality and understanding of student performance. Experimental results validate its superiority in identifying student issues, while offering potential directions for future research expansion.