Showing Academic Performance Predictions during Term Planning: Effects on Students' Decisions, Behaviors, and Preferences

Title of the Paper

Predicting Academic Performance in Semester Planning: Impacts on Student Decisions, Behaviors, and Preferences

Bibliographic Information

• Subject Area: Academic decision support systems, particularly the intersection of course selection and academic performance prediction
• Keywords: Visual learning analytics, academic performance prediction, framing effects, course selection, course recommendation, learning analytics dashboards

Research Background and Issues

• Main Problems or Challenges:

  • Course selection directly affects students' workload and academic performance, yet this process is often time-consuming, subjective, and based on incomplete information.
  • Current academic advising primarily relies on the knowledge and experience of advisors, which, due to time constraints, can be prone to errors and personal biases.
  • Higher education institutions use GPA as the primary metric for evaluating academic performance, but GPA does not fully reflect students' learning outcomes or abilities.
  • In designing academic decision-making tools, presenting data and predictions in an appropriate manner to avoid potential negative impacts (e.g., over-reliance on GPA) is a significant challenge.

• Significance of the Research:

  • Improving academic and course recommendation tools can help students plan their semester courses more efficiently and with more comprehensive information.
  • Exploring the impact of data-driven tools on student behaviors and decision-making processes is a crucial research direction in the field of learning analytics.

• Motivation and Related Work:

  • Existing studies indicate that exposing students to GPA and historical performance data may lead them to choose courses that are easier to score highly in, fostering an overemphasis on GPA.
  • Research shows that different data visualization methods influence users' understanding and decision-making, and studying this effect can help optimize the design of academic visualization systems.

Solution

• Methods and Solutions:

  • Proposed iCoRA (Interactive Course Recommendation Assistant), an interactive visualization tool to support students in semester planning.
  • The tool uses historical data and predictive models to display course performance predictions to students, along with explanations of the predictions and related course information.
  • iCoRA's predictive model is trained using Gradient Boosting Trees (GBT), incorporating multidimensional data such as course workload and past performance, and visualizes prediction intervals.

• Innovations:

  • Investigated the design methods of predictive visualizations that influence student decisions by creating a "specific-to-ambiguous" visual representation spectrum to explore how visualization styles shape decision-making processes.
  • Provided an in-depth discussion of potential ethical issues related to GPA and performance prediction tools in academic contexts.

• Implementation Steps and Techniques:

  • Conducted a two-phase study on the tool's impact:
    1. Qualitative study: Analyzed students' course selection behaviors when using tools with range-based predictions.
    2. Quantitative study: Tested eight different predictive visualization styles to determine whether they led to changes in decision-making behaviors.
  • Employed semantically similar visual representations (e.g., colors, lines, icons, and text) and validated findings through task data analysis and survey results.

Research Outcomes

• Specific Findings:

  • Found that when students viewed GPA and performance predictions, they often treated course selection as a "grade maximization problem," potentially overlooking other factors (e.g., workload).
  • Identified the influence of certain visualization types (e.g., specific vs. ambiguous) on students' trust in the tool and clarified preferences for visualization designs (e.g., range-based predictions and textual descriptions).
  • Analysis revealed that "specific" predictions (e.g., single values) prompted deeper thinking and more interactive choices from students.

• Advantages Over Existing Solutions:

  • Compared to traditional GPA-driven recommendation tools, iCoRA provides comprehensive predictive explanations and integrates multidimensional factors.
  • Addressed the lack of transparency in explanations and student engagement in many existing learning analytics dashboards.

• Experimental Results and Evaluation:

  • Experiments confirmed that the visualization style of predictions (specific vs. ambiguous) significantly influenced students' course selection and behavioral processes.
  • Students preferred range-based predictions for their ability to reflect uncertainty in predictions more realistically, while textual predictions were highly rated for their directness.

• Limitations and Future Directions:

  • Due to experimental constraints, virtual course history data and non-real courses were used.
  • Using GPA as the sole dimension for predicting academic performance is limiting; future research should explore more comprehensive performance metrics (e.g., learning outcomes, course engagement).
  • Before practical deployment, further discussions with university administrators are needed to address ethical concerns and better balance the roles of tools and advisors in course recommendations.