A Human-Centered Review of Algorithms for Decision-Making in Higher Education

Title of the Paper

A Human-Centered Review of Algorithms in Decision-Making in Higher Education

Paper Information

• Subject Area: Algorithmic decision-making in higher education and its human-centered design framework
• Keywords: Human-centered machine learning, artificial intelligence, literature review, higher education, algorithm design, participatory design, fairness, learning analytics

Research Background and Issues

• Identified Problems or Challenges:
  • As algorithms are increasingly used in decision-making within higher education, the potential for data misuse and harmful decisions grows, particularly concerning data privacy, fairness, and algorithmic transparency.
  • Current technological designs primarily focus on improving model performance, often neglecting human-centered perspectives and potential societal impacts.
  • Few existing studies adopt a human-centered design framework to comprehensively examine the application of decision-making algorithms in higher education.
• Significance:
  • Algorithmic decision-making in higher education can have profound implications for high-stakes areas such as course planning, academic support, and admissions decisions.
  • Issues like fairness, interpretability, and potential biases will spark complex and far-reaching ethical discussions among students, universities, and the academic community.
• Research Motivation:
  • To develop more human-centered approaches to the design and implementation of educational algorithms, grounded in theoretical, participatory, and forward-looking design principles.
  • To investigate trends in existing literature on algorithms used in higher education, including input data, design objectives, and their relationship to human-centered design.

Solutions

• Proposed Methods or Solutions:
  • Conducting a systematic review of 62 studies on decision-making algorithms in higher education published between 2010 and 2021, examining their input data, target outcomes, and computational methods, and integrating the analysis with a human-centered design framework.
  • The human-centered algorithm design framework includes:
    1. Theoretical Design: Incorporating behavioral and social science theories to guide algorithm design.
    2. Participatory Design: Involving stakeholders (e.g., students, faculty) in the design and evaluation process of algorithms.
    3. Forward-Looking Design: Anticipating the potential societal impacts and bias risks of algorithms.
• Innovative Aspects:
  • Introducing a comprehensive human-centered design perspective in the literature analysis to explore pathways for achieving fair decision-making and enhancing algorithm transparency.
  • Providing a literature review centered on human-centered algorithm design, addressing a gap in research related to higher education.
• Implementation Steps and Techniques:
  • Using the PRISMA methodology to collect literature and conduct coding analysis based on input data, computational methods, and target variables.
  • Analyzing trends in algorithm versions over the years and observing improvements or regressions in interpretability and transparency.
  • Cross-analyzing the relationships between input data, target outcomes, computational methods, and human-centered design elements.

Research Outcomes

• Specific Findings:
  • Current algorithms are gradually shifting from rule-based systems to more complex models such as neural networks and natural language processing.
  • Increasing reliance on personalized student data (e.g., GPA, online learning activities) and protected attributes (e.g., age, gender, race).
  • While model complexity has increased, data transparency has declined, and most algorithms fail to systematically incorporate human-centered design dimensions.
  • Only a small number of studies involve educational or social theoretical support or include stakeholders such as students and faculty in the design process.
• Advantages Over Existing Solutions:
  • Systematically highlights the need for greater social responsibility among researchers, such as attention to the use of protected attributes and algorithmic fairness.
  • Provides a highly referential framework for future researchers to build transparent and interpretable educational algorithms.
• Experimental or Evaluation Results:
  • Over 75% of the reviewed studies employed machine learning, with 29 studies comparing multiple algorithms to enhance predictive capabilities.
  • Since 2018, the rapid adoption of natural language processing and deep learning has significantly reduced model interpretability.
  • In terms of target outcomes, most algorithms focus on predicting student performance and retention rates, while new objectives such as pathway recommendations and admissions management are emerging.
• Limitations and Future Directions:
  • Limitations:
    • Literature collection was limited to the ACM Digital Library, potentially overlooking relevant studies in other fields.
    • Did not fully explore the long-term or secondary impacts of such algorithms in actual higher education scenarios.
  • Future Directions:
    • Future research should involve more interdisciplinary collaboration (combining educational theory and data science).
    • Investigate the educational theoretical background and relevance of specific input variables (e.g., LMS behavioral data).
    • Establish stronger governance frameworks, particularly to encourage institutions to conduct comprehensive evaluations and transparent oversight of algorithm applications.

Conclusion

This paper systematically reviews the current state and potential issues of algorithmic decision-making in higher education, proposing methods to improve algorithm transparency and fairness through human-centered design. The study recommends integrating theoretical, participatory, and forward-looking design approaches to ensure that algorithm design and implementation align with complex social contexts. It also provides specific pathways for improvement to mitigate the adverse impacts of algorithms on student decision-making.