Toward Scalable and Responsible Integration of Course-Specific AI Tutors: Instructor Experiences with a Campus-Wide Platform

Paper Title

Toward Scalable and Responsible Integration of Course-Specific AI Tutors: Instructor Experiences with a Campus-Wide Platform

Publication Info

• Topic area: Integration of generative AI tutors in higher education.
• Keywords: AI tutors, higher education, generative AI, instructional design, responsible AI, scalability, pedagogy, ethics, multimodal learning, faculty development.

Background and Problem

• Problem / challenge: While generative AI tools are increasingly adopted in higher education, there is limited empirical research on how instructors design, evaluate, and implement course-specific AI tutors. Existing platforms often lack guidance, infrastructure, and professional development for instructors, creating barriers to scalable adoption.
• Significance: Understanding how instructors use AI tutors can inform the design of platforms that align with pedagogical goals, address diverse educational challenges, and ensure responsible AI integration.
• Motivation and related work: Prior research on Intelligent Tutoring Systems (ITS) and generative AI tools has focused on student learning outcomes rather than instructor practices. Studies like Yoo et al. (2022) explored K–12 chatbot use but lacked relevance for higher education. This study fills the gap by examining instructor-driven adoption of AI tutors at scale.

Solution

• Proposed approach: UT Sage, a course-specific AI tutor creation platform, designed to support both student learning and instructor-driven pedagogical goals.
• Novelty:
  1. Empirical insights into how instructors design, evaluate, and implement course-specific AI tutors.
  2. Identification of pedagogical, technical, and ethical challenges in AI tutor adoption.
  3. Design recommendations for scalable, responsible AI tutor platforms.
  4. Exploration of discipline-specific and contextual variations in AI tutor use.
• Procedure and key techniques:
  • Conducted interviews with 20 instructors, TAs, and instructional designers across STEM and humanities disciplines.
  • Analyzed instructor experiences with UT Sage, focusing on tutor creation, evaluation, and implementation.
  • Used thematic analysis to identify patterns in instructor practices, challenges, and ethical considerations.

Results

• Concrete findings:
  • Instructors used UT Sage to address cognitive (e.g., knowledge gaps), affective (e.g., anxiety), and social (e.g., group dynamics) challenges.
  • Evaluation practices included informal testing, discipline-specific stress tests, and benchmarking against instructional values.
  • Implementation revealed technical barriers (e.g., lack of multimodal support, LMS integration) and ethical concerns (e.g., fairness, data governance).
• Advantage over baselines: UT Sage provided pedagogical guardrails and discipline-specific customization, addressing gaps in general-purpose AI tools like ChatGPT.
• Experiments / evaluation:
  • Participants included 20 educators across disciplines and roles, with varied teaching experience and AI familiarity.
  • Data collection involved interviews, iterative tutor creation, and follow-up reflections.
  • Analysis identified patterns in tutor design, evaluation, and implementation across contexts.
• Limitations and future work:
  • Single-institution study limits generalizability.
  • Lack of direct student data restricts insights into learning outcomes.
  • Early-stage findings require longitudinal validation.
  • Future research should explore multimodal capabilities, scalable tutor design, and equitable AI access.

Summary

This study investigates how instructors design, evaluate, and implement course-specific AI tutors using UT Sage, a platform tailored for higher education. Findings reveal that instructors adapt AI tutors to address diverse cognitive, affective, and social challenges, but face technical and ethical barriers such as multimodal limitations and fairness concerns. Evaluation practices remain inconsistent, highlighting the need for platform-supported workflows and faculty development. The study underscores the potential of AI tutors to enhance learning when integrated responsibly, with recommendations for scalable, discipline-aware, and ethically grounded platform design.