Rethinking Teaching Evaluation Reports: Designing AI-transformed Student Feedback for Instructor Engagement

Student evaluations of teaching (SETs) represent a valuable yet often underutilized resource, as many instructors struggle with the substantial time, cognitive, and emotional demands of processing this feedback effectively. While these evaluations contain crucial insights into students' learning experiences that could enhance instruction, their potential remains largely untapped. Our work explores how to redesign SET reports using language models (LMs) to distill, highlight, and present student feedback in more engaging and actionable ways. We systematically explored a $4 \times 4$ strategy-presentation design space, creating six representative mock-ups that integrate different analytical strategies with various presentation formats. Through interviews with 16 post-secondary instructors, we learned how and when they engage with current SETs, and how they would perceive and use the LM-augmented redesigned SET mock-ups. Our findings revealed that instructors' preferences for different redesigns aligned with distinct goals: whether improving their teaching practices, gaining quick insights into their teaching effectiveness, or preparing summative teaching performance reports. These findings shed light on new opportunities for designing dynamic SET systems where AI can adaptively process and present feedback based on instructors' specific needs and contexts.