12:15 - 13:00
Parallel sessions 4
12:15 - 13:00
Submission 39
Implementing Generative AI Tutoring at Scale: Student Adoption and Impacts
Presented by: Caitlin Kirby
Caitlin Kirby
Michigan State University
University of Tübingen

Generative AI tutoring could provide scalable support for students, yet real-world evaluations remain limited. A large United States university piloted 790 licenses for Khanmigo, a generative AI tutoring tool, targeting academically at-risk students and those enrolled in the university’s lowest level mathematics course. We analyzed usage logs, demographics, grades, and student reflections to examine student adoption, relationships with mathematics grades, and student experiences. Student adoption was low: 17% of students used Khanmigo at least once. Course-based integration produced the highest usage (98%), while email-only outreach resulted in substantially lower engagement. First-generation students engaged in significantly more chats than continuing-generation peers. Among students in the mathematics course, regression models controlling for cumulative GPA, race/ethnicity, and course section showed no significant relationship between Khanmigo interactions and course grades. Qualitative responses reflected mixed experiences, with students citing benefits such as step-by-step practice and improved conceptual understanding, alongside frustrations with login barriers and limitations with visuals. Course-based implementation appears critical for adoption, but improved implementation and tool design are necessary to support meaningful impact at scale.