16:30 - 17:30
Parallel sessions 11
Submission 195
Saved by the Bot! A Comparative Analysis of Human vs. AI Tutors on Formative Assessment Performance of Computer Science Students in Higher Education
Presented by: Mark P. McCormack
Mark P. McCormackJohn G. Keating
Maynooth University

Generative AI in Higher Education has become a prevalent point of discussion for practitioners and researchers alike. Current literature looks at such technologies either on negative aspects such as plagiarism and cheating [1], or on the positive uses of this technology for assessment [2], with little discussion focusing on the integration of such technologies as digital tutors. In this study, we investigate the performance different in Computer Science students results in a formative assessment activity between groups using a human tutor and another using AI tutors. The AI tutors in this study were constructed using Microsoft Copilot and were prompted using the CRTRF framework [3]. Findings suggest that there are negligible differences between these groups in post-test results, suggesting that AI tutors are just as effective as human tutors, and can be effectively leveraged and scaled to assist in resolving Blooms “Two Sigma Problem” [4].