Submission 4
AI-Supported Course-Level Quality Assurance in Higher Education
Presented by: Jason Myrick
Artificial intelligence (AI) is increasingly proposed as a means of supporting quality assurance (QA) and learning design review in higher education. While prior research has examined AI applications in learning analytics and student-facing systems, there remains limited empirical work investigating how AI interacts with professional judgment during course-level QA processes. In particular, little is known about how educators interpret, negotiate, or contest AI-generated analyses when conducting evaluative review tasks.
This paper presents an exploratory, human-centered microstudy examining AI-supported course-level quality assurance in undergraduate education. The study focuses on a single, authentic QA task: reviewing alignment between course learning outcomes (CLOs) and course assessments. Two review conditions are compared: a human-only review using established QA criteria and professional judgment, and an AI-supported review in which the same task is conducted with the assistance of AI-generated analytical prompts. In the AI-supported condition, AI outputs are explicitly positioned as advisory, with reviewers retaining full authority over all evaluative decisions.
Rather than treating agreement between human and AI judgments as an indicator of correctness, the study treats divergence as analytically meaningful. A discrepancy classification framework is used to examine patterns of agreement, disagreement, and interpretive negotiation between review conditions. Data collection is currently underway and includes alignment judgments, reviewer rationales, reflective notes, and time-on-task measures.
By foregrounding professional reasoning and contextual interpretation, this work contributes a structured empirical approach for studying human-AI interaction in quality assurance contexts. The study demonstrates the feasibility of small-scale comparative designs in under-researched domains and contributes to ongoing discussions around human-centered AI, responsible adoption, and the role of professional judgment in higher education quality assurance processes.