12:15 - 13:00
Parallel sessions 4
Submission 19
ThinkAI: Supporting Critical GenAI Evaluation Through Simulation
Presented by: Netta Soreq
Netta Soreq 1, Gila Kurtz 1, Alon Bartal 2, Yael Dubinsky 3, Eran Gal 1, Sigal Tifferet 4
1 Faculty of Instructional Technologies, Holon Institute of Technology (HIT), Israel
2 The Graduate School of Business Administration, Bar-Ilan University, Ramat Gan, Israel
3 Faculty of Technology Ramat Gan College RGC, Ramat Gan, Israel
4 Dept. of Business Administration, Ruppin Academic Center, Israel

The integration of generative artificial intelligence (GenAI) tools in higher education has led to increased attention to pedagogical approaches that foreground human judgment, critical thinking, and responsible engagement with AI-generated content (Yan et al., 2024). Although students frequently use GenAI systems, prior research documents gaps in competencies related to evaluation, verification, and reflective decision making within AI-supported tasks (Kasneci et al., 2023; Miao et al., 2024).

This paper presents ThinkAI, an AI-driven, simulation-based learning environment designed to support undergraduate students’ critical thinking in contexts involving GenAI use. Grounded in experiential learning theory (Kolb, 1984), simulation-based learning research (Chernikova et al., 2020), and the Promoting Expertise Through Simulation (PETS) framework (Jossberger et al., 2022), the ThinkAI simulation structures engagement through preparation, guided practice, feedback, and reflection. The design aligns with design-based research principles, positioning AI as a resource embedded within a human-centered decision-making process (Bannan-Ritland, 2003).

The paper reports findings from an ongoing, formative empirical study involving 76 undergraduate students who engaged with the ThinkAI simulation across multiple iterative deployments. Data sources included system-generated interaction logs and self-reported measures capturing engagement, confidence, and perceived learning outcomes. Analysis focuses on patterns of learner interaction, critical evaluation practices, and design-related insights that inform ongoing refinement of the simulation and research design, rather than on summative claims of effectiveness.

This work is presented as a study in progress and provides empirical, design-oriented, and methodological insights relevant to ongoing work on simulation-based learning and critical thinking in generative AI-supported higher education contexts