PRE-SERVICE TEACHERS’ EVALUATIVE JUDGMENTS AND THE "HUMAN-IN-THE-LOOP" PARADIGM IN AI-ASSISTED ASSESSMENT DESIGN
DOI:
https://doi.org/10.35631/IJEPC.1162079Keywords:
Classroom Assessment, Generative AI, Human-in-the-loop, Qualitative Method, University StudentsAbstract
The paradigm for education has changed from a traditional teacher-tool approach to a hybrid intelligence framework due to the quick rise of Generative Artificial Intelligence (GenAI). This study examines the assessments of pre-service teachers at a public university in Malaysia as they utilise GenAI to generate subjective assessment items, including essay questions and interpretative exercises. Even though GenAI provides previously unheard-of efficiency for structural scaffolding and brainstorming, research shows that raw AI outputs frequently fall back on Lower-Order Thinking Skills (LOTS) and lack contextual empathy for learning demands. This study examines the ways in which aspiring teachers evaluate AI-generated content through a qualitative reflective thematic analysis of 35 teacher-designed artefacts. The results show that although AI provides a good "starting point" for draughting, human interaction is necessary to guarantee cognitive calibration, cultural sensitivity, and marking reliability. In order to close the gap between generic AI outputs and regional educational realities, pre-service teachers served as "cultural mediators," manually incorporating ideals like Adl (Justice) and Sejahtera into evaluations. In order to guarantee that automated content creation promotes deep conceptual learning rather than unthinking dependency, this study indicates that the teacher must continue to serve as the "essential architect" and maintain the pedagogical compass.
Downloads
References
Addison, L., Ramdial, P. T., & Hosein, P. (2024). An investigation into the use of large language models for adaptive grading of course projects. In 2024 IEEE International Conference on Technology Management, Operations and Decisions (ICTMOD 2024). https://www.scopus.com/pages/publications/86000016073?origin=scopusAI
Banawan, M., Kukas, E. J., Tano, J. R., & Villegas, R. (2024). Transformative approach to fairness and transparency in classroom participation assessment. In L@S 2024: Proceedings of the 11th ACM Conference on Learning @ Scale. https://www.scopus.com/pages/publications/85199860877?origin=scopusAI
Bui, P., Korhonen, T., Kontkanen, S., Karme, S., Piispa-Hakala, S., & Veermans, M. (2025). Exploring pre-service teachers’ generative AI readiness and behavioral intentions: A pilot study. LUMAT: International Journal on Math, Science and Technology Education, 13(1), 8. https://doi.org/10.31129/LUMAT.13.1.2755
Busuttil, L. (2025). Leveraging generative AI in pre-service teacher training: Insights into assessment as learning. In Communications in Computer and Information Science. https://doi.org/10.1007/978-3-031-94002-6_4
Chen, X., & Pi, X. (2025). Human-in-the-loop: A conceptual framework for Business English teachers’ AI‑empowered assessment. In Lecture Notes in Computer Science. https://www.scopus.com/pages/publications/105003621373?origin=scopusAI
Dayagbil, F. T., Boholano, H. B., & Sumalinog, G. G. (2025). Are they in or out? Exploring pre-service teachers’ knowledge, perceptions, and experiences regarding artificial intelligence (AI) in teaching and learning. Frontiers in Education. https://www.scopus.com/pages/publications/105018501353?origin=scopusAI
Dockens, A. L., & Shelton, K. (2025). AI for formative and summative assessment: A balanced approach. In Emerging Trends, Global Perspectives, and Systemic Transformation in AI. https://www.scopus.com/pages/publications/105017950051?origin=scopusAI
Erdem Coşgun, G. (2025). Artificial intelligence literacy in assessment: Empowering pre-service teachers to design effective exam questions for language learning. British Educational Research Journal. https://www.scopus.com/pages/publications/105002217074?origin=scopusAI
Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/10.1111/bjet.13544
Ishmuradova, I. I., Zhdanov, S. P., Kondrashev, S. V., Erokhova, N. S., Grishnova, E. E., & Volosova, N. Y. (2025). Pre-service science teachers’ perception on using generative artificial intelligence in science education. Contemporary Educational Technology, 17(3), ep579. https://doi.org/10.30935/cedtech/16207
Jho, H., & Lee, B. (2025). Analysis of pre-service science teachers’ inquiry design through interaction with an AI inquiry assistant. New Physics: Sae Mulli. https://www.scopus.com/pages/publications/105017055644?origin=scopusAI
Kohnke, L., Zou, D., Ou, A. W., & Gu, M. M. (2025). Preparing future educators for AI-enhanced classrooms: Insights into AI literacy and integration. Computers and Education: Artificial Intelligence. https://www.scopus.com/pages/publications/105000935937?origin=scopusAI
Lampropoulos, G., & Papadakis, S. (2025). The educational value of artificial intelligence and social robots. In The educational value of artificial intelligence and social robots. (pp 3-15). Springer. https://doi.org/10.1007/978-3-031-82915-4_1
May, T. A., Provinzano, K., Koskey, K. L. K., & Rimkunas, N. (2025). Expanding the team: Integrating generative artificial intelligence into the assessment development process. Applied Sciences (Switzerland). https://www.scopus.com/pages/publications/105017225799?origin=scopusAI
Miao, F., & Holmes, W. (2023). Guidance for generative AI in education and research. UNESCO Publishing.
Naidu, K., & Sevnarayan, K. (2025). Culturally relevant assessment and generative AI: A co-creation framework for pre-service teachers. Journal of Applied Learning & Teaching, 8(2). https://doi.org/10.37074/jalt.2025.8.2.20
Navío-Inglés, M., Guzmán Mora, J., O’Connor-Jiménez, P., & García González, A. (2025). What’s next for feedback in writing instruction? Pre-service teachers’ perceptions of assessment practices and the role of generative AI. Education Sciences, 15(11), 1534. https://doi.org/10.3390/educsci15111534
Nguyen, A., Duong, A. T., Nguyen, D. T. B., Lai, V. T. T., & Dang, B. (2025). Guidelines for learning design and assessment for generative artificial intelligence-integrated education: A unified view. Information and Learning Sciences, 126(7–8), 491–512. https://doi.org/10.1108/ILS-11-2024-0148
O’Dea, X. (2024). Generative AI: is it a paradigm shift for higher education? Studies in Higher Education, 49(5), 811–816. https://doi.org/10.1080/03075079.2024.2332944
OECD. (2023). Generative AI in the classroom: From hype to reality? OECD Publishing.
Oh, S. (2025). Integration of MATH41 and generative AI in pre-service mathematics teacher education: An empirical study on lesson design competency. IEEE Access, 13. https://doi.org/10.1109/ACCESS.2025.3586593
Oktay, O. S., & Kilinç, H. (2025). Pre-service teachers' perspectives on the use of artificial intelligence in learning processes: Biases, advantages, and disadvantages. In Policy and Practice for Ethical AI Integration in Education. https://www.scopus.com/pages/publications/105024037881?origin=scopusAI
Öz, B., Wang, R., Chandler, C., & Léger, P.-M. (2022). Responsible artificial intelligence in knowledge work: User experience design problems and implications. In Lecture Notes in Computer Science. https://www.scopus.com/pages/publications/85144404160?origin=scopusAI
Prilop, C. N., Mah, D.-K., Jacobsen, L. J., Hansen, R. R., Weber, K. E., & Hoya, F. (2025). Generative AI in teacher education: Educators’ perceptions of transformative potentials and the triadic nature of AI literacy explored through AI-enhanced methods. Computers and Education: Artificial Intelligence, 9, 100471. https://doi.org/10.1016/j.caeai.2025.100471
Puerta-Beldarrain, M., Gómez-Carmona, O., Sánchez-Corcuera, R., Casado-Mansilla, D., López-de-Ipiña, D., & Chen, L. (2025). A multifaceted vision of the human-AI collaboration: A comprehensive review. IEEE Access. https://doi.org/10.1109/ACCESS.2025.3536095
Rojas‑Contreras, M., Orjuela Duarte, A., & Marina Santos Jaimes, L. (2025). Human‑in‑the‑loop (HITL) as a verification and validation strategy for knowledge generated by generative artificial intelligence. In Proceedings of the LACCEI International Multi‑Conference for Engineering, Education and Technology. https://www.scopus.com/pages/publications/105019318086?origin=scopusAI
Tanik Önal, N., Önal, N., & Ilhan, A. G. (2026). Artificial intelligence in early childhood classrooms: Insights from in-service and pre-service teachers. European Early Childhood Education Research Journal. https://www.scopus.com/pages/publications/105026582759?origin=scopusAI
Tocchetti, A., Bianchi, M., Campi, R., & Brambilla, M. (2025). On the principles and effectiveness of gamification in bidirectional artificial intelligence and explainable AI. In Bi‑directionality in Human‑AI Collaborative Systems. https://www.scopus.com/pages/publications/105019729263?origin=scopusAI
Uğraş, H., Uğraş, M., Papadakis, S., & Kalogiannakis, M. (2024). Innovative early childhood STEM education with ChatGPT: Teacher perspectives. Technology, Knowledge and Learning, 30(2), 809–831. https://doi.org/10.1007/s10758-024-09804-8
