Generative AI in the Software Modeling Classroom: An Experience Report With ChatGPT and Unified Modeling Language

Abstract

Formative assessment in software modeling typically involves students solving problems with limited support from instructors or on their own without any automated support. However, the irruption of generative AI chatbots, such as ChatGPT, has entirely changed the landscape in the modeling classroom, triggering the need to re-evaluate how both formative and summative assessments are designed and administered to students. This article describes an experience report on the use of ChatGPT in formative assessment for software modeling in an undergraduate software engineering course. Our study indicates that the use of generative AI chatbots in formative assessment can contribute to effectively gauge learning progress, increase the academic performance of students, compared to a traditional methodology, and also raise student awareness about the tradeoffs of employing generative AI in their work.