RT Journal Article
T1 Fast and Accurate Incremental Feedback for Students’ Software Tests Using Selective Mutation Analysis.
A1 Kazerouni, Ayaan M.
A1 Davis, James C.
A1 Basak, Arinjoy
A1 Shaffer, Clifford A.
A1 Servant-Cortés, Francisco Javier
A1 Edwards, Stephen H.
A2 Wong, W. Eric
K1 Software - Evaluación
AB As incorporating software testing into programming assignments becomes routine, educators have begun to assess not only the correctness of students’ software, but also the adequacy of their tests. In practice, educators rely on code coverage measures, though its shortcomings are widely known. Mutation analysis is a stronger measure of test adequacy, but it is too costly to be applied beyond the small programs developed in introductory programming courses. We demonstrate how to adapt mutation analysis to provide rapid automated feedback on software tests for complex projects in large programming courses. We study a dataset of 1389 student software projects ranging from trivial to complex. We begin by showing that although the state-of-the-art in mutation analysis is practical for providing rapid feedback on projects in introductory courses, it is prohibitively expensive for the more complex projects in subsequent courses. To reduce this cost, we use a statistical procedure to select a subset of mutation operators that maintains accuracy while minimizing cost. We show that with only 2 operators, costs can be reduced by a factor of 2–3 with negligible loss in accuracy. Finally, we evaluate our approach on open-source software and report that our findings may generalize beyond our educational context.
PB Elsevier
YR 2021
FD 2021
LK https://hdl.handle.net/10630/35532
UL https://hdl.handle.net/10630/35532
LA eng
NO Ayaan M. Kazerouni, James C. Davis, Arinjoy Basak, Clifford A. Shaffer, Francisco Servant, Stephen H. Edwards, Fast and accurate incremental feedback for students’ software tests using selective mutation analysis, Journal of Systems and Software, Volume 175, 2021, 110905, ISSN 0164-1212, DOI: https://doi.org/10.1016/j.jss.2021.110905.
NO NSF DUE-1625425 and DLR-1740765.
DS RIUMA. Repositorio Institucional de la Universidad de Málaga
RD 2 mar 2026