Technical Report MSC-2021-08

Title: Better Prediction of Mutation Score Dor
Authors: Dor David Ma’ayan
Supervisors: Yossi Gil
PDFCurrently accessibly only within the Technion network
Abstract: Mutation score is widely accepted to be a reliable measurement for the effectiveness of software tests. Recent studies, however, show that mutation analysis is extremely costly and hard to use in practice. The research community invested significant efforts to reduce these costs; however, no practical and real-time solutions were proposed so far. We present a novel direct prediction model of mutation score using neural networks. Relying solely on static code features that do not require generation of mutants or execution of the tests, we predict mutation score with an accuracy better than a quintile. When we include statement coverage as a feature, our accuracy rises to about a decile. Using a similar approach, we also gain a 4% improvement in the state-of-the-art results for binary test effectiveness prediction. Intending to propose a practical approach, we introduce an intuitive, easy-to-calculate set of features based on a simple analysis of the Code’s Abstract Syntax Tree (AST). Our new set of features is superior to previously studied sets of features in its prediction performance, and it also significantly reduces the engineering trade-offs involved in the extraction process. We publish the largest dataset of test-class level mutation score and static code features to date, for future research. Finally, we discuss how our approach could be integrated into real-world systems, IDEs, Continuous Integration (CI) tools, and testing frameworks and the implication of our findings for future studies in the field.
CopyrightThe above paper is copyright by the Technion, Author(s), or others. Please contact the author(s) for more information

Remark: Any link to this technical report should be to this page (, rather than to the URL of the PDF files directly. The latter URLs may change without notice.

To the list of the MSC technical reports of 2021
To the main CS technical reports page

Computer science department, Technion