Prioritization of model smell refactoring using a covariance matrix-based adaptive evolution algorithm

Context: The refactoring process enhances the software design by modifying the structure of design parts impaired with model smells without altering the overall software behavior. However, handling these smells without proper prioritization will not produce the anticipated effects. Objective: In this paper, we solve the prioritization of the model smell refactoring using a multi-objective optimization (MOO) algorithm called the multi-objective (MO) covariance matrix adaptation evolution strategy (MO[sbnd]CMA-ES). Our formulation relies on the refactoring of unified modeling language (UML) class diagrams to mitigate the negative effect of design smells. Method: We treat the prioritization problem as a real-valued MOO where we propose novel data encoding procedures. We use two conflicting objectives, quality, and maintainability, to balance the refactoring. We first build a new solution representation that guarantees smell fixing and eliminates the rejection limitation. Furthermore, we suggest a custom mapping scheme to properly encode real-valued quantities using unique representations. For performance evaluation purposes, we developed a large custom dataset with more than 30,000 class records, using seven popular open-source software projects. A novel relative coverage metric is proposed to mitigate the limitations associated with the standard coverage. For benchmarking purposes, we also consider an improved version of the nondominated sorting genetic algorithm (NSGA-II(. Results: The reported performance scores confirm the superiority of the MO[sbnd]CMA-ES algorithm over NSGA-II. The former successfully identified the refactoring sequences that lead to the best improvements in software quality and maintainability while it is able to fix all identified design smells. These improvements are quantified in terms of hypervolume, coverage, spacing metrics, and execution time. Conclusion: The MO[sbnd]CMA-ES attained the highest average maximum quality score of 1149 while keeping the average maintainability at the lowest score of 13.8. In all experiment settings, the proposed solution leads to longer refactoring sequences at no additional computational cost.