Deep learning approaches for bad smell detection: a systematic literature review

Context: Bad smells negatively impact software quality metrics such as understandability, reusability, and maintainability. Reduced costs and enhanced software quality can be achieved through accurate bad smell detection. Objective: This review aims to summarize and synthesize the studies that used deep learning (DL) techniques for bad smell detection. Given the rapid growth of DL techniques, we believe that reviewing and analyzing the current body of knowledge would facilitate the development of new techniques and help researchers identify research gaps in this area. Method: We followed a systematic approach to identify 67 studies on DL-based bad smell detection published until October 2021. We collected and analyzed quantitative and qualitative data to obtain our results. Results: Code Clone was the most recurring smell. Supervised learning is the most adopted learning approach for DL-based bad smell detection. Convolutional neural network (CNN), Artificial neural network (ANN), Deep neural network (DNN), Long short-term memory (LSTM), Attention model, and recursive autoencoder (RAE) are the most popularly used DL models. DL models that efficiently detect bad smells, such as Tree-based CNN (TBCNN) and the Abstract syntax tree-based LSTM (AST-LSTM), tend to be specifically designed to encode features for bad smell detection. Conclusion: Many factors can affect the detection performance of DL models. Although studies exist on DL-based bad smell detection, more works that use other DL models than those already studied are needed. In this SLR, we provide a summary of existing research and recommendations for further research directions on DL-based bad smell detection.