Metaheuristic Approach to Artificial Neural Network Optimization

The performance of Artificial Neural Networks (ANNs) depends on the tuning quality of the parameters, such as weights, learning rate, hyperparameters, number of layers and number of neurons. Typical tuning approaches include gradient descent based methods, which are applicable for smooth functions and perform at their best for convex problems. However, ANNs are non-convex and they may benefit from non-smooth functions. Thus, finding new innovative techniques to overcome these limitations is a priority for researchers in the field. Metaheuristic (MH) algorithms can be an alternate option to estimate the optimal parameters of ANNs. Since most MH algorithms have a high capability in exploration or exploitation of non-convex problems, developing customized MH algorithms will be a fruitful area for parameter search of ANNs. In this work, our focus is on developing a Variable Neighborhood Search (VNS) approach that overcomes the limitations of conventional methods in the parameter search of ANNs.