Constrained Stochastic Non-Convex Optimization Algorithm for Machine Learning Problems in Electromagnetic Signal Processing

This work introduces a novel optimization algorithm, Constrained STochastic Recursive Momentum Successive Convex Approximation (CoSTA), designed for solving general stochastic non-convex learning problems with non-convex constraints. It finds applications across diverse domains, including electromagnetic signal processing, radar target identification, and signal classification, with human detection using UWB radar being one such example. CoSTA provides a robust solution to these problems. The proposed framework leverages successive convex approximation (SCA) techniques combined with momentum-based gradient tracking to achieve near-optimal stochastic first-order (SFO) complexity of Õ(ϵ^-3/2), almost matching theoretical lower bounds for unconstrained problems with an adaptive step size. By iteratively constructing convex surrogates for both non-convex objectives and constraints, CoSTA ensures efficient and scalable optimization. Furthermore, it introduces a parameterized Mangasarian-Fromovitz Constraint Qualification (MFCQ) condition to guarantee bounded dual variables and convergence to an ϵ-stationary point. We detail the steps to perform human detection using UWB radar. The proposed method outperforms specialized methods, such as the Level-Constrained Proximal Point (LCPP) algorithm, and converges faster than it. To demonstrate its effectiveness, CoSTA is applied to sparsity-constrained binary classification tasks on Human detection dataset where it is shown to converge faster than LCPP, and on the gisette and MNIST datasets, where it achieves classification accuracies as high as 99.7%. As such, CoSTA holds significant potential for machine learning applications in radar, satellite, and space-borne systems.