Skip to main navigation Skip to search Skip to main content

Constrained Stochastic Recursive Momentum Successive Convex Approximation

  • Basil M. Idrees
  • , Lavish Arora
  • , Ketan Rajawat*
  • *Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

We consider stochastic optimization problems with non-convex functional constraints, such as those arising in trajectory generation, sparse approximation, and robust classification. To this end, we put forth a recursive momentum-based accelerated successive convex approximation (SCA) algorithm. At each iteration, the proposed algorithm entails constructing convex surrogates of the stochastic objective and the constraint functions, and solving the resulting convex optimization problem. A recursive update rule is employed to track the gradient of the stochastic objective function, which contributes to variance reduction and hence accelerates the algorithm convergence. A key ingredient of the proof is a new parameterized version of the standard Mangasarian-Fromowitz Constraints Qualification, that allows us to bound the dual variables and hence obtain problem-dependent bounds on the rate at which the iterates approach an \epsilon-stationary point. Remarkably, the proposed algorithm achieves near-optimal stochastic first-order (SFO) complexity with adaptive step sizes closely matching that achieved by state-of-the-art stochastic optimization algorithms for solving unconstrained problems. As an example, we detail an obstacle-avoiding trajectory optimization problem that can be solved using the proposed algorithm and show that its performance is superior to that of the existing algorithms used for trajectory optimization. The performance of the proposed algorithm is also shown to be comparable to that of a specialized sparse classification algorithm applied to a binary classification problem.

Original languageEnglish
Pages (from-to)3331-3345
Number of pages15
JournalIEEE Transactions on Signal Processing
Volume73
DOIs
StatePublished - 2025
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 1991-2012 IEEE.

Keywords

  • Stochastic optimization
  • accelerated gradient descent
  • successive convex approximation

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Constrained Stochastic Recursive Momentum Successive Convex Approximation'. Together they form a unique fingerprint.

Cite this