Skip to main navigation Skip to search Skip to main content

Integrating Risk-Averse and Constrained Reinforcement Learning for Robust Decision-Making in High-Stakes Scenarios

  • Moiz Ahmad
  • , Muhammad Babar Ramzan
  • , Muhammad Omair
  • , Muhammad Salman Habib*
  • *Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

2 Scopus citations

Abstract

This paper considers a risk-averse Markov decision process (MDP) with non-risk constraints as a dynamic optimization framework to ensure robustness against unfavorable outcomes in high-stakes sequential decision-making situations such as disaster response. In this regard, strong duality is proved while making no assumptions on the problem’s convexity. This is necessary for some real-world issues, e.g., in the case of deprivation costs in the context of disaster relief, where convexity cannot be ensured. Our theoretical results imply that the problem can be exactly solved in a dual domain where it becomes convex. Based on our duality results, an augmented Lagrangian-based constraint handling mechanism is also developed for risk-averse reinforcement learning algorithms. The mechanism is proved to be theoretically convergent. Finally, we have also empirically established the convergence of the mechanism using a multi-stage disaster response relief allocation problem while using a fixed negative reward scheme as a benchmark.

Original languageEnglish
Article number1954
JournalMathematics
Volume12
Issue number13
DOIs
StatePublished - Jul 2024
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2024 by the authors.

Keywords

  • Markov risk
  • augmented Lagrangian
  • constrained reinforcement learning
  • dynamic decision-making
  • non-convexities
  • robust decision-making

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • General Mathematics
  • Engineering (miscellaneous)

Fingerprint

Dive into the research topics of 'Integrating Risk-Averse and Constrained Reinforcement Learning for Robust Decision-Making in High-Stakes Scenarios'. Together they form a unique fingerprint.

Cite this