Deep Reinforcement Learning-Based Model-Free Secondary Frequency Control of Widespread Islanded Microgrid with Stability Constraints

This paper proposes a novel model-free control of an islanded microgrid (MG) using value- and policy-based deep reinforcement learning (DRL) for secondary frequency regulation, considering the nonlinear behavior of renewable energy sources (RES), load variations, and limitations of model-based techniques. The proposed DRL incorporates both value-based deep-Q learning (DQN) and policy-based proximal policy optimization (PPO) agents. The employed MG test-bench environment includes 300 small-capacity distributed energy resources. The proposed model-free control is compared with the existing distributed model predictive controller (MPC), adaptive linear quadratic regulator (LQR), and proportional–integral–derivative with a first-order filter (PIDF). The results depict a 91.9% and 61.6% reduction in average secondary frequency deviation, obtained using value- and policy-based proposed DRL methodology as compared to model-based optimal control techniques. Additionally, the proposed technique optimizes the operational cost of the system by merging observations of both agents. Also, the system stability is validated by comparing the integral square error (ISE) with prior model-based techniques.