Decentralized collaborative optimal scheduling for EV charging stations based on multi-agent reinforcement learning

Charging behaviours of electric vehicles (EVs) exhibit substantial randomness, making accurate prediction or modelling challenging. Furthermore, as the number of EVs continues to increase, charging stations are diversifying their offerings to accommodate distinct charging characteristics, addressing a wide spectrum of EV charging needs. Previous research mostly focused on the randomness of EVs while neglecting the heterogeneity in charging infrastructure. Therefore, this paper introduces a decentralized collaborative optimal method for EV charging stations, taking into account the varying facility types and the power limitations. First, a decentralized collaborative framework is proposed. The energy boundary model and the average laxity of EVs contribute to transforming the optimization problem into a Markov Decision Process (MDP) with uncertain transitions. Then, multi-agent deep deterministic policy gradient multi-individuals (MADDPG-MI) algorithm is developed to train several heterogeneous agents presenting different types of charging facilities. Each agent makes decisions for multiple homogenous charging piles. Numerous simulation studies validate that the proposed method can effectively reduce charging costs and manages in scenarios involving either homogeneous or multiple heterogeneous charging facilities. Moreover, the MADDPG-MI algorithm demonstrates performance consistency among multiple decision-making units while consuming lower training resources offering enhanced scalability.