Cooperative Variable Speed Limit Control using Multi-agent Reinforcement Learning and Evolution Strategy for Improved Throughput in Mixed Traffic

Improving the traffic throughput in mixed traffic scenarios including both human-driving vehicles and Connected and Automated Vehicles (CAVs) has long been a hot spot in automated driving. In recent years, variable speed limit (VSL) has been a promising solution and attracts considerable attention from both industry and academy. In this paper, a multi-agent reinforcement learning model and evolution strategy-based approach is proposed to provide both macroscopic and microscopic control in mixed traffic scenarios. In this approach, Graph Attention Networks (GATs) are introduced into Deep Q-Networks for vehicles' decision making. The architecture of the VSL network is designed using an evolution strategy to provide real-time speed limit. A dedicated reward function has been implemented to consider both the actions and speed limit. Extensive experiments are conducted focusing on Bottleneck networks. The experimental results show that the proposed approach has demonstrated superior performance compared with other baselines in terms of several metrics such as throughput, average speed, and safety.