ETA-Hysteresis-Based Reinforcement Learning for Continuous Multi-Target Hunting of Swarm USVs

Swarm unmanned surface vehicles (USVs) have been increasingly explored for maritime defense and security operations, particularly in scenarios requiring the rapid detection and interception of multiple attackers. The target detection reliability and defender–target assignment stability are significantly crucial to ensure quick responses and prevent mission failure. A key challenge in such missions lies in the assignment of targets among multiple defenders, where frequent reassignment can cause instability and inefficiency. This paper proposes a novel ETA-hysteresis-guided reinforcement learning (RL) framework for continuous multi-target hunting with swarm USVs. The approach integrates estimated time of arrival (ETA)-based task allocation with a dual-threshold hysteresis mechanism to balance responsiveness and stability in multi-target assignments. The ETA module provides an efficient criterion for selecting the most suitable defender–target pair, while hysteresis prevents oscillatory reassignments triggered by marginal changes in ETA values. The framework is trained and evaluated in a 3D-simulated water environment with multiple continuous targets under static and dynamic water environments. Experimental results demonstrate that the proposed method achieves substantial measurable improvements compared to basic MAPPO and MAPPO-LSTM, including faster convergence speed (+20–30%), higher interception rates (improvement of +9.5% to +20.9%), and reduced mean time-to-capture (by 9.4–19.0%), while maintaining competitive path smoothness and energy efficiency. The findings highlight the potential of integrating time-aware assignment strategies with reinforcement learning to enable robust, scalable, and stable swarm USV operations for maritime security applications.