Distributed Deep Reinforcement Learning for Hybrid Beamforming in Cooperative UAV-ISAC Networks

The Cooperative Integrated Sensing and Communication (C-ISAC) paradigm has emerged as a key enabler of unmanned aerial vehicle (UAV)-assisted sixth-generation (6G) networks, where precise target localization and high-rate data transmission must be achieved simultaneously under mobility, power, and backhaul constraints. The optimization problem is formulated as a multi-objective problem to jointly optimize UAV trajectories, hybrid beamforming strategies, and orthogonal frequency-division multiple access (OFDMA) resource allocation. The resulting mixed-integer non-convex problem is modeled as a cooperative Markov decision process (MDP) with constraint-aware reward shaping. A fusion center orchestrates subcarrier allocation and integrates multi-static echo information to support Extended Fisher Information Matrix (EFIM)-based sensing. To address this complex optimization, a distributed deep reinforcement learning (DDRL) framework is proposed, based on a centralized-training-decentralized-execution multi-agent deep deterministic policy gradient (MADDPG) algorithm. Simulation results demonstrate improvements of 20%-35% in spectral efficiency and sub-meter localization accuracy compared with centralized DDPG, greedy, and heuristic approaches.