Multi-Agent Reinforcement Learning for Trajectory and Handover Optimization in UAV-Assisted 6G HetNets

UAV-assisted heterogeneous networks (HetNets) in urban environments face significant challenges in optimizing trajectory, rate, and handover (HO) decisions under dense mobility and interference conditions. Existing approaches often fail to jointly address these factors, leading to inefficient resource allocation, frequent HO failures, and degraded user experience. This paper presents a multi-agent reinforcement learning (MARL) framework that enables cooperative UAV agents to optimize trajectory planning, network rate, and HO performance simultaneously. The model integrates signal-to-interference-plus-noise ratio (SINR), achievable rate, and HO-related parameters into the probability of identification (POI), ensuring accurate user association and sustained connectivity. A reward function is designed to balance rate maximization, HO frequency reduction, and trajectory efficiency, guiding agents toward optimal network performance. Simulation results, validated with ray tracing (RT) in realistic urban scenarios and benchmarked against 3GPP and NYM models, demonstrate that the proposed MARL framework achieves higher rate, lower HO failures, improved POI, and enhanced energy efficiency in complex urban HetNets.