Deep Reinforcement Learning-Based Joint Optimization for UAV-Mounted STAR-RIS Networks with Energy Harvesting

Reconfigurable intelligent surfaces (RIS) are increasingly recognized as an enabling component of sixthgeneration (6 G) wireless systems, as they allow controllable modification of the propagation environment to improve both sum-rate and energy efficiency. This work investigates a UAVmounted simultaneous transmitting and reflecting RIS (STARRIS) network, where an energy-harvesting-enabled aerial platform functions as a reconfigurable relay to support users located on both the reflection and transmission sides. A joint nonconvex optimization problem is formulated to optimize the UAV trajectory, base station beamforming, STAR-RIS reflection and transmission coefficients, and the energy harvesting powersplitting ratio. The objective is to maximize a weighted sum of energy efficiency and harvested energy while satisfying users' quality-of-service (QoS) requirements under practical mobility and transmit power constraints. To address this challenging control problem, deterministic policy-gradient-based deep reinforcement learning algorithms, namely deep deterministic policy gradient (DDPG) and twin delayed DDPG (TD3), are employed within a continuous-control environment that incorporates realistic Rician fading channels. Simulation results demonstrate that TD3 achieves more stable convergence behavior and superior energy efficiency compared to the baseline DDPG approach while effectively optimizing the weighted objective of energy efficiency and harvested energy.