Dynamic spectrum sharing in heterogeneous wireless networks using deep reinforcement learning

The rapid expansion of wireless networks demands efficient spectrum allocation. Dynamic Spectrum Sharing (DSS) is a technology that allows multiple wireless networks to share the same frequency spectrum dynamically. It is an effective technique for optimizing spectrum use, particularly in heterogeneous environments where multiple wireless technologies with diverse spectrum access requirements coexist, often leading to interference challenges and increased spectrum competition. This research proposes an enhanced DSS technique based on Deep Reinforcement Learning (DRL). The proposed method enables an effective sharing of the available spectrum between two access technologies, namely Long Term Evolution (LTE) and Narrowband IoT (NB-IoT). The study optimizes throughput through DRL methods, including Deep Q-Networks (DQN), conducting experiments in three phases: LTE-DRL coexistence, NB-IoT-DRL coexistence, and LTE-NB-IoT coexistence. Results show that deep learning enhances the LTE-DRL system's convergence rate and throughput, achieving over 85% throughput with convergence times as low as 24 milliseconds (ms). The study highlights the trade-offs between parameters such as probabilities (arrival, successful transmission, and retransmission), packet expiry duration, learning rate, discount factor, fairness index, and the neural network architecture as well as the parameters’ impact on the overall system throughput. NB-IoT coexistence with DRL shows similar results with a slight decrement in throughput and negligibly longer convergence rate, while the coexistence of LTE and NB-IoT results in throughput of around 70% for each of the LTE and NB-IoT systems due to increased spectrum competition and increased complexity of the operating environment. This work offers insights into optimizing spectrum sharing using DRL and underscores the balance between various parameters for efficient spectrum management.