Deep Q-Learning–Based Resource Allocation for Solar-powered Users in Cognitive Radio Networks

다국어 초록 (Multilingual Abstract)

This paper considers an uplink solar-powered cognitive radio networks (CRNs) where multiple secondary users (SUs) transmit data to a secondary base station (SBS) by sharing a licensed channel of a primary system. A deep Q-learning (DQL) algorithm, which combines non-orthogonal multiple access (NOMA) and time division multiple access (TDMA) techniques, is proposed to maximize the long-term throughput of the system. By using our scheme, the agent (i.e. the SBS) can obtain the optimal decision by interacting with the environment to learn about system dynamics. Simulation results validate the superiority of the performance under the proposed scheme, compared with traditional schemes.

참고문헌 (Reference)

1 M.B. Shahab, "User pairing schemes for capacity maximization in non-orthogonal multiple access systems" 16 (16): 2884-2894, 2016

2 Xin Liu, "Uplink Resource Allocation for NOMA-Based Hybrid Spectrum Access in 6G-Enabled Cognitive Internet of Things" Institute of Electrical and Electronics Engineers (IEEE) 8 (8): 15049-15058, 2021

3 C.I.C. Rowell, "Toward green and soft:A 5G perspective" 52 (52): 66-73, 2014

4 A. Cuadras, "Thermal energy harvesting through pyroelectricity" 158 (158): 132-139, 2010

5 Qian Yu Liau, "Successive User Relaying in Cooperative NOMA System" Institute of Electrical and Electronics Engineers (IEEE) 8 (8): 921-924, 2019

6 F. Zhou, "State of the art, taxonomy, and open issues on cognitive radio networks with NOMA" 25 (25): 100-108, 2018

7 J. Ma, "Soft combination and detection for cooperative spectrum sensing in cognitive radio networks" 7 (7): 4502-4507, 2008

8 Yanan Liu, "Situation-Aware Resource Allocation for Multi-Dimensional Intelligent Multiple Access: A Proactive Deep Learning Framework" Institute of Electrical and Electronics Engineers (IEEE) 39 (39): 116-130, 2021

9 R.S. Sutton, "Reinforcement Learning: An Introduction" MIT Press 2018

10 Z. Wei, "Power-efficient resource allocation for MC-NOMA with statistical channel state information" GLOBECOM 1-7, 2016

11 S. M. Riazul Islam, "Power-Domain Non-Orthogonal Multiple Access (NOMA) in 5G Systems: Potentials and Challenges" Institute of Electrical and Electronics Engineers (IEEE) 19 (19): 721-742, 2017

12 Khai Nguyen Doan, "Power Allocation in Cache-Aided NOMA Systems: Optimization and Deep Reinforcement Learning Approaches" Institute of Electrical and Electronics Engineers (IEEE) 68 (68): 630-644, 2020

13 Yen Kheng Tan, "Optimized wind energy harvesting system using resistance emulator and active rectifier for wireless sensor nodes" 26 (26): 38-50, 2011

14 A. Zafar, "On multiple users scheduling using superposition coding over Rayleigh fading channels" 17 (17): 733-736, 2013

15 H. Al-Obiedollah, "On energy harvesting of hybrid TDMANOMA systems" GLOBECOM 1-6, 2019

16 Y. Yuan, "Nonorthogonal transmission technology in LTE evolution" 68-74, 2016

17 Y. Liu, "Nonorthogonal multiple access in large-scale underlay cognitive radio networks" 65 (65): 10152-10157, 2016

18 K. Higuchi, "Non-orthogonal multiple access (NOMA)with successive interference cancellation for future radio access" E98-B (E98-B): 403-414, 2015

19 Yali Wu, "Non-Orthogonal Random Access and Data Transmission Scheme for Machine-to-Machine Communications in Cellular Networks" Institute of Electrical and Electronics Engineers (IEEE) 8 : 27687-27704, 2020

20 Li Ping Qian, "NOMA-Enabled Mobile Edge Computing for Internet of Things via Joint Communication and Computation Resource Allocations" Institute of Electrical and Electronics Engineers (IEEE) 7 (7): 718-733, 2020

21 Zhenni Li, "NOMA-Enabled Cooperative Computation Offloading for Blockchain-Empowered Internet of Things: A Learning Approach" Institute of Electrical and Electronics Engineers (IEEE) 8 (8): 2364-2378, 2021

22 C.L. Wang, "Low-complexity resource allocation for downlink multicarrier NOMA systems" 1-6, 2018

23 Wenjun Xu, "Joint Sensing Duration Adaptation, User Matching, and Power Allocation for Cognitive OFDM-NOMA Systems" Institute of Electrical and Electronics Engineers (IEEE) 17 (17): 1269-1282, 2018

24 Pham Duy Thanh, "Joint Resource Allocation and Transmission Mode Selection Using a POMDP-Based Hybrid Half-Duplex/Full-Duplex Scheme for Secrecy Rate Maximization in Multi-Channel Cognitive Radio Networks" Institute of Electrical and Electronics Engineers (IEEE) 20 (20): 3930-3945, 2020

25 Chaofan He, "Joint Power Allocation and Channel Assignment for NOMA With Deep Reinforcement Learning" Institute of Electrical and Electronics Engineers (IEEE) 37 (37): 2200-2210, 2019

26 Zhiguo Ding, "Impact of User Pairing on 5G Nonorthogonal Multiple-Access Downlink Transmissions" Institute of Electrical and Electronics Engineers (IEEE) 65 (65): 6010-6023, 2016

27 A.B. Rozario, "Hybrid TDMA-NOMA based M2M communications over cellular networks with dynamic clustering and 3D channel models" ISAECT 1-6, 2019

28 Y. Zhang, "Energy-efficient transmission design in non-orthogonal multiple access" 66 (66): 2852-2857, 2017

29 Zeming Li, "Energy-Efficient Resource Allocation With Hybrid TDMA–NOMA for Cellular-Enabled Machine-to-Machine Communications" Institute of Electrical and Electronics Engineers (IEEE) 7 : 105800-105815, 2019

30 Xue Wang, "Energy-Efficient Power Allocation and Q-Learning-Based Relay Selection for Relay-Aided D2D Communication" Institute of Electrical and Electronics Engineers (IEEE) 69 (69): 6452-6462, 2020

31 L. Lv, "Design of cooperative non-orthogonal multicast cognitive multiple access for 5G systems: User scheduling and performance analysis" 65 (65): 2641-2656, 2017

32 Jonathan van de Belt, "Defining and Surveying Wireless Link Virtualization and Wireless Network Virtualization" Institute of Electrical and Electronics Engineers (IEEE) 19 (19): 1603-1627, 2017

33 G. Gui, "Deep learning for an effective nonorthogonal multiple access scheme" 67 (67): 8440-8450, 2018

34 M. Liu, "Deep cognitive perspective: Resource allocation for NOMA-based heterogeneous IoT with imperfect SIC" 6 (6): 2885-2894, 2019

35 Yunzeng Li, "Deep Reinforcement Learning for Dynamic Spectrum Sensing and Aggregation in Multi-Channel Wireless Networks" Institute of Electrical and Electronics Engineers (IEEE) 6 (6): 464-475, 2020

36 I. Goodfellow, "Deep Learning" MIT Press 2016

37 Gang Liu, "Cooperative NOMA Broadcasting/Multicasting for Low-Latency and High-Reliability 5G Cellular V2X Communications" Institute of Electrical and Electronics Engineers (IEEE) 6 (6): 7828-7838, 2019

38 T. Theodorou, "CORAL-SDN: A software-defined networking solution for the Internet of Things" 1-2, 2017

39 Zhiguo Ding, "Application of Non-Orthogonal Multiple Access in LTE and 5G Networks" Institute of Electrical and Electronics Engineers (IEEE) 55 (55): 185-191, 2017

40 L. Zhang, "A survey of advanced techniques for spectrum sharing in 5G networks" 24 (24): 44-51, 2017

41 A.S. Marcano, "A novel method for improving the capacity in 5G mobile networks combining NOMA and OMA" VTC Spring 1-5, 2017

42 Ayaz Ahmad, "A Survey on Radio Resource Allocation in Cognitive Radio Sensor Networks" Institute of Electrical and Electronics Engineers (IEEE) 17 (17): 888-917, 2015

43 H.T.H. Giang, "A POMDP-based long-term transmission rate maximization for cognitive radio networks with wireless-powered ambient backscatter" 32 : e3993-, 2019