A Reinforcement Learning Framework for Autonomous Cell Activation and Customized Energy-Efficient Resource Allocation in C-RANs = A Reinforcement Learning Framework for Autonomous Cell Activation and Customized Energy-Efficient Resource Allocation in C-RANs

다국어 초록 (Multilingual Abstract)

Cloud radio access networks (C-RANs) have been regarded in recent times as a promising concept in future 5G technologies where all DSP processors are moved into a central base band unit (BBU) pool in the cloud, and distributed remote radio heads (RRHs) compress and forward received radio signals from mobile users to the BBUs through radio links. In such dynamic environment, automatic decision-making approaches, such as artificial intelligence based deep reinforcement learning (DRL), become imperative in designing new solutions. In this paper, we propose a generic framework of autonomous cell activation and customized physical resource allocation schemes for energy consumption and QoS optimization in wireless networks. We formulate the problem as fractional power control with bandwidth adaptation and full power control and bandwidth allocation models and set up a Q-learning model to satisfy the QoS requirements of users and to achieve low energy consumption with the minimum number of active RRHs under varying traffic demand and network densities. Extensive simulations are conducted to show the effectiveness of our proposed solution compared to existing schemes.

참고문헌 (Reference)

1 J. Lofberg, "YALMIP: a toolbox for modeling and optimization in MATLAB" 2004

2 A. Moubayed, "Wireless resource virtualization with device-to-device communication underlaying LTE Network" 61 (61): 734-740, 2015

3 Q. Ye, "User association for load balancing in heterogeneous cellular networks" 12 (12): 2706-2716, 2013

4 C. Xu, "Self-organized dynamic caching space sharing in virtualized wireless networks" 1-6, 2016

5 R. S. Sutton, "Reinforcement learning: an introduction" MIT Press 1998

6 Y. Xu, "QoE-aware mobile association and resource allocation over wireless heterogeneous networks" 4695-4701, 2014

7 Lin, Yicheng, "Optimizing user association and spectrum allocation in HetNets: a utility perspective" 33 (33): 1025-1039, 2015

8 X. Zhang, "On the study of fundamental trade-offs between QoE and energy efficiency in wireless networks" 24 (24): 259-265, 2013

9 D. A. Duwaer, "On the deep reinforcement learning for data-driven traffic control"

10 N. Bhushan, "Network densification : the dominant theme for wireless evolution into 5G" 52 (52): 82-89, 2014

11 Baxter A. Laurence, "Markov decision processes : discrete stochastic dynamic programming" 37 (37): 353-, 1995

12 Wei-Sheng Lai, "Joint power and admission control for spectral and energy efficiency maximization in heterogeneous OFDMA networks" 15 : 3531-3547, 2016

13 Y. Shi, "Group sparse beamforming for green cloud-RAN" 13 (13): 2809-2823, 2014

14 Y. Shi, "Enhanced group sparse beamforming for green cloud-RAN : a random matrix approach" 17 (17): 2511-2524, 2017

15 M. Peng, "Energy-efficient resource assignment and power allocation in heterogeneous cloud radio access networks" 64 (64): 5275-5287, 2015

16 F. Shams, "Energy-efficient power control for multiple-relay cooperative networks using Q-learning" 14 (14): 1567-1580, 2015

17 A. Mesodiakaki, "Energy-efficient context-aware user association for outdoor small cell heterogeneous networks" 1614-1619, 2014

18 B. Dai, "Energy efficiency of downlink transmission strategies for cloud radio access networks" 34 (34): 1037-1050, 2016

19 Guolin Sun, "Energy Efficient Cell Management by Flow Scheduling in Ultra Dense Networks" 한국인터넷정보학회 10 (10): 4108-4122, 2016

20 H. Zhang, "Downlink energy efficiency of power allocation and wireless backhaul bandwidth allocation in heterogeneous small cell networks" 66 (66): 1705-1716, 2018

21 S. Luo, "Downlink and uplink energy minimization through user association and beam forming in C-RAN" 14 (14): 494-508, 2015

22 H. Li, "Deep reinforcement learning: framework, applications and embedded implementations" 847-854, 2017

23 J. Tang, "Cross-layer optimization for end-to-end rate allocation in multi-radio wireless mesh networks" 15 (15): 53-64, 2009

24 Checko A., "Cloud RAN for mobile networks—A technology overview" 17 (17): 405-426, 2015

25 Michael Grant, "CVX: Matlab software for disciplined convex programming, version 2.0 beta"

26 China Mobile, "C-RAN: the road towards green RAN" 2011

27 S. Buzzi, "A survey of energy-efficient techniques for 5G networks and challenges ahead" 34 (34): 697-709, 2016

28 X. Zhiyuan, "A deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs" 1-6, 2017

29 Koudouridis, G.P., "A centralised approach to power on-off optimization for heterogeneous networks" 3-6, 2012