Resource Allocation for Heterogeneous Service in Green Mobile Edge Networks Using Deep Reinforcement Learning = Resource Allocation for Heterogeneous Service in Green Mobile Edge Networks Using Deep Reinforcement Learning

다국어 초록 (Multilingual Abstract)

The requirements for powerful computing capability, high capacity, low latency and low energy consumption of emerging services, pose severe challenges to the fifth-generation (5G) network. As a promising paradigm, mobile edge networks can provide services in proximity to users by deploying computing components and cache at the edge, which can effectively decrease service delay. However, the coexistence of heterogeneous services and the sharing of limited resources lead to the competition between various services for multiple resources. This paper considers two typical heterogeneous services: computing services and content delivery services, in order to properly configure resources, it is crucial to develop an effective offloading and caching strategies. Considering the high energy consumption of 5G base stations, this paper considers the hybrid energy supply model of traditional power grid and green energy. Therefore, it is necessary to design a reasonable association mechanism which can allocate more service load to base stations rich in green energy to improve the utilization of green energy. This paper formed the joint optimization problem of computing offloading, caching and resource allocation for heterogeneous services with the objective of minimizing the on-grid power consumption under the constraints of limited resources and QoS guarantee. Since the joint optimization problem is a mixed integer nonlinear programming problem that is impossible to solve, this paper uses deep reinforcement learning method to learn the optimal strategy through a lot of training. Extensive simulation experiments show that compared with other schemes, the proposed scheme can allocate resources to heterogeneous service according to the green energy distribution which can effectively reduce the traditional energy consumption.

참고문헌 (Reference)

1 Z. Tan, "Virtual resource allocation for heterogeneous services in full duplex-enabled small cell networks with cache and MEC" 163-168, 2017

2 Y. Wei, "User Scheduling and Resource Allocation in HetNets With Hybrid Energy Supply : An Actor-Critic Reinforcement Learning Approach" 17 (17): 680-692, 2018

3 Y. Zhou, "Resource Allocation for Information-Centric Virtualized Heterogeneous Networks With In-Network Caching and Mobile Edge Computing" 66 (66): 11339-11351, 2017

4 L. Wang, "RL-Based User Association and Resource Allocation for Multi-UAV enabled MEC" 741-746, 2019

5 C. Li, "QoE-Driven Mobile Edge Caching Placement for Adaptive Video Streaming" 20 (20): 965-984, 2018

6 E. Baccour, "Proactive Video Chunks Caching and Processing for Latency and Cost Minimization in Edge Networks" 1-7, 2019

7 T. Hou, "Proactive Content Caching by Exploiting Transfer Learning for Mobile Edge Computing" 1-6, 2017

8 Y. Jin, "Optimal Transcoding and Caching for Adaptive Streaming in Media Cloud : an Analytical Approach" 25 (25): 1914-1925, 2015

9 S. Sardellitti, "Optimal Association of Mobile Users to MultiAccess Edge Computing Resources" 1-6, 2018

10 B. Wang, "On Efficient Utilization of Green Energy in Heterogeneous Cellular Networks" 11 (11): 846-857, 2017

11 W. Jiang, "Multi-Agent Reinforcement Learning Based Cooperative Content Caching for Mobile Edge Networks" 7 : 61856-61867, 2019

12 Y. Lan, "Mobile-Edge Computation Offloading and Resource Allocation in Heterogeneous Wireless Networks" 1-6, 2019

13 T. X. Tran, "Joint Task Offloading and Resource Allocation for Multi-Server Mobile-Edge Computing Networks" 68 (68): 856-868, 2019

14 J. Zhou, "Joint Resource Allocation and User Association for Heterogeneous Services in Multi-Access Edge Computing Networks" 7 : 12272-12282, 2019

15 Y. Wei, "Joint Optimization of Caching, Computing, and Radio Resources for Fog-Enabled IoT Using Natural Actor–Critic Deep Reinforcement Learning" 6 (6): 2061-2073, 2019

16 Y. Dai, "Joint Computation Offloading and User Association in Multi-Task Mobile Edge Computing" 67 (67): 12313-12325, 2018

17 S. Seng, "Joint Access Selection and Heterogeneous Resources Allocation in UDNs with MEC Based on Non-Orthogonal Multiple Access" 1-6, 2018

18 Y. He, "Integrated Networking, Caching, and Computing for Connected Vehicles : A Deep Reinforcement Learning Approach" 67 (67): 44-55, 2018

19 V. Mnih, "Human-level control through deep reinforcement learning" 518 (518): 529-533, 2015

20 Q. Fan, "Green energy aware user association in heterogeneous networks" 1-6, 2016

21 L. Sun, "Fogmed : a fog-based framework for disease prognosis based medical sensor data streams" 66 (66): 603-619, 2021

22 L. Li, "Efficient virtual resource allocation in mobile edge networks based on machine learning" 2 (2): 141-150, 2020

23 M. Merluzzi, "Dynamic Joint Resource Allocation and User Assignment in Multi-access Edge Computing" 4759-4763, 2019

24 Y. Wei, "Deep q-learning based computation offloading strategy for mobile edge computing" 59 (59): 89-104, 2019

25 K. Zhang, "Cooperative Content Caching in 5G Networks with Mobile Edge Computing" 25 (25): 80-87, 2018

26 S. Bi, "Computation Rate Maximization for Wireless Powered Mobile-Edge Computing With Binary Computation Offloading" 17 (17): 4177-4190, 2018

27 M. M. Amiri, "Cache-aided data delivery over erasure broadcast channels" 1-6, 2017

28 Z. G. Qu, "Analysis and improvement of steganography protocol based on bell states in noise environment" 59 (59): 607-624, 2019

29 Z. G. Qu, "A Secure Controlled Quantum Image Steganography Algorithm" 19 (19): 1-25, 2020

30 L. Wang, "A New Look at Physical Layer Security, Caching, and Wireless Energy Harvesting for Heterogeneous Ultra-Dense Networks" 56 (56): 49-55, 2018