신뢰성 있는 지구 저궤도 위성 통신망 최적화를 위한 강화학습 기반 동적 라우팅 알고리즘 = Reinforcement Learning-Based Dynamic Routing for Robust Optimization of Low Earth Orbit (LEO) Satellite Communication Networks

다국어 초록 (Multilingual Abstract)

Recently, satellite communication has garnered significant attention as a novel industry capable of providing global internet access in conjunction with the next-generation communication system, 6G. Notably, low-Earth orbit satellites, operating at comparatively lower altitudes, offer an advantage in communication system configuration due to their closer proximity to Earth. The inherent characteristics of LEO satellites, such as their high orbital speed and deployment of numerous satellites in the same orbit, necessitate research into inter-satellite routing technology for enhanced communication performance. Consequently, this study presents a routing algorithm aimed at optimizing the LEO satellite communication network by employing reinforcement learning, a machine learning technique. By applying various reinforcement learning algorithms to satellite topologies that may arise in space environments, the superiority of the algorithm is assessed, and simultaneously, the feasibility of implementing inter-satellite routing in space is demonstrated.

참고문헌 (Reference)

1 이현수 ; 김중헌, "저궤도 인공위성에 적용되는 심층강화학습 기술 동향" 한국통신학회 48 (48): 196-205, 2023

2 박찬영 ; 김규선 ; 이경진 ; 윤일수, "자율적인 UAM 시스템의 효율적인 무인 정보수집 및 감시를 위한 멀티 에이전트 기반 심층 강화학습" 한국통신학회 48 (48): 176-184, 2023

3 B. Di, "Ultra-Dense LEO : Integration of satellite access networks into 5G and beyond" 26 (26): 62-69, 2019

4 K. E. Eichensehr, "Ukraine, cyberattacks, and the lessons for international law" 116 : 145-149, 2022

5 정택현 ; 김상원 ; 김기천, "N-DQN: 계층화된 병렬 강화학습 모델의 구현 및 연구" 한국통신학회 44 (44): 1961-1974, 2019

6 Guillen-Perez Antonio, "Multi-Agent Deep Reinforcement Learning to Manage Connected Autonomous Vehicles at Tomorrow's Intersections" Institute of Electrical and Electronics Engineers (IEEE) 71 (71): 7033-7043, 2022

7 K. Q. Nguyen, "Monte carlo tree search for collaboration control of ghosts in Ms. Pac-Man" 5 (5): 57-68, 2013

8 V. Mnih, "Human-level control through deep reinforcement learning" 518 (518): 529-533, 2015

9 M. Kaloev, "Experiments focused on exploration in deep reinforcement learning" 351-355, 2021

10 R. Bellman, "Dynamic programming" 153 (153): 34-37, 1966

11 Lilian Del Consuelo Hernandez Ruiz Gaytan, "Dynamic Scheduling for High Throughput Satellites Employing Priority Code Scheme" Institute of Electrical and Electronics Engineers (IEEE) 3 : 2044-2054, 2015

12 Y. Su, "Broadband LEO satellite communications : Architectures and key technologies" 26 (26): 55-61, 2019

13 P. Zuo, "An intelligent routing algorithm for leo satellites based on deep reinforcement learning" 1-5, 2021

14 M. Neinavaie, "Acquisition, doppler tracking, and positioning with starlink LEO satellites : First results" 58 (58): 2606-2610, 2022

15 V. B. Ajabshir, "A low-cost q-learning-based approach to handle continuous space problems for decentralized multi-agent robot navigation in cluttered environments" 10 : 35287-35301, 2022

16 N. Gholizadeh, "A comparative study of reinforcement learning algorithms for distribution network reconfiguration with deep q-learning-based action sampling" 11 : 13714-13723, 2023