A Reinforcement Learning-Based Intelligent Route Update Scheme for Future Generation IoT Networks

다국어 초록 (Multilingual Abstract)

Future generation Internet of Things (IoT) communication infrastructure is expected to pave the path for innovative applications like smart cities, smart grids, smart industries, and smart healthcare. To support these diverse applications, the communication protocols are required to be adaptive and intelligent.\par
At the network layer, an efficient and lightweight algorithm known as trickle-timer is designed to perform the route updates and it utilizes control messages to share the updated route information between IoT nodes. Trickle-timer tends to generate higher control overhead ratio and achieve lower reliability.
Therefore, this dissertation aims to propose an RL-based Intelligent Adaptive Trickle-Timer Algorithm (RIATA). The proposed algorithm performs three-fold optimization of the trickle-timer algorithm. Firstly, the RIATA assigns a higher probability to control message transmission to nodes that have received an inconsistent control message in the past intervals. Secondly, the RIATA utilizes RL to learn the optimal policy to transmit or suppress a control message in the current network environment. Lastly, the RIATA selects an adaptive redundancy constant value to avoid unnecessary transmissions of control messages.
Simulation results show that RIATA outperforms the other state-of-the-art mechanisms in terms of reducing the control overhead ratio by an average of 21%, decreasing the average total power consumption by 10%, and increasing the packet delivery ratio by 4% on an average.

국문 초록 (Abstract)

스마트시티, 스마트그리드, 스마트산업, 사마트헬스 등에 차세대 사물인터넷
통신기술 사용이 검토되고 있다. 이처럼다양한 응용분야를지원하기 위해서는
통신 프로토콜이 적응적이고 지능적이어야 한다. 네트워크계층에서는경로갱신
을 위한 효율적이고 경량의 알고리즘인 트리클타이머가 사용되고 있다. 트리클
타이머는사물인터넷 단말사이에서 갱신된 경로정보를제어 메시지를사용하여
공유한다. 트리클타이머는제어 오버헤드의 증가로 인하여 신뢰성이 낮아지는
문제가 발생한다. 본 연구에서는이러한 문제를해결하기 위한 강화학습 기반의
지능적이고적응적인트리클타이머알고리즘을제안한다. 제안된알고리즘에서는
트리클타이머알고리즘에대한세단계의최적화를수행한다. 첫번째단계에서과
거 기간동안 일관되지 않은 제어 메시지를수신한 노드에게 제어 메시지 송신에서
우선 순위를준다. 두번째 단계에서는현재 네트워크상태에서 제어 메시지 전송
여부에 대한 판단을 강화학습을 사용하여 결정한다. 마지막단계에서 불필요한
제어 메시지의 전송을 줄이기 위하여 적응적인 중복 상수를선택한다. 기존의
방법과비교하였을때제안된방법은제어오버헤드는21% 감소, 전력소비는10%
감소, 패킷전달율은 4% 향상시키는시뮬레이션 결과를보여주었다

목차 (Table of Contents)

• 1 Introduction and Motivation 1
• 1.1 Introduction 1
• 1.2 Motivation 4
• 1.3 Objectives of the dissertation 6
• 1.4 Organization 7
• 2 Related Work 9
• 2.1 Trickle-F 9
• 2.2 Optimized Trickle 10
• 2.3 Adaptive Trickle 10
• 2.4 Enhanced Trickle 10
• 2.5 Trickle-plus 11
• 2.6 FL-Trickle 11
• 2.7 I-Trickle 12
• 2.8 Drizzle 12
• 3 Proposed Reinforcement Learning-Based Adaptive Trickle-Timer Algorithm 14
• 3.1 System Model 14
• 3.2 Reinforcement Learning 15
• 3.3 Proposed RIATA Algorithm 16
• 3.3.1 Intelligent decision for DIO transmission or suppression 18
• 3.3.2 Higher transmission probability 21
• 3.3.3 Adaptive redundancy constant selection 23
• 4 Performance Evaluation and Experimental Results 24
• 4.1 Introduction 24
• 4.2 Analysis under fixed data rate scenario 27
• 4.3 Analysis under variable data rate scenario 31
• 4.4 Analysis with variable redundancy constant 34
• 4.5 Analysis with variable exploration rates 37
• 4.6 Analysis of adaptive redundancy constant ck selection under various network sizes 38
• 4.7 Analysis of computation and memory complexity 39
• 4.8 Analysis of the three fold optimizations of the proposed algorithm 40
• 4.9 Potential Application Areas of the Proposed Method 43
• 5 Conclusion 45
• Bibliography 47

참고문헌 (Reference)

1. RPL : Ipv6 routing protocol for low power and lossy networks, T. Winter et al, , 2012

2. Reinforcement learning based routing in networks : Review and classification of approaches, Z. Mammeri, , 2019