Coordinated spatial reuse with curriculum learning-based multi-agent deep reinforcement learning major in information and communication engineering

다국어 초록 (Multilingual Abstract)

Next-generation Wi-Fi standards, such as IEEE 802.11bn, emphasize Multi- Access Point Coordination (MAPC)—particularly Coordinated Spatial Reuse (CSR)—to enhance performance in dense wireless environments. However, existing machine learning–based CSR techniques often lack adaptability to dynamic network conditions, such as random station (STA) positions per transmission and variable inter-AP distances, and typically rely on centralized computation. This thesis proposes a novel hierarchical CSR framework based on multi-agent deep reinforcement learning (MARL) to address these limitations. The proposed frame- work decomposes the complex coordination task into three stages: (1) cooperative AP group formation, (2) transmission power optimization, and (3) carrier-sensing threshold adjustment. Notably, it adopts a Centralized Training with Decentralized Execution (CTDE) architecture and integrates a Curriculum Learning (CL) strategy, enabling agents to progressively adapt from fixed to highly dynamic environments, thereby enhancing environmental adaptability. Simulation results demonstrate that the proposed MARL-based CSR scheme achieves significant performance gains—improving system throughput by 94% and 81%, and capacity by 95% and 80%, while reducing the average queue length by 68% and 66% compared to the conventional Distributed Coordination Function (DCF) and standard Spatial Reuse (SR) schemes. Moreover, the proposed method improves instantaneous fairness by a factor of four over DCF and three over SR. When the CL strategy is applied, all performance metrics exhibit further improvement, confirming the robustness and adaptability of the proposed framework for next- generation high-density WLANs.

목차 (Table of Contents)

• 1 Introduction 1
• 1.1 Motivation 1
• 1.2 Contribution of Thesis 4
• 1.3 Chapter Organization 6
• 2 Related Works 7
• 3 Multi Access Point Coordination 11
• 3.1 DCF 11
• 3.2 SR as OBSS/PD 13
• 3.3 Coordinated Spatial Reuse 15
• 4 Reinforcement Learning 18
• 4.1 Markov Decision Process 19
• 4.2 Q Learning 20
• 4.3 Deep Q Network 21
• 5 Multi Agent Reinforcement Learning 23
• 5.1 Cooperative Learning 24
• 5.2 Competitive Learning 26
• 5.3 Adaptability Concern 27
• 5.4 Curriculum Learning 27
• 6 System Model 29
• 6.1 Propagation Model 29
• 6.2 Traffic Model 31
• 6.3 Objective Function 33
• 6.3.1 MAPC Scheduling 33
• 6.3.2 Spatial Reuse Optimization 34
• 7 Proposed Method 36
• 7.1 Top-Down MDP 36
• 7.2 Algorithm 40
• 7.3 Adaptation 42
• 8 Experimental Results and Discussion 45
• 8.1 Training 48
• 8.2 Testing 53
• 8.3 Adaptation as CL 58
• 8.4 Supplementary 65
• 8.4.1 Additional Overhead 66
• 8.4.2 Extreme Case 1: Extremely Close Deployment (2m Inter- AP Distance) 69
• 8.4.3 Extreme Case 2: Large Separation (80 m Inter-AP Distance). 71
• 9 Conclusion 75
• Bibliography 77
• 요약 83