Adaptive Demand and Resource Management in On-Demand Delivery Platforms with Multi-Agent Reinforcement Learning Pohang University of Science and Technology = 다중 에이전트 강화학습을 이용한 주문형 배달 플랫폼의 적응형 수요 및 자원 관리

다국어 초록 (Multilingual Abstract)

Modern on-demand meal delivery platforms pose significant management challenges due to complex interactions among strategically behaving decentralized stakeholders. Existing approaches struggle to manage these decentralized decisions effectively and lead to inefficient resource utilization and operational shortcomings. To overcome these limitations, this dissertation proposes an actionable framework based on multi-agent reinforcement learning (MARL) for adaptive demand and resource management in decentralized operational environments. The dissertation contains two distinct yet complementary studies. The first study focuses on a dynamic order assignment algorithm which anticipate the strategic behavior of heterogeneous couriers. It employs a MARL-inspired interaction model integrated with a reinforcement learning-driven bipartite matching approach to optimize real-time task allocation. Simulation experiments demonstrate substantial improvements in target dispatch time compliance, order backlogs, and workload distribution across couriers. The second study addresses dynamic service area sizing through a cooperative MARL approach using the QMIX algorithm. This method dynamically adapts Customer Service Areas (CSA) and Driver Dispatch Areas (DDA), explicitly considering spatial spillover effects and real-time fluctuations in demand and courier availability. A key methodological innovation is the integration of a flexible constraint-handling mechanism without explicit penalties, supported by a novel best-effort action selection strategy, ensuring robust operational adaptability. Empirical validation through simulation experiments exhibits significant enhancements in real-time responsiveness and overall system efficiency compared to independent optimization approach. This dissertation integrates strategic agent interactions into centralized adaptive management strategies and advances the field by proposing a multi-agent approach to managing complex systems involving stakeholder interactions, thereby enhancing the practical foundations of adaptive resource management in decentralized service environments.

목차 (Table of Contents)

• I. Introduction 1
• II. Order Assignment Considering Diverse Strategic Couriers 9
• 2.1 Overview 10
• 2.2 Related Work 14
• 2.3 Problem Description 20
• 2.3.1 Spatial Representation 20
• 2.3.2 Order Dynamics 23
• 2.3.3 Orders 23
• 2.3.4 Agents 24
• 2.4 Methodology 26
• 2.4.1 MARL Interaction Modeling 27
• 2.4.2 Courier Decision-Making Model 29
• 2.4.3 Operator Decision-Making Model 32
• 2.5 Numerical Experiments 47
• 2.5.1 Target Time Dispatch Rate 49
• 2.5.2 Unfulfilled Orders 52
• 2.5.3 Workload and Locational Balance 52
• 2.5.4 Personalization and Earnings 54
• 2.5.5 Sensitivity Analysis: Effect of Staffing Level 57
• 2.6 Summary 59
• III. Dynamic Service Area Sizing Using Multi-Agent Reinforcement Learning 61
• 3.1 Overview 62
• 3.2 Related Work 66
• 3.3 Problem Description 70
• 3.3.1 Zone-level Service Area Sizing and Demand Spillover 71
• 3.3.2 Service Quality Metric (UTR) and Its Operational Challenges 72
• 3.3.3 Operational Stability Constraint 74
• 3.4 Methodology 74
• 3.4.1 Multi-Agent Markov Decision Process Formulation 75
• 3.4.2 QMIX Implementation 77
• 3.4.3 Operational Constraints and Best-Effort Action
• Selection 78
• 3.4.4 Baseline Methodologies 80
• 3.5 Numerical Experiments 83
• 3.5.1 Experiment Settings 83
• 3.5.2 Baseline Methods for Numerical Comparison 86
• 3.5.3 Unconstrained Scenario 87
• 3.5.4 Impact of Constraints 91
• 3.5.5 Sensitivity Analyses 94
• 3.5.6 Alternative Demand Scenario Analysis 97
• 3.6 Summary 98
• IV. Conclusion 100
• 4.1 Dissertation Summary 101
• 4.2 Future Research 102
• 4.3 Personal Perspective 103
• Appendices 104
• A MARL-based Interaction Model 105
• B Learning Process for Operator and Courier Agents 107
• Summary (in Korean) 109