혼합 교차로 환경에서의 협력적 자율주행을 위한 불확실성 대응 그래프 기반 다중 에이전트 강화학습 = Uncertainty-Aware Graph-Based Multi-Agent Reinforcement Learning for Cooperative Autonomous Driving at Mixed Traffic Intersections

다국어 초록 (Multilingual Abstract)

Signal-based intersection control inherently causes unnecessary delays and limits overall traffic efficiency. Autonomous Intersection Management (AIM) has therefore gained attention as a promising alternative. Yet, fully autonomous traffic remains a long-term vision, and mixed traffic—where autonomous vehicles (AVs) operate alongside human-driven vehicles (HDVs)—will be unavoidable for the foreseeable future. In such environments, unpredictable HDV behavior becomes a major challenge, especially near intersections where interaction complexity and conflict points amplify the effects of behavioral uncertainty. These factors can destabilize decision-making and increase collision risks, underscoring the need for a control strategy that remains reliable even under uncertain interactions. To address this challenge, this study proposes a multi-agent reinforcement learning (MARL)–based control framework, which is well-suited for the cooperative decision-making required at complex intersections, designed to enhance robustness and scalability in mixed traffic. The key component is a Predicted Occupancy Grid Map (POGM) that accumulates future occupancy information over time and selectively incorporates only potential collision-risk vehicles into the graph structure. By relying on temporally accumulated occupancy rather than single-step predictions, the framework mitigates the impact of momentary prediction errors and enables stable decision-making under uncertainty. This selective graph construction reduces redundant information, lowers computational burden, and preserves essential interaction dynamics. The framework is evaluated in dynamic multi-lane intersection scenarios with varying traffic densities. Experimental results show faster policy convergence, higher success rates, and more stable speed regulation compared to non-selective observation methods. An additional ablation study confirms that the accumulated POGM representation maintains safety performance even when short-term prediction errors occur near intersection entry zones, while single-step structures experience notable degradation. Overall, the experimental results demonstrate the effectiveness of POGM-based selective graph construction in achieving robust and cooperative autonomous driving within complex mixed traffic environments.

목차 (Table of Contents)

• 제 1 장 서론 1
• 제 2 장 배경 이론 6
• 2.1 강화학습 6
• 2.1.1 마르코프 결정 과정 7
• 2.1.2 벨만 방정식 8
• 2.1.3 Twin Delayed Deep Deterministic Policy Gradient 9
• 2.1.4 Prioritized Experience Replay 12
• 2.2 그래프 신경망 14
• 제 3 장 프레임워크 제안 15
• 3.1 프레임워크 개요 15
• 3.1.1 Geofence 기반 경로 계획 17
• 3.1.2 HDV 불확실성 통합 18
• 3.1.3 선택적 그래프 구성 21
• 3.2 그래프 신경망 기반 강화학습 24
• 3.2.1 상태공간 24
• 3.2.2 행동공간 26
• 3.2.3 보상함수 27
• 3.2.4 신경망 구조 29
• 제 4 장 실험 및 결과 33
• 4.1 시뮬레이션 구성 33
• 4.1.1 동적 교통 환경 구성 34
• 4.1.2 인간 운전 차량 설계 37
• 4.2 제안 기법 성능 평가 (Scenario 1: Scalability) 39
• 4.2.1 학습 수렴 특성 비교 41
• 4.2.2 성공률 및 주행 성능 비교 43
• 4.2.3 실행 시간 비교 45
• 4.3 Ablation Study (Scenario 2: Robustness) 47
• 4.3.1 실험 환경 및 Predictor 조건 47
• 4.3.2 예측 누적 유무에 따른 성능 비교 49
• 제 5 장 결론 51
• 참고문헌 53