Adaptive Inference Caching and Offloading for Resource-Constrained Edge Systems = 제한된 자원 환경의 에지 시스템을 위한 적응형 추론 캐싱 및 오프로딩 기법

다국어 초록 (Multilingual Abstract)

This paper systematically evaluates the practicality of adaptive split inference and communication optimization techniques in edge device-based autonomous robot environments. Edge devices like Raspberry Pi struggle with real-time inference of large deep learning models such as YOLOv8 segmentation due to limited computational resources. While previous studies have proposed techniques like model partitioning, adaptive split point selection, and communication skip, comprehensive performance evaluations under real-world network variability and studies on the combined effects of these techniques remain scarce. This research conducts a comprehensive experimental analysis across three dimensions. First, we perform performance profiling at each segmentation point across the 22 layers of the YOLOv8n model and experimentally derive the optimal segmentation point based on network state (Good/Moderate/Poor). We implement an adaptive segmentation strategy based on a pre-measured lookup table and quantify the performance improvement over fixed segmentation in network fluctuation scenarios such as steps, sine waves, and bursts. Second, we measure the effectiveness of the coordinate-change-based communication skip logic. We analyze the trade-offs in communication reduction rate, control accuracy, and driving stability for vari- ous threshold (θ) settings. Third, we evaluate a periodic transmission technique (100ms, 1000ms) for environments with relaxed real-time requirements and experimentally verify its combined effect with adaptive segmentation and communication skipping. Actual line-tracing experiments using the AlphaBot2 robot demonstrated that the pro- posed system achieved a 72.5% reduction in end-to-end latency and a 58.0% reduction in driving time compared to Full Local, while achieving an 81.7% reduction in network usage compared to Full Remote. Adaptive distributed inference showed an 8.8% reduction in latency and a 9.4% improvement in FPS compared to Fixed Split, while coordinate-based communication skipping (θ=2) achieved a 10.4% reduction in network usage. Periodic transmis- sion experiments confirmed that a 500ms cycle represents the optimal balance point, achieving an 89.3% reduction in communication while maintaining a 100% driving completion rate. This study provides practical guidelines for edge AI system designers and presents optimal system configuration strategies considering network conditions, real-time requirements, and energy efficiency.

목차 (Table of Contents)

• List of Tables vii
• List of Figures ix
• List of Algorithms x
• Abstract
• 1 Introduction 1
• 1.1 Research Background 1
• 1.2 Research Motivation 2
• 1.3 Research Objectives and Approach 2
• 1.4 Main Contributions 4
• 1.5 Paper Organization 4
• 2 Related Work 6
• 2.1 Edge Computing and Distributed DNN Inference 6
• 2.2 DNN Model Partitioning and Optimization 7
• 2.3 Object Detection and Semantic Segmentation Models 8
• 2.3.1 YOLO series evolution 8
• 2.3.2 Transformer-based segmentation 8
• 2.3.3 Real-time segmentation networks 8
• 2.3.4 Domain-specific optimizations 9
• 2.4 Communication Optimization and Resource Management 9
• 2.5 Resource-Constrained Deployment Strategies 10
• 2.6 Real-World Applications and Benchmarks 11
• 2.7 Research Gaps and Contributions 11
• 3 System Design 13
• 3.1 Overall System Architecture 13
• 3.1.1 Client-Server Split Inference Structure 13
• 3.1.2 Hardware Configuration 15
• 3.1.3 Software Stack Configuration 16
• 3.2 YOLOv8 Segmentation Model 17
• 3.2.1 Model Selection and Suitability Analysis 17
• 3.2.2 YOLOv8n Architecture Structure 18
• 3.2.3 Layer-wise Tensor Characteristics and Split Point Analysis 20
• 3.2.4 Model Customization for Line Tracing 22
• 3.3 Model split and Execution Mechanism 27
• 3.3.1 ONNX-based Client Model split 27
• 3.3.2 PyTorch-based Server Model Execution 28
• 3.3.3 Intermediate Tensor Serialization and Network Transmission 29
• 3.4 Adaptive Split Point Selection Mechanism 30
• 3.4.1 Network Condition Monitoring 30
• 3.4.2 Performance Profile-based Split Point Selection 31
• 3.4.3 Dynamic Split Point Switching Strategy 32
• 3.5 Control System Implementation 32
• 3.5.1 Coordinate Extraction and Filtering 32
• 3.5.2 PID Controller 34
• 3.5.3 AlphaBot2 Motor Control 35
• 3.5.4 Special Situation Handling 36
• 3.5.5 Control System Performance 36
• 4 Performance Profiling Experiment 38
• 4.1 Experimental Environment and Settings 38
• 4.1.1 Hardware and Software Configuration 38
• 4.1.2 Network Condition Settings 39
• 4.1.3 Measurement Methodology 39
• 4.2 Layer-wise Client Inference Performance 40
• 4.2.1 Client Inference Time Measurement 40
• 4.2.2 Client Inference Time Analysis 40
• 4.3 Moderate Tensor Size and Transmission Overhead 41
• 4.3.1 Layer-wise Intermediate Tensor Characteristics 41
• 4.3.2 Tensor Size Variation Pattern Analysis 42
• 4.3.3 Network Condition-specific Transmission Time 42
• 4.4 Server Inference Performance Analysis 43
• 4.4.1 Layer-wise Server Inference Time 43
• 4.4.2 FP16 Mixed Precision Inference Effect 45
• 4.5 End-to-End Latency and Optimal Split Points 45
• 4.5.1 End-to-End Latency Measurement 45
• 4.5.2 Optimal Split Point Derivation 45
• 4.5.3 Split Point Performance Characteristics Analysis 47
• 4.6 Baseline Comparison and Performance Improvement Verification 48
• 4.6.1 Baseline Performance Measurement 48
• 4.6.2 Performance Improvement of Proposed Approach 48
• 4.6.3 FPS and Real-time Performance Analysis 49
• 4.7 FP16 Quantization Effect Verification 50
• 4.7.1 Quantization Overhead Measurement 50
• 4.7.2 Accuracy Impact Analysis 51
• 4.7.3 Memory Efficiency 51
• 4.8 Experimental Results Summary and Discussion 51
• 4.8.1 Key Experimental Results Summary 51
• 4.8.2 Necessity of Adaptive Mechanism 52
• 4.8.3 Experimental Limitations and Future Research 52
• 4.8.4 Conclusion 53
• 5 Adaptive Split Inference Mechanism 54
• 5.1 Real-time Network Condition Monitoring 54
• 5.1.1 Network Delay Time Measurement 54
• 5.1.2 Stabilization through Moving Average Filter 55
• 5.1.3 Network Condition Classification Algorithm 55
• 5.1.4 State Transition Hysteresis 56
• 5.2 Dynamic Split Point Selection Strategy 56
• 5.2.1 Lookup Table-based Selection Method 56
• 5.2.2 Rationale for Split Point Selection 57
• 5.2.3 Scalability of Lookup Table 57
• 5.2.4 Split Point Transition Overhead 58
• 5.3 Split Inference Execution Process 58
• 5.3.1 Overall Execution Flow 58
• 5.3.2 Client-side Implementation 59
• 5.3.3 Server-side Implementation 59
• 5.3.4 Communication Protocol Design 60
• 5.4 Experiments and Performance Verification 60
• 5.4.1 Experimental Design and Comparison Groups 60
• 5.4.2 Dynamic Network Scenario Design 61
• 5.4.3 Three Strategy Comparison Experiment 62
• 5.4.4 Split Point Transition Frequency Analysis 64
• 5.4.5 Network Condition Prediction Accuracy 65
• 5.5 Actual Navigation Experiment 65
• 5.5.1 Experimental Setup 65
• 5.5.2 Navigation Performance Comparison 65
• 5.5.3 Real-time FPS Analysis 66
• 5.5.4 Adaptive Mechanism Dynamic Transition Analysis 67
• 5.6 Conclusion and Discussion 69
• 5.6.1 Key Achievements Summary 69
• 5.6.2 Advantages of Adaptive Mechanism 69
• 5.6.3 Limitations and Future Improvements 70
• 5.6.4 Extensibility to Other Applications 71
• 5.6.5 Conclusion 71
• 6 Communication Optimization 73
• 6.1 Coordinate-based Communication Skip Mechanism 73
• 6.1.1 Temporal Continuity of Segmentation Results 73
• 6.1.2 Threshold-based Communication Skip Logic 73
• 6.1.3 Trade-offs in Threshold Selection 74
• 6.2 Performance Evaluation per Threshold 75
• 6.2.1 Experimental Design 75
• 6.2.2 Communication Reduction Rate Results 75
• 6.2.3 Navigation Performance Impact Analysis 76
• 6.2.4 Center Point Deviation Analysis 76
• 6.2.5 Recommended Threshold Setting 77
• 6.3 Integration with Chapter 5 and Conclusion 78
• 6.3.1 Synergy with Adaptive Distributed Inference 78
• 6.3.2 Conclusion 78
• 7 Experiments and Evaluation 80
• 7.1 Experimental Environment and Settings 80
• 7.1.1 Hardware and Software Configuration 80
• 7.1.2 Experimental Track and Measurement Metrics 80
• 7.2 Ablation Study: Technique-specific Contribution Analysis 81
• 7.2.1 Experimental Design 81
• 7.2.2 Performance Comparison Results 81
• 7.3 Baseline Comparison: Full System Performance 82
• 7.3.1 Comparison Targets 82
• 7.3.2 Comprehensive Performance Comparison 82
• 7.3.3 Comparison in Network Fluctuation Environments 83
• 7.4 Periodic Transmission Experiment 83
• 7.4.1 Experimental Motivation and Design 83
• 7.4.2 Performance Comparison by Period 84
• 7.4.3 Recommended Period and Application Scenarios 84
• 7.5 Robustness Verification under Various Environments 85
• 7.5.1 Extended Track Experiment 85
• 7.5.2 Lighting and Speed Condition Verification 85
• 7.6 Energy Efficiency Analysis 85
• 7.7 Experimental Results Summary 86
• 8 Results and Discussion 87
• 8.1 Comprehensive Experimental Results 87
• 8.1.1 Summary of Key Performance Metrics 87
• 8.1.2 Implications of Experimental Results 88
• 8.2 Effectiveness Analysis of Adaptive Distributed Inference 88
• 8.2.1 Why Adaptive split is Effective 88
• 8.2.2 Impact of Split Point Transition Overhead 89
• 8.2.3 Limitations of Fixed Split 89
• 8.3 Practicality Analysis of Communication Optimization 89
• 8.3.1 Effect of Coordinate-based Communication Skip 89
• 8.3.2 Trade-offs in Threshold Selection 90
• 8.3.3 Application Conditions for Periodic Transmission 90
• 8.3.4 Synergy between Adaptive split and Communication Skip 90
• 8.4 System Limitations and Improvement Directions 91
• 8.4.1 Identified Limitations 91
• 8.4.2 Future Improvement Directions 91
• 8.4.3 Expandability to Other Applications 92
• 8.5 Conclusion 92
• 9 Conclusion 94
• 9.1 Research Summary 94
• 9.2 Key Contributions 94
• 9.3 Research Limitations and Future Directions 95
• 9.4 Closing Remarks 96
• Bibliography 97
• Abstract Korean 105