개인정보 보호형 밀리미터파 레이더-열영상 융합 기반 인체 추적 기술 개발 = mmWave Radar-Thermal Fusion for Privacy-Preserving Indoor Human Tracking

다국어 초록 (Multilingual Abstract)

We present a privacy-preserving indoor human tracking framework that
fuses millimeter-wave (mmWave) radar and long-wave infrared (thermal)
imaging without using RGB cameras. RGB images are used only for offline
annotation of the UTM multimodal dataset; inference relies solely on
mmWave radar depth, thermal images, and thermal-based pseudo depth.
Using Faster R-CNN with a ResNet50-FPN backbone, we evaluate three
input configurations for human detection and headcount estimation: (1)
radar only, (2) radar + thermal, and (3) radar + thermal + thermal
pseudo depth. The third configuration adds a pseudo-depth channel
generated by a pre-trained monocular thermal depth network
(MonoTher-Depth). On UTM, radar + thermal consistently improves mAP
and counting MAE over radar-only inputs, with larger gains in
multi-person and small/partially occluded scenes. Adding thermal pseudo
depth yields further improvements, especially in complex backgrounds
prone to false positives.
For qualitative tracking analysis, we connect frame-wise detections into
trajectories through a lightweight association step based on bounding-box
overlap and center-point distance, and we examine temporal stability
across the three configurations. These results suggest the feasibility of
practical indoor monitoring—presence, location, headcount, and coarse
trajectory-level analysis—using only mmWave radar, thermal imaging, and
thermal-based pseudo depth.

목차 (Table of Contents)

• Ⅰ. 서론 1
• Ⅱ. 관련 연구 3
• Ⅲ. 연구방법 및 실험구성 10
• 3.1 실험 환경 10
• 3.2 데이터셋 및 라벨 구조 11
• 3.2.1 UTM 데이터셋 개요 11
• 3.2.2 라벨 구조와 재실자 카운팅 정의 13
• 3.2.3 열영상 기반 의사 깊이 생성 13
• 3.3 전처리와 정합 14
• Ⅳ. 제안 방법 14
• 4.1 입력 표현 및 채널 구성 14
• 4.2 재실자 탐지 모델 구조 16
• 4.3 재실자 카운팅 및 점유 기반 위치 변화 분석 16
• Ⅴ. 실험 및 결과 분석 18
• 5.1 실험 시나리오 18
• 5.2 평가 지표 및 분석 19
• 5.3 성능 비교 및 고찰 20
• Ⅵ. 결론 22
• 참고문헌 25
• ABSTRACT 26