Real-Time Multi-Target Tracking for Sparse 3D Point Cloud Objects

국문 초록 (Abstract)

자율 주행 자동차와 같은 자율 시스템 및 로봇틱스 플랫폼에선 그 주변 환경을 인식하는 것이 매우 중요하다. 이를 위해 거리정확도가 매우 높고 조도에 영향을 받지 않는 3차원 LiDAR 센서가 각광받고 있는 추세이다. 3차원 LiDAR는 주변 물체의
표면 정보를 point cloud 형태로 제공하고 이 데이터는 군집화, 물체 탐지 등과 같은 기술에 의해 유의미한 데이터로 변화 된다. 그 중 주변의 움직이는 물체들의 움직임을 예측, 회피하기 위해선 물체 추적 및 속도 추정 기술이 필수적이며 이를 모두 포함하여 다중 물체 추적 기술이라고 부른다. LiDAR 기반 다중 물체 추적 기술을위해 이미 많은 연구가 진행되었지만 센서 크기나 처리 속도 등의 원인으로 LiDAR 의 레이저 채널 수가 적어져 취득되는 point cloud의 밀도가 낮아지고 있다. 본 학위 논문에선 저밀도 3차원 point cloud 물체를 고려하는 새로운 다중 물체 추적 방법을 제안한다.
우선 EVD라고 불리는 3차원 보간에 기반한 저밀도 point cloud 물체의 정확한 속도 추정을 위한 새로운 단일 물체 추적 알고리즘을 제안한다. 물체 속도를 예측하기 위해선 점들을 직접 비교하는 방법이 일반적으로 사용된다. 하지만 저밀도 물체의 경우, 정확한 위치에서도 포인트들끼리 매칭되지 않기 때문에 점들을 직접 비교하는 방법은 물체의 위치차이를 구하는데 정확하지 않다. 따라서 수직 분포 예측에 기반한 물체 표면 정보 보강방법을 제안하여 저밀도 물체에 대해 정확한 속도 추정을 가능하게 한다.
저밀도 물체에 대해서 정확한 속도 추정이 가능해졌다고 하더라도 이는 같은 물체가 올바르게 연결 되었을 때만 가능하다. 하지만 다중물체추적에 입력으로 사용되는 군집화 알고리즘의 under/over-segmentation 에러는 잘못된 association 문제를 야기하고, 이 현상은 저밀도 point cloud에서 더 두드러진다. 이 문제를 해결하기 위해 신뢰할수 있는 물체를 구별하고 under/over-segmentation 에러가난 물체들을 이를 바탕으로 개선한다. 이후에 association score에 두 물체 사이에 겹치는 영역의 크기를 고려하여 전체적인 다중 물체 추적 기술의 성능을 높인다.
움직이는 로봇 플랫폼을 위한 다중 물체 추적 기술의 목적은 자차량의 움직임을 계획하기위한 안정적이고 강건한 판단을 내리도록 도와주는 것이다. 이를 위한 첫 번째 단계는 다른 주변 물체들이 움직이고 있는지 판별하는 것이다. 물체가 움직이고 있는지 판단하는 일반적인 방법은 예측된 속도를 이용하는 것이다. 하지만 저밀도 물체들의 경우, 그 추정된 속도값이 정확하지 않을 수 있다. 그러므로 정지한 물체는 0의 속력을 가지고, 이 0의 속력으로 물체의 point cloud를 누적한 물체가
가장 촘촘하다는 생각에 입각해 정지된 물체를 분류하기 위한 새로운 방법론을 제안한다.
전체적으로 이 학위논문에선 3차원 저밀도 point cloud 입력으로 인해 생기는 문제점에 대해 얘기하고, 그러한 문제점들을 고려한 새로운 다중 물체 추적 기술과 정지 물체 판별법을 제안한다. 제시된 프레임워크를 바탕으로 움직이는 플랫폼은 주변 물체에 대해 정확하게 예측할 수 있게 되고, 더욱더 정확한 판단이 가능해진다.

다국어 초록 (Multilingual Abstract)

In autonomous systems and robotics platforms, such as autonomous driving cars, it is very important to recognize their surrounding environment. To this end, 3D LiDAR sensors with very high distance accuracy and no illumination effects are in the spotlight. The 3D LiDAR provides surface information of surrounding objects in the form of a point cloud, and this data is transformed into meaningful data by some technologies such as segmentation and object detection. Among them, multi-target tracking and velocity estimation are essential to predict and avoid surrounding moving objects. Many researches have already been done for the multi-target tracking based on 3D LiDAR, but the number of laser channels in 3D LiDAR is decreasing due to sensor size and processing time, thus density of acquired point cloud becomes sparse. In this dissertation, a new multi-target tracking method considering sparse 3D point cloud object inputs is proposed.
First, a new single target tracking algorithm to estimate precise velocity of sparse point cloud object based on 3D interpolation, named EVD, is proposed. In order to estimate object velocity, direct point comparison method is commonly utilized. However, for low-density objects, the direct point comparison method is inaccurate to find a difference in the position of objects since points are not matched even on the actual position. Therefore, augmentation method for object’s surface information based on estimating vertical distribution is suggested so precise velocity estimation for sparse point cloud objects can be possible.
Although accurate velocity estimation becomes possible for sparse point cloud objects, the estimation is only applicable when the same objects are associated correctly in the multi-target tracking framework. However, under/over-segmentation errors from point segmentation which is generally utilized as an measurement input for 3D object tracking cause false data association problem and this problem becomes worse when a point cloud input is more spare. To solve the problem, confident tracklets are distinguished and the under/over-segmented objects are refined based on the confident objects. After that, using a new association scoring function additionally considering intersection of between two objects, overall multi-target tracking performance is increased.
A purpose of the multi-target tracking for a mobility platform is making stable and robust decisions to plan the ego vehicle’s future movement. The very first step for the decision is distinguishing whether other surrounding objects are willing to move or stay. How to decide the object status is generally based on velocity estimate, but for sparse point cloud objects, velocity estimates could not be accurate. Therefore, a new methodology to classify stopped objects is proposed based on the idea that a stopped object has zero speed so its accumulated point cloud by the zero speed should have the best compactness.
Overall, in this dissertation, problems caused by 3d sparse point cloud input on a mobility platform such as a self-driving car, is discussed and a new multi-target tracking method and its further usage, stopped object classification, are proposed considering the problems. The presented framework ensures that a mobility platform can precisely estimate and predict surrounding objects, so more accurate decision for the vehicle could be made.

목차 (Table of Contents)

• 1 INTRODUCTION 1
• 1.1 Background and Motivations . . . . . . . . . . . . . . . . . . . . . . 1
• 1.2 Contributions and Outline of the Dissertation . . . . . . . . . . . . . 3
• 1.2.1 Real-Time Object Tracking in Sparse Point Clouds based on Estimation of Vertical Distribution . . . . . . . . . . . . . . . 3
• 1.2.2 Online Multi-Target Tracking based on Tracklet Confidence and Virtual Measurement . . . . . . . . . . . . . . . . . . . . 5
• 1.2.3 Stopped Object Classification using Compactness of Accumulated 3D Point Cloud . . . . . . . . . . . . . . . . . . . . . . 5
• 2 Real-Time Object Tracking in Sparse Point Clouds based on Estimation of Vertical Distribution 7
• 2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
• 2.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
• 2.3 Overall Tracking Framework . . . . . . . . . . . . . . . . . . . . . . 12
• 2.3.1 Indirect Observation Model based on Motion Difference . . . 12
• 2.3.2 Dynamic System Model . . . . . . . . . . . . . . . . . . . . 15
• 2.4 Estimation of Vertical Distributions . . . . . . . . . . . . . . . . . . 17
• 2.4.1 Derivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
• 2.4.2 Trick to Speed Up EVD . . . . . . . . . . . . . . . . . . . . 19
• 2.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 19
• 2.5.1 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
• 2.5.2 Sensor Configuration . . . . . . . . . . . . . . . . . . . . . . 21
• 2.5.3 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . 23
• 2.5.4 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
• 2.5.5 Evaluation for Simulation Dataset . . . . . . . . . . . . . . . 25
• 2.5.6 Evaluation for Stationary Dataset . . . . . . . . . . . . . . . 26
• 2.5.7 Evaluation for Dynamic Dataset . . . . . . . . . . . . . . . . 26
• 2.5.8 Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
• 2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
• 3 Online Multi-Target Tracking based on Tracklet Confidence and Virtual Measurements 33
• 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
• 3.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
• 3.3 Online Tracking Framework with Tracklet Confidence . . . . . . . . 40
• 3.3.1 Tracklet Confidence . . . . . . . . . . . . . . . . . . . . . . 42
• 3.3.2 Affinity Score . . . . . . . . . . . . . . . . . . . . . . . . . . 43
• 3.3.3 Bounding Box Estimation . . . . . . . . . . . . . . . . . . . 47
• 3.4 Virtual Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . 52
• 3.4.1 Split Operation . . . . . . . . . . . . . . . . . . . . . . . . . 54
• 3.4.2 Merge Operation . . . . . . . . . . . . . . . . . . . . . . . . 57
• 3.5 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 58
• 3.5.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
• 3.5.2 Sensor Configuration . . . . . . . . . . . . . . . . . . . . . . 60
• 3.5.3 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
• 3.5.4 Implementation Details . . . . . . . . . . . . . . . . . . . . . 61
• 3.5.5 Performance Evaluation for Segmentation . . . . . . . . . . . 61
• 3.5.6 Performance Evaluation for MTT . . . . . . . . . . . . . . . 62
• 3.5.7 Performance Evaluation for Velocity Estimation . . . . . . . 64
• 3.5.8 Qualitative Discussion . . . . . . . . . . . . . . . . . . . . . 65
• 3.5.9 Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
• 3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
• 4 Stopped Object Classification using Compactness of Accumulated 3D Point Cloud 70
• 4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
• 4.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
• 4.3 Stopped Object Classification (SOC) . . . . . . . . . . . . . . . . . . 75
• 4.3.1 Overall Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 75
• 4.3.2 Point Cloud Accumulation . . . . . . . . . . . . . . . . . . . 79
• 4.3.3 Compactness of Point Cloud . . . . . . . . . . . . . . . . . . 83
• 4.4 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 84
• 4.4.1 Dataset and Evaluation Metrics . . . . . . . . . . . . . . . . 85
• 4.4.2 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
• 4.4.3 Evaluation for SOC . . . . . . . . . . . . . . . . . . . . . . . 86
• 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
• 5 Conclusion 90
• Bibliography 92
• Abstract (In Korean) 103