반도체 칩의 정밀한 높이 측정을 위한 스테레오 비전 알고리즘

김영두(Youngdoo Kim),조태훈(Tai-Hoon Cho) 한국지능시스템학회 2011 한국지능시스템학회 학술발표 논문집 Vol.21 No.1

많은 2D 비전 알고리즘들이 반도체 부품의 불량 검출에 적용되어 왔다. 그러나 이런 2D 비전 알고리즘으로는 높이정보를 통해 불량의 유무를 검출해야하는 공정에 적용하는데 한계가 있다. 스테레오 비전은 카메라와 측정 물체간의 거리 정보를 획득하기 위해 이용된다. 그러나 이런 3D 비전 방법을 공정과정에 적용하는 것은 스테레오 비전이 가진 측정 오차로 인해 한계가 있다. 이에 본 논문에서는 스테레오 비전을 통해 측정된 높이 데이터를 재조정함으로써 기존의 스테레오 비전 방법이 가진 측정 오차를 줄이는 알고리즘을 제안한다. 본 논문에서 제안된 알고리즘은 반도체 칩의 높이 측정 실험을 통해 검증되었다. Lots of 2D vision algorithms have been applied for inspection. However, these 2D vision algorithms have limitation in inspection applications which require 3D information data such as the height of semiconductor chips. Stereo vision is a well known method to measure the distance from the camera to the object to be measured. But it is difficult to apply for inspection directly because of its measurement error. In this paper, we propose an adjustment method to reduce the error of the measured height data for stereo vision. This algorithm has been proved through the experiment which measures the height of semiconductor chips.

스테레오 비전 기반 가상 모델 확장형 칼만 필터를 이용한 안정된 상태 추정 방법

임영철(Young-Chul Lim),이충희(Chung-Hee Lee),이종훈(Jong-Hoon Lee) 大韓電子工學會 2011 電子工學會論文誌-SC (System and control) Vol.48 No.3

본 논문은 스테레오 비전 시스템에서 객체의 기동 상태에 상관없이 안정된 거리 및 속도를 추정할 수 있는 방법을 제안한다. 스테레오 비전은 좌우 영상의 시차를 이용하여 거리를 추정할 수 있지만, 영상 화소의 양자화 오차로 인해 거리 오차가 발생할 수 있다. 부화소 보간법은 이러한 양자화 오차를 최소화하여 실수를 갖는 정밀 시차를 추정할 수 있다. 확장형 칼만 필터는 추정된 정밀 시차의 공분산을 최소화하고 객체의 속도를 추정하기 위하여 사용되어진다. 하지만, 시스템 모델의 불확실성으로 인해 기동이 발생했을 때, 발산 문제가 생기고 이는 오히려 추정 오차를 증가시킨다. 본 논문에서는 연산 시간을 최소화하면서, 객체의 기동 상태에 상관없이 안정된 상태 추정 성능을 제공할 수 있는 가상 모델 확장형 칼만 필터를 제안한다. 모의실험 및 실제 도로 환경에서의 실험 결과는 제안한 방법이 기존 추정 필터들에 비하여, 다양한 기동 상태에서 안정된 추정 성능과 향상된 연산시간을 제공한다는 것을 보여준다. This paper presents a method that estimates distance and velocity of an object with reliability regardless of maneuver status of the target in stereo vision system. A stereo vision system can calculate a distance with disparity from left and right images. However, the distance estimation error may occur due to quantization error of image pixel. A sub-pixel interpolation method minimizes the quantization error and estimates accurate disparity with real value. Extended Kalman filter (EKF) was used to minimize the error covariance and estimate the object’s velocity. However, divergence problem occurs due to model uncertainty when a target maneuvers highly, which makes the estimation error increase. In this paper, we propose a virtual model extended Kalman filter (VMEKF) method that minimizes the processing time and provides reliable estimation ability regardless of maneuver status. Computer simulations and experimental results in real road environments demonstrate that the proposed method gives a reliable estimation performance and reduces processing time under various maneuver status while comparing other estimation filters.

Stereo Vision과 AlphaPose를 이용한 다중 객체 거리 추정 방법에 관한 연구

이주민 ( Ju-min Lee ),배현재 ( Hyeon-jae Bae ),장규진 ( Gyu-jin Jang ),김진평 ( Jin-pyeong Kim ) 한국정보처리학회 2021 정보처리학회논문지. 소프트웨어 및 데이터 공학 Vol.10 No.7

최근 COVID-19 확산 방지를 위한 공공장소에서는 최소 1m 이상을 유지하는 물리적 거리두기 정책을 실행하고 있다. 본 논문에서는 드론과 CCTV가 취득한 스테레오 영상에서 실시간으로 사람들 간의 거리를 추정하는 방법과 추정된 거리에서 1m 이내의 객체를 인식하는 자동화 시스템을 제안한다. 기존의 CCTV를 이용하여 다중 객체 간의 거리 추정에 사용되었던 방법의 문제점으로는 한 대의 CCTV만을 이용하여 객체의 3차원 정보를 얻지 못한다는 것이다. 선, 후행하거나 겹쳐진 사람 간의 거리를 구하기 위해서는 3차원 정보가 필요하기 때문이다. 또한, 일반적인 Detected Bounding Box를 사용하여 영역 안에서 사람이 존재하는 정확한 좌표를 얻지 못한다. 따라서 사람이 존재하는 정확한 위치 정보를 얻기 위해 스켈레톤 추출하여 관절 키포인트의 2차원 좌표를 획득한 후, Stereo Vision을 이용한 카메라 캘리브레이션을 적용하여 3차원 좌표로 변환한다. 3차원으로 변환된 관절 키포인트의 중심좌표를 계산하고 객체 간 사이의 거리를 추정한다. 3차원 좌표의 정확성과 객체(사람) 간의 거리 추정 실험을 수행한 결과, 1m 이내에 존재하는 다수의 사람 간의 거리 추정에서 0.098m 이내 평균오차를 보였다. Recently, We are carrying out a policy of physical distancing of at least 1m from each other to prevent the spreading of COVID-19 disease in public places. In this paper, we propose a method for measuring distances between people in real time and an automation system that recognizes objects that are within 1 meter of each other from stereo images acquired by drones or CCTVs according to the estimated distance. A problem with existing methods used to estimate distances between multiple objects is that they do not obtain three-dimensional information of objects using only one CCTV. his is because three-dimensional information is necessary to measure distances between people when they are right next to each other or overlap in two dimensional image. Furthermore, they use only the Bounding Box information to obtain the exact coordinates of human existence. Therefore, in this paper, to obtain the exact two-dimensional coordinate value in which a person exists, we extract a person's key point to detect the location, convert it to a three-dimensional coordinate value using Stereo Vision and Camera Calibration, and estimate the Euclidean distance between people. As a result of performing an experiment for estimating the accuracy of 3D coordinates and the distance between objects (persons), the average error within 0.098m was shown in the estimation of the distance between multiple people within 1m.

스테레오 비전 기반 전방 충돌 경고 및 회피 시스템

이윤희(Yunhee Lee),김병주(Byungjoo Kim),정호기(Hogi Jung),윤팔주(Paljoo Yoon) 한국자동차공학회 2006 한국자동차공학회 춘 추계 학술대회 논문집 Vol.- No.-

This paper describes stereo vision based forward collision warning and avoidance system which consists of forward obstacle monitoring sensor and brake actuator for collision avoidance. Several tools can be used for forward obstacle monitoring system : radar, laser, camera, sensor fusion. The proposed system in this paper uses stereo vision. Stereo vision method can acquire distance information that cannot be acquired by using single camera. The proposed system acquires ROI(Region Of Interest) through ego-lane detection. Therefore, the proposed system has advantage of detecting ego-lane obstacle and reduction of computational time by removing no interest region. In addition, the proposed system uses feature-based stereo matching. Feature-based stereo matching provides relatively insensitive to noise and low computational time. The proposed system uses adaptive threshold for disparity histogram since disparity quantity depends on distance. Also the proposed system uses active braking for collision avoidance, and the effectiveness of active braking using ESC has been proven for avoiding dangerous situations in case of imminent rear end collision.

스테레오 영상을 이용한 상대항법 실험적 연구

변수영,이동진,방효충 한국항공우주학회 2012 한국항공우주학회 학술발표회 논문집 Vol.2012 No.4

본 논문에서는 스테레오 영상을 이용하여 두 대의 비행체 사이의 위치관계를 추정하는 구체적인 방법을 제안한다. 두 대의 비행체가 도킹임무를 수행하기 위해서 근접거리에서 기동하고 있을 때 영상정보를 통해 정밀한 상대항법을 구현할 수 있다. 스테레오 영상은 3차원 영상정보를 얻기 위한 기법으로, 복수의 카메라를 이용해 카메라와 촬영된 물체 간의 거리정보를 추정하는데 활용될 수 있다. 영상정보를 바탕으로 거리 정보를 포함한 3차원 공간에서의 상대위치를 구하기 위한 실험을 설계하고 그 방법론과 결과를 정리했다. This paper concern with a methodology of relative navigation using stereo vision. By using imagery data, we can navigate two flight vehicles very precisely to conduct docking in a close range. Stereo vision is an one of computer vision technique and its purpose is to get three dimensional imagery data from multiple cameras. An experiment in this paper shows that how to acquire three dimensional relative navigation information of two vehicles using stereo vision in a short distance.

Development of Integrated Algorithm Based on Deep Learning and Stereo Vision Technology for Real Time Worker Tracking in Decommissioning Site

Byung Hee Won,Chaehun Lee,Seonkwang Yoon,Sang-Bum Hong 한국방사성폐기물학회 2022 한국방사성폐기물학회 학술논문요약집 Vol.20 No.1

It is important to ensure worker’s safety from radiation hazard in decommissioning site. Real-time tracking of worker’s location is one of the factors necessary to detect radiation hazard in advance. In this study, the integrated algorithm for worker tracking has been developed to ensure the safety of workers. There are three essential techniques needed to track worker’s location, which are object detection, object tracking, and estimating location (stereo vision). Above all, object detection performance is most important factor in this study because the performance of tracking and estimating location is depended on worker detection level. YOLO (You Only Look Once version 5) model capable of real-time object detection was applied for worker detection. Among the various YOLO models, a model specialized for person detection was considered to maximize performance. This model showed good performance for distinguishing and detecting workers in various occlusion situations that are difficult to detect correctly. Deep SORT (Simple Online and Realtime Tracking) algorithm which uses deep learning technique has been considered for object tracking. Deep SORT is an algorithm that supplements the existing SORT method by utilizing the appearance information based on deep learning. It showed good tracking performance in the various occlusion situations. The last step is to estimate worker’s location (x-y-z coordinates). The stereo vision technique has been considered to estimate location. It predicts xyz location using two images obtained from stereo camera like human eyes. Two images are obtained from stereo camera and these images are rectified based on camera calibration information in the integrated algorithm. And then workers are detected from the two rectified images and the Deep SORT tracks workers based on worker’s position and appearance between previous frames and current frames. Two points of workers having same ID in two rectified images give xzy information by calculating depth estimation of stereo vision. The integrated algorithm developed in this study showed sufficient possibility to track workers in real time. It also showed fast speed to enable real-time application, showing about 0.08 sec per two frames to detect workers on a laptop with high-performance GPU (RTX 3080 laptop version). Therefore, it is expected that this algorithm can be sufficiently used to track workers in real decommissioning site by performing additional parameter optimization.

스테레오 비전 뎁스 맵을 이용한 쿼드로터 영상기반 자동착륙

박종호,박철우,김유단 한국항공우주학회 2011 한국항공우주학회 학술발표회 논문집 Vol.2011 No.11

쿼드로터와 같은 무인항공기(UAV)는 인간이 접근할 수 없는 위험 지역을 탐색할 수 있어 최근 큰 관심을 받고 있다. 무인항공기가 주어진 임무를 수행하기 위해서는 무인항공기의 모델링이 필수적인데, 쿼드로터는 복잡한 비선형 동역학을 가지고 있어 일부분을 생략 또는 왜곡하여 모델링하는 사례가 많았다. 본 논문에서는 영상기반 자동착륙을 위한 뎁스 맵 모델링과 쿼드로터의 모델링, 그리고 자동착륙을 위한 제어 기법을 다룬다. 궤환선형화와 선형최적 제어기법을 이용해 제어기를 설계하였고, 수치 시뮬레이션을 통하여 성능을 검증하였다. Unmanned aerial vehicle such as a quadrotor has advantage of accessing dangerous environments. Previous works on quadrotor used some approximations for quadrotor modeling because of its complicated nonlinear behavior. This paper mainly describes depth map modeling for vision-based automatic landing, quadrotor modeling, and controller design. Feedback linearization and linear quadratic regulator have been applied for designing the controller. Numerical simulation is performed to validate the performance of the proposed modeling and controller.

VEHICLE DETECTION SYSTEM DESIGN BASED ON STEREO VISION SENSORS

황준연,허건수 한국자동차공학회 2009 International journal of automotive technology Vol.10 No.3

Vehicle detection is a crucial issue for driver assistance system as well as for autonomous vehicle guidance function and it has to be performed with high reliability to avoid any potential collision. The vision-based vehicle detection systems are regarded promising for this purpose because they require little infrastructure on a highway. However, the feasibility of these systems in passenger car requires accurate and robust sensing performance. In this paper, a vehicle detection system using stereo vision sensors is developed. This system utilizes feature extraction, epipoplar constraint and feature matching in order to robustly detect the initial corresponding pairs. The proposed system can detect a leading vehicle in front and can estimate its position parameters such as the distance and heading angle. After the initial detection, the system executes the tracking algorithm for the vehicles in the lane. The proposed vehicle detection system is implemented on a passenger car and its performances are verified experimentally. Vehicle detection is a crucial issue for driver assistance system as well as for autonomous vehicle guidance function and it has to be performed with high reliability to avoid any potential collision. The vision-based vehicle detection systems are regarded promising for this purpose because they require little infrastructure on a highway. However, the feasibility of these systems in passenger car requires accurate and robust sensing performance. In this paper, a vehicle detection system using stereo vision sensors is developed. This system utilizes feature extraction, epipoplar constraint and feature matching in order to robustly detect the initial corresponding pairs. The proposed system can detect a leading vehicle in front and can estimate its position parameters such as the distance and heading angle. After the initial detection, the system executes the tracking algorithm for the vehicles in the lane. The proposed vehicle detection system is implemented on a passenger car and its performances are verified experimentally.

Vision-based Object Detection for Passenger’s Safety in Railway Platform

Seh-Chan Oh,Gil-Dong Kim,Woo-Tae Jeong,Young-Tae Park 제어로봇시스템학회 2008 제어로봇시스템학회 국제학술대회 논문집 Vol.2008 No.10

In this paper, we propose a vision-based object detection algorithm for railway passenger’ safety. The proposed algorithm uses three-dimensional position information with stereo cameras for minimizing various illuminant effects in railway platform environment, such as ambient illumination changes due to train arrival/departure in the scene. The detection process analyzes scene and detects both four different train status and fallen objects in preset monitoring area. To solve the detection problem in conventional two-dimensional detection system, the system detects object in three dimensionally by using stereo vision algorithm. We verify the system performance with extensive experimental results in a metro station. We expect the proposed algorithm will play a key role in establishing highly intelligent monitoring system for passenger’ safety for future railway environment.

스테레오 비전과 YOLOv3 를 이용한 드론의 3 차원 실내 위치 추정 알고리즘 개발

변영훈(Younghun Byeon),강민재(Minjae Kang),임현준(Hyeon Jun Lim),김한솔(Han Sol Kim) 제어로봇시스템학회 2021 제어·로봇·시스템학회 논문지 Vol.27 No.11

In this paper, we propose a three-dimensional indoor position estimation algorithm for drones using stereo vision and YOLOv3. First, we find the bounding box of a drone in the image using a deep-learning-based object detection algorithm called YOLOv3. To this end, we collect the training dataset consisting of drone images. In addition, the object detection performance of the YOLOv3 algorithm is improved by dividing object class labels of the same drone based on the angle of the drone seen from the camera. Then, the three-dimensional relative position of the drone is estimated based on the camera internal parameters, the bounding box information, and the depth map taken by the stereo vision. In addition, the Kalman filter is employed to estimate the position of the drone continuously. Finally, the position estimation performance of the proposed algorithm is evaluated through the experiments.