AMR를 이용한 가변적인 관심 대상 객체 추적을 위한 딥러닝 기반 프레임워크

곽정훈,양견모,구재완,서갑호 제어·로봇·시스템학회 2022 제어·로봇·시스템학회 논문지 Vol.28 No.12

AMR(Autonomous Mobile Robot) is being used to improve working environment through collaboration such as transporting goods between workers. For collaboration such as transporting goods, AMR tracks the workers and carries out goods transport. Object tracking is possible based on a deep learning model trained using big data, built as an object to be tracked. When the worker changes frequently, such as in a work environment, there is a problem in that big data construction and deep learning model learning are required whenever an object to be tracked is changed. There is a need for a method for tracking objects that change frequently while providing small amounts of data. This paper proposes a deep learning-based framework for tracking changeable object. An object to be tracked, such as a worker, is defined as a ToI (Target-of-Interest) object. The proposed framework utilizes a two-stage deep learning model to track a changeable ToI object. In the deep learning model of the first stage, an object of the same type as the ToI object is tracked. In the deep learning model of the second stage, the ToI object is found among the objects being tracked. The position of the ToI object is transformed into the coordinate system of the AMR so that the AMR can track the ToI object. In the experiment, the results of tracking the ToI object by using the proposed method were verified. When tracking ToI objects with a single-stage deep learning model with a small amount of data, the accuracy of tracking the ToI objects decreased according to the amount of data. In the case of the proposed method, the tracking of the ToI object was not affected by the amount of data. .

OnBoard Vision Based Object Tracking Control Stabilization Using PID Controller

Vinayagam Mariappan,Minwoo Lee,Juphil Cho,Jaesang Cha 국제문화기술진흥원 2016 International Journal of Advanced Culture Technolo Vol.4 No.4

In this paper, we propose a simple and effective vision-based tracking controller design for autonomous object tracking using multicopter. The multicopter based automatic tracking system usually unstable when the object moved so the tracking process can’t define the object position location exactly that means when the object moves, the system can’t track object suddenly along to the direction of objects movement. The system will always looking for the object from the first point or its home position. In this paper, PID control used to improve the stability of tracking system, so that the result object tracking became more stable than before, it can be seen from error of tracking. A computer vision and control strategy is applied to detect a diverse set of moving objects on Raspberry Pi based platform and Software defined PID controller design to control Yaw, Throttle, Pitch of the multicopter in real time. Finally based series of experiment results and concluded that the PID control make the tracking system become more stable in real time.

정확히 재가중되는 온라인 전체 에러율 최소화 기반의 객체 추적

장세인,박충식 한국지능정보시스템학회 2019 지능정보연구 Vol.25 No.4

Object tracking is one of important steps to achieve video-based surveillance systems. Object tracking is considered as an essential task similar to object detection and recognition. In order to perform object tracking, various machine learning methods (e.g., least-squares, perceptron and support vector machine) can be applied for different designs of tracking systems. In general, generative methods (e.g., principal component analysis) were utilized due to its simplicity and effectiveness. However, the generative methods were only focused on modeling the target object. Due to this limitation, discriminative methods (e.g., binary classification) were adopted to distinguish the target object and the background. Among the machine learning methods for binary classification, total error rate minimization can be used as one of successful machine learning methods for binary classification. The total error rate minimization can achieve a global minimum due to a quadratic approximation to a step function while other methods (e.g., support vector machine) seek local minima using nonlinear functions (e.g., hinge loss function). Due to this quadratic approximation, the total error rate minimization could obtain appropriate properties in solving optimization problems for binary classification. However, this total error rate minimization was based on a batch mode setting. The batch mode setting can be limited to several applications under offline learning. Due to limited computing resources, offline learning could not handle large scale data sets. Compared to offline learning, online learning can update its solution without storing all training samples in learning process. Due to increment of large scale data sets, online learning becomes one of essential properties for various applications. Since object tracking needs to handle data samples in real time, online learning based total error rate minimization methods are necessary to efficiently address object tracking problems. Due to the need of the online learning, an online learning based total error rate minimization method was developed. However, an approximately reweighted technique was developed. Although the approximation technique is utilized, this online version of the total error rate minimization could achieve good performances in biometric applications. However, this method is assumed that the total error rate minimization can be asymptotically achieved when only the number of training samples is infinite. Although there is the assumption to achieve the total error rate minimization, the approximation issue can continuously accumulate learning errors according to increment of training samples. Due to this reason, the approximated online learning solution can then lead a wrong solution. The wrong solution can make significant errors when it is applied to surveillance systems. In this paper, we propose an exactly reweighted technique to recursively update the solution of the total error rate minimization in online learning manner. Compared to the approximately reweighted online total error rate minimization, an exactly reweighted online total error rate minimization is achieved. The proposed exact online learning method based on the total error rate minimization is then applied to object tracking problems. In our object tracking system, particle filtering is adopted. In particle filtering, our observation model is consisted of both generative and discriminative methods to leverage the advantages between generative and discriminative properties. In our experiments, our proposed object tracking system achieves promising performances on 8 public video sequences over competing object tracking systems. The paired t-test is also reported to evaluate its quality of the results. Our proposed online learning method can be extended under the deep learning architecture which can cover the shallow and deep networks. Moreover, online learning methods, that need the exact reweighting pro... 영상 기반의 보안 시스템의 증가함에 따라 각 용도마다 다른 다양한 객체들에 대한 처리들이 중요해지고 있다. 객체 추적은 객체 인식, 검출과 같은 작업들과 함께 필수적인 작업으로 다뤄진다. 이 객체 추적을 달성하기위해서 다양한 머신러닝이 적용될 수 있다. 성공적인 분류기로써 전체 에러율 최소화(total-error-rate minimization) 기반의 방법론이 사용될 수 있다. 이 전체 에러율 최소화 기반의 방법론은 오프라인 학습을 기반으로 하고 있다. 객체 추적은 실시간으로 처리하며 갱신해야하는 것이 필수적이므로 온라인 학습(online learning)을 기반으로 하는 것이 적합하다. 온라인 전체 에러율 최소화 방법론이 개발되었지만 점근적으로 재가중되는(approximately reweighted) 작업이 포함되어 에러를 누적시킬 수 있다는 단점이 있다. 본 논문에서는 정확하게 재가중되는(exactly reweighted) 방법론을 제안하면서 온라인 전체 에러율 최소화가 달성되었다. 이 제안된온라인 학습 방법론을 객체 추적에 적용하여 총 8개의 데이터베이스에서 다른 추적 방법론들 보다 좋은 성능이달성되었다.

A Particular Object Tracking in an Environment of Multiple Moving Objects

Hyung-Bok Kim,Kwee-Bo Sim 제어로봇시스템학회 2010 제어로봇시스템학회 국제학술대회 논문집 Vol.2010 No.10

Usually, the video based object tracking deal with non-stationary image stream that changes over time. Robust and Real time moving object tracking is a problematic issue in computer vision research area. Multiple object tracking has many practical applications in scene analysis for automated surveillance. If we can track a particularly selected object in an environment of multiple moving objects, then there will be a variety of applications. In this paper, we introduce a particular object tracking in an environment of multiple moving objects. When tracking, we need to analyze video sequences to track object in each frame. In this paper, we use a differential image of region-based tracking method for the detection of multiple moving objects. In other to ensure accurate object detection in unconstrained environment, we also use a method of background image update. There are problems in tracking a particular object through a sequence of video. It can’t rely only on image processing techniques. Thus we solved these problems using a probabilistic framework. Particle filter has been proven to be a robust algorithm to deal with the nonlinear, non-Gaussian problems. In this paper, the particle filter provides a robust object tracking framework under ambiguity conditions and greatly improved estimation accuracy for complicated tracking problems.

가변 윈도우 마스크와 광 상관기를 이용한 스테레오 추적 시스템의 구현

이재수,김은수 연세대학교 전파통신연구소 2001 電波通信論文誌 Vol.5 No.1

본 논문에서는 가변 윈도우 마스크의 기준 영상과 스테레오 입력 영상간에 광 BPEJTC를 실행하여 추적 물체의 위치 값을 추출하고, 이 값으로 스테레오 카메라를 제어하는 새로운 스테레오 물체추적 알고리즘을 제안하였다. 즉, 스테레오 비젼 시스템의 구성 요소에 의해 추적 물체까지의 거리 정보를 쉽게 구할 수 있고, 이 거리 정보로 윈도우 마스크를 가변 시켜 추적물체 영역을 추출할 수 있다. 이 가변 윈도우 마스크의 추적물체 영역은 다음 기준 영상으로 갱신하여 사용된다. 그리고 이 가변 윈도우 마스크의 기준영상과 스테레오 입력 영상간에 광 BPEJTC를 실행하여 추적 물체의 위치 값을 구하고, 이 값으로 스테레오 카메라의 주시각과 팬/틸트를 제어함으로써 스테레오 물체 추적이 이루어 진다 실험 결과 제안한 알고리즘은 스테레오 입력 영상에서 배경잡음과 관계없이 추적 물체영역을 추출하여 스테레오로 추적할 수 있었고, 이의 구현으로 스테레오 원격작업 시스템이나 적응적인 스테레오 물체 추적기 등의 구현 가능성을 제시하였다. In this paper, we proposed a new stereo object tracking algorithm that can extract location's values of a tracking object by applying variable window mask and optical BPEJTC to stereo input image, and can be in control of stereo camera using these values. That is, the distance information from stereo camera to tracking object can be acquiring easily by elements of a streo vision system, we can extract area of a tracking object by vary window mask as this distance information. It use an extractive tracking object's area by the variable window mask as next updated reference image. And, the location's coordinate of the tracking object can be acquiring as it carry out an optical BPEJTC between a reference image of variable window mask and a stereo input image, it can be accomplish stereo object tracking by a controlling convergence angle and pan/tilt of stereo camera as this values. From the experimental results, the proposed algorithm is found to extract the area of the target object from input image independent of the background noise in the stereo input image, and a possibility of implementation of the stereo tele-working and adaptive stereo object tracker using the proposed algorithm is also suggested.

Object Motion Tracking using a Moving Direction Estimate and Color Updates

Samuel Henry Chang,심덕선,김희영,최광남 제어·로봇·시스템학회 2012 International Journal of Control, Automation, and Vol.10 No.1

This paper presents a direction detection and tracking object color update algorithm used to track moving objects that change colors. Different from traditional color-based tracking methods, which use an initial color distribution in order to track objects as long as the object carries the full or partial initial color, this method introduces a color update method used to quickly find the new object color in a new location if the object changes its color partially or completely; the updated color is then used to locate the object. In our algorithm, an initial color pattern is used to track an object using the color. During the tracking, an object’s new location is at first estimated and then used to detect any color change. If the color has changed, a new color pattern is updated based on the changes in the previous color distribution, and then the new color pattern is used to calculate the current location of the object. This algorithm utilizes the property that the movement of an object can be estimated either by using the object’s shadow or by background subtraction. The implementation of our algorithm results in an effective real-time object tracking. The validity of the approach is illustrated by the presentation of experiment results obtained using the methods described in this paper.

모바일 환경 Homography를 이용한 특징점 기반 다중 객체 추적

한우리,김영섭,이용환 한국반도체디스플레이기술학회 2015 반도체디스플레이기술학회지 Vol.14 No.3

This paper proposes an object tracking system based on keypoints using homography in mobile environments. The proposed system is based on markerless tracking, and there are four modules which are recognition, tracking, detecting and learning module. Recognition module detects and identifies an object to be matched on current frame correspond to the database using LSH through SURF, and then this module generates a standard object information. Tracking module tracks an object using homography information that generate by being matched on the learned object keypoints to the current object keypoints. Then update the window included the object for defining object’s pose. Detecting module finds out the object based on having the best possible knowledge available among the learned objects information, when the system fails to track. The experimental results show that the proposed system is able to recognize and track objects with updating object’s pose for the use of mobile platform.

Specified Object Tracking Problem in an Environment of Multiple Moving Objects

Park, Seung-Min,Park, Jun-Heong,Kim, Hyung-Bok,Sim, Kwee-Bo Korean Institute of Intelligent Systems 2011 INTERNATIONAL JOURNAL of FUZZY LOGIC and INTELLIGE Vol.11 No.2

Video based object tracking normally deals with non-stationary image streams that change over time. Robust and real time moving object tracking is considered to be a problematic issue in computer vision. Multiple object tracking has many practical applications in scene analysis for automated surveillance. In this paper, we introduce a specified object tracking based particle filter used in an environment of multiple moving objects. A differential image region based tracking method for the detection of multiple moving objects is used. In order to ensure accurate object detection in an unconstrained environment, a background image update method is used. In addition, there exist problems in tracking a particular object through a video sequence, which cannot rely only on image processing techniques. For this, a probabilistic framework is used. Our proposed particle filter has been proved to be robust in dealing with nonlinear and non-Gaussian problems. The particle filter provides a robust object tracking framework under ambiguity conditions and greatly improves the estimation accuracy for complicated tracking problems.

Specified Object Tracking Problem in an Environment of Multiple Moving Objects

Seung-Min Park,Junheong Park,Hyung-Bok Kim,Kwee-Bo Sim 한국지능시스템학회 2011 INTERNATIONAL JOURNAL of FUZZY LOGIC and INTELLIGE Vol.11 No.2

Video based object tracking normally deals with non-stationary image streams that change over time. Robust and real time moving object tracking is considered to be a problematic issue in computer vision. Multiple object tracking has many practical applications in scene analysis for automated surveillance. In this paper, we introduce a specified object tracking based particle filter used in an environment of multiple moving objects. A differential image region based tracking method for the detection of multiple moving objects is used. In order to ensure accurate object detection in an unconstrained environment, a background image update method is used. In addition, there exist problems in tracking a particular object through a video sequence, which cannot rely only on image processing techniques. For this, a probabilistic framework is used. Our proposed particle filter has been proved to be robust in dealing with nonlinear and non-Gaussian problems. The particle filter provides a robust object tracking framework under ambiguity conditions and greatly improves the estimation accuracy for complicated tracking problems.

객체 검출 인식률 향상을 위한 다중 객체 추적 기반 강건한 트랙 관리 기법

김민기,이동석,최병인 대한전자공학회 2023 전자공학회논문지 Vol.60 No.12

Recently, deep learning-based object detection and multi-object tracking used in autonomous driving technology have been widely studied. The two technologies consist of one sequence. Taking advantage of this, we propose a robust track management based on multi-object tracking technology that complements the limitations of object detection using only a single image. This paper propose the following method. Hungarian algorithm cost matrix for inter-class using class information,, Track management technique that induces data association through Template Matching, Track update that improves reliability for object detection by utilizing additional information of class and score. Through three types of robust track management, it compensate for non-detection or mis-classification problems that occur in object detection. Additionally, it show stable results in object detection and multi-object tracking. As a result, compared to the model using only object detection, the mAP increase by about 4%, and the precision result increased by about 10%. we were tested in an actual autonomous driving environment and recorded high performance improvement in objects with little learning data or small sizes. In addition, it enabled stable object detection and tracking even in sudden image shaking such as bumps. 최근 자율주행 기술에 활용되는 딥러닝 기반 객체 검출과 다중 객체 추적은 많은 연구가 이루어지고 있다. 두 개의 기술은 하나의 시퀀스로 이루어져 있고, 이 점을 이용하여 단일 영상만을 사용하여 객체 검출의 한계점을 보완하는 다중 객체 추적 기반 강건한 트랙 관리 기법을 제안한다. 본 논문은 클래스 정보를 활용한 클래스 간의 헝가리안 알고리즘 코스트 매트릭스(cost matrix), 템플릿 매칭을 통한 데이터 연관 유도 트랙 관리 기법, 클래스와 스코어의 추가 정보를 활용하여 검출 신뢰성을 향상하는 트랙 업데이트를 제안한다. 3가지 강건한 트랙 관리를 통해 객체 검출에서 생기는 미검출 혹은 오분류의 문제를 보완하고 객체 검출과 다중 객체 추적의 안정화된 결과를 보여준다. 그 결과, 객체 검출만을 사용한 모델과 비교하여 mAP가 약 4% 증가했고, 정밀도(Precision)의 결과는 약 10% 증가했다. 본 논문에서 제안한 기법은 실제 도로 주행 환경에서 테스트 되었고, 학습 데이터의 수가 적거나 작은 크기의 객체에서 높은 성능 향상을 기록했다. 또한, 방지턱과 같은 급격한 영상의 흔들림에서도 안정적인 객체 검출 및 추적을 가능하게 한다.