Visual Attention Servo Control for Task-specific Robotic Applications

Dong Liu,Ming Cong,Yu Du,Yunfei Zhang,Clarence W. de Silva 제어·로봇·시스템학회 2013 International Journal of Control, Automation, and Vol.11 No.6

This paper proposes a visual attention servo control (VASC) method which uses the Gaussian mixture model (GMM) for task-specific applications of mobile robots. In particular, low dimensional bias feature template is obtained using GMM to get an efficient attention process. An image-based visual servo (IBVS) controller is used to search for a desired object in a scene through an attention system which forms a task-specific state representation of the environment. First, task definition and object representation in semantic memory (SM) are proposed, and bias feature template is obtained using GMM deduction for features from high dimension to low dimension. Second, the features intensity, color, size and orientation are extracted to build the feature set. Mean shift method is used to segment the visual scene into discrete proto-objects. Given a task-specific object, top-down bias attention is evaluated to generate the saliency map by combining with the bottom-up saliency-based attention. Third, a visual attention servo controller is developed to integrate the IBVS controller and the attention system for robotic cognitive control. A rule-based arbitrator is proposed to switch between the episodic memory (EM)-based controller and the IBVS controller depending on whether the robot obtains the desired attention point on the image. Finally, the proposed method is evaluated on task-specific object detection under different conditions and visual attention servo tasks. The obtained results validate the applicability and usefulness of the developed method for robotics.

A Platform Stabilization Algorithm Based on Feedforward Visual-Inertial Servoing

Ho Quoc Phuong Nguyen,강희준,서영수,Ole Jacob Elle 한국정밀공학회 2012 International Journal of Precision Engineering and Vol. No.

This paper presents a method of improving performance of visual servoing system by integrating inertial sensors to the system. The method is applied to a roll and pitch rotated platform stabilization in high control frequency. For the purpose,an inertial measurement unit is attached to the platform to provide its dynamics information. A new inertial information based feedforward control is used along with the conventional visual feedback control. Two contributions are realized: first,it helps solve the remaining limitation of static-object assumption in conventional visual servoing. Second, it helps drastically improve the response rate of the servoing system due to the utilization of a high-speed inertial measurement unit. Stability of the control system is analyzed such that the error of the system is proved to be bounded. Control algorithm was simulated using Matlab Aerospace Toolbox as well as Robotics Toolbox. Then, experiments were implemented to verify the feasibility of the proposed methodology.

A Platform Stabilization Algorithm Based on Feedforward Visual-Inertial Servoing

Nguyen, Ho Quoc Phuong,Kang, Hee-Jun,Suh, Young-Soo,Elle, Ole Jacob 한국정밀공학회 2012 International Journal of Precision Engineering and Vol.13 No.4

This paper presents a method of improving performance of visual servoing system by integrating inertial sensors to the system. The method is applied to a roll and pitch rotated platform stabilization in high control frequency. For the purpose, an inertial measurement unit is attached to the platform to provide its dynamics information. A new inertial information based feed forward control is used along with the conventional visual feedback control. Two contributions are realized: first, it helps solve the remaining limitation of static-object assumption in conventional visual servoing. Second, it helps drastically improve the response rate of the servoing system due to the utilization of a high-speed inertial measurement unit. Stability of the control system is analyzed such that the error of the system is proved to be bounded. Control algorithm was simulated using Matlab Aerospace Toolbox as well as Robotics Toolbox. Then, experiments were implemented to verify the feasibility of the proposed methodology.

무인기 이착륙 충격 감소를 위한 비주얼 서보잉기반 모션플랫폼 임피던스 제어

강형엽,임재준,최우영 한국동력기계공학회 2024 동력시스템공학회지 Vol.28 No.6

본 논문에서는 무인기의 이/착륙 시 발생하는 충격을 감소하기 위한 6축 모션 플랫폼 제어 시스템을 제안한다. 이/착륙 시 돌풍 등으로 인해 기체의 전도 및 충돌이 발생할 수 있으며, 이를 방지하기 위해 비주얼 서보잉 기반 임피던스 제어기를 통한 충격 감소 시스템을 개발하였다. 제안된 시스템은 6개의 선형 액추에이터로 구성된 플랫폼과 플랫폼 중앙에 설치된 카메라로 이루어져 있으며, 제어기는 역기구학 기반의 피드백 PID 제어, 비전 기반 임피던스 제어기, 그리고 비주얼 서보잉 기술을 활용하여 설계되었다. 제안된 방법은 MATLAB/Simulink 시뮬레이션 실험을 통해 추종 정확도 RMS 97%의 비주얼 서보잉을 수행하였으며, 플랫폼이 11.5 cm 하강하여 안전한 착륙공간을 제공하여 무인기의 착륙 충격을 효과적으로 감소시킴을 확인하였다. 향후 무인기 안전성 향상에 기여할 것으로 기대된다. This paper proposes a six-axis motion platform control system designed to reduce shock during unmanned aerial vehicle (UAV) takeoff and landing. During these phases, factors such as gusts of wind can cause UAV tipping or collision, and to address this, a shock reduction system by visual servoing-based impedance controller is developed. The proposed system consists of a platform with six linear actuators and a centrally positioned camera, and the controller is designed using inverse kinematics-based feedback PID control, vision-based impedance control, and visual servoing techniques. The proposed method was verified through MATLAB/Simulink simulation experiments to perform visual serving with a tracking accuracy of 97% RMS, and the platform descended 11.5 cm to provide a safe landing space, effectively reducing the landing shock of the UAV. This is expected to contribute to improving the safety of UAVs in the future.

혼합 비주얼 서보 제어 기법을 이용한 이동로봇의 목표물 추종

이호원(Howon Lee),권지욱(Ji-Wook Kwon),홍석교(Suk-Kyo Hong),좌동경(Dongkyoung Chwa) 대한전기학회 2011 전기학회논문지 Vol.60 No.6

This paper proposes a target tracking algorithm for wheeled mobile robots using in various fields. For the stable tracking, we apply a vision system to a mobile robot which can extract targets through image processing algorithms. Furthermore, this paper presents an algorithm to position the mobile robot at the desired location from the target by estimating its relative position and attitude. We show the problem in the tracking method using the Position-Based Visual Servo(PBVS) control, and propose a tracking method, which can achieve the stable tracking performance by combining the PBVS control with Image-Based Visual Servo(IBVS) control. When the target is located around the outskirt of the camera image, the target can disappear from the field of view. Thus the proposed algorithm combines the control inputs with of the hyperbolic form the switching function to solve this problem. Through both simulations and experiments for the mobile robot we have confirmed that the proposed visual servo control method is able to enhance the stability compared to of the method using only either PBVS or IBVS control method.

An Enhanced IBVS Controller of a 6DOF Manipulator Using Hybrid PDSMC Method

Shutong Li,Ahmad Ghasemi,Wen-Fang Xie,Yanbin Gao 제어·로봇·시스템학회 2018 International Journal of Control, Automation, and Vol.16 No.2

The accuracy and stability are two fundamental concerns of the visual servoing control system. This paper presents an enhanced image based visual servoing (IBVS) method for increasing the accuracy of a 6DOF manipulator. The controller is designed to combine proportional derivative (PD) control with sliding mode control (SMC) on a 6DOF manipulator. The properly tuned PD controller can ensure the fast tracking performance and SMC can deal with the external disturbance and uncertainties due to the depth. The enhanced IBVS controller benefits from simple structure and easy implementation of PD control and good robustness to uncertainties of SMC. The stability of the proposed method is proven by using Lyapunov method. Simulation and experimental results are used to demonstrate the effectiveness of the proposed controller.

Finite-time Control for Image-based Visual Servoing of a Quadrotor Using Nonsingular Fast Terminal Sliding Mode

Guanglei Zhao,Guangbo Chen,Jiannan Chen,Chang-chun Hua 제어·로봇·시스템학회 2020 International Journal of Control, Automation, and Vol.18 No.9

This paper investigates finite-time control for image-based visual servoing (IBVS) of a quadrotor subjects to image dynamics uncertainties and external disturbances. The quadrotor model is firstly proposed, based on this model and suppose that sensors on quadrotor only use a monocular camera and an inertial measurement unit, the image dynamics is developed by defining appropriate image features on virtual image plane and by means of perspective projection. The constructed visual model of the quadrotor, which includes 12 states, is decomposed into 6 subsystems in order to design the finite-time controller. Then, nonsingular fast terminal sliding model (NFTSM) finite-time controller is designed, and Lyapunov-based finite-time convergence and disturbance rejection properties are proved. Finally, simulation results are provided to demonstrate the effectiveness of the proposed results.

Image Based Visual Servoing for an Autonomous Quadrotor with Adaptive Backstepping Control

Suseong Kim,Deawon Lee,H. Jin Kim 제어로봇시스템학회 2011 제어로봇시스템학회 국제학술대회 논문집 Vol.2011 No.10

This paper proposes a quadrotor control scheme using image-based visual servo integrated with adaptive backstepping control. Image-based visual servo provides the reference velocities by using image-Jacobian matrix and errors of feature positions on the image plane. Adaptive backstepping control generates input signals for propellers to track the reference velocity accurately even under uncertain effects. The integrated system enables a quadrotor robot to fly to the desired place by minimizing the errors of image feature positions. The performance of the overall system is validated by numerical simulation. Also, to demonstrate the superiority of the proposed system, we compare the performance to that of image-based visual servo integrated with PI control.

영상 정보를 이용한 ROBOKER 팔 위의 역진자 시스템의 지능 밸런싱 제어 구현

김정섭,정슬 한국지능시스템학회 2011 한국지능시스템학회논문지 Vol.21 No.5

This paper presents balancing control of inverted pendulum on the ROBOKER arm using visual information. The angle of the inverted pendulum placed on the robot arm is detected by a stereo camera and the detected angle is used as a feedback and tracking error for the controller. Thus, the overall closed loop forms a visual servoing control task. To improve control performance, neural network is introduced to compensate for uncertainties. The learning algorithm of radial basis function(RBF) network is performed by the digital signal controller which is designed to calculate floating format data and embedded on a field programmable gate array(FPGA) chip. Experimental studies are conducted to confirm the performance of the overall system implementation. 본 논문에서는 영상 정보를 이용하여 로보커 팔위의 역진자의 밸런싱 제어를 한다. 로봇 팔위에 놓인 역진자의 각도는 카메라로 검출하고 검출된 각도 값은 제어기로 귀환되어 오차를 생성한다. 따라서 전체 제어루프는 폐회로 루프를 형성한다. 제어 성능을 높이기 위해 기존 선형제어기에 신경망 제어기를 더하였다. RBF 네트워크의 학습 알고리즘은 FPGA에 설계된 부동소수점 연산이 가능한 디지털 제어기에 의해 수행된다. 실험을 통하여 전체 시스템 성능을 검증하였다.

전방향 모바일 매니퓰레이터의 강인 영상기반 비주얼 서보잉

조강민(Kangmin Jo),좌동경(Dongkyoung Chwa) 대한전기학회 2022 전기학회논문지 Vol.71 No.1

This paper proposes robust image based visual servoing of an omnidirectional mobile manipulator using integral sliding mode control to compensate for the uncertainty that may occur while performing the image-based visual servoing. The stability of the overall omnidirecitonal mobile manipulator system using the proposed method is proven by the Lyapunov stability analysis by showing that the error stays on the sliding surface always and asymptotically converges to zero. In order to solve the singularity problem that can occur when controlling the omnidirectional mobile robot and the manipulator at the same time, a switching technique is proposed in such a way that the omnidirectional mobile robot and manipulator operate sequentially to prevent the abnormal operation of the manipulator and solve the singularity problem. The validity of the proposed robust image based visual servoing method is verified through simulation results.