딥 러닝을 이용한 영상 디블러링을 위한 새로운 U-Net

정우열,김성주,이창우 대한전기학회 2023 전기학회논문지 Vol.72 No.7

Many studies have been conducted for image deblurring, which is classified into non-blind and blind image deblurring techniques. Many iterative methods have been studied based on the maximum-a-posteriori (MAP) framework for image deblurring. Recently, deep learning methods for blind image deblurring have attracted a lot of attention for their excellent performance. In this paper, a method for improving the performance of the blind image deblurring using deep learning is proposed by introducing a new structure of U-Net. U-Net is used as a deep neural network for deep learning in various image processing fields. We propose a new U-Net by using short cut and parallel structure in each stage of contractive and expansive path for U-Net, and pre-processing and post-processing are used for the proposed new U-Net to improve the deblurring performance. Extensive computer simulations are performed to evaluate the image deblurring performance for motion blur and Gaussian blur, and it is shown that the proposed U-Net shows superior image deblurring performance compared to the conventional U-Net.

자율 운항을 위한 U-Net 기반의 deblurring 안정성 연구

박창희,정혜영 한국지능시스템학회 2023 한국지능시스템학회논문지 Vol.33 No.5

항공기 자율 운항(UAV: Unmanned Aerial Vehicle)은 주로 GPS(Global PositioningSystem) 기반의 통신 기술에 의존한다. 악천후와 같은 비상 상황 발생 시 외부 통신 이전에항공기 내에서 기장이 상황을 인지할 수 있다면 더욱 효율적인 대처가 가능할 수 있다. 이미무인 항공기 자율 운항에서는 동적인 외부 상황을 딥러닝 기법인 deblurring에 의해 인지하고 있다. 실제 유인 항공기에서 기내 보조 장치로써 deblurring 기술을 활용하기 위해서는안정성 연구가 필수적이다. 따라서 본 논문은 실제 항공 이미지를 통해 U-Net의 deblurring안정성을 확인하고, 안정성 향상을 위해 blur augmentation을 적용한 새로운 U-Net알고리즘을 제시한다. The autonomous operation of aircraft mainly relies on GPS-based communicationtechnology. In the event of an emergency such as bad weather, more efficientresponse may be possible if the captain can recognize the situation within theaircraft before external communication. Already, in UAV autonomous operation,dynamic external situations are recognized by the deblurring deep learningtechnique. Stability research is essential to use deblurring technology as an in-flightauxiliary device in a manned aircraft. Therefore, this paper confirms the deblurringstability of U-Net through real aerial images, and proposes a novel U-Netalgorithm applying blur augmentation to improve stability.

Faster Deblurring for Digital Images using an Ameliorated Richardson-Lucy Algorithm

Zohair Al-Ameen 대한전자공학회 2018 IEIE Transactions on Smart Processing & Computing Vol.7 No.4

Blur is one popular artifact that degrades digital images due to various unavoidable reallife limitations. Image deblurring is the recovery of an acceptable-quality image from a blurry image. This topic has been a major research focus because of the noticeable upsurge in the use of digital images and imaging systems in many real-world applications. Many intricate and simple algorithms exist for image deblurring. Accordingly, the Richardson–Lucy (RL) algorithm is highly renowned in the field of image deblurring. However, it requires numerous iterations in many situations to produce results with sharp attributes. Hence, an ameliorated RL algorithm is proposed in this article to accelerate the deblurring process by reducing the number of required iterations. The key novelties of the proposed algorithm lie in the addition of a specially designed acceleration factor and in raising part of the algorithm to the power of two. These modifications are achieved experimentally, in which they significantly reduced the number of required iterations to obtain the desired results. Intensive experiments on naturally and synthetically-blurred images reveal that the proposed algorithm performs better than the original and accelerated counterparts in terms of recorded accuracy, perceived quality, and processing speed.

3축 가속도 센서의 흔들림 정보를 이용한 영상의 Deblurring

박상용(Sang-Yong Park),박은수(Eun-Soo Park),김학일(Hakil Kim) 대한전자공학회 2008 電子工學會論文誌-SC (System and control) Vol.45 No.3

본 논문은 모바일 단말기에 탑재된 카메라를 이용하여 정지영상을 획득할 때 발생할 수 있는 blur현상을 3축 가속도 센서를 이용하여 실시간 보정 할 수 있는 방법을 제안한다. Blur현상은 획득한 이미지에서 발생하는 번짐 효과이다. 소형의 모바일 단말기는 사용자의 미세한 손 떨림에도 크게 흔들릴 수 있기 때문에 blur현상이 크게 나타나며, 이를 적절하게 보정할 수 있는 알고리즘이 필요하다. 본 논문에선 3축 가속도센서를 진자운동에 적용하여 출력결과의 신뢰성을 확보하였고, blur현상을 Uniform 분포와 Gaussian 분포로 모델링하였다. 실험을 통하여 실제 blur 현상이 Non-Gaussian 형태로 모델링됨을 확인하였고, 이 blur모델의 역과정인 deblurring 특성함수를 설계하였다. 이 특성함수에 3축 가속도센서에서 발생하는 미세한 떨림 정보를 적용하여 실험 이미지를 deblurring한 결과, 이미지 blur현상을 적절하게 보정할 수 있었다. This paper proposes a real-time method using a 3-axis accelerometer to enhance blurred images taken from a camera loaded in mobile devices. Blurring phenomenon is a smoothing effect occurring in photo images. Algorithms to cope with blurring phenomenon is essential since small-size mobile devices tremble severely by even a tiny hand-shaking of a user. In this paper, accurate sensing characteristics of the 3-axis accelerometer is acquired by applying the sensor in pendulum motion and the blurring phenomenon is modeled as a uniform distribution and Gaussian distribution. Also, non-Gaussian distributed model is observed in the experiment of real blurring phenomenon and a particular deblurring function is designed by reversing the model. It has been demonstrated that the application of trembling information to the deblurring function adequately removes the blurring phenomenon.

PARALLEL PERFORMANCE OF THE G`-PCG METHOD FOR IMAGE DEBLURRING PROBLEMS

윤재헌 한국전산응용수학회 2018 Journal of applied mathematics & informatics Vol.36 No.3

We rst provide how to apply the global preconditioned conju- gate gradient (G`-PCG) method with Kronecker product preconditioners to image deblurring problems with nearly separable point spread functions. We next provide a coarse-grained parallel image deblurring algorithm using the G`-PCG. Lastly, we provide numerical experiments for image deblur- ring problems to evaluate the eectiveness of the G`-PCG with Kronecker product preconditioner by comparing its performance with those of the G`-CG, CGLS and preconditioned CGLS (PCGLS) methods.

1/4 선택 필터를 이용한 번짐 영상의 외곽선 복원

정우진(Woo-Jin Jeong),이종민(Jong-Min Lee),김재영(Chaeyoung Kim),문영식(Young-Shik Moon) 한국컴퓨터정보학회 2015 韓國컴퓨터情報學會論文誌 Vol.20 No.1

본 논문에서는 번짐 영상의 외곽선 복원을 위한 1/4 선택 필터를 제안한다. 일반적인 열화 제거 방법들은 연산량이 많아 수행시간이 오래 걸리는 단점을 가지고 있다. 따라서 속도 향상을 위해서 1/4 선택 필터를 새롭게 제안하고, 1/4 선택 필터를 이용한 번짐 영상의 외곽선 복원 방법을 제안한다. 1/4 선택 필터는 영상의 외곽선을 복원하는 기능이 있으나 세밀한 부분을 잃어버리는 단점이 있다. 이를 보완하기 위하여 영상의 주요 외곽선은 1/4 선택 필터로 복원하고 영상의 세밀한 정보는 DOG(Difference of Gaussian) 필터를 이용하여 복원하는 번짐 현상 제거 방법을 제안한다. 실험 결과를 통해 제안하는 방법이 번짐 영상에서 외곽선을 빠르고 효과적으로 복원함을 확인하였다. In this paper, we propose a deblurring method using 1/4 selective filter. Deblurring methods require a lot of processing time for deblurring. In order to enhance execution speed, we propose a novel 1/4 selective filter. The proposed 1/4 selective filter restores major edge, but it distorts minor edge and texture. To solve this problem, we apply 1/4 selective filter to restore major edge and DOG(Difference of Gaussian) filter to restore minor edge and texture. Experimental results show that the proposed method removes the blur effectively.

A blind-deblurring method based on a compressed-sensing scheme in digital breast tomosynthesis

Kim, K.,Kim, W.,Kang, S.,Park, C.,Lee, D.,Cho, H.,Seo, C.,Lim, H.,Lee, H.,Kim, G.,Park, S.,Park, J.,Jeon, D.,Lim, Y.,Woo, T.,Oh, J. Elsevier 2018 Optics and lasers in engineering Vol.110 No.-

<P><B>Abstract</B></P> <P><B>Background and objective</B></P> <P>Digital breast tomosynthesis (DBT) is a well-established multiplanar imaging modality in breast examinations designed to overcome the limitations of conventional mammography. However, reconstructed DBT images from the acquired projection data are often limited in image performance due mainly to blur artifacts resulting from inherent aspects of imaging systems, including detector resolution and the finite focal spot of the x-ray tube.</P> <P><B>Methods</B></P> <P>We investigated an effective blind-deblurring method based on a compressed-sensing scheme in an attempt to solve the blurring problem in DBT. We implemented the proposed algorithm and performed a systematic simulation and an experiment to demonstrate its viability. In both simulation and experiment, all of the projection data were taken with a tomographic angle of <I>θ</I> = 32° and an angle step of Δ<I>θ</I> = 2°. The proposed deblurring algorithm was then applied to the projection data before performing the common filtered-backprojection-based DBT reconstruction process.</P> <P><B>Results</B></P> <P>The deblurred projection images showed much better image performance compared with the blurred projection images, demonstrating the viability of the proposed blind-deblurring scheme in conventional radiography. The PSNR and RMSE characteristics of the deblurred DBT image improved by factors of approximately 1.63 and 0.37, respectively, compared with those of the blurred DBT image.</P> <P><B>Conclusions</B></P> <P>Our results indicate that the proposed blind-deblurring method was effective in reducing the blurring problem in both DBT and in conventional radiography, excluding additional measurement of the system response function.</P> <P><B>Highlights</B></P> <P> <UL> <LI> It is investigated for an effective blind-deblurring method. </LI> <LI> We perform the systematic simulation and experiment. </LI> <LI> All of the projection data are applied by the proposed deblurring algorithm. </LI> <LI> The blind-deblurring method appears to be effective for the blurring problem. </LI> </UL> </P>

Blind Image Deblurring based on Deep Image Prior

Changwoo Lee,Jinwon Choi 대한전자공학회 2022 IEIE Transactions on Smart Processing & Computing Vol.11 No.2

Many studies on image deblurring have been conducted, and deep learning methods for blind image deblurring have received considerable attention due to their good performance. Recently, the SelfDeblur method was proposed for blind image deblurring based on deep image prior (DIP). In the SelfDeblur method, two neural networks for an image generator and a blur kernel generator are learned simultaneously with only one blurry image. This shows the feasibility of blind image deblurring using unsupervised learning, since it requires no training process. In this paper, we propose a method to maximize the performance of blind image deblurring based on DIP. The optimal loss function for deep learning is studied for the SelfDeblur method, and the deblurring performance of the proposed method is stabilized and maximized using the image prior and the kernel prior for the total loss function. Extensive computer simulations show that the proposed method yields superior performance compared to conventional methods.

HD 해상도에서 실시간 구동이 가능한 딥러닝 기반 블러 제거 알고리즘

심규진(Kyujin Shim),고강욱(Kangwook Ko),윤성준(Sungjoon Yoon),하남구(Namkoo Ha),이민석(Minseok Lee),장현성(Hyunsung Jang),권구용(Kuyong Kwon),김은준(Eunjoon Kim),김창익(Changick Kim) 한국방송·미디어공학회 2022 방송공학회논문지 Vol.27 No.1

Image deblurring aims to remove image blur, which can be generated while shooting the pictures by the movement of objects, camera shake, blurring of focus, and so forth. With the rise in popularity of smartphones, it is common to carry portable digital cameras daily, so image deblurring techniques have become more significant recently. Originally, image deblurring techniques have been studied using traditional optimization techniques. Then with the recent attention on deep learning, deblurring methods based on convolutional neural networks have been actively proposed. However, most of them have been developed while focusing on better performance. Therefore, it is not easy to use in real situations due to the speed of their algorithms. To tackle this problem, we propose a novel deep learning-based deblurring algorithm that can be operated in real-time on HD resolution. In addition, we improved the training and inference process and could increase the performance of our model without any significant effect on the speed and the speed without any significant effect on the performance. As a result, our algorithm achieves real-time performance by processing 33.74 frames per second at 1280x720 resolution. Furthermore, it shows excellent performance compared to its speed with a PSNR of 29.78 and SSIM of 0.9287 with the GoPro dataset.

다중 영상 기반의 고속 처리용 디블러링 기법

손창환,박형민 大韓電子工學會 2012 電子工學會論文誌-SP (Signal processing) Vol.49 No.4

본 논문은 디블러링(Deblurring) 계산 시간을 단축하면서 복원된 영상의 텍스처 및 에지의 선명도를 동시에 강화할 수 있는 다중 영상 기반의 고속 처리용 디블러링 기법을 제안하고자 한다. 먼저 상대적으로 긴 노출 시간에서 촬영된 번짐(Blurring) 결함이 발생한 두 장의 번짐 영상과 짧은 노출에서 촬영된 번짐이 없지만 잡음 성분이 많은 한 장의 잡음 영상을 취득한다. 그리고 처리 속도 개선을 위해 촬영된 다중 입력 영상을 두 배로 다운 샘플링 한 후, 전체 영상에서 추출된 영상 패치 또는 에지 패치에 기반한 점 확산 함수(PSF: Point Spread Function) 추정 기법을 도입해서 점 확산 함수 추정에 소요되는 계산 시간을 효과적으로 단축할 것이다. 입력 영상의 다운 샘플링으로 인해 열하된 미세한 텍스처 성분의 표현 능력을 보완하고 번짐 현상이 제거된 복원 영상을 재현하기 위해 텍스처 향상을 위한 디블러링 기법을 개발 및 적용할 것이다. 마지막으로 입력 영상과 동일한 영상 크기로 복구하기 위해 잡음 영상의 선명한 에지 성분을 활용한 업 샘플링 기법을 적용할 것이다. 제안된 방법을 통해 기존의 디지털 카메라 적용에 걸림돌이 되었던 디블러링 처리 속도 시간을 단축할 수 있었고 동시에 텍스처 및 에지의 미세한 성분도 복원할 수 있었다. This paper presents a fast multiple-image-based deblurring method that decreases the computation loads in the image deblurring, enhancing the sharpness of the textures or edges of the restored images. First, two blurred images with some blurring artifacts and one noisy image including severe noises are consecutively captured under a relatively long and short exposures, respectively. To improve the processing speeds, the captured multiple images are downsampled at the ratio of two, and then a way of estimating the point spread function(PSF) based on the image or edge patches extracted from the whole images, is introduced. The method enables to effectively reduce the computation time taken in the PSF prediction. Next, the texture-enhanced image deblurring method of supplementing the ability of the texture representation degraded by the downsampling of the input images, is developed and then applied. Finally, to get the same image size as the original input images, an upsampling method of utilizing the sharp edges of the captured noisy image is applied. By using the proposed method, the processing times taken in the image deblurring, which is the main obstacle of its application to the digital cameras, can be shortened, while recovering the fine details of the textures or edge components.