기온 데이터 초해상화를 위한 Super-Resolution Convolutional Neural Network 모델 구축

김용훈,임효혁,하지훈,박건우,김용혁 한국융합학회 2020 한국융합학회논문지 Vol.11 No.8

Meteorology and climate are closely related to human life. By using high-resolution weather data, services that are useful for real-life are available, and the need to produce high-resolution weather data is increasing. We propose a method for super-resolution temperature data using SRCNN. To evaluate the super-resolution temperature data, the temperature for a non-observation point is obtained by using the inverse distance weighting method, and the super-resolution temperature data using interpolation is compared with the super-resolution temperature data using SRCNN. We construct an SRCNN model suitable for super-resolution of temperature data and perform super-resolution of temperature data. As a result, the prediction performance of the super-resolution temperature data using SRCNN was about 10.8% higher than that using interpolation. 기상과 기후는 인간의 생활과 밀접하게 연관되어 있다. 특히 고해상도 기상 데이터를 활용하여 정밀한 연구나 실생활에 유용한 서비스가 가능하므로, 고해상도 기상·기후 데이터를 생산해야할 필요성이 증가하고 있다. 기존의 고해상도 기상 데이터는 적절한 보간법에 따라 데이터를 생산하지만, 본 논문에서는 SRCNN을 이용하여 기온 데이터를 초해상화 하는 방안을 제안한다. 기온 데이터 초해상화에 가장 적절한 SRCNN 모델을 구축하고, 기온 데이터를 초해상화 한다. 결과 데이터를 평가하기 위해 역거리 가중법을 이용하여 비 관측 지점에 대한 기온을 구하고, 제안한 방법을 적용한 기온 데이터와 보간법을 이용한 기온 데이터를 비교한다. 비교 결과, 기온 데이터를 초해상화하기 위한 적절한 SRCNN 모델을 구축하였고, 제안한 방법이 보간법을 이용한 방법보다 약 10.8% 더 높은 예측 성능을 보였다.

움직임영역을 고려한 연속영상 초해상도 영상 복원

조성민(Sung Min Cho),정우진(Woo Jin Jeong),장경현(Kyung Hyun Jang),최병인(Byung In Choi),문영식(Young Shik Moon) 한국컴퓨터정보학회 2017 韓國컴퓨터情報學會論文誌 Vol.22 No.3

In this paper, we propose a consecutive-frame super-resolution method to tackle a moving object problem. The super-resolution is a method restoring a high resolution image from a low resolution image. The super-resolution is classified into two types, briefly, single-frame super-resolution and consecutive-frame super-resolution. Typically, the consecutive-frame super-resolution recovers a better than the single-frame super-resolution, because it use more information from consecutive frames. However, the consecutive-frame super-resolution failed to recover the moving object. Therefore, we proposed an improved method via moving object detection. Experimental results showed that the proposed method restored both the moving object and the background properly.

MAP 추정법과 Huber 함수를 이용한 초고해상도 영상복원

장재용(Jae-Lyong Jang),조효문(Hyo-Moon Cho),조상복(Sang-Bok Cho) 대한전자공학회 2009 電子工學會論文誌-SD (Semiconductor and devices) Vol.46 No.5

1984년 처음 SR 알고리즘이 제안된 이후, 많은 SR 복원 알고리즘이 제안되었다. SR의 접근방법 중에서도 공간적 접근방법은 저해상도 이미지의 픽셀 값을 고해상도 이미지 격자에 매핑 함으로써 이루어진다. 이때, 저해상도 이미지들 간의 각각 다른 노이즈와 다른 PSF(Point Spread Function) 함수, 왜곡으로 인해 매핑 시 문제가 된다. 때문에 저해상도 이미지들의 노이즈 성분을 최소화하는 방법이 필요하다. 본 논문에서는 노이즈 성분을 최소화하는 방법으로 L1 norm의 방법을 사용하고 이와 동시에 이미지의 경계를 보완해주는 Huber norm을 사용하는 SR의 구조를 제안한다. 실험에서는 타 알고리즘과의 비교를 통해 제안한 알고리즘이 저해상도 이미지 상에 존재하는 노이즈를 줄이고 이미지 경계부분의 보완을 확인하였다. Many super-resolution reconstruction algorithms have been proposed since it was the first proposed in 1984. The spatial domain approach of the super-resolution reconstruction methods is accomplished by mapping the low resolution image pixels into the high resolution image pixels. Generally, a super-resolution reconstruction algorithm by using the spatial domain approach has the noise problem because the low resolution images have different noise component, different PSF, and distortion, etc. In this paper, we proposed the new super-resolution reconstruction method that uses the L1 norm to minimize noise source and also uses the Huber norm to preserve edges of image. The proposed algorithm obtained the higher image quality of the result high resolution image comparing with other algorithms by experiment.

Super-resolution Techniques based on Temporal Recursion using Optimal and Real Parameters

S. Kathiravan,J. Kanakaraj 한국산학기술학회 2014 SmartCR Vol.4 No.1

The major goal of a super-resolution image reconstruction method is to construct a single in-depth high-resolution image from a group of numerous low-resolution images of the scene taken from diverse positions. Since every low-resolution image retains a different view of the scene, it is possible to reconstruct an in-depth high-resolution image. Thus image super-resolution is a key to overcoming the material precincts of hardware competence. An enormous amount of video is still in traditional formats or at an even lower resolution. Some also has relentless coding artifacts. Hence, there is a need for techniques that can improve video quality and that show all the traditional and low-resolution videos on panels with high-resolution grids. Iterative super-resolution reconstruction algorithms can accomplish this exigent chore by using internal image models and an incorporated feedback loop to control output quality, thereby improving resolution and lessening artifacts. This article portrays the prospects of iterative reconstruction algorithms and establishes a new super-resolution algorithm that is computationally very strong and efficient against motion estimation errors.

Stage-GAN with Semantic Maps for Large-scale Image Super-resolution

( Zhensong Wei ),( Huihui Bai ),( Yao Zhao ) 한국인터넷정보학회 2019 KSII Transactions on Internet and Information Syst Vol.13 No.8

Recently, the models of deep super-resolution networks can successfully learn the non-linear mapping from the low-resolution inputs to high-resolution outputs. However, for large scaling factors, this approach has difficulties in learning the relation of low-resolution to high-resolution images, which lead to the poor restoration. In this paper, we propose Stage Generative Adversarial Networks (Stage-GAN) with semantic maps for image super-resolution (SR) in large scaling factors. We decompose the task of image super-resolution into a novel semantic map based reconstruction and refinement process. In the initial stage, the semantic maps based on the given low-resolution images can be generated by Stage-0 GAN. In the next stage, the generated semantic maps from Stage-0 and corresponding low-resolution images can be used to yield high-resolution images by Stage-1 GAN. In order to remove the reconstruction artifacts and blurs for high-resolution images, Stage-2 GAN based post-processing module is proposed in the last stage, which can reconstruct high-resolution images with photo-realistic details. Extensive experiments and comparisons with other SR methods demonstrate that our proposed method can restore photo-realistic images with visual improvements. For scale factor ×8, our method performs favorably against other methods in terms of gradients similarity.

대규모 결측 영역에 강인한 Super Resolution 기반 Image Inpainting

이지은,정승원,심종화,황인준 한국정보과학회 2022 정보과학회논문지 Vol.49 No.9

Image Inpainting은 이미지의 누락된 영역을 그럴듯한 이미지로 채우는 기법이다. 최근 딥러닝의 도입으로 인해 복원 성능이 크게 향상되었으나 누락된 영역이 클 경우, 복잡한 장면을 담고 있는 경우, 그리고 고해상도일 경우에는 부자연스러운 복원 결과를 얻는다. 본 논문에서는 고해상도 이미지보다 저해상도 이미지에서 복원이 더 잘 된다는 점을 활용하여 Super Resolution 기반의 2단계 Image Inpainting 기법을 제안한다. 첫 번째 단계에서 고해상도 이미지를 저해상도로 변환하여 복원을 수행하고, 두 번째 단계에서 Super Resolution 모델을 통해 원래의 고해상도로 복원한다. 제안하는 기법의 효과를 검증하기 위해 고해상도의 Urban100 데이터셋을 사용하여 정량 및 정성 평가를 수행하였다. 또한, 누락된 영역의 크기에 따른 복원 성능을 분석하고, 제안하는 기법이 자유로운 형태의 마스크에서 만족할 만한 복원 결과를 생성할 수 있음을 입증하였다. Image inpainting is a method of filling missing regions of an image with plausible imagery. Even though the performance of recent inpainting methods has been significantly improved owing to the introduction of deep learning, unnatural results can be obtained when an input image has a large-scale missing region, contains a complex scene, or is a high-resolution image. In this study, we propose a super resolution-based two-stage image inpainting method, motivated by the point that inpainting performance in low-resolution images is better than in high-resolution images. In the first step, we convert a high-resolution image into a low-resolution image and then perform image inpainting, which results in the initial output image. In the next step, the initial output image becomes the final output image, with the same resolution as the original input image using the super resolution model. To verify the effectiveness of the proposed method, we conducted quantitative and qualitative evaluations using the high-resolution Urban100 dataset. Furthermore, we analyzed the inpainting performance depending on the size of the missing region and demonstrated that the proposed method could generate satisfactory results in a free-form mask.

딥러닝의 초해상화 기술 리뷰

Astha Adhikari,Sang-Woong Lee 한국차세대컴퓨팅학회 2022 한국차세대컴퓨팅학회 학술대회 Vol.2022 No.05

Photo-realistic high-resolution image reconstruction from its counterpart low-resolution image is still a long, challenging task in computer vision. Estimating a High Resolution (HR) image from a single Low Resolution (LR) is referred to as Super-Resolution (SR). When deep learning with a learning and noise immunity capability has been applied, the LR images are generally obtained by down-sampling HR images with additional noise and blur. Single Image Super-Resolution (SISR) is more respected and better in efficiency. Based on models and architectures, deep learning super-resolution techniques can be categorized into Convolutional Neural Networks (CNNs), Adversarial Networks, and Transformer models. This paper summarizes the development history and characteristics of the state-of-the-art papers, discusses the trend and challenges, and provides a compact review of current development and deep learning-based super resolution trends.

광학 위성 영상 기반 선박탐지의 정확도 개선을 위한 딥러닝 초해상화 기술의 영향 분석

박성욱,김영호,김민식,Park, Seongwook,Kim, Yeongho,Kim, Minsik 대한원격탐사학회 2022 大韓遠隔探査學會誌 Vol.38 No.5

광학 위성 영상의 공간해상도가 낮게 되면 크기가 작은 객체들의 경우 객체 탐지의 어려움이 따른다. 따라서 본 연구에서는 위성 영상의 공간해상도를 향상시키는 초해상화(Super-resolution) 기술이 객체 탐지 정확도 향상에 대한 영향이 유의미한지 알아보고자 하였다. 쌍을 이루지 않는(unpaired) 초해상화 알고리즘을 이용하여 Sentinel-2 영상의 공간해상도를 3.2 m로 향상시켰으며, 객체 탐지 모델인 Faster-RCNN, RetinaNet, FCOS, S²ANet을 활용하여 초해상화 적용 유무에 따른 선박 탐지 정확도 변화를 확인했다. 그 결과 선박 탐지 모델의 성능 평가에서 초해상화가 적용된 영상으로 학습된 선박 탐지 모델들에서 Average Precision (AP)가 최소 12.3%, 최대 33.3% 향상됨을 확인하였고, 초해상화가 적용되지 않은 모델에 비해 미탐지 및 과탐지가 줄어듦을 보였다. 이는 초해상화 기술이 객체 탐지에서 중요한 전처리 단계가 될 수 있다는 것을 의미하고, 객체 탐지와 더불어 영상 기반의 다른 딥러닝 기술의 정확도 향상에도 크게 기여할 수 있을 것으로 기대된다. When a satellite image has low spatial resolution, it is difficult to detect small objects. In this research, we aim to check the effect of super resolution on object detection. Super resolution is a software method that increases the resolution of an image. Unpaired super resolution network is used to improve Sentinel-2's spatial resolution from 10 m to 3.2 m. Faster-RCNN, RetinaNet, FCOS, and S²ANet were used to detect vessels in the Sentinel-2 images. We experimented the change in vessel detection performance when super resolution is applied. As a result, the Average Precision (AP) improved by at least 12.3% and up to 33.3% in the ship detection models trained with the super-resolution image. False positive and false negative cases also decreased. This implies that super resolution can be an important pre-processing step in object detection, and it is expected to greatly contribute to improving the accuracy of other image-based deep learning technologies along with object detection.

CNN을 이용한 Quad Tree 기반 2D Smoke Super-resolution

홍병선,박지혁,최명진,김창헌 (사)한국컴퓨터그래픽스학회 2019 컴퓨터그래픽스학회논문지 Vol.25 No.3

Physically-based fluid simulation takes a lot of time for high resolution. To solve this problem, there are studies that make up the limitation of low resolution fluid simulation by using deep running. Among them, Super-re solution, which converts low-resolution simulation data to high resolution is under way. However, traditional techniques require to the entire space where there are no density data, so there are problems that are inefficient in terms of the full simulation speed and that cannot be computed with the lack of GPU memory as input resolution increases. In this paper, we propose a new method that divides and classifies 2D smoke simulation data into the space using the quad tree, one of the spatial partitioning methods, and performs Super-resolution only required space. This technique accelerates the simulation speed by computing only necessary space. It also processes the divided input data, which can solve GPU memory problems. 물리 기반 유체 시뮬레이션은 고해상도 연산을 위해 많은 시간이 필요하다. 이 문제를 해결하기 위해 저해상도 유체 시뮬레이션의 한계를 딤 러닝으로 보완하는 연구들이 있으며, 그중에서는 저해상도의 시뮬레이션 데이터 를 고해상도로 변환해주는 Super-resolution 분야가 있다. 하지만 기존 기법들은 전체 데이터 공간에서 밀도 데이터가 없는 부분까지 연산하므로 전체 시뮬레이션 속도 면에서 효율성이 떨어지며, 입력 해상도가 큰 경우 에는 GPU 메모리가 부족해 연산할 수 없는 경우가 발생할 수 있다. 본 연구에서는 공간 분할 법 중 하나인 쿼 드 트리를 활용하여 시뮬레이션 공간을 분할 및 분류하여 Super-resolution 하는 기법을 제안한다. 본 기법은 필요 공간만 Super-resolution 하므로 전체 시뮬레이션 가속화가 가능하고, 입력 데이터를 분할 연산하므로 GPU 메모리 문제를 해결할 수 있게 된다.

Investigation of the super-resolution methods for vision based structural measurement

Zhi Cong Chen,Lijun Wu,Zhouwei Cai,Chenghao Lin,Shuying Cheng,Peijie Lin 국제구조공학회 2022 Smart Structures and Systems, An International Jou Vol.30 No.3

The machine-vision based structural displacement measurement methods are widely used due to its flexible deployment and non-contact measurement characteristics. The accuracy of vision measurement is directly related to the image resolution. In the field of computer vision, super-resolution reconstruction is an emerging method to improve image resolution. Particularly, the deep-learning based image super-resolution methods have shown great potential for improving image resolution and thus the machine-vision based measurement. In this article, we firstly review the latest progress of several deep learning based super-resolution models, together with the public benchmark datasets and the performance evaluation index. Secondly, we construct a binocular visual measurement platform to measure the distances of the adjacent corners on a chessboard that is universally used as a target when measuring the structure displacement via machine-vision based approaches. And then, several typical deep learning based super resolution algorithms are employed to improve the visual measurement performance. Experimental results show that super-resolution reconstruction technology can improve the accuracy of distance measurement of adjacent corners. According to the experimental results, one can find that the measurement accuracy improvement of the super resolution algorithms is not consistent with the existing quantitative performance evaluation index. Lastly, the current challenges and future trends of super resolution algorithms for visual measurement applications are pointed out.