RGB 합성영상을 이용한 적조탐지 기법 연구

박수호(Suho Bak),윤홍주(Hong Joo Yoon) 한국생태공학회 2016 한국생태공학회지 Vol.5 No.1

In this study, we propose a method of separating red pixels by producing RGB composite image using GOCI (Geostationary Ocean Color Imager) image of COMS (Communication, Ocean and Meteorological Satellite). The RGB composite images were generated by applying the additive color mixture after representing the computational result of the optical characteristics of the red tide zone in red, green, and blue images. In the case of red tide, all of the computation results of the three formulas made using optical characteristics are used as high pixels, and they are shown in white on the composite image. Comparisons of red tide pixels in RGB composite images and red tides data of the National Fisheries Research and Development Institute showed similar distributions, and one of three expressions tended to be underestimated in turbid sea water such as Yeoja Bay. In order to improve the quality of synthesized images, further study on the optical characteristics of the red tide area is needed, and it is expected that a higher level of detection ability can be secured by improving the formula causing underestimation.

딥러닝 기반 다중 객체 추적 모델을 활용한 조식성 무척추동물 현존량 추정 기법 연구

박수호 ( Suho Bak ),김흥민 ( Heung-min Kim ),이희원 ( Heeone Lee ),한정익 ( Jeong-ik Han ),김탁영 ( Tak-young Kim ),임재영 ( Jae-young Lim ),장선웅 ( Seon Woong Jang ) 대한원격탐사학회 2022 大韓遠隔探査學會誌 Vol.38 No.3

본 연구에서는 딥러닝 기반 다중 객체 추적 모델을 활용하여 수중드론으로 촬영된 영상으로부터 특정 해역의 조식동물 현존량을 추정하는 방법을 제안한다. 수중드론 영상 내에 포함된 조식동물을 클래스 별로 탐지하기 위해 YOLOv5 (You Only Look Once version 5)를 활용하였으며, 개체수 집계를 위해 DeepSORT (Deep Simple Online and real-time tracking)를 활용하였다. GPU 가속기를 활용할 수 있는 워크스테이션 환경에서 두 모델의 성능 평가를 수행하였으며, YOLOv5 모델은 평균 0.9 이상의 모델의 정확도(mean Average Precision, mAP)를 보였으며, YOLOv5s 모델과 DeepSORT 알고리즘을 활용하였을 때, 4 k 해상도 기준 약 59 fps의 속도를 보이는 것을 확인하였다. 실해역 적용 결과 약 28%의 과대 추정하는 경향이 있었으나 객체 탐지 모델만 활용하여 현존량을 추정하는 것과 비교했을 때 오차 수준이 낮은 것을 확인하였다. 초점을 상실한 프레임이 연속해서 발생할 때와 수중드론의 조사 방향이 급격히 전환되는 환경에서의 정확도 향상을 위한 후속 연구가 필요하지만 해당 문제에 대한 개선이 이루어진다면, 추후 조식동물 구제 사업 및 모니터링 분야의 의사결정 지원자료 생산에 활용될 수 있을 것으로 판단된다. In this study, we propose a method to estimate the biomass of invertebrate grazers from the videos with underwater drones by using a multi-object tracking model based on deep learning. In order to detect invertebrate grazers by classes, we used YOLOv5 (You Only Look Once version 5). For biomass estimation we used DeepSORT (Deep Simple Online and real-time tracking). The performance of each model was evaluated on a workstation with a GPU accelerator. YOLOv5 averaged 0.9 or more mean Average Precision (mAP), and we confirmed it shows about 59 fps at 4 k resolution when using YOLOv5s model and DeepSORT algorithm. Applying the proposed method in the field, there was a tendency to be overestimated by about 28%, but it was confirmed that the level of error was low compared to the biomass estimation using object detection model only. A follow-up study is needed to improve the accuracy for the cases where frame images go out of focus continuously or underwater drones turn rapidly. However, should these issues be improved, it can be utilized in the production of decision support data in the field of invertebrate grazers control and monitoring in the future.

수중영상을 이용한 저서성 해양무척추동물의 실시간 객체 탐지: YOLO 모델과 Transformer 모델의 비교평가

박강현 ( Ganghyun Park ),박수호 ( Suho Bak ),장선웅 ( Seonwoong Jang ),공신우 ( Shinwoo Gong ),곽지우 ( Jiwoo Kwak ),이양원 ( Yangwon Lee ) 대한원격탐사학회 2023 大韓遠隔探査學會誌 Vol.39 No.5

저서성 해양무척추동물은 해양 바닥에서 서식하는 무척추동물로서 해양 생태계의 중요한 구성원이지만, 이 중에서 조식동물이나 해적생물이 과도하게 번식하면 연안어장 생태계에 피해를 야기할 수 있다. 본 연구에서는 저서성 해양무척추동물을 대상으로 우리나라 연안에서 수중 촬영한 영상을 활용하여, 실시간 객체 탐지를 위한 딥러닝 모델로 가장 널리 사용되는 You Only Look Once Version 7 (YOLOv7)과 트랜스포머 계열 모델인 detection transformer (DETR)을 비교평가 하였다. YOLOv7이 mean average precision at 0.5 (mAP@0.5) =0.899, DETR이 mAP@0.5=0.862를 기록하였는데, YOLOv7은 멀티스케일(multiscale)로 바운딩 박스(bounding box)를 생성하는 구조이기 때문에 작은 객체를 포함하여 다양한 크기의 객체 탐지에 보다 적합한 것으로 사료된다. 두 모델 모두 30 frames per second (FPS) 이상의 처리속도를 보였기 때문에, 잠수부 촬영영상 및 수중드론 영상에 대한 실시간 객체탐지가 가능할 것으로 기대된다. 이를 통해 조식동물 구제, 바다 사막화 방지를 위한 바다숲 조성 등 연안어장 생태계 피해 방지 및 복원에 활용될 수 있을 것이다. Benthic marine invertebrates, the invertebrates living on the bottom of the ocean, are an essential component of the marine ecosystem, but excessive reproduction of invertebrate grazers or pirate creatures can cause damage to the coastal fishery ecosystem. In this study, we compared and evaluated You Only Look Once Version 7 (YOLOv7), the most widely used deep learning model for real-time object detection, and detection tansformer (DETR), a transformer-based model, using underwater images for benthic marine invertebrates in the coasts of South Korea. YOLOv7 showed a mean average precision at 0.5 (mAP@0.5) of 0.899, and DETR showed an mAP@0.5 of 0.862, which implies that YOLOv7 is more appropriate for object detection of various sizes. This is because YOLOv7 generates the bounding boxes at multiple scales that can help detect small objects. Both models had a processing speed of more than 30 frames per second (FPS), so it is expected that real-time object detection from the images provided by divers and underwater drones will be possible. The proposed method can be used to prevent and restore damage to coastal fisheries ecosystems, such as rescuing invertebrate grazers and creating sea forests to prevent ocean desertification.

Study on Dimensionality Reduction for Sea-level Variations by Using Altimetry Data around the East Asia Coasts

( Do-hyun Hwang ),( Suho Bak ),( Min-ji Jeong ),( Na-kyeong Kim ),( Mi-so Park ),( Bo-ram Kim ),( Hong-joo Yoon ) 대한원격탐사학회 2021 大韓遠隔探査學會誌 Vol.37 No.1

Recently, as data mining and artificial neural network techniques are developed, analyzing large amounts of data is proposed to reduce the dimension of the data. In general, empirical orthogonal function (EOF) used to reduce the dimension in the ocean data and recently, Self-organizing maps (SOM) algorithm have been investigated to apply to the ocean field. In this study, both algorithms used the monthly Sea level anomaly (SLA) data from 1993 to 2018 around the East Asia Coasts. There was dominated by the influence of the Kuroshio Extension and eddy kinetic energy. It was able to find the maximum amount of variance of EOF modes. SOM algorithm summarized the characteristic of spatial distributions and periods in EOF mode 1 and 2. It was useful to find the change of SLA variable through the movement of nodes. Node 1 and 5 appeared in the early 2000s and the early 2010s when the sea level was high. On the other hand, node 2 and 6 appeared in the late 1990s and the late 2000s, when the sea level was relatively low. Therefore, it is considered that the application of the SOM algorithm around the East Asia Coasts is well distinguished. In addition, SOM results processed by SLA data, it is able to apply the other climate data to explain more clearly SLA variation mechanisms.

Abnormal behaviour in rock bream (Oplegnathus fasciatus) detected using deep learning-based image analysis

Jang, Jun-Chul,Kim, Yeo-Reum,Bak, SuHo,Jang, Seon-Woong,Kim, Jong-Myoung The Korean Society of Fisheries and Aquatic Scienc 2022 Fisheries and Aquatic Sciences Vol.25 No.3

Various approaches have been applied to transform aquaculture from a manual, labour-intensive industry to one dependent on automation technologies in the era of the fourth industrial revolution. Technologies associated with the monitoring of physical condition have successfully been applied in most aquafarm facilities; however, real-time biological monitoring systems that can observe fish condition and behaviour are still required. In this study, we used a video recorder placed on top of a fish tank to observe the swimming patterns of rock bream (Oplegnathus fasciatus), first one fish alone and then a group of five fish. Rock bream in the video samples were successfully identified using the you-only-look-once v3 algorithm, which is based on the Darknet-53 convolutional neural network. In addition to recordings of swimming behaviour under normal conditions, the swimming patterns of fish under abnormal conditions were recorded on adding an anaesthetic or lowering the salinity. The abnormal conditions led to changes in the velocity of movement (3.8 ± 0.6 cm/s) involving an initial rapid increase in speed (up to 16.5 ± 3.0 cm/s, upon 2-phenoxyethanol treatment) before the fish stopped moving, as well as changing from swimming upright to dying lying on their sides. Machine learning was applied to datasets consisting of normal or abnormal behaviour patterns, to evaluate the fish behaviour. The proposed algorithm showed a high accuracy (98.1%) in discriminating normal and abnormal rock bream behaviour. We conclude that artificial intelligence-based detection of abnormal behaviour can be applied to develop an automatic bio-management system for use in the aquaculture industry.

HRNet-OCR과 Swin-L 모델을 이용한 조식동물 서식지 수중영상의 의미론적 분할

김형우 ( Hyungwoo Kim ),장선웅 ( Seonwoong Jang ),박수호 ( Suho Bak ),공신우 ( Shinwoo Gong ),곽지우 ( Jiwoo Kwak ),김진수 ( Jinsoo Kim ),이양원 ( Yangwon Lee ) 대한원격탐사학회 2022 大韓遠隔探査學會誌 Vol.38 No.5

이 연구에서는 국내 연안어장을 대상으로 조식동물 및 서식지에 대한 수중영상 기반의 인공지능 학습자료를 구축하고, state-of-the-art (SOTA) 모델인 High Resolution Network-Object Contextual Representation (HRNet-OCR)과 Shifted Windows-L (Swin-L)을 이용하여, 조식동물 서식지 수중영상의 의미론적 분할을 수행함으로써 화소 또는 화소군 간의 공간적 맥락(상관성)을 반영하는 보다 실제적인 탐지 결과를 제시하였다. 조식동물 서식지인 감태, 모자반의 수중영상 레이블 중 1,390장을 셔플링(shuffling)하여 시험평가를 수행한 결과, 한국수산자원공단의DeepLabV3+ 사례에 비해 약 29% 향상된 정확도를 도출하였다. 모든 클래스에 대해 Swin-L이 HRNet-OCR보다 판별율이 더 좋게 나타났으며, 특히 데이터가 적은 감태의 경우, Swin-L이 해당 클래스에 대한 특징을 더 풍부하게 반영할 수 있는 것으로 나타났다. 영상분할 결과 대상물과 배경이 정교하게 분리되는 것을 확인되었는데, 이는 Transformer 계열 백본을 활용하면서 특징 추출능력이 더욱 향상된 것으로 보인다. 향후 10,000장의 레이블 데이터베이스가 완성되면 추가적인 정확도 향상이 가능할 것으로 기대된다. In this paper, we presented a database construction of undersea images for the Habitats of Ecklonia cava and Sargassum and conducted an experiment for semantic segmentation using state-of-the-art (SOTA) models such as High Resolution Network-Object Contextual Representation (HRNet-OCR) and Shifted Windows-L (Swin-L). The result showed that our segmentation models were superior to the existing experiments in terms of the 29% increased mean intersection over union (mIOU). Swin-L model produced better performance for every class. In particular, the information of the Ecklonia cava class that had small data were also appropriately extracted by Swin-L model. Target objects and the backgrounds were well distinguished owing to the Transformer backbone better than the legacy models. A bigger database under construction will ensure more accuracy improvement and can be utilized as deep learning database for undersea images.

위성 및 드론 영상을 이용한 해안쓰레기 모니터링 기법 개발

김흥민,박수호,한정익,예건희,장선웅,Kim, Heung-Min,Bak, Suho,Han, Jeong-ik,Ye, Geon Hui,Jang, Seon Woong 대한원격탐사학회 2022 大韓遠隔探査學會誌 Vol.38 No.6

This study proposes a marine debris monitoring methods using satellite and drone multispectral images. A multi-layer perceptron (MLP) model was applied to detect marine debris using Sentinel-2 satellite image. And for the detection of marine debris using drone multispectral images, performance evaluation and comparison of U-Net, DeepLabv3+ (ResNet50) and DeepLabv3+ (Inceptionv3) among deep learning models were performed (mIoU 0.68). As a result of marine debris detection using satellite image, the F1-Score was 0.97. Marine debris detection using drone multispectral images was performed on vegetative debris and plastics. As a result of detection, when DeepLabv3+ (Inceptionv3) was used, the most model accuracy, mean intersection over union (mIoU), was 0.68. Vegetative debris showed an F1-Score of 0.93 and IoU of 0.86, while plastics showed low performance with an F1-Score of 0.5 and IoU of 0.33. However, the F1-Score of the spectral index applied to generate plastic mask images was 0.81, which was higher than the plastics detection performance of DeepLabv3+ (Inceptionv3), and it was confirmed that plastics monitoring using the spectral index was possible. The marine debris monitoring technique proposed in this study can be used to establish a plan for marine debris collection and treatment as well as to provide quantitative data on marine debris generation.

HRNet 기반 해양침적쓰레기 수중영상의 의미론적 분할

김대선,김진수,장성웅,박수호,공신우,곽지우,배재구,Kim, Daesun,Kim, Jinsoo,Jang, Seonwoong,Bak, Suho,Gong, Shinwoo,Kwak, Jiwoo,Bae, Jaegu 대한원격탐사학회 2022 大韓遠隔探査學會誌 Vol.38 No.6

Destroying the marine environment and marine ecosystem and causing marine accidents, marine debris is generated every year, and among them, submerged marine debris is difficult to identify and collect because it is on the seabed. Therefore, deep-learning-based semantic segmentation was experimented on waste fish nets and waste ropes using underwater images to identify efficient collection and distribution. For segmentation, a high-resolution network (HRNet), a state-of-the-art deep learning technique, was used, and the performance of each optimizer was compared. In the segmentation result fish net, F1 score=(86.46%, 86.20%, 85.29%), IoU=(76.15%, 75.74%, 74.36%), For the rope F1 score=(80.49%, 80.48%, 77.86%), IoU=(67.35%, 67.33%, 63.75%) in the order of adaptive moment estimation (Adam), Momentum, and stochastic gradient descent (SGD). Adam's results were the highest in both fish net and rope. Through the research results, the evaluation of segmentation performance for each optimizer and the possibility of segmentation of marine debris in the latest deep learning technique were confirmed. Accordingly, it is judged that by applying the latest deep learning technique to the identification of submerged marine debris through underwater images, it will be helpful in estimating the distribution of marine sedimentation debris through more accurate and efficient identification than identification through the naked eye.

원격탐사 기법 적용을 통한 대청호 상류 유입 부유쓰레기 조사 및 현존량 추정 연구

김영민,장선웅,김흥민,김탁영,박수호,Youngmin Kim,Seon Woong Jang,Heung-Min Kim,Tak-Young Kim,Suho Bak 대한원격탐사학회 2023 大韓遠隔探査學會誌 Vol.39 No.5

Floating debris in large quantities from land during heavy rainfall has adverse social, economic, and environmental impacts, but the monitoring system for the concentration area and amount is insufficient. In this study, we proposed an efficient monitoring method for floating debris entering the river during heavy rainfall in Daecheong Lake, the largest water supply source in the central region, and applied remote sensing techniques to estimate the standing-stock of floating debris. To investigate the status of floating debris in the upper of Daecheong Lake, we used a tracking buoy equipped with a low-orbit satellite communication terminal to identify the movement route and behavior characteristics, and used a drone to estimate the potential concentration area and standing-stock of floating debris. The location tracking buoys moved rapidly during the period when the cumulative rainfall for 3 days increased by more than 200 to 300 mm. In the case of Hotan Bridge, which showed the longest distance, it moved about 72.8 km for one day, and the maximum moving speed at this time was 5.71 km/h. As a result of calculating the standing-stock of floating debris using a drone after heavy rainfall, it was found to be 658.8 to 9,165.4 tons, with the largest amount occurring in the Seokhori area. In this study, we were able to identify the main concentrations of floating debris by using location-tracking buoys and drones. It is believed that remote sensing-based monitoring methods, which are more mobile and quicker than traditional monitoring methods, can contribute to reducing the cost of collecting and processing large amounts of floating debris that flows in during heavy rain periods in the future.

YOLOv5와 YOLOv7 모델을 이용한 해양침적쓰레기 객체탐지 비교평가

박강현,윤유정,강종구,김근아,최소연,장선웅,박수호,공신우,곽지우,이양원,Park, Ganghyun,Youn, Youjeong,Kang, Jonggu,Kim, Geunah,Choi, Soyeon,Jang, Seonwoong,Bak, Suho,Gong, Shinwoo,Kwak, Jiwoo,Lee, Yangwon 대한원격탐사학회 2022 大韓遠隔探査學會誌 Vol.38 No.6

Deposited Marine Debris(DMD) can negatively affect marine ecosystems, fishery resources, and maritime safety and is mainly detected by sonar sensors, lifting frames, and divers. Considering the limitation of cost and time, recent efforts are being made by integrating underwater images and artificial intelligence (AI). We conducted a comparative study of You Only Look Once Version 5 (YOLOv5) and You Only Look Once Version 7 (YOLOv7) models to detect DMD from underwater images for more accurate and efficient management of DMD. For the detection of the DMD objects such as glass, metal, fish traps, tires, wood, and plastic, the two models showed a performance of over 0.85 in terms of Mean Average Precision (mAP@0.5). A more objective evaluation and an improvement of the models are expected with the construction of an extensive image database.