YOLO와 EfficientNetV2를 사용한 지능적 제품 탐지 및 추천을 위한 End-to-End AI 기반 서비스 모델 개발 = End-to-End AI-Based Solution for Intelligent Product Detection and Recommendation Using YOLO and EfficientNetV2

국문 초록 (Abstract)

최근, 동영상 콘텐츠 소비와 제품구매 행동이 자연스럽게 이어지는 마케팅 연구가 활발하게 이루어지고있다. 본 논문에서는 유튜브의 영상 시청중에 자연스럽게 인터넷 쇼핑으로 이어질 수 있도록 관련 제품을 탐지하고유사 제품을 추천해주는 서비스 기술을 제안하고 구현하였다. 제안한 기술은 YOLOv8과 미리 학습된 EfficientNetV2 를 사용하였고, 제한적으로 탐지할 수 있는 제품 클래스의 개수를 해결하기 위해 CNN모델을 추가함으로써 단일객체 탐지 모델의 한계를 극복하였다. 또한 탐지할 수 있는 제품의 분류 성능을 높이기 위해 Weighted Box Fusion기법을 적용하였다. 테스트 모델에 사용된 제품 검출 성능 평가 결과에서 mAP는 최고 91.2%의 높은 검출률을 보였다. 구현결과, 제안한 기술은 시청중에 판매가 이루어질 수 있는 다양한 동영상 플랫폼의 End-to-End 서비스에 활용될 수 있을 것으로 기대한다.

다국어 초록 (Multilingual Abstract)

Recently, there has been active research in marketing connecting video content consumption to product purchasing behaviors seamlessly. In this paper, we propose and implement a service technology that detects related products during YouTube video viewing and recommends similar products, enabling a smooth transition to online shopping. Our proposed technology utilizes YOLOv8 and a pre-trained EfficientNetV2, and overcomes the limitations of a single object detection model by adding a CNN model to address the issue of a limited number of detectable product classes. Additionally, the Weighted Box Fusion technique is applied to enhance the product classification performance of detectable products. The product detection performance of the test model showed a high mAP of up to 91.2%. Based on our results, we anticipate that the proposed technology can be applied to various video platforms where sales can be made during viewing, offering an end-to-end service.

참고문헌 (Reference)

1 P. Samothai, "The evaluation of bone fracture detection of YOLO series" 1054-1057, 2022

2 A. O. Philip, "Smart standalone edge IoT device for traffic volume counting in smart cities" 1255-1259, 2023

3 E. Hoffer, "Similarity-Based Pattern Recognition" Springer 2015

4 G. Koch, "Siamese neural networks for one-shot image recognition" 2015

5 R. Zhou, "Self-attention mechanism enhanced user interests modeling for personalized recommendation services in cyber-physicalsocial systems" 9 (9): 3835-3846, 2022

6 유영중 ; 문상호 ; 심상진 ; 박성호, "Recognition of license plate number for web camera input using deep learning technique" 4 (4): 565-572, 2020

7 T. Sun, "Personalized learning resource recommendation using differential evolution-based graph neural network: A GraphSAGE approach" 636-639, 2023

8 이성현, "Personal identification method using partial fingerprint image" 6 (6): 2096-2102, 2022

9 L. Fei-Fei, "One-shot learning of object categories" 28 (28): 594-611, 2006

10 L. Lamei, "Mask defect detection algorithm based on improved EfficientNetV2" 507-510, 2022

11 L. Qin, "ID-YOLO : Real-time salient object detection based on the driver’s fixation region" 23 (23): 15898-15908, 2022

12 T. A. N. Mingxing, "Efficientnetv2: Smaller models and faster training" 10096-10106, 2021

13 G. A. Rani, "E-commerce product recommendation system using collaborative filtering" 1521-1525, 2023

14 I. Y. Tyukin, "Demystification of few-shot and one-shot learning" 1-7, 2021

15 W. X. Zhao, "Connecting social media to e-commerce : Cold-start product recommendation using microblogging information" 28 (28): 1147-1159, 2016

16 송다은 ; 전동빈 ; 하태성 ; 이형원 ; 김경이, "Comparison of Korean facial expression classification performance between deep learning based image filters" 6 (6): 767-774, 2022

17 F. Voron., "Building data science applications with FastAPI: Develop, manage, and deploy efficient machine learning applications with Python" Packt Publishing 2023

18 X. Sun, "A defect detection method for a boiler inner wall based on an improved YOLO-v5network and data augmentation technologies" 10 : 93845-93853, 2022