A Versatile Fruit and Vegetable Image Recognition Method based on Deep Convolutional Neural Networks = A Versatile Fruit and Vegetable Image Recognition Method based on Deep Convolutional Neural Networks

다국어 초록 (Multilingual Abstract)

Due to the increasing labor costs and shortage of labor in the agricultural industry, automation in agriculture has become ever more important. This paper proposes a versatile and automatic fruit and vegetable recognition method through the use of computer vision and deep neural networks. The proposed method allows for detection, recognition, and localization of selected fruits and vegetables via images or video streams. Therefore, the method can be used in various applications in agriculture such as robotic harvest, greenhouse management, or crop phenotyping. To detect fruits or vegetables in images, traditional image processing algorithms have some limitations due to occlusions and background variations. Different fruits or vegetables may require different algorithms. However, deep convolutional neural networks have brought about a breakthrough in dealing with this problem. The significance of deep neural networks in imaging processing is that features are no longer extracted by image processing algorithms. Instead, the network will learn by itself from the input data and extract the important features, called deep features. Therefore, we apply deep convolutional neural networks with You Only Look Once (YOLO), a real-time object detection algorithm, to build a versatile image recognition model for selected fruits and vegetables. Using YOLO, the models are trained with five kinds of fruits and vegetables: apple, tomato, cucumber, orange and strawberry. There are two kinds of models developed: ‘one vs. all’ and ‘one vs. one’ models. These models are compared to obtain the ensemble model. In addition, the effects of different phenotype between training data sets and testing data sets are also evaluated. Finally, the optimized model is applied in the recognition system and multiple kinds of fruits are recognized. We also tested the method with images and video streams acquired from greenhouses to evaluate the performance of the method.