An Insect Counting and Recognition Method for Greenhouse Insect Pests on Sticky Paper using Convolutional Neural Network

( Lin-ya Chiu ),( Ya-fang Wu ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

Greenhouses are built to promote crop growth, but the same environment also allows pests to reproduce and affect crops. Placing sticky papers in the greenhouse allows monitoring of the number of insect pests in order to predict the breakout of insects and to take appropriate action. Currently, the insect pests on sticky papers are counted and identified by manual inspection which is very time consuming and laborious. In this study, an automatic counting and recognition system for whiteflies and thrips using a convolutional neural network (CNN) approach was developed. The system makes use of a scanner to scan the images of sticky papers with insects and to recognize the objects as whiteflies or thrips. A graphical user interface (GUI) is provided; it was designed using Qt with an OpenCV image processing library. To apply CNN for object detection and recognition, we used the YOLO (You Only Look Once) real-time object detection tool. The sticky papers were collected from several greenhouses and preserved by using cellophane sheets as cover. They were then scanned to obtain high resolution images. Insect image samples were labeled from the scanned sticky paper images to train the CNN object detector model. The object detector model was further optimized in terms of iteration time, detection threshold level, and sample image size. The optimized average recognition accuracies were 86.30% and 90.45% for whitefly and thrip, respectively. This work can be used for automated insect sticky paper checking which is necessary for quarantine purposes.

Applying Hyperspectral and Fluorescence Imaging to Assess Characteristics of Animal and Plant Oil Mixture

( Li-da Chen ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

Milk fat is one of the most important animal oils in the food industry due to its high nutritional value and market cost. However, when affected by adulteration, it leads to economic loss for food manufacturers, and moreover, causing harm to human health. This research demonstrates the application of hyperspectral and fluorescence imaging in assessing the concentration of animal and plant oil mixture. Milk fat samples were adulterated with different plant oil samples at different concentrations. Spectrum analysis was performed using hyperspectral imaging and fluorescence imaging with light source excitation obtained from an Excitation Emission Matrix (EEM). A model was developed using Artificial Neural Networks (ANN) to determine what kind of plant oil is being adulterated, while Support Vector Regression (SVR) was used to determine the actual concentration of the oil mixture. This technique can serve as a rapid detection tool to help speed up the traditional inspection process.

A Versatile Fruit and Vegetable Image Recognition Method based on Deep Convolutional Neural Networks

( Yi-hsuan Huang ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

Due to the increasing labor costs and shortage of labor in the agricultural industry, automation in agriculture has become ever more important. This paper proposes a versatile and automatic fruit and vegetable recognition method through the use of computer vision and deep neural networks. The proposed method allows for detection, recognition, and localization of selected fruits and vegetables via images or video streams. Therefore, the method can be used in various applications in agriculture such as robotic harvest, greenhouse management, or crop phenotyping. To detect fruits or vegetables in images, traditional image processing algorithms have some limitations due to occlusions and background variations. Different fruits or vegetables may require different algorithms. However, deep convolutional neural networks have brought about a breakthrough in dealing with this problem. The significance of deep neural networks in imaging processing is that features are no longer extracted by image processing algorithms. Instead, the network will learn by itself from the input data and extract the important features, called deep features. Therefore, we apply deep convolutional neural networks with You Only Look Once (YOLO), a real-time object detection algorithm, to build a versatile image recognition model for selected fruits and vegetables. Using YOLO, the models are trained with five kinds of fruits and vegetables: apple, tomato, cucumber, orange and strawberry. There are two kinds of models developed: ‘one vs. all’ and ‘one vs. one’ models. These models are compared to obtain the ensemble model. In addition, the effects of different phenotype between training data sets and testing data sets are also evaluated. Finally, the optimized model is applied in the recognition system and multiple kinds of fruits are recognized. We also tested the method with images and video streams acquired from greenhouses to evaluate the performance of the method.

An Evaluation System for Agricultural Ergonomics using Multiple Inertial Measurement Units

( Chi-fang Hsu ),( Po-jung Ho ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

Recently, ergonomics has gained more attention from people in different fields of work. Especially in agricultural workplaces, most tasks cause discomfort or a decline in work efficiency since traditional machines and robotic operations still cannot fully assist farmers. Sustained or frequent trunk flexion may lead to low back disorder, one of the top health-challenging problems in agricultural operations. This study provides a system that permits a real-time ergonomic assessment of manual tasks in agricultural workplaces. Multiple Inertial Measurement Units (IMU) are used to measure the acceleration and angle by using sensors placed at different locations on the upper and lower limbs. Based on this system, the body movement model can be built and visualized in real time. The status of the subject can be evaluated by ergonomic evaluation called Rapid Upper Limb Assessment (RULA). To test the feasibility of the system, it was worn by a subject and the movements of the subject were observed to determine whether or not it was ergonomic; the resulting information may be used to reduce the possibility of bone disease musculoskeletal disorders. This work can be applied not only to agricultural work, but also to healthcare, construction work, and other related work fields.

Study of Pear Orchard Environment Monitoring using Lora Wireless Sensor Network

( Pei-wen Huang ),( Yi-chich Chiu ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

In this study, a system for monitoring orchard environments was developed using long range (LoRa) communication technology, where environmental data were captured and analyzed remotely to establish an orchard environment database and achieve the goals of orchard environmental monitoring and early warnings. Because pear quality is susceptible to the effects of climate change, to prevent them from sustaining damage, an environmental monitoring system was constructed in three orchards in Yilan County’s Sanxing Township, Taiwan. Sensors were used to monitor various environmental data including temperature, humidity, illuminance, soil temperature, soil moisture, and wind speed. An Arduino Uno control interface card was used to collect sensor data, which were transmitted to a LoRa Gateway as well as to the message queuing telemetry transport protocol using LoRa wireless communication modules. Integrate environmental data into an orchard data database for fruit farmers to promptly understand the growth situation of their shangjiang pears, perform effective detection and analyses, and implement a crop cultivation early warning system as well as corresponding contingency plans. This study elevated the stability of outdoor environmental monitoring stations, ensured that equipment such as sensors could collect outdoor environmental data for an extended period of time, and integrated environmental parameters monitored before performing subsequent investigations.

A Real-time Multi-class Insect Pest Identification Method using Cascaded Convolutional Neural Networks

( Dan Jeric Arcega Rustia ),( Chien Erh Lin ),( Jui-yung Chung ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

Insect pest identification is very important for greenhouse management. Having the knowledge of what insects exist in their greenhouse, farmers will be able to determine which pesticide will be more effective to prevent insect pest outbreaks and protect their crops. The most common technique to monitor insect pests is the use of strips of yellow sticky papers. Insects trapped on these yellow sticky papers are usually counted by human inspection without the assistance of any machine or device. To replace this inefficient method, this work presents a multi-class insect identification method for yellow sticky paper, obtained from wireless cameras using cascaded convolutional neural networks (CNN). The designed algorithm makes use of a marker-based image segmentation technique for object detection. The objects are sorted using an insect vs. non-insect filter CNN model to remove non-insect objects such as glare, dirt, and water droplets with 88-95% counting accuracy, while the multi-class insect classifier has an accuracy of 86-92%. The CNN models are optimized based on accuracy and computation time for real-time insect pest monitoring application. The combined algorithm can process each yellow sticky paper image with an average processing time of 13-15 seconds and 2-3 seconds using a quad-core Cortex A53 1.2GHz CPU and GTX1080 2.2GHz GPU, respectively. This work can be applied for real-time and remote insect pest monitoring using wireless camera networks and for observing insect population dynamics of different species.

Application of an image and environmental sensor network for automated greenhouse insect pest monitoring

Dan Jeric Arcega Rustia,Chien Erh Lin,Jui-Yung Chung,Yi-Ji Zhuang,Ju-Chun Hsu,Ta-Te Lin 한국응용곤충학회 2020 Journal of Asia-Pacific Entomology Vol.23 No.1

This work presents an automated insect pest counting and environmental condition monitoring system using integrated camera modules and an embedded system as the sensor node in a wireless sensor network. The sensor node can be used to simultaneously acquire images of sticky paper traps and measure temperature, humidity, and light intensity levels in a greenhouse. An image processing algorithm was applied to automatically detect and count insect pests on an insect sticky trap with 93% average temporal detection accuracy compared with manual counting. The integrated monitoring system was implemented with multiple sensor nodes in a greenhouse and experiments were performed to test the system’s performance. Experimental results show that the automatic counting of the monitoring system is comparable with manual counting, and the insect pest count information can be continuously and effectively recorded. Information on insect pest concentrations were further analyzed temporally and spatially with environmental factors. Analyses of experimental data reveal that the normalized hourly increase in the insect pest count appears to be associated with the change in light intensity, temperature, and relative humidity. With the proposed system, laborious manual counting can be circumvented and timely assessment of insect pest and environmental information can be achieved. The system also offers an efficient tool for long-term insect pest behavior observations, as well as for practical applications in integrated pest management (IPM).

Assessment of Pesticide Effect on Honey Bees Behavior using a Real-time Imaging System

( Thi Nha Ngo ),( Kung-chin Wu ),( En-cheng Yang ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

Honey bees (Apis mellifera) are ecologically and economically important insects. However, massive deaths of honey bees have been reported all around the world. There is much evidence linking the decline of their population to pesticides. However, the impact of lethal doses of pesticide on honey bees is still under debate. Monitoring honey bees passing frequencies at hive entrance is an efficient method to verify the health condition of a beehive. In this study, a real-time imaging system based on a GPU processor for automatic tracking of in-and-out frequencies is presented. The imaging system includes: (1) a dark acrylic box with a transparent pathway. This restricts bees to pass into the image capturing area, (2) an LED light source, (3) a webcam, and (4) an embedded system with GPUs (NVIDIA Jetson TX2) for real-time image acquisition and processing. Background subtraction is applied to remove unnecessary objects from the video. In order to track the in-and-out frequencies of multiple honey bees, and integrated Kalman Filter (KF) and Hungarian algorithm is implemented. KF is used to estimate the object position on each frame. Meanwhile, the Hungarian algorithm is used for the detection of multiple honey bees. Based on the honey bees’ trajectories, a counting algorithm is used to determine their in-and-out activities. The detection algorithms and real-time automatic tracking accuracy rates were evaluated and the system performance was tested with field experiments. The imaging system was further applied to assess the effect of pesticides on honey bee colonies. Five healthy honey bee colonies were treated with different levels of pesticide concentration in contaminated food. The experimental results, which demonstrate the feasibility of our monitoring and tracking system to determine honey bee frequencies, are useful for assessing the pesticide effect on honey bee colonies.

Assessing Pesticide Effects on Honeybee Movement Behavior using an In-hive Imaging System

( Kung-chin Wu ),( Jun-jee Chao ),( Thi Nha Ngo ),( En-cheng Yang ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

Food and cash crops in the world naturally depend on honeybees for delivering pollen. In recent years, the occurrence of the honeybee colony collapse disorder (CCD) caused a large number of honeybee populations to disappear. This phenomenon is causing a significant impact on agricultural production. Therefore, this study aims to monitor the behavior of honeybees and establish effective analysis tools to understand the causes of CCD. Honeybee interaction behavior inside the beehives offers important behavioral information. In order to analyze honeybee behavior, each individual honeybee was affixed with waterproof text labels for observation. Using image processing techniques, such as label recognition and tracking, the movement of the honeybee inside the hive are recorded. By using label tags, the honeybee can be labeled by groups, and the trajectories can be used to classify them into different groups. After obtaining the honeybee trajectories, the states and transformation conditions were determined and used to create a finite state machine (FSM) model. The FSM model was used to analyze the trajectories of the honeybee: it was divided into multiple secondary trajectories by different conditions and state transitions. The model could also be used to transform the trajectories into patterns of behavior and were combined into a sequence of behavioral patterns. Using the data obtained, it was found that in-hive and foraging bees have different trajectory and behavioral patterns. It was also found that the behavioral pattern sequences and trajectories can be used to train models using machine learning and deep learning techniques to classify and recognize different groups of honeybees. Experiments were performed using the imaging system to record and analyze long-term observation of honeybee movement behavior after treatment with pesticide in contaminated food. We further demonstrated this technique in assessing the effect of pesticide on the change of movement behaviors of honeybees.

An Automatic Environmental Monitoring and Cooling System to Reduce Heat Stress in Dairy Cows

( Yu-chi Tsai ),( Chen-yu Cheng ),( Jih-tay Hsu ),( Shih-torng Dnig ),( Ta-te Lin ) 한국농업기계학회 2018 한국농업기계학회 학술발표논문집 Vol.23 No.1

In tropical and subtropical regions with high temperature and humidity conditions, one of the problems in dairy farm management is the effect of heat stress on animal productivity. Due to this problem, the utilization of an automatic environmental monitoring system for cooling, labor-saving and heat stress reduction is necessary. In this study, an environmental monitoring and automatic cooling system based on an embedded system for dairy farms was designed. The system aims to regulate the environmental condition by means of direct feedback depending on the monitoring of heat stress index (HSI) augmented by dairy cow drinking behavior. The system is composed of two sub-systems: data acquisition and environmental control. The data acquisition module collects images, temperature and humidity data. The cooling system is controlled based on the real-time environmental data, and the historical data are recorded and uploaded to the server for display and analysis. The data are analyzed on the environmental monitoring panel on the farm and on the website, which can be remotely accessed. The control parameters are adjusted using the image results from the dairy cow drinking behavior with a fixed time interval. The results of the conducted experiments show that there is labor saving improvement and an increment in milk production compared to the same period in the previous year when the system was not installed. The automatic environmental monitoring cooling system exhibits high efficiency and can perform real-time control and self-adjustment. The system can be used for fully automated dairy cow system application, as well as dairy cow behavior analysis through both environmental and image information.