NeRF 기반 3차원 포인트 클라우드를 활용한 딸기 잎 자동 분할 파이프라인 개발 = A 3D Leaf Segmentation Pipeline for Strawberry Plants Using NeRF-Based Point Clouds

다국어 초록 (Multilingual Abstract)

This study aimed to construct a three-dimensional (3D) pipeline of strawberry plants and to develop a machine learning–based 3D leaf segmentation framework. Strawberry plants cultivated in a controlled greenhouse at Jeonbuk National University were used as the study target. A 3D leaf segmentation model was developed by applying a Random Forest (RF) classifier and the Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) algorithm, and the results are summarized as follows.
1. Video data were acquired on May 12 and May 21, 2025, to generate a 3D model of strawberry plants. The recorded videos were processed using COLMAP (Structure-from-Motion and Multi-View Stereo) to extract 300 keyframes, from which camera pose information, including position and orientation, was obtained through frame alignment. The extracted keyframes and corresponding pose data were then used as inputs to a Neural Radiance Field (NeRF)– based algorithm to reconstruct a 3D strawberry model.
2. The generated 3D model was scaled using reference points, and background regions excluding the target plants were manually removed. Noise was further reduced by applying filtering techniques.
Instance-level segmentation annotation was then performed using a web-based tool provided by Segments.ai, with four classes defined:
background (0), leaf (1), petiole (2), and runner (3).
3. To reflect the geometric characteristics of strawberry plants, global planarity, local planarity, and verticality were extracted in addition to conventional XYZ spatial coordinates and RGB color information.
These features were combined to construct point-wise feature vectors, which were used together with labeled data to train the RF model for leaf candidate detection. The leaf candidate prediction achieved high performance, with an average F1-score of 0.9879 and a minimum F1-score of 0.9633.
4. The leaf candidate points predicted by the RF model were clustered using the HDBSCAN algorithm to perform instance-level segmentation. The initial segmentation results showed limited performance, with an average mean Intersection over Union (mean IoU) of 0.6766 and a minimum mean IoU of 0.2724.
Under-segmentation, where multiple leaves were merged into a single instance, and over-segmentation, where a single leaf was divided into multiple instances, were observed in some cases. To mitigate these issues, abnormal clusters were detected based on instance size, volume, and density, followed by re-segmentation. In addition, Principal Component Analysis (PCA)–based shape descriptors were used to identify and filter anomalous clusters. After filtering, the average mean IoU increased to 0.7371, while the minimum mean IoU showed only a marginal improvement to 0.2975.
5. Despite the relatively low mean IoU values, the segmentation results were found to provide sufficient information for practical phenotypic trait extraction, demonstrating the feasibility of automated 3D phenotyping. In this study, strawberry leaf segmentation was performed by applying RF and HDBSCAN algorithms to NeRF-based 3D strawberry models. While the leaf candidate detection achieved high accuracy, the segmentation performance remained limited. Nevertheless, the results indicate the potential for extracting phenotypic indicators for subsequent analysis. Future work will focus on improving segmentation performance through the incorporation of supervised learning approaches and integrating the segmentation framework with comprehensive 3D phenotypic analysis models. Such advancements are expected to enable early detection of abnormal plant conditions and the application of phenotype-based control strategies, with further potential expansion to agricultural AI applications such as plant growth monitoring, environmental control, and crop stress analysis.

목차 (Table of Contents)

• Ⅰ. 서 론 1
• Ⅱ. 연구사 4
• 2.1. 현형 분석 기술의 발전과 한계 4
• 2.2. 3D 기반 표현형 분석 연구의 확장 5
• 2.3. 식물 3D segmentation 연구의 흐름 6
• 2.4. 기존 딸기 3D segmentation 연구 7
• Ⅲ. 재료 및 방법 8
• 3.1. 알고리즘 구성 개요 8
• 3.2. 데이터 수집 9
• 3.3. 3D 재구성 12
• 3.4. 포인트 클라우드 전처리 14
• 3.5. Segmentation 학습용 Annotation 구축 16
• 3.6. 3D Segmentation 모델 구축 17
• 3.6.1. 기하학적 특징 추출 17
• 3.6.2. Random Forest 기반 잎 후보군 검출 19
• 3.6.3. HDBSCAN 기반 인스턴스 분할 21
• 3.6.4. 이상 인스턴스 감지 및 재분할 23
• 3.7. 모델 성능 평가 25
• Ⅳ. 결과 및 고찰 28
• 4.1. NeRF 기반 3D 재구성 결과 28
• 4.2. 3D 모델 전처리 결과 30
• 4.3. Segments.ai 기반 Annotation 결과 31
• 4.4. Random Forest 기반 잎 후보군 검출 성능 32
• 4.5. HDBSCAN 기반 Segmentation 결과 33
• 4.6. 이상 인스턴스 판정 및 재segmentation 결과 35
• 4.7. NeRF-Segmentation 파이프라인의 구조적 영향 분석 37
• 4.8. IoU 기반 평가의 실용적 재해석 38
• Ⅴ. 결론 및 요약 39
• 참고문헌 42