강화학습을 활용한 연안선망어업 투망 제어 연구

다국어 초록 (Multilingual Abstract)

Purse seine fishing is a key sector in Korea's commercial fisheries; however, the industry faces structural sustainability challenges, including a declining number of vessels and an aging workforce. Effective purse seine operations require the accurate prediction of fish school behavior while accounting for complex environmental factors such as wind, currents, and waves. Traditionally, these decisions have relied heavily on the tacit knowledge of experienced captains. As this expertise is at risk of being lost, there is a critical need for automated decision support systems based on artificial intelligence. This study presents a hybrid reinforcement learning framework for automated trajectory planning in purse seine operations. A high-fidelity virtual simulation environment was developed, incorporating vessel dynamics via the Nomoto first-order steering model, fish school behavior patterns, net deployment mechanics, and dynamic oceanographic conditions. The simulation achieved 97.8% trajectory accuracy in circular patterns and maintained 96.8% circularity under combined current, wave, and wind conditions. Initially, three pure reinforcement learning algorithms, namely Q-Learning, Double DQN, and PPO, were evaluated. However, all approaches achieved 0% success rates after 10,000 episodes due to the sparse reward problem inherent in purse seine operations, where positive rewards require successful completion of extended sequential actions from search to encirclement. To address this, a rule-based control system was integrated to guide initial exploration and provide demonstration data. A curriculum learning strategy was adopted to ensure stable convergence across increasing levels of difficulty. Experiments were conducted in two environment scales: 1,000 m × 1,000 m and 300 m × 300 m, the latter corresponding to actual sonar detection range. In the 300 m environment, the hybrid PPO model achieved an average success rate of 66.7% with all random seeds exceeding 50%, demonstrating consistent performance regardless of initialization conditions. The hybrid Double DQN model achieved 35.3% average success rate but exhibited high initialization sensitivity with performance ranging from 0% to 96%. PPO showed superior stability with progressive improvement through curriculum stages from 45.5% to 75.6%, while Double DQN displayed policy collapse patterns in later training phases. This research validates the feasibility of employing hybrid reinforcement learning for decision automation in purse seine fishing and demonstrates that rule-based guidance is essential for overcoming sparse reward challenges in complex maritime operations. The findings provide a technological foundation for preserving expert maritime knowledge and supporting sustainable fisheries management.

목차 (Table of Contents)

• I. 서 론 1
• Ⅱ. 선망어업 가상환경 구현 4
• 1. 서론 4
• 2. 재료 및 방법 6
• 2.1. 어선 거동 모델 설계 6
• 2.2. 어구 거동 모델 설계 8
• 2.3. 어군 행동 및 어군 감지 모델 설계 11
• 2.4. 해양환경 모델 설계 13
• 2.5. 상태·행동 공간 정의 15
• 3. 결과 및 고찰 17
• 3.1. 어선 거동 구현 17
• 3.2. 어구 거동 구현 19
• 3.3. 어군 행동 및 어군 탐지 시스템 구현 21
• 3.4. 해양환경 조건에 따른 가상환경 구현 23
• 4. 결론 28
• Ⅲ. 강화학습 기반 선망어업 투망 모델 개발 29
• 1. 서론 29
• 2. 재료 및 방법 31
• 2.1. 학습환경 설정 31
• 2.2. Q-Learning 알고리즘 적용 투망 모델 설계 37
• 2.3. Double DQN 알고리즘 적용 투망 모델 설계 43
• 2.4. PPO 알고리즘 적용 투망 모델 설계 47
• 3. 결과 및 고찰 53
• 3.1. Q-Learning 알고리즘 기반 투망 제어 결과 분석 53
• 3.2. Double DQN 알고리즘 기반 투망 제어 결과 분석 58
• 3.3. PPO 알고리즘 기반 투망 제어 결과 분석 62
• 4. 결론 69
• Ⅳ. 규칙 기반 강화학습을 활용한 선망어업 투망 시스템 개발 70
• 1. 서론 70
• 2. 재료 및 방법 72
• 2.1. 규칙 기반 투망 시스템 설계 73
• 2.2. 강화학습과 규칙 기반 통합 시스템 설계 82
• 3. 결과 및 고찰 87
• 3.1. Double DQN 기반 통합 제어 결과 분석 87
• 3.2. PPO 기반 통합 제어 결과 분석 96
• 4. 결론 106
• V. 종합 고찰 107
• 5.1 연구 결과에 대한 종합적 고찰 107
• 5.2 연구의 한계 및 향후 연구 방향 110