SAFETY RISK ASSESSMENT IN AVIATION USING BIG DATA, NATURAL LANGUAGE PROCESSING (NLP) AND LARGE LANGUAGE MODELS (LLM)

다국어 초록 (Multilingual Abstract)

The aviation industry is facing rapid growth, making safety risk assessment more critical than ever. Traditional Safety Management Systems (SMS), though foundational, struggle with the increasing volume and complexity of safety data. This research explores how Big Data analytics, Natural Language Processing (NLP), and Large Language Models (LLMs) can improve the automation and reliability of safety risk classification. Using the U.S. Aviation Safety Reporting System (ASRS) dataset, which contains narrative reports of incidents, two classification tasks were carried out: single-label classification of primary problems and multi-label classification of contributing factors. DistilBERT, a transformer-based model, was fine-tuned for both tasks, with baseline performance compared against a probability thresholding approach to better handle class imbalance. The findings demonstrate that threshold-adjusted classification improved precision and F1-scores for dominant categories such as Human Factors and Aircraft, while maintaining balanced performance across smaller classes. In the multi-label task, the model achieved a micro-F1 of 0.82 and a weighted-F1 of 0.82, highlighting its capacity to identify multiple contributing factors within a single report. These results confirm that NLP and LLM-based methods can streamline aviation safety risk assessment, reduce reliance on manual analysis, and provide timely insights for regulators and operators. The study offers practical evidence that advanced machine learning tools can support data-driven decision-making and strengthen proactive safety management in aviation.

Keywords: Aviation Safety - Risk Assessment - Natural Language Processing - Large Language Models - Transformer Models (DistilBERT - Multi-label Classification).

목차 (Table of Contents)

• I. INTRODUCTION 1
• 1.1 Background of Aviation Safety Risk Assessment 1
• 1.2 Importance of Automation in Safety Analysis 2
• 1.3 Research Problem 2
• 1.4 Research Questions 2
• 1.5 Objectives 3
• II. LITERATURE REVIEW 4
• 2.1. Predictive Aircraft Maintenance 4
• 2.2. Air Traffic Management and Optimization 5
• 2.3. Passengers’ Experience and Customization 6
• 2.4. Fuel Efficiency and Environmental Impact Reduction 7
• 2.5. Aviation Safety 8
• 2.6. Aviation Data and Safety Reporting Systems 9
• 2.6.1. European Coordination Centre for Accident and Incident Reporting Systems (ECCAIRS & 2) 9
• 2.6.2. The Aviation Safety Reporting System (ASRS) 10
• 2.7. Safety and Risk Management 14
• 2.8. Current Gaps in using NLP in Safety and Risk Assessment 16
• III. THEORETICAL BACKGROUND 19
• 3.1. Safety Management System (SMS) in Aviation. 19
• 3.1.1 Reactive Safety Management Approach 21
• 3.1.2 Proactive Safety Management Approach 22
• 3.1.3 Predictive Safety Management Approach. 22
• 3.2 Big Data, Natural Language Processing, Machine Learning and Large Language Models 22
• 3.2.1 Big Data 22
• 3.2.2 Machine Learning 24
• 3.2.3 Natural Language Processing (NLP) 30
• 3.2.4 Large Language Models 34
• 3.2.4.1 Evolution from Traditional NLP to Transformer-Based Architectures 35
• 3.2.4.2 Transformer Architecture Overview 35
• 3.2.4.3 Theories of Attention Mechanism 38
• 3.2.4.4 BERT and its Role in Transformer Architectures 42
• IV. RESEARCH MODEL 45
• V. RESEARCH DESIGN AND METHODOLOGY 51
• 1. Data Source and Description 51
• 2. Data Preprocessing and Preparation 53
• 2.1 Data Processing for Primary Problem Classification 56
• 2.2 Data Processing for Contributing Factors Classification 58
• 3. Model Selection 59
• 3.1 Model Selection for Primary Problem (Single-Label Classification) 60
• 3.2 Model Selection for Contributing Factors (Multi-Label Classification) 61
• 4. Training Procedure 63
• 4.1 Primary Problem Classification (Single-Label Task) 63
• 4.1.1 Baseline Approach 63
• 4.1.2 Probability-Threshold Adjustment Approach 64
• 4.2 Contributing Factors Classification (Multi-Label Task) 65
• 5. Evaluation Metrics 66
• 6. Tools Used 68
• 6.1 Programming Language 68
• 6.2 Machine Learning and NLP Frameworks 68
• 6.3 Development Environment 69
• 6.4 Hardware 69
• VI. RESULT AND DISCUSSION 70
• 1. Exploratory Data Analysis. 70
• 2. Baseline Performance for Primary Factors 74
• 3. Threshold-Adjusted Results for Primary Factors 77
• 4. Comparison of Baseline and Adjusted Threshold Performance 79
• 5. Performance for Contributing Factors 81
• VII. CONCLUSION AND RECOMMENDATIONS. 85
• 1. Summary of Findings 85
• 2. Answers to Research Questions 86
• 3. Implications 87
• 4. Recommendations 89
• 5. Challenges and Limitations 90
• 6. Future Studies 92
• REFERENCES 95
• APPENDIX: PYTHON CODES SNIPPET 102
• Appendix A: Data Cleaning Preparation 102
• Appendix B: Code for Primary Factor Classification 104
• Appendix C: Code for Contributing Factors Classification 108