Bi-LSTM과 Attention Mechanism을 결합한 감성 분석 = Sentiment analysis combined Bi-LSTM and AttentionMechanism

국문 초록 (Abstract)

인터넷 기술의 급속한 발전으로 많은 인터넷 사용자들이 소셜 네트워크 서비스를 통해 사용자 리뷰 등과 같은 감성적인 의견을 많이 포함하는 데이터를 생성하고 있다. 데이터로부터 사용자의 감성 상태를 분석하는 것이 연구의 주요 이슈가 되고 있다. 감성 분석은 일반적으로 한 단락의 텍스트에 표현된 감성 상태를 분석하는 것을 말한다. 본 논문에서는 자연어 처리 기술을 적용해 주관적인 감성 정보를 인식하고 감성을 예측한다.
먼저, 현재의 감성 분석 기술에 대한 조사 연구를 통해, 텍스트로부터 잠재적인 감성 정보를 발굴하는 방법을 연구하고, TextCNN, Bi-LSTM등의 모델을 사용하여 텍스트 데이터 감정 분류의 정확성을 향상시킨다. TextCNN과 Bi-LSTM을 각각 Attention을 결합하여 모델을 구축한다. 실험을 통하여 WeiBo 리뷰에 대한 예측 분석을 실시하고, 모든 결과를 교차 검증하여 모델의 예측 결과의 정확도를 높인다. 마지막으로, WeiBo 리뷰 데이터를 사용하여 해당 모델의 성능을 비교하고 모델의 유효성을 검증한다. 본고에서는 Bi-LSTM에 Attention을 가입함으로써 Bi-LSTM 모델의 정확도를 높이고 정확도는 96.8%를 달성하였으며, 테스트 손실률은 8%로 감소한다.

다국어 초록 (Multilingual Abstract)

With the rapid development of Internet technology, many Internet users are generating data containing a large amount of emotional opinions, such as user comments, through social networking services. Analyzing the emotional state of users from the data has become a major research problem. Sentiment analysis generally refers to analyzing the emotional state expressed by a text. In this paper, natural language processing techniques are applied to identify subjective sentiment information and predict sentiment.
First, through research on the current sentiment analysis technology, a method of excavating potential emotional information from text is studied, and the accuracy of text data emotion classification is improved using models such as TextCNN and Bi-LSTM. TextCNN and Bi-LSTM are each combined with attention to build a model. Through experiments, predictive analysis is performed on WeiBo reviews, and all results are cross-validated to increase the accuracy of the model's prediction results. Finally, the WeiBo review data is used to compare the performance of the models and validate the models. In this paper, by subscribing attention to Bi-LSTM, the accuracy of the Bi-LSTM model was increased, and the accuracy was achieved to 96.8%, and the test loss rate was reduced to 8%.

목차 (Table of Contents)

• CONTENTS
• LIST OF TABLES ⅳ
• LIST OF FIGURES ⅴ
• 1. Introduction 1
• 1.1 Background and Significance 1
• 1.2 Research status and development trend 3
• 1.2.1 Sentiment analysis based on sentiment dictionary 3
• 1.2.2 Sentiment analysis based on traditional machine learning 6
• 1.2.3 Sentiment analysis based on deep learning 9
• 1.3 Purpose and significance of the study 14
• 2. Related Theory and Methods 16
• 2.1 Word vectors 16
• 2.1.1 Static word vector approach 17
• 2.1.2 Dynamic word vector approach 20
• 2.2 Deep neural network related techniques 21
• 2.2.1 Convolutional neural network 21
• 2.2.2 Recurrent Neural Network 22
• 2.2.3 Attentional mechanisms 24
• 3. Sentiment Analysis Methods 28
• 3.1 TextCNN 28
• 3.2 TextCNN-Attention 30
• 3.3 LSTM 31
• 3.4 Bi-LSTM-Attention 33
• 4. Experimental Results and Evaluations 36
• 4.1 Model design flow chart 36
• 4.2 Data sets and evaluation metrics 37
• 4.2.1 Introduction to the data set 37
• 4.2.2 Introduction of evaluation indicators 38
• 4.3 Model Training 40
• 4.3.1 Data processing 40
• 4.3.2 Text pre-processing 41
• 4.3.3 Word vectors 41
• 4.3.4 Model building 42
• 4.3.5 Training of the model 42
• 4.4 Experimental results and Evaluations 43
• Conclusion 51
• References 52
• Abstract 55