자연어 처리 기반 멀티 소스 이벤트 로그의 보안 심각도 다중 클래스 분류 = A Multiclass Classification of the Security Severity Level of Multi-source Event Log Based on Natural Language Processing

다국어 초록 (Multilingual Abstract)

Log data has been used as a basis in understanding and deciding the main functions and state of information systems. It has also been used as an important input for the various applications in cybersecurity. It is an essential part to get necessary information from log data, to make a decision with the information, and to take a suitable countermeasure according to the information for protecting and operating systems in stability and reliability, but due to the explosive increase of various types and amounts of log, it is quite challenging to effectively and efficiently deal with the problem using existing tools. Therefore, this study has suggested a multiclass classification of the security severity level of multi-source event log using machine learning based on natural language processing. The experimental results with the training and test samples of 472,972 show that our approach has archived the accuracy of 99.59%.

국문 초록 (Abstract)

로그 데이터는 정보 시스템의 주요 동작과 상태를 이해하고 판단하는 근거로 사용되어 왔으며, 여러 보안 분야 응용에서도 중요한 입력 데이터로 사용된다. 로그 데이터로부터 필요한 정보를 얻어 이를 근거로 의사 결정을 하고, 적절한 대응 방안을 취하는 것은 시스템을 보호하고 안정적으로 운영하는 데 있어 필수적인 요소이지만, 로그의 종류와 양이 폭발적으로 증가함에 따라 기존 도구들로는 효과적이고 효율적인 대응이 쉽지 않은 상황이다. 이에 본 연구에서는 자연어 처리 기반의 머신 러닝을 이용해 멀티 소스 이벤트 로그의 보안 심각도를 여러 단계로 분류하는 방법을 제안하였으며, 472,972건의 훈련 및 테스트 샘플을 이용하여 실험을 수행한 결과 99.59%의 정확도를 달성하였다.

참고문헌 (Reference)

1 Y. Zhang, "Understanding bag-of-words model : A statistical framework" 1 (1): 43-52, 2010

2 J. Zhu, "Tools and benchmarks for automated log parsing" 121-130, 2019

3 M. Landauer, "System log clustering approaches for cyber security applications : A survey" 92 : 101739-101756, 2020

4 K. Erk, "Representing words as regions in vector space" 57-65, 2009

5 Y. Liu, "New machine learning algorithm : Random forest" 246-252, 2012

6 "NLP based log analysis test"

7 J. Raffety, "Multi-source log clustering in distributed systems" 31-41, 2020

8 W. Meng, "LogAnomaly : Unsupervised detection of sequential and quantitative anomalies in unstructured logs" 4739-4745, 2019

9 Z. Chen, "Experience report : Deep learning-based system log analysis for anomaly detection"

10 P. He, "Drain: An online log parsing approach with fixed depth tree" 33-40, 2017

11 M. Du, "Deeplog: Anomaly detection and diagnosis from system logs through deep learning" 1285-1298, 2017

12 T. van Ede, "DEEPCASE : Semi-supervised contextual analysis of security events" 522-539, 2022

13 C. Wan, "Composite feature extraction and selection for text classification" 7 : 35208-35219, 2019

14 R. Yang, "An online log template extraction method based on hierarchical clustering" 2019 (2019): 882-895, 2019

15 Z. Liu, "An integrated method for anomaly detection from massive system logs" 6 : 30602-30611, 2018

16 "AI competition for predicting security risk level through log analysis"

17 S. He, "A survey on automated log analysis for reliability engineering" 54 (54): 1-37, 2021