한국어 형태소 분석을 위한 3단계 확률 모델 = Three-Step Probabilistic Model for Korean Morphological Analysis

국문 초록 (Abstract)

확률 모델을 기반으로 만들어진 형태소 분석기는 형태소 품사 부착 말뭉치의 다양한 언어 현상과 태깅 원칙을 바로 학습할 수 있으므로 다양한 분야에 대한 적응력이 높다. 본 논문에서는 한국어 형태소 분석을 위한 3단계 확률 모델을 제안한다. 이 모델은 분석 단계를 형태소 복원, 분리, 태깅의 3단계로 나누어 독립된 모듈로 처리함으로써 기존의 2단계 확률 모델보다 처리 복잡도를 줄였다. 또한, 음절 대신 자소 단위의 처리를 하고, 형태소 전이 확률을 이용하여 형태소 분리를 함으로써 다양한 품사 태깅 원칙을 학습할 수 있도록 했다. 모델의 성능 평가는 세종 계획 프로젝트에서 개발한 문어체 및 구어체 형태소 부착 말뭉치에 대해 실험하였고 기존의 방법들과 비교하였다.

다국어 초록 (Multilingual Abstract)

A morphological analyzer based on probabilistic model can learn easily various language phenomena and tagging principles used in morpheme-tagged corpus, so that it is very portable to various domains. In this paper, we propose a three-step probabilistic model for Korean morphological analysis which consists of original form restoring step, morpheme segmentation step and morpheme tagging step. The three-step method, which uses modular approach, reduces processing complexity compared with two-step probabilistic model. Processing in Jaso unit rather than syllable unit and using morpheme transition probability for morpheme segmentation increase portability for various tagging principles. Experiment on Sejong tagged corpus, both of written text corpus and spoken text corpus, was done to show the performance of the model and compare it with other methods.

목차 (Table of Contents)

• 요약
• Abstract
• 1. 서론
• 2. 관련 연구
• 3. 3단계 확률 모델
• 4. 실험 및 평가
• 5. 결론
• 참고문헌

참고문헌 (Reference)

1 이재성, "효과적인 외래어 이형태 생성을 위한 확률 문맥 의존 치환 방법" 한국콘텐츠학회 7 (7): 73-83, 2007

2 P. F. Brown, "Using Large Corpora" The MIT Press 1994

3 A. V. Aho, "The Theory of Parsing, Translation and Compiling" Prentice-Hall 1972

4 P. F. Brown, "The Mathematics of Statistical Machine Translation: Parameter Estimation" 19 (19): 263-311, 1993

5 R. A. Sharman, "Syllable-based Phonetic Trans cription by Maximum Likelihood Methods"

6 F. Jelinek, "Statistical Methods for Speech Recognition" The MIT Press 1997

7 E. Charniak, "Statistical Language Learning" The MIT Press 1993

8 D. Lee, "Probabilistic Models for Korean Morphological Analysis and Part-of-Speech Tagging" Korea Univ 2005

9 D. Lee, "Probabilistic Modeling of Korean Morphology"

10 S. Kang, "Multi-level Morphology and Morphological Analysis Model for Korean" 140-145, 1994

11 J. S. Lee, "Morphological Analyzer and Tagger Evaluation Contest(MATEC 99) Overview" 13-22, 1999

12 O. Kwon, "Korean Morphological Analyzer and Part- Of-Speech Tagger Based on CYK Algorithm using Syllable Information" 76-88, 1999

13 S. Kang, "Korean Morphological Analysis using Syllable Information and Multi-word Unit Information" Seoul National University 1993

14 J. Kim, "Korean Morphological Analysis using Inflected-Word-Dictionary" 813-816, 1994

15 S. S. Kang, "Korean Morphological Analysis and Information Retrieval" Hongreung Science Publisher 2002

16 C. Manning, "Foundations of Statistical Natural Language Processing" The MIT Press 1999

17 U. Germann, "Fast Decoding and Optimal Decoding for Machine Translation" 228-235, 2001

18 J. Kim, "Evaluation Method for Korean Morphological Analysis System and its Application to MATEC 99" 44-49, 1999

19 S. Kang, "Encoding of Morphological Analysis Result and Eojeol Dictionary Construction" 112-117, 2004

20 S. Lee, "A Two-level Morphological Analysis of Korean, Korea Advanced Institute of Science and Technology" Dept. of Computer Science 1992

21 S. Lee, "A Two-level Approach to Korean Verb Morphology" 19 (19): 993-996, 1992

22 W. A. Gale, "A Program for Aligning Sentences in Bilingual Corpora" The MIT Press 75-102, 1994

23 S. Lee, "A Predictive Morphological Analyzer, A Part-of- Speech Tagger Based on Joint Independence Model, and A Fast Noun Extractor" 145-150, 1999

24 J. Hong, "A New Korean Morphological Analyzer using Eojeol Pattern Dictionary" 35 (35): 279-284, 2008

25 S. Y. Kim, "A Morphological Analyzer for Korean Language with Tabular Parsing Method and Connectivity Information" 133-147, 1987

26 K. Koskenniemi, "A General Computational Model for Word-form Recognition and Production" Association for Computational Linguistics 178-181, 1984

27 K. Lee, "A Compound Noun Processing in the Two-level Morphological Analysis of Korean" 29 (29): 505-507, 2002

28 The National Institute of the Korean Language, "21st Century Sejong Project Final Result - 2009. 12. Revised Edition" The National Institute of the Korean Language 2009