인공지능 : 부분어절 조건부확률 기반 동형이의어 태깅 모델 = Artificial Intelligence : Korean Homograph Tagging Model based on Sub-Word Conditional Probability

다국어 초록 (Multilingual Abstract)

In general, the Korean morpheme analysis procedure is divided into two steps. In the first step as an ambiguity generation step, an Eojeol is analyzed into many morpheme sequences as candidates. In the second step, one appropriate candidate is chosen by using contextual information. Hidden Markov Model(HMM) is typically applied in the second step. This paper proposes Sub-word Conditional Probability(SCP) model as an alternate algorithm. SCP uses sub-word information of adjacent eojeol first. If it failed, then SCP use morpheme information restrictively. In the accuracy and speed comparative test, HMM`s accuracy is 96.49% and SCP`s accuracy is just 0.07% lower. But SCP reduced processing time 53%.

참고문헌 (Reference)

1 신준철, "한국어 품사 및 동형이의어 태깅을 위한 단계별 전이모델" 한국정보과학회 39 (39): 889-901, 2012

2 박민서, "차세대 시퀀싱을 위한 짧은 서열 정렬에 대한 기술조사" 한국정보과학회 38 (38): 503-511, 2011

3 박희근, "어절별 중의성 해소 규칙을 이용한 혼합형 한국어 품사 태깅 시스템" 한국정보과학회 13 (13): 427-431, 2007

4 허정, "상호정보량과 복합명사 의미사전에 기반한 동음이의어 중의성 해소" 한국정보과학회 33 (33): 1073-1089, 2006

5 이용구, "사전 정보를 이용한 단어 중의성 해소 모형에 관한 실험적 연구" 한국정보관리학회 24 (24): 321-342, 2007

6 김준수, "가중치를 이용한 통계 기반 한국어 동형이의어 분별 모델" 한국정보과학회 30 (30): 1112-1123, 2003

7 David Yarowsky, "Word-sense disambiguation using statistical models of Roget’s categories trained on large corpora" 454-460, 1992

8 Soojong Lim, "Word Sense Disambiguation of Korean Verbs Using Weight Information from Context" 425-429, 1998

9 Wang Woo Lee, "Word Sense Disambiguation System Using Lexical Co-occurrencing Set and Thesaurus(in Korean)" Ulsan university 2003

10 Jin-dong Kim, "Twoply HMM: A Part-of-Speech Tagging Model based on Morpheme-Unit considering the Characteristics of Korean" 24 (24): 1502-1512, 1997

11 Eric Brill, "Transformation-based error-driven learning and natural language processing: A case study in part of speech tagging" 21 (21): 543-565, 1995

12 Dong Myung Kim, "Simultaneous Korean POS and Homonym Tagging System using HMM(in Korean)" Ulsan University 2009

13 Seung-Hoon Na, "Semi-CRF or Linear-chain CRF? A comparative Study of Joint Models for Korean Morphological Analysis and POS Tagging" 9-12, 2013

14 Young-Jun Base, "Semantic Analysis of Korean Compound Noun using Lexical Semantic Network(U-WIN)" Ulsan University 2013

15 Dan Roth, "Part of speech tagging using a network of linear separators" 1136-1142, 1998

16 Joon-Choul Shin, "Comparison between Markov Model and Hidden Markov Model for Korean Part-of-Speech and Homograph Tagging" 152-155, 2013

17 Ho Suk Lee, "A Survey of conditional Random Fields and Applications" 36 (36): 2009

18 Scott M. Thede, "A Second-Order Hidden Markov Model for Part-of-Speech Tagging" 175-182, 1999

19 Joon-Choul Shin, "A Korean Morphological Analyzer using a Pre-analyzed Partial Word-phrase Dictionary(in Korean)" 39 (39): 2012