지식 그래프 확장을 위한 파스 트리 기반 트리플 추출과 논리 속성 보존 임베딩 = Parse Tree based Triple Extraction and Logical Property Preserving Embedding for Knowledge Graph Enrichment

다국어 초록 (Multilingual Abstract)

Knowledge graph is a very important key factor in data-driven applications such as QA (Question & Answering), Chatbot, etc. Although there are various public knowledge graphs which contain a massive amount of triples, they are still far from perfection compared to infinite real-world facts. This lack of knowledge greatly affects the performance of data-driven applications. Therefore, this problem should be solved by creating triples infinitely.
This dissertation proposes two models for knowledge graph enrichment which extract explicit knowledge and implicit knowledge. In order to extract explicit knowledge, a pattern-based relation extraction approach is proposed. This model adopts a parse tree pattern representation and a semantic similarity based pattern filtering function. Parse tree patterns are superior to lexical patterns used commonly in many previous studies in that they can manage long distance dependencies among words. In addition, the proposed semantic filter which is a combination of WordNet-based similarity and word embedding similarity removes patterns that are semantically irrelevant to the meaning of a target relation.
A Logical Property Preserving (LPP) embedding method is proposed for extracting implicit knowledge. Previous translation based embedding methods could not handle two crucial logical properties of relations which are transitivity and symmetricity. The embedding space generated by existing translation-based embedding models cannot represent triples which have transitive and symmetric relations perfectly, because they ignore the role of entities in a triple. This dissertation describes the aftereffect of this phenomenon and introduce a solution which is named a role-specific projection. A role-specific projection overcomes the limitation of previous methods by mapping an entity to distinct vectors according to its role in a triple. That is, a head entity is projected onto an embedding space by a head projection operator, and a tail entity is projected by a tail projection operator. This idea can be applied to previous translation based embedding models, easily. In this dissertation, lppTransE, lppTransR, and lppTransD based on TransE, TransR, and TransD, respectively, are introduced.
According to experiments, two proposed knowledge extraction models showed outstanding performance. In an explicit knowledge extraction task, a proposed explicit knowledge extraction model achieved 60.1% of the average accuracy of the newly extracted triples in an English knowledge extraction task. This is 28.9% higher than baseline which is lexical sequence pattern-based. The proposed model also works wonderfully in a multi-lingual environment. In addition, a proposed model showed much more stable performance in comparison with neural network based approaches. These results prove that the proposed model produces more relevant patterns for the relations of seed knowledge, and thus more accurate triples are generated by the patterns. The performance of implicit knowledge extraction models were measured with two kinds of tasks, link prediction, and triple classification. The proposed lpp-models achieved state-of-the-art performance at both tasks. Especially, there was significant improvement with a N-to-N relation category which contains transitivity and symmetricity relations. These results prove that it is critical to preserve logical properties of relations while embedding knowledge graphs, and the proposed method does it effectively.

목차 (Table of Contents)

• 1 서 론 1
• 1.1 연구 배경 1
• 1.2 연구 내용 5
• 1.3 논문의 기여와 구성 9
• 2 관련 연구 11
• 2.1 외부 말뭉치를 활용한 명시적 지식 추출 11
• 2.2 추론 기반의 암시적 지식 추출 18
• 3 지식 그래프 확장을 위한 관계 추출 모델 27
• 3.1 지식 그래프 확장을 위한 통합 프레임워크 27
• 3.2 명시적 지식 추출을 위한 패턴 기반의 트리플 추출 29
• 3.2.1 명시적 지식 추출 아키텍쳐 29
• 3.2.2 지식 추출을 위한 패턴 생성 31
• 3.2.3 패턴 기반의 신규 지식 추출 44
• 3.3 암시적 지식 추출을 위한 논리적 속성 보존 지식 그래프 임베딩 48
• 3.3.1 논리 속성 보존 임베딩 모델 53
• 3.3.2 논리 속성 보존 임베딩의 구현 55
• 3.3.3 논리 속성 보존 임베딩 모델 학습 58
• 3.3.4 기존 translation 기반 임베딩 모델들과의 복잡도 비교 59
• 4 실험 및 성능평가 61
• 4.1 명시적 지식 추출 모델 실험 61
• 4.1.1 영어 말뭉치를 이용한 지식 추출 61
• 4.1.2 한국어 말뭉치를 이용한 지식 추출 80
• 4.1.3 딥러닝 기반 모델들과의 비교 82
• 4.2 암시적 지식 추출 모델 실험 87
• 4.2.1 링크 예측 실험 87
• 4.2.2 트리플 분류 실험 95
• 4.2.3 모델 복잡도 영향성 검증 실험 102
• 5 결 론 107
• 5.1 제안 모델 요약 108
• 5.2 실험 결과 요약 109
• 5.3 향후 과제 112
• 참고 문헌 113
• 초 록 128