신경망의 분석과 적용: 생성적 적대 신경망과 그래프 분류에 대하여 = Analysis and Applications of Neural Networks: the Generative Adversarial Networks and the Graph Classification

국문 초록 (Abstract)

이 박사학위 논문은 신경망을 적용한 알고리즘에 대한 두 가지의 연구로 구성된다. 첫 번째는 와서스타인 생성적 적대 신경망에서 경사 페널티를 주었을 때, 이 학습 과정의 국소수렴성 분석 결과에 대한 연구이다. 이 연구는 기존의 경사 페널티에 관련되어 제시되었던 방법들을 포괄하는 문제에 와서스타인 생성적 적대 신경망을 학습하는 과정 중 경사 페널티를 일반적인 추상측도 형태로 부여하였을 때 어떤 특정 조건을 만족시켰을 경우 국소수렴성이 보장됨을 증명하였다. 더불어, 특정 분포를 만들어내거나 그림을 만들어내는 실험을 통해 우리가 제시한 특정 조건에 대한 실험적 검증을 수행하였다. 두 번째는 그래프를 분류하는 문제에서 간선에 대한 정보를 수집하는 그래프 신경망 구조를 새로이 제시한 연구이다. 이 논문에서는 기존의 그래프 동형 네트워크를 비롯하여 인접한 꼭지점의 정보만을 수집하는 경우 구분할 수 없는 그래프의 군집을 새로 보이고, 이를 해결하기 위해 이웃한 꼭지점들끼리의 연결 관계를 수집하는 구조를 제시하였다.

다국어 초록 (Multilingual Abstract)

This doctoral dissertation contains two studies related to the recent topics of neural networks. The first part analyses the local stability of the Wasserstein Generative Adversarial Networks (WGANs) with a gradient penalty, where the penalty measure is given as an abstract parametric form. We provide an analysis of GANs to reveal what conditions for gradient penalty methods should be satisfied to achieve successful convergence. The second part proposes a new edge-aggregating framework for a graph classification task with graph neural networks (GNN). Our proposed framework gives integrated information that describes which nodes in the neighborhood are connected. This can be discovered with a family of graphs that are not distinguished by the previous 1-hop based algorithms. For both topics, our experiment results substantiate our arguments on various generative tasks and graph classification tasks.

목차 (Table of Contents)

• Contents
• I. Local Stability of Wasserstein GANs with Abstract Gradient Penalty 1
• 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
• 1.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
• 1.2.1 Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . 3
• 1.2.2 Generative Adversarial Networks . . . . . . . . . . . . . . . 4
• 1.2.3 Wasserstein GANs and Gradient Penalty . . . . . . . . . . . 6
• 1.2.4 Measure-Valued Differentiation . . . . . . . . . . . . . . . . 7
• 1.3 Problem formulation and some examples . . . . . . . . . . . . . . . . 11
• 1.3.1 Optimization process as a dynamic system . . . . . . . . . . 12
• 1.3.2 Toy examples . . . . . . . . . . . . . . . . . . . . . . . . . . 13
• 1.4 Main convergence theorem . . . . . . . . . . . . . . . . . . . . . . . 17
• 1.4.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . 17
• 1.4.2 Main convergence theorem . . . . . . . . . . . . . . . . . . . 19
• 1.4.3 Detailed proof . . . . . . . . . . . . . . . . . . . . . . . . . 21
• 1.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
• 1.5.1 2D examples . . . . . . . . . . . . . . . . . . . . . . . . . . 29
• 1.5.2 CIFAR-10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
• 1.5.3 CelebA-HQ 128 . . . . . . . . . . . . . . . . . . . . . . . . 36
• 1.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
• II. NEAR: Neighborhood Edge AggregatoR for Graph Classification 39
• 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
• 2.2 Related Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
• 2.3 Graph Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . 42
• – II –
• 2.3.1 Graph Isomorphism Networks . . . . . . . . . . . . . . . . . 43
• 2.3.2 Toy example . . . . . . . . . . . . . . . . . . . . . . . . . . 44
• 2.4 Proposed method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
• 2.4.1 Proposed Variants of NEAR, Computational Cost, and Related
• Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
• 2.4.2 How powerful is NEAR?: A comparison with 3-WL test . . . 53
• 2.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
• 2.5.1 Model Configuration of GIN . . . . . . . . . . . . . . . . . . 54
• 2.5.2 Toy example . . . . . . . . . . . . . . . . . . . . . . . . . . 55
• 2.5.3 Graph classification tasks . . . . . . . . . . . . . . . . . . . . 58
• 2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
• Summary (in Korean) 64
• References 65

참고문헌 (Reference)

1. A. Approximation theory of the mlp model in neural networks, PINKUS, numerica 8 (, , 1999