TRAIL: Trajectory-based Representation And Integration for Limiting Over-smoothing

다국어 초록 (Multilingual Abstract)

Graph Neural Networks (GNNs) are powerful tools for processing structured graph data but often suffer from over-smoothing, where nodes become indistinguishable as the network depth increases. To address this challenge, we present TRAIL (Trajectory-based Representation And Integration for Limiting over-smoothing), a novel framework that tracks how local embeddings change across layers and combines them to preserve informative node features. Extensive evaluations demonstrates that TRAIL significantly increases Dirichlet Energy, a key metric for quantifying over-smoothing, ensuring that feature differences between nodes remain distinguishable even in deep networks. These improvements translate into superior performance across diverse datasets, particularly excelling in heterophily graphs which are harsh condition, such as Cornell, Chameleon, and Wisconsin, with accuracy improvements of up to 15% over baseline models. Furthermore, TRAIL consistently outperforms existing spectral and spatial GNNs on high-homophily datasets like Cora, Citeseer, and Pubmed. Overall, TRAIL offers an effective and generalizable solution for improving GNN performance across a wide range of graph types. Keywords: Graph Neural Network, Over-smoothing, Node classification, Node embedding, Trajectory

목차 (Table of Contents)

• 1. Introduction 9
• 2. Related Work 11
• 3. Formalization 13
• 3.1 Graph Neural Network 13
• 3.2 Over-smoothing 14
• 3.3 Dirichlet energy 15
• 3.4 Graph Distance Ratio 16
• 4. Proposed Approach 18
• 4.1 Average of Embedding Trajectory 20
• 4.2 Intuition of how TRAIL mitigates over-smoothing 21
• 4.3 TRAIL on Dirichlet energy and Graph Ratio Distance 24
• 4.4 Batch Normalization on Node Embedding 27
• 4.5 Runtime Analysis 27
• 5. Experiment 29
• 5.1 Hyperparameter 30
• 5.2 Result 33
• 6. Ablation Study 36
• 7. Conclusion 39
• Reference 40
• Appendix 44