Towards Interpretability of GPT-Style Models in Step-by-Step Games Through First-Order Logic = 1차 논리를 활용한 단계별 게임 내 GPT 계열 모델의 해석 가능성 연구

다국어 초록 (Multilingual Abstract)

Understanding how transformer-based large language models make decisions re- mains a continuous challenge in artificial intelligence. While these models achieve im- pressive performance, their internal workings and reasoning processes remain unclear black boxes, especially on the attention level. In this thesis, I explore mechanistic in- terpretability methods and attempts to combine its findings with First-Order Logic to propose a framework idea that could systematically characterize strategic reason- ing in game-playing transformer models and formally express them. I focus on models trained to play games like Othello and Chess, which provides a controlled domain where rules and optimal strategies are fully known, which proves ideal for intepreting the transformer’s reasoning at the attention level. I synthesize insights from circuit- level interpretability, probing methodologies, neuro-symbolic systems with First-Order Logic, and emergence of world models to manifest a way to identify computational pathways, detect encoded strategy heuristics, and translate attention patterns into explicit First-Order Logic formulas.

목차 (Table of Contents)

• Abstract （English） i
• List of Contents iii
• 1 Introduction 1
• 1.1 Introduction 1
• 1.2 Research Questions 2
• 2 Background Knowledge 3
• 2.1 The Transformer Model and GPT 3
• 2.2 First-Order Logic 8
• 3 Proposal Idea 12
• 3.1 Problem Definition 12
• 3.2 Conceptual Framework 13
• 3.3 Required Components 14
• 4 Circuit-Level Interpretability 16
• 4.1 Fundamental Frameworks 16
• 4.2 Beyond Toy Models 21
• 4.3 Automating Discovery 23
• 4.4 Conclusion 25
• 5 Probing and Representation Learning 27
• 5.1 Discovering What Models Know 27
• 5.2 Why First-Order Logic instead of Dependency Parsing to interpret At-
• tention 31
• 5.3 What Do Probes Actually Tell Us 32
• 5.4 Connecting Probing with Mechanistic Interpretability 35
• 5.5 Conclusion 37
• 6 First-Order Logic with Neural Network Interpretability 39
• 6.1 Integrating Logic with Neural Computation 39
• 6.2 Learning and Extracting Symbolic Knowledge 43
• 6.3 Verification 46
• 6.4 Relating Theorem Proving 48
• 6.5 Logic-Guided Neural Learning 49
• 6.6 Adding Game Description Language 51
• 6.7 Conclusion 52
• 7 Emergence of World Models 54
• 7.1 Evidence of Emerging Models 54
• 7.2 Robustness and Latent Variables 57
• 7.3 Evidence of Look-Ahead 59
• 7.4 Emergent Strategy from Next-Token Prediction 61
• 7.5 Discovery Beyond Activation Patching 62
• 7.6 Conclusion 64
• 8 Discussion 66
• 8.1 Limitations of Current Interpretability Methodologies 66
• 8.2 A Potential Framework for Interpreting Game-Playing Transformers Through
• First-Order Logic 67
• 8.2.1 Novel Potential 69
• 8.3 Experimental Requirements 70
• 8.4 Addressing Multiple Strategies and Opponent Modeling 71
• 8.5 Expected Insights and Contributions 72
• 9 Conclusion 74
• References 76
• Acknowledgements 79
• 0.1 Croatian Translation Hrvatski Prijevod Zahvale 80