대규모 언어 모델에서 맞춤형 전문가 네트워크 및 어댑터를 통한 모듈식 편집 = Modular Editing via Customized Expert Networks and Adaptors in Large Language Models.

다국어 초록 (Multilingual Abstract)

Efficiently updating language models to incorporate new knowledge while retaining their pre-existing capabilities remains a formidable challenge particularly when aiming to preserve pre-existing capabilities. This thesis presents a novel Modular Editing framework that addresses this
challenge by integrating new information through a unified approach that combines customized expert networks and adaptors. At the core of the framework is an module level deferral router , which evaluates incoming queries fall within the scope of existing edit requests. Queries are then dynamically routed to either customized editing experts or key-value adaptors. This modular strategy ensures that updates are localized, thereby mitigating risks of unintended alterations on unrelated outputs. The proposed method not only emphasizes the precise integration of novel content but also prioritizes the preservation of the model’s original performance across previously mastered tasks. Comprehensive experimental evaluations reveal that this framework consistently surpasses existing state of the art knowledge editing techniques, achieving a superior balance between incorporating new information and maintaining overall model stability. Moreover, this modular strategy is shown to
effectively manage the inherent trade-offs between generalization and locality, paving the way for more adaptable and resilient language
models. This research contributes a robust and scalable solution for updating language models without the need for expensive full-model
retraining. By focusing on localized modifications, the framework offers a practical path forward for keeping language models current in
rapidly evolving information landscapes. The insights and methodologies presented herein lay a solid foundation for future advancements in modular editing techniques and adaptive model updating, ultimately enhancing the dynamic integration of knowledge within
large language models.

목차 (Table of Contents)

• 1 Introduction 1
• 2 Literature Review 13
• 3 Modular Editing via Customized Expert Networks and Adaptors in Large Language Models 25
• 4 Conclusion 74