Code Search and Generation with Agentic Language Models = 코드 검색 및 생성을 위한 에이전틱 언어모델 연구

다국어 초록 (Multilingual Abstract)

Recent advances in large language models have enabled code agents to perform complex tasks through interaction with environments.
Yet their effectiveness remains limited by challenges across the core capabilities of perception, reasoning, action, and learning, as well as their evaluation.
This dissertation investigates the challenges associated with these areas—including information deficiency, shallow reasoning, budget inefficiency, feedback deficiency, and limited interactions—and introduces targeted methods to address them.

First, to mitigate information deficiency in perception, we propose CT-BERT, a structure-aware representation model that injects template-level signals to improve compositional generalization in code search.
Second, to address shallow reasoning, ArchCode introduces an intermediate step that extracts and interprets implicit or verbose user requirements.
Third, to reduce budget inefficiency in action, EffiCode estimates problem solvability to guide efficient tool usage.
Fourth, to overcome feedback deficiency in learning, CIRL applies reinforcement learning with code-level intervention to align code search with execution feedback.
Lastly, to address limited agent–environment interactions, we present ConvCodeWorld, a reproducible benchmark with nine feedback combinations for cost-effective and diverse assessment.

Finally, we summarize the contributions and discuss future directions of agentic code search and generation.

국문 초록 (Abstract)

대규모 언어 모델의 발전은 코드 에이전트가 외부 환경과 상호작용함으로써 복잡한 작업을 수행할 수 있도록 만들었지만, 여전히 지각, 추론, 행동, 학습이라는 네 가지 핵심 역량과 이를 평가하는 체계 전반에서 다양한 한계에 직면해 있다. 본 논문은 이 다섯 영역에 대한 주요 과제—정보 부족, 얕은 추론, 예산 비효율, 피드백 부족, 제한적인 상호작용—를 중심으로 분석하고, 이를 해결하기 위한 특화된 기법들을 제안한다.

우선, 지각 단계에서의 정보 부족 문제를 완화하기 위해, 템플릿 수준의 구조 신호를 주입하여 코드의 구성 일반화를 향상시키는 구조 인식 표현 모델 CT-BERT를 제안한다.
다음으로, 추론 과정의 얕은 처리를 해결하기 위해, 사용자 요구가 암시적이거나 장황할 때 이를 중간 추론 단계를 통해 명시적으로 추출하고 해석하는 ArchCode를 제안한다.
셋째로, 행동 선택에서의 예산 비효율 문제를 줄이기 위해, 문제의 해결 가능성을 사전 추정하여 도구 사용을 효율적으로 조정하는 EffiCode를 제안한다.
넷째로, 학습 과정에서의 피드백 부족 문제를 극복하기 위해, 실행 기반 피드백을 활용하고 코드 수준 개입을 적용하는 강화학습 기반 접근인 CIRL을 제안한다.
마지막으로, 평가 단계에서 환경과 에이전트 간의 상호작용이 제한적인 문제에 대응하기 위해, 아홉 가지 피드백 조합을 포함하여 에이전트-환경 상호작용을 다양화하고, 비용 효율적으로 평가할 수 있는 재현 가능한 벤치마크 ConvCodeWorld를 구축한다.

끝으로, 본 논문은 이러한 기법들의 기여를 정리하고, 에이전트 기반 코드 검색 및 생성의 미래 방향에 대해 논의한다.

목차 (Table of Contents)

• Acknowledgements 1
• Chapter 1 Introduction 3
• 1.1 Thesis Organization 5
• Chapter 2 Background 10
• 2.1 Preliminary 10
• 2.1.1 Code-Related Tasks 10
• 2.1.2 Agentic AI 11
• 2.2 Challenges in Code Agents 13
• 2.2.1 Perception 13
• 2.2.2 Reasoning 15
• 2.2.3 Action 16
• 2.2.4 Learning 17
• 2.2.5 Evaluation 18
• 2.3 Related Work 19
• 2.3.1 Chapter 3: Perception: Compositional Generalization 19
• 2.3.2 Chapter 4: Reasoning: Requirement Generation 20
• 2.3.3 Chapter 5: Action: Budget Awareness 21
• 2.3.4 Chapter 6: Learning: Execution Feedback 22
• 2.3.5 Chapter 7: Evaluation: Diverse Interactions 24
• Chapter 3 Perception: Compositional Generalization 26
• 3.1 Introduction 26
• 3.2 CTBERT 28
• 3.2.1 Mining Templates 29
• 3.2.2 Pretraining with Templates 30
• 3.3 Experiments 33
• 3.3.1 Main Results 34
• 3.3.2 RQ1: Do Templates Help Compositional Generalization? 34
• 3.3.3 RQ2: Do Templates Complement Other Structural Tasks? 35
• 3.3.4 RQ3: Do Templates Contribute To Lexical/Structural Generalization? 36
• 3.3.5 Template Span Task 36
• 3.3.6 Impact of Initialization on Model Performance 37
• 3.4 Summary 37
• Chapter 4 Reasoning: Requirement Generation 41
• 4.1 Introduction 41
• 4.2 The ArchCode Framework 45
• 4.2.1 Delineating Software Requirements 46
• 4.2.2 Requirements-aware Generation 47
• 4.2.3 Pursuing Requirements Satisfaction 48
• 4.3 Experiments 49
• 4.3.1 Experimental Setup 49
• 4.3.2 HumanEval-NFR: Embracing NFR Evaluation 50
• 4.3.3 HumanEval and CodeContests: Public Benchmarks for FR Evaluation 51
• 4.4 Analysis and Discussion 52
• 4.4.1 Efficiency and Effectiveness of Requirement-aware Test Case Generation 52
• 4.4.2 Conditioning Code Generation on Test Cases 53
• 4.4.3 Preference over Requirements 54
• 4.4.4 ArchCode under Diverse Settings 55
• 4.4.5 Gains from Prompt Engineering 58
• 4.4.6 Correctness of Generated Requirements 59
• 4.4.7 Analysis of Generated Test Cases by ArchCode 59
• 4.4.8 NFR Preference Control 62
• 4.5 Ethical and Social Implications 63
• 4.6 Limitations 64
• 4.7 Summary 64
• Chapter 5 Action: Budget Awareness 66
• 5.1 Introduction 66
• 5.2 Sample Efficiency 69
• 5.3 EffiCode 70
• 5.3.1 Code Sampling as Discrete Search 70
• 5.3.2 Solvability Estimation 72
• 5.3.3 Adaptive Decoding 74
• 5.4 Experimental Setup 75
• 5.4.1 Evaluation Metrics 75
• 5.4.2 Benchmarks 76
• 5.5 Experimental Results 76
• 5.5.1 Sample Efficiency 76
• 5.5.2 Functional Correctness 77
• 5.6 Summary 77
• Chapter 6 Learning: Execution Feedback 80
• 6.1 Introduction 80
• 6.2 Supervised Code Search Training 83
• 6.2.1 Code Search 83
• 6.2.2 Supervised Baseline 84
• 6.3 Methodology 85
• 6.3.1 Contribution I: Reinforcement Learning with Intervention 85
• 6.3.2 Contribution II: Structural Perturbation 87
• 6.3.3 CIRL 89
• 6.4 Experiments 90
• 6.4.1 Datasets 90
• 6.4.2 Code Search Baselines 92
• 6.4.3 Code Intervention Baselines 93
• 6.4.4 Main Results 94
• 6.5 Discussion 96
• 6.5.1 Is CIRL Perturbation Sample-efficient? 96
• 6.5.2 Does CIRL Generalize Over Counterfactual Perturbation? 96
• 6.5.3 Does CIRL Generalize Over Different Programming Languages? 97
• 6.5.4 Can Augmentation Negatively Affect Performance? 98
• 6.5.5 Can LLMs Align better with EF? 98
• 6.6 Limitations 99
• 6.7 Summary 99
• Chapter 7 Evaluation: Diverse Interactions 101
• 7.1 Introduction 101
• 7.2 ConvCodeWorld: Reproducible Feedback Environments 103
• 7.2.1 Feedback Categorization 104
• 7.2.2 Feedback Combinations 107
• 7.3 ConvCodeBench: A Static Benchmark for Efficient Evaluation 108
• 7.4 Experiments 110
• 7.4.1 Setup 111
• 7.4.2 Results on ConvCodeWorld 112
• 7.4.3 Results on ConvCodeBench 118
• 7.4.4 Verbal Feedback 119
• 7.4.5 Verbal Feedback by Open-Source LLMs 123
• 7.5 Summary 123
• Chapter 8 Conclusion 125
• 8.1 Summary of Contributions 125
• 8.2 Future Work 126
• 요약 129
• Appendix A Supplementary Details for Perception: Compositional Generalization 130
• A.1 Implementation Details 130
• A.2 Mining Templates 132
• A.3 Dataset Statistics 133
• Appendix B Supplementary Details for Reasoning: Requirement Generation 134
• B.1 Implementation Details 134
• B.1.1 Open-sourced Backbone Models and Java Language 136
• B.2 HumanEval-NFR Construction 136
• B.2.1 Quality Control for FR Test Cases 137
• B.2.2 Quality Control for NFR Test Cases 138
• B.3 Varying Difficulty Levels of NFRs in HumanEval-NFR 139
• B.4 In-Context Learning Examples 148
• B.5 In-Context Learning Prompt Templates 151
• B.6 Case Study 153
• Appendix C Supplementary Details for Action: Budget Awareness 159
• C.1 Implementation Detail 159
• Appendix D Supplementary Details for Learning: Execution Feedback 161
• D.1 Motivation of RL framework 161
• D.2 Implementation Detail 163
• D.2.1 Code Search Baselines 163
• D.2.2 Code Intervention Approaches 163
• D.2.3 Test Time Code Augmentation by CIRL 165
• D.2.4 Misalignment in LLMs 165
• D.3 Dataset 169
• D.4 CIRL: Case Study 170
• D.5 GPT-3.5: Case Study 171
• Appendix E Supplementary Details for Evaluation: Diverse Interactions 175
• E.1 DeekSeek-R1-Distill Results on ConvCodeWorld 175
• E.2 Distinction of ConvCodeWorld 178
• E.3 Implementation Details 179
• E.4 ConvCodeWorld 181
• E.5 ConvCodeBench 184
• E.5.1 MRR and Recall Results 184
• E.6 Rank Correlations between ConvCodeBench and ConvCodeWorld 186
• E.6.1 Reference Model: CodeLlama-7B-Instruct-hf 186
• E.7 In-Context Learning Examples 191
• E.7.1 Novice-Level Verbal Feedback 191
• E.7.2 Expert-Level Verbal Feedback 193
• E.8 Case Study 196