경량 언어 모델에서 의미적 연결 향상을 위한 Text-to-SQL 방법 = A Text-to-SQL Framework for Enhancing Schema Linking in SLM

다국어 초록 (Multilingual Abstract)

Text-to-SQL research focuses on translating natural language queries into SQL, reducing the time and cost of data access. While recent studies have achieved high accuracy using large-scale models like GPT-4, these approaches neglect resource constraints, limiting practical deployment on end-user environments where fast inference and low latency are critical.
To address these challenges and deliver an optimized solution for end-user environments, this thesis proposes URFine (Understanding Relations via Fine-tuning), a Supervised Fine-Tuning framework designed to enhance the Text-to-SQL performance of SLMs. The URFine framework systematically optimizes the model through a process that includes prompt structure selection, database schema extraction, and supervised fine-tuning. This study verifies the effectiveness of URFine using the Spider dataset, employing both execution accuracy and total inference time as key evaluation metrics to assess time efficiency.
The experimental results demonstrate significant improvements in both accuracy and efficiency. First, comparing the model before and after supervised fine-tuning revealed that fine-tuning improved accuracy by up to 3 times and time efficiency by up to 11 times compared to the base model. This confirms that supervised fine-tuning is essential for maximizing the inference capabilities of lightweight LLMs.
Furthermore, this thesis identifies the optimal prompt structure for SLMs by analyzing three distinct configurations: Full (role-playing with full schema), Intermediate (role-playing with extracted schema), and Minimal (extracted schema without role-playing). The results indicate that the Minimal structure yields the highest performance, achieving 1.4 times higher accuracy and time efficiency compared to the Full structure. This finding proves that while role-playing and extensive context benefit large models, they act as noise for SLM. Instead, removing unnecessary role assignments and providing only the relevant extracted schema is the most effective strategy.
In a comprehensive comparison with existing Text-to-SQL research, URFine achieved up to a 67-fold improvement in accuracy and a 19-fold improvement in time efficiency. These results highlight that the example-based and role-playing techniques effective for GPT-4 often degrade the performance of SLM.
The contributions of this thesis are as follows:
Introduction of Time Efficiency Metrics: It systematically analyzes the Text-to-SQL process by adopting total elapsed time as a core metric, which was overlooked in prior studies.
Optimization of Prompt Structure: It proves that for SLM, excluding role assignment and focusing on precise schema extraction significantly contributes to performance, contrary to practices for large models.
Development of URFine Pipeline: It demonstrates that the proposed supervised fine-tuning pipeline drastically enhances accuracy and speed in constrained environments.
Analysis of Lightweight Model Characteristics: It provides a theoretical analysis of why conventional large-model techniques fail on SLM, offering a practical framework for future mobile AI research.

목차 (Table of Contents)

• Ⅰ. 서 론 1
• Ⅱ. 관련 연구 7
• 2.1. 문맥 기반 학습 연구 7
• 2.2. 지도 미세 조정 연구 9
• Ⅲ. 문제 정의 13
• Ⅳ. 제안하는 프레임워크: URFine 15
• 4.1. 제안하는 지도 미세 조정 방법론 구조 15
• 4.2. 프롬프트 구조 선정 단계 16
• 4.3. 스키마 컬럼 추출 단계 20
• 4.4. 지도 미세 조정 단계 22
• 4.5. 추론 및 결과 후처리 단계 22
• Ⅴ. 실험 결과 25
• 5.1. 환경 설정 25
• 5.1.1. 실험 환경 26
• 5.1.2. 활용한 경량 LLM 모델 26
• 5.1.3. 학습 시 파라미터 설정 26
• 5.1.4. 데이터셋 27
• 5.1.5. 평가 지표 28
• 5.2. 성능 평가 결과 29
• 5.2.1. 지도 미세 조정 전후 정확도, 소요 시간 비교 29
• 5.2.2. 프롬프트 구조 간 정확도, 소요 시간 비교 36
• 5.2.3. 기존 연구와 정확도, 소요 시간 비교 38
• Ⅵ. 결론 44
• Ⅶ. 향후 연구 46
• 참고 문헌 48