거대언어모델 기반 국토교통 분야 데이터 질의응답 챗봇 시스템 설계 및 구현 = Design and Implementation of Large Language Model-based Question-Answering Chatbot System for Ministry of Land, Infrastructure and Transport Data

다국어 초록 (Multilingual Abstract)

This paper presents the development and evaluation of an on-premises intelligent question-answering (QA) system designed to provide reliable and secure access to national land, infrastructure, and transport policy information. To address the limitations of existing civil inquiry services-such as restricted call center operating hours, long wait times, and the rigidity of rule-based chatbots-the proposed system integrates a domain-specialized large language model (LLM) with Retrieval-Augmented Generation (RAG) technology. A customized version of Gemma-3-1B, named molit-gemma, was fine-tuned using official datasets from the Ministry of Land, Infrastructure and Transport (MOLIT), and combined with an OpenSearch-based retrieval pipeline to generate grounded, document-consistent responses. Experimental results demonstrate that the proposed system achieves superior performance compared to pretrained and commercial models. The molit-gemma + RAG configuration attained a BLEU of 0.6258 and an LLM-as-a-Judge overall score of 4.34 out of 5, confirming its strong domain suitability and factual accuracy. Deploying the system in an on-premises environment resolves critical security concerns associated with cloud-based LLMs, preventing the transmission of sensitive administrative data to external servers. Furthermore, the RAG architecture significantly mitigates hallucination risks by explicitly grounding responses in retrieved policy documents, thereby enhancing reliability and explainability-key requirements for public-sector AI services. This research not only demonstrates the feasibility of applying LLM-RAG hybrid architectures to government administrative services but also offers a scalable model for AI-driven policy communication. The proposed system provides a foundation for 24/7 citizen-centered policy consultation and contributes to the broader advancement of AI-based digital government. Future work will explore multimodal extensions, continuous quality improvement using Human-in-the-Loop pipelines, and cross-agency integration to expand the system's applicability across the public sector.

목차 (Table of Contents)

• Ⅰ. 서 론 1
• 1.1 연구 배경 1
• 1.2 연구 목적 4
• 1.3 논문의 구성 6
• Ⅱ. 관련 연구 8
• 2.1 챗봇 개발 방법론 8
• 2.2 LLM 기반 챗봇 아키텍처 10
• 2.3 도메인 특화 LLM 커스터마이징 기법 13
• 2.4 검색 증강 생성(RAG)의 역할 15
• Ⅲ. 국토교통분야 챗봇 시스템 설계 및 구현 18
• 3.1 시스템 아키텍처 설계 18
• 3.2 국토교통 데이터 수집 및 구성 22
• 3.3 데이터 전처리 및 정제 25
• 3.4 QA 데이터셋 생성을 위한 Qwen3 RAG 모듈 29
• 3.5 Gemma-3 모델 파인튜닝 32
• 3.6 챗봇 RAG 구현 40
• 3.7 메모리 아키텍처 49
• Ⅳ. 시험 및 평가 51
• 4.1 평가 데이터셋 구성 51
• 4.2 성능 평가 지표 및 방법 52
• 4.3 국토교통 데이터 질의 응답 결과 분석 61
• Ⅴ. 결론 및 향후 연구 63
• 참고문헌 65