망분리 환경 내 온프레미스 LLM의 데이터 유출 위험 검증 프레임워크 연구 = A Data Leakage Risk Validation Framework for On-Premises LLMs in Network Isolation Environments

다국어 초록 (Multilingual Abstract)

The adoption of Large Language Models （LLMs） in the financial sector has increased rapidly with the advancement of generative AI technologies. Due to strict regulatory requirements and the sensitive nature of financial data, many institutions are introducing on-premises LLMs within network-isolated environments. However, network isolation alone cannot completely prevent data leakage risks because LLMs process and generate information through complex data flows.
This study analyzes data leakage risks in on-premises LLM environments and proposes a validation framework tailored to the financial sector. Based on existing standards and guidelines, including the NIST AI Risk Management Framework （AI RMF） and OWASP Top 10 for LLM Applications, data leakage risks were identified and analyzed from the perspectives of prompt input, retrieval, response generation, data storage, and operational management.
The proposed framework consists of five validation domains: Prompt Input Validation, RAG and Vector Database Validation, Response Generation and Output Validation, Data Storage and Retention Validation, and Insider Activity and Operational Validation. In addition, attack scenario-based test cases and a quantitative assessment model using Attack Success Rate （ASR）, Data Sensitivity（DS）, Exposure Level （EL）, and Control Level （CL） are presented to evaluate data leakage risks objectively.
The results indicate that data leakage risks remain present even in on-premises LLM environments and cannot be addressed solely
through network separation. The proposed framework provides a practical approach for identifying, validating, and managing data leakage risks throughout the LLM lifecycle. It is expected to support secure and compliant deployment of on-premises LLMs in financial institutions.

목차 (Table of Contents)

• 제1장 서 론 ··········································································································1
• 제1절 연구의 배경 및 목적 ·········································································1
• 제2절 연구의 범위 및 방법 ·········································································3
• 1. 연구의 범위 ·····························································································3
• 2. 연구의 방법 ·····························································································4
• 제2장 망분리 정책과 온프레미스 LLM 현황 ············································6
• 제1절 국내 금융권 망분리 정책 현황 ······················································6
• 1. 망분리 정책의 도입 배경 및 법적 근거 ·········································6
• 2. 생성형 AI 도입에 따른 망분리 규제의 변화 추이 ····················· 7
• 제2절 국내 금융권 온프레미스 LLM의 도입 현황 ······························8
• 1. 온프레미스 LLM 도입 개요 ·······························································8
• 2. 온프레미스 LLM과 상용 SaaS AI 운영 방식 비교 ···················9
• 3. 온프레미스 LLM 도입 현황 ····························································11
• 제3장 온프레미스 LLM의 데이터 유출 위험 ·········································· 15
• 제1절 온프레미스 LLM 데이터 처리 구조 및 유출 가능 지점 ····· 15
• 제2절 온프레미스 LLM 데이터 유출 위협 유형 분석 ······················24
• 1. 사용자 행위 기반 데이터 유출 （User-driven Leakage） ······· 25
• 2. 프롬프트 조작 기반 데이터 유출 （Prompt Exploitation） ····· 25
• 3. 모델 추론 및 정보 재구성 기반 유출 （Inference Leakage） · 26
• 4. 생성 결과 기반 데이터 유출 （Output Leakage） ······················ 27
• 5. 데이터 축적 기반 데이터 유출 （Persistence Leakage） ········· 27
• 제3절 기존 프레임워크의 한계 ································································ 29
• 1. 기존 프레임워크의 기능적 특성과 적용 범위 ···························· 29
• 2. 데이터 유출 검증 관점의 구조적 한계 ·········································33
• 제4장 온프레미스 LLM 데이터 유출 위험 검증 프레임워크 ············· 43
• 제1절 프레임워크 설계 기준 및 재구성 방향 ····································· 43
• 제2절 온프레미스 LLM 데이터 유출 검증 프레임워크 구성 ········· 45
• 1. 프롬프트 입력 검증 영역 ··································································46
• 2. RAG·벡터DB 검색 검증 영역 ·························································47
• 3. 응답 생성 및 출력 검증 영역 ··························································49
• 4. 데이터 저장·순환 검증 영역 ····························································50
• 5. 내부자 행위 및 망분리 운영 검증 영역 ·······································51
• 제3절 공격 시나리오 기반 검증 항목 구체화 ····································· 52
• 제4절 정량적 평가 모델 및 적용 방안 ·············································55
• 1. 평가 지표 정의 ·····················································································56
• 2. 위험 점수 산정 방식 ··········································································57
• 3. 영역별 평가 방식 ················································································58
• 4. 위험 등급 기준 ·····················································································59
• 5. 프레임워크 적용 절차 ········································································59
• 제5장 결 론 ········································································································62
• 참 고 문 헌 ··································································································64
• ABSTRACT ······································································································67