Toward Multimodal Electronic Health Record Foundation Models = 멀티모달 전자건강기록 파운데이션 모델에 관한 연구

국문 초록 (Abstract)

전자건강기록(Electronic health record, EHR) 파운데이션 모델(Foundation model, FM)은 순차적 의료 기록으로부터 포괄적인 임상 맥락을 학습함으로써 임상 과제 수행 능력을 향상시켜 왔다. 그러나 여전히 몇 가지 중요한 한계점이 존재한다. 첫째, 대부분의 현재 전자건강기록 파운데이션 모델은 의료 코딩 시스템의 차이로 인해 호환성이 부족하다. 둘째, 멀티모달 접근법이 등장했음에도 불구하고, 기존 방법들은 각 모달리티에 대해 표현형을 별도로 생성하고 최종 예측 단계에서만 이를 통합하여, 데이터 유형 간 시간 관계를 포착하지 못한다. 본 학위논문에서는 네 가지 핵심 기여를 통해 이러한 한계점을 해결한다. 첫째, OMOP(Observational Medical Outcomes Partnership) 공통 데이터 모델(Common Data Model, CDM) 기반 전자건강기록 데이터에 특화된 전자건강기록 파운데이션 모델인 CDM-BERT를 제안한다. 둘째, 실제 환경에서 CDM-BERT의 임상 검증 연구를 제시한다. 셋째, 일반 EHR FM을 위해 대규모 언어 모델이 생성한 개념 설명의 텍스트 정보와 OMOP 그래프 온톨로지의 관계 정보를 통합하여 생성한 표현형인 MedRep을 소개한다. 마지막으로, 전자건강기록 파운데이션 모델이 시간적 맥락을 보존하면서 정형 데이터와 비정형 데이터를 동시에 분석할 수 있게 하는 새로운 멀티모달 접근법인 TRIM을 제안한다. CDM-BERT, MedRep, TRIM은 전자건강기록 파운데이션 모델의 현재 한계점을 해결하고 다양한 임상 과제에서 성능, 일반화 가능성, 해석 가능성을 향상시킨다.

다국어 초록 (Multilingual Abstract)

Electronic health record (EHR) foundation models (FMs) have improved clinical task performance by learning comprehensive clinical context from sequential medical records. However, several critical limitations remain: First, most current EHR FMs lack compatibility due to disparities in medical coding systems. Second, while multimodal approaches have emerged, existing methods generate representations separately for each modality and integrate them only at the final prediction stage, without capturing temporal relationships across data types. This dissertation addresses these limitations through four key contributions. First, this dissertation proposes CDM-BERT, an EHR FM specialized for observational medical outcomes partnership (OMOP) common data model (CDM)-based EHR data. Second, this dissertation presents a clinical implementation study demonstrating the effectiveness of CDM-BERT in practical clinical settings. Third, this dissertation introduces MedRep, medical concept representations for general EHR FMs that integrates textual information from concept descriptions generated by large language models with relational information from the graph ontology of OMOP vocabulary. Finally, this dissertation proposes TRIM (Text-based Representation for Integrated Multimodal patient trajectories), a novel multimodal approach that enables EHR FMs to simultaneously analyze structured and unstructured longitudinal data while preserving temporal context. Together, CDM-BERT, MedRep and TRIM address current limitations of EHR FMs and enhance performance, generalizability, and interpretability across diverse clinical tasks.

목차 (Table of Contents)

• Abstract i
• Contents ii
• List of Figures vi
• List of Tables viii
• Chapter 1 Introduction 1
• 1.1. OMOP CDM 4
• 1.2. Attention mechanism and Transformer 6
• 1.3. Current EHR FMs 8
• 1.4. Multimodal AI approaches for EHR 12
• 1.5. Overview of the dissertation 14
• Chapter 2 Improving Performance on Clinical Tasks using EHR FMs Specialized for OMOP CDM 17
• 2.1. Introduction 19
• 2.2. Methods 23
• 2.2.1. Data curation 23
• 2.2.2. ADE prediction task 23
• 2.2.3. Record tokenization 24
• 2.2.4. CDM-BERT and domain embedding 25
• 2.2.5. Trajectory construction and embedding 26
• 2.2.6. Pretraining process 26
• 2.2.7. Finetuning process 27
• 2.2.8. Feature importance 27
• 2.2.9. Statistical analysis 28
• 2.2.10. Training and computing informatition 29
• 2.3. Results 30
• 2.3.1. Study population 30
• 2.3.2. Pretraining results 31
• 2.3.3. Performance evaluation 31
• 2.3.4. Analysis of different prediction timepoints 34
• 2.3.5. Model interpretation 35
• 2.4. Discussion 40
• Chapter 3 Clinical Validation of EHR FMs: Cutaneous Adverse Drug Reaction Prediction 45
• 3.1. Introduction 47
• 3.2. Methods 49
• 3.2.1. Data curation 49
• 3.2.2. Nursing statements and reports 50
• 3.2.3. Prediction timepoint 50
• 3.2.4. Model training 51
• 3.2.5. Feature importance 53
• 3.2.6. Statistical analysis 54
• 3.3. Results 55
• 3.3.1. Study population 55
• 3.3.2. Prediction performance 56
• 3.3.3. Feature importance 59
• 3.3.4. Sub-analysis of immediate and delayed CADR types 61
• 3.4. Discussion 67
• Chapter 4 Improving Generalizability of EHR FMs through Medical Concept Representation Learning 73
• 4.1. Introduction 74
• 4.2. Methods 77
• 4.2.1. Datasets and outcome labeling 77
• 4.2.2. Concept curation 78
• 4.2.3. Trajectory construction 79
• 4.2.4. Medical concept representation (MedRep) 79
• 4.2.5. Text-based representation 80
• 4.2.6. Complementing relational information 82
• 4.2.7. Experimental settings 83
• 4.2.8. Pretraining and finetuning details 84
• 4.2.9. MedTok training 85
• 4.2.10. Qualitative analysis 85
• 4.3. Results 87
• 4.3.1. Baseline characteristics 87
• 4.3.2. Downstream task performance 87
• 4.3.3. Impact of graph ontology 91
• 4.3.4. Qualitative analysis 91
• 4.3.5. Scalability of MedRep 92
• 4.4. Discussion 95
• Chapter 5 Multimodal Approach for EHR FMs: Using Text-based Representations for Integrated Multimodal Patient Trajectories 100
• 5.1. Introduction 101
• 5.2. Methods 103
• 5.2.1. Data curation 103
• 5.2.2. Data preprocessing and outcomes 105
• 5.2.3. Multimodal EHR FM framework to predict CVDs 105
• 5.2.4. Generating descriptions for ECG diagnoses 107
• 5.2.5. Model and training details 108
• 5.2.6. Model interpretation 109
• 5.2.7. Statistical analysis 109
• 5.3. Results 111
• 5.3.1. Baseline characteristics 111
• 5.3.2. TRIM representation analysis 112
• 5.3.3. CVD prediction performance evaluation 114
• 5.3.4. Model interpretation of the MIMIC-IV dataset 115
• 5.3.5. Model interpretation of the SNUH dataset 118
• 5.3.6. Survival analysis for model decision validation 121
• 5.4. Discussion 124
• Chapter 6 Conclusion 129
• 6.1. Dissertation summary 129
• 6.2. Limitations 131
• 6.2.1. Clinical implementation 131
• 6.2.2. Collaboration across multiple institutions 132
• 6.2.3. Difficulties in interpretation 132
• 6.3. Future work 133
• 6.3.1. Discovering new biomarkers and phenotypes 133
• 6.3.2. Merging EHR FMs from multiple institutions 133
• 6.3.3. Generating human-understandable explanations 134
• Bibliography 135
• Appendices 152
• A. Supplement for Chapter 2 152
• B. Supplement for Chapter 3 161
• C. Supplement for Chapter 4 168
• D. Supplement for Chapter 5 173
• D.1. Detailed investigation of the causal and pathophysiologic relationship between the top 20 EHR concepts with the highest average attention scores and target CVDs (MIMIC-IV) 173
• D.2. Detailed investigation of the causal and pathophysiologic relationship between the top 20 EHR concepts with the highest average attention scores and target CVDs (SNUH) 178