Multimodal and Longitudinal Osteoporotic Fracture Risk Prediction using DXA and Chest Radiography = DXA와 흉부X선을 활용한 멀티모달 시계열 기반 골다공증성 골절 위험도 예측

다국어 초록 (Multilingual Abstract)

Osteoporotic fractures impose a substantial clinical and societal burden, contributing to excess morbidity, mortality, disability, and healthcare costs worldwide. Despite their widespread use, conventional risk assessment tools based on dual-energy X-ray absorptiometry–derived bone mineral density and the Fracture Risk Assessment Tool provide only modest accuracy for identifying individuals at highest risk of future fracture. In particular, a large proportion of fractures occur in individuals without densitometric osteoporosis, underscoring fundamental limitations of low-dimensional risk representations. Recent advances in deep learning for survival analysis suggest that routinely acquired medical images may encode latent structural, geometric, and systemic biomarkers that can substantially improve fracture risk prediction without additional imaging, radiation exposure, or cost.
This dissertation develops a unified deep learning framework for imaging-based, time-to-event prediction of osteoporotic fractures and systematically evaluates its performance across single-modality, multimodal snapshot, and longitudinal modeling paradigms. All models are formulated within a discrete-time survival framework, enabling direct estimation of individualized fracture risk trajectories over clinically relevant time horizons.
In Chapter 2, image-only deep survival models based on chest radiography and dual-energy X-ray absorptiometry images are developed to predict incident major osteoporotic fractures. Across large, multi-institutional cohorts, both imaging modalities demonstrate superior discrimination, calibration, and risk stratification compared with bone mineral density T-scores and the Fracture Risk Assessment Tool. The chest radiography model shows that radiographs acquired for non-osteoporosis indications nonetheless encode prognostic signals of systemic skeletal fragility, enabling opportunistic population-level screening. The dual-energy X-ray absorptiometry image–based model captures microarchitectural and geometric features beyond scalar bone mineral density measurements, with particularly strong gains in osteopenic populations where conventional tools are least precise.
Chapter 3 extends this framework to multimodal snapshot models that integrate imaging features with basic clinical variables routinely available in bone densitometry workflows. Multiple fusion strategies—including feature-level concatenation, hazard-level late fusion, cross-attention–based image fusion, and universal architectures with explicit support for missing modalities—are systematically evaluated. Across all configurations, multimodal models consistently outperform image-only and tabular-only baselines, highlighting the complementary prognostic value of structural imaging and clinical risk factors.
In Chapter 4, the modeling framework is further extended to longitudinal survival analysis using irregular sequences of repeated dual-energy X-ray absorptiometry examinations and time-varying clinical information. By encoding visit-level multimodal observations with recurrent and transformer-based temporal architectures, the longitudinal models capture dynamic changes in skeletal health. These models substantially outperform snapshot approaches, demonstrating that fracture risk is inherently time-dependent and more accurately estimated through explicit temporal modeling.
Chapter 5 evaluates the external clinical validity of the proposed approach using independently collected chest radiography and dual-energy X-ray absorptiometry datasets from separate institutions. Image-only models maintain strong discrimination, robust calibration, and clinically interpretable subgroup performance despite substantial domain shifts in patient populations and imaging characteristics, supporting their generalizability and real-world applicability.
Collectively, this dissertation establishes a principled and generalizable deep survival learning framework for opportunistic fracture risk prediction from routinely acquired medical images. The findings support the integration of imaging-derived prognostic signals into routine osteoporosis care and provide a methodological foundation for future multimodal, longitudinal, and cross-population research aimed at precision fracture prevention.

국문 초록 (Abstract)

골다공증성 골절은 고령 인구에서 사망률을 증가시키고 기능 저하, 장기 요양, 의료비 부담을 초래하는 대표적인 공중보건 문제이다. 현재 임상에서 널리 사용되는 dual-energy X-ray absorptiometry 기반 골밀도(bone mineral density, BMD)와 골절 위험 평가 도구(Fracture Risk Assessment Tool, FRAX)는 근거 기반의 표준 도구로 자리 잡고 있으나, 실제로는 상당수의 골절이 진단적 골다공증 기준에 해당하지 않는 집단에서 발생한다는 한계를 지닌다. 이는 골절 취약성이 단일 수치로 요약되기 어려운 현상임을 나타낸다. 최근 딥러닝 기반 생존분석 기법의 발전은 일상 진료 과정에서 획득되는 의료영상에 내재된 구조적·기하학적·전신적 정보를 정량화하여, 추가 촬영이나 비용 부담 없이 골절 위험을 보다 정밀하게 예측할 수 있는 가능성을 제시하고 있다.
본 논문은 영상 기반 골절 위험 예측을 위한 통합적 딥러닝 생존모델링 프레임워크를 제안하고, 단일모달, 다중모달, 그리고 장기 추적 기반 모델로 확장하여 그 임상적 유효성과 일반화 가능성을 체계적으로 평가하였다. 모든 모델은 시간-사건(time-to-event) 구조를 명시적으로 반영하는 이산형 생존모형(discrete-time survival model)으로, 개인별 골절 위험 궤적을 임상적으로 의미 있는 시간 범위에서 추정하도록 설계되었다.
제2장에서는 흉부 X선 영상(chest radiography, CXR)과 DXA 영상만을 입력으로 사용하는 이미지 단독(image-only) 딥러닝 생존모델을 구축하여 주요 골다공증성 골절 발생을 예측하였다. 대규모 다기관 개발 및 검증 코호트 전반에서 두 모델은 BMD T-점수 및 FRAX 대비 우수한 예측력, 보정도, 위험도 층화 성능을 일관되게 보였다. 특히 CXR 기반 모델은 골다공증 평가를 목적으로 촬영되지 않은 영상에서도 골절 취약성과 연관된 잠재적 특징이 존재함을 입증하여, 영상 기회검진(opportunistic screening)의 가능성을 제시하였다. DXA 영상 기반 모델은 T-점수로는 포착되지 않는 미세구조적 및 기하학적 특성을 학습함으로써, 진단적 불확실성이 큰 골감소증 집단에서 특히 두드러진 성능 향상을 보였다.
제3장에서는 영상 특징과 기본 임상 변수를 통합한 다중모달 스냅샷 모델을 제안하였다. 특징 수준 결합(feature concatenation), 위험도 수준 결합(late fusion), 교차 어텐션(cross-attention), 결측 모달리티에 강인한 범용(universal) 구조 등 다양한 융합 전략을 비교한 결과, 구조적 영상 정보와 역학적 위험 요인을 결합함으로써 단일모달 모델 대비 예측 성능이 지속적으로 향상됨을 확인하였다.
제4장에서는 DXA 반복 검사와 임상 변수의 불규칙한 시계열 구조를 반영하기 위해 순환 신경망과 트랜스포머 기반 시간 인코더를 도입한 다중모달 장기 추적(longitudinal) 생존모델을 개발하였다. 장기 추적 모델은 스냅샷 모델을 일관되게 능가하는 성능을 보였으며, 골밀도 변화 등 시간에 따라 변화하는 위험 요인을 효과적으로 포착함을 입증하였다.
제5장에서는 외부 기관에서 독립적으로 수집된 CXR 및 DXA 자료를 활용하여 이미지 기반 생존모델의 외부 임상 검증을 수행하였다. 인구학적 특성과 촬영 환경이 상이한 조건에서도 모델은 우수한 판별력과 안정적인 보정도를 유지하였으며, 다양한 하위집단 분석에서도 일관된 성능을 보여 높은 임상 일반화 가능성을 시사하였다.
종합하면, 본 논문은 영상 기반 딥러닝 생존모델을 단일모달, 다중모달, 그리고 장기 추적 구조로 체계적으로 확장하고 외부 검증을 통해 그 실질적 활용 가능성을 제시하였다. 본 연구의 결과는 일상적으로 획득되는 의료영상을 활용한 기회검진, 정밀 위험 층화, 그리고 시간적 변화에 기반한 골절 예방 전략 개발을 위한 방법론적 토대를 제공한다.

목차 (Table of Contents)

• Abstract i
• Contents iii
• List of tables ix
• List of figures xii
• Chapter 1. Introduction 15
• 1.1 Clinical Burden of Osteoporotic Fractures 16
• 1.2 Conventional Fracture Risk Assessment and Its Limitations 17
• 1.3 Clinical Problem and Imaging Opportunities 18
• 1.3.1 Imaging-based BMD Analysis or Osteoporosis Detection for Fracture Prediction 19
• 1.3.2 Imaging-based Fracture Detection for Future Fracture Prediction 21
• 1.3.3 Imaging-based Detection of Other Properties Related to Fracture Risk 22
• 1.3.4 Imaging-based Direct Fracture Prediction (End-to-End Models) 24
• 1.4 Time-to-Event Deep Learning for Imaging-based Risk Prediction 27
• 1.5 Research Motivation 30
• 1.6 Overall Study Design 31
• Chapter 2. Single-Modality Imaging and Clinical Models for Fracture Risk Prediction 34
• 2.1 Cohort and Global Preprocessing 34
• 2.1.1 Discrete-time survival setup 34
• 2.1.2 Tabular preprocessing 35
• 2.1.3 DXA image preprocessing 35
• 2.1.4 Chest radiograph preprocessing 36
• 2.1.5 Evaluation Metrics 36
• 2.1.6 Statistical analysis and model selection 38
• 2.2 Tabular-only Baseline Models 38
• 2.2.1 Survival Model Formulations 39
• 2.2.2 Training Configuration 42
• 2.2.3 Results of tabular-only models 43
• 2.3 Image-only Models 48
• 2.3.1 Imaging Data and Survival Model Formulation 48
• 2.3.2 Training Configuration 50
• 2.3.3 Results of image-only models 51
• 2.4 Discussion 54
• Chapter 3. Multimodal Snapshot Models for Fracture Risk Prediction 56
• 3.1 Overview and Motivation 56
• 3.2 Image-Clinical Fusion Models 58
• 3.2.1 Feature-Level Fusion Architectures 59
• 3.2.2 Late Fusion at the Hazard Level 61
• 3.2.3 Results of Image–Clinical Fusion 62
• 3.3 Multi-image Fusion of Hip and Spine DXA 64
• 3.3.1 Late Fusion 64
• 3.3.2 Feature Concatenation of Image Representations 66
• 3.3.3 Cross-attention Fusion on Feature Maps 66
• 3.3.4 Clinical-Gated Fusion 68
• 3.3.5 Results of Multi-image Fusion 69
• 3.4 CXR-augmented Multimodal Snapshot Models 70
• 3.4.1 Motivation and Cohort Construction 70
• 3.4.2 Image Preprocessing and Model Architecture 72
• 3.4.3 Model Configurations 72
• 3.4.4 Results of CXR-augmented Multimodal Models 73
• 3.5 Multimodal Models with Missing-Modality Support 74
• 3.5.1 Motivation 74
• 3.5.2 Modality Masks and Encoding Framework 74
• 3.5.3 Late Fusion for Arbitrary Modality Sets 75
• 3.5.4 Gated Fusion with Learned Modality Weights 76
• 3.5.5 Token-Based Transformer Fusion 77
• 3.5.6 Results of Multimodal Model with Missing-Modality Support 78
• 3.6 Discussion 79
• Chapter 4. Longitudinal Multimodal Models for Fracture Risk Prediction 81
• 4.1 Overview and Motivation 81
• 4.2 Temporal Modeling for Tabular-only Inputs 83
• 4.2.1 Longitudinal Tabular Structure 83
• 4.2.2 History Pooling Baseline 84
• 4.2.3 Universal GRU for Longitudinal Tabular Data 85
• 4.2.4 Universal Transformer for Temporal Tabular Modeling 86
• 4.2.5 Temporal Token Transformer 87
• 4.3 Temporal Modeling for Combined DXA and Tabular Inputs 88
• 4.3.1 Longitudinal DXA Representation and Irregular Sampling 88
• 4.3.2 Multimodal Visit-level Fusion of DXA and Tabular Features 90
• 4.3.3 Temporal Encoders for Multimodal Longitudinal Sequences 91
• 4.4 Results of Longitudinal Models 93
• 4.4.1 Tabular-only Longitudinal Models 93
• 4.4.2 Longitudinal DXA + Tabular Models 95
• 4.5 Model Selection and Comparative Analysis 97
• 4.5.1 Performance Comparison of Selected Survival Models 97
• 4.5.2 Computational Efficiency and Resource Considerations 98
• 4.6 Discussion 99
• Chapter 5. External Clinical Validation of Fracture Risk Prediction Models 102
• 5.1 Overview 102
• 5.2 Materials and Methods 103
• 5.2.1 External Validation Cohorts 103
• 5.2.2 Outcome Labeling 105
• 5.2.3 Evaluation Metrics 106
• 5.2.4 Additional Optimization of Imaging-Based Models for Clinical Analysis 106
• 5.3 External Validation of Selected Survival Models 107
• 5.3.1 Evaluation Protocol 107
• 5.3.2 Performance of Selected Models in External Dataset 108
• 5.4 External Validation of the CXR Model 112
• 5.4.1 Baseline Characteristics 112
• 5.4.2 Predictive Performance and Comparison with FRAX 119
• 5.4.3 Subgroup Analyses 124
• 5.4.4 Calibration in the External CXR Cohort 134
• 5.4.5 Risk Stratification using Kaplan–Meier Estimates 135
• 5.4.6 Decision-Level Comparison between CXR DL and FRAX 137
• 5.4.7 Grad-CAM Visualization 140
• 5.5 External Validation of the DXA Model 143
• 5.5.1 Baseline Characteristics 143
• 5.5.2 Predictive Performance and Comparison with FRAX 147
• 5.5.3 Subgroup Analyses 152
• 5.5.4 Calibration in the External DXA Cohort 163
• 5.5.5 Risk Stratification using Kaplan–Meier Estimates 164
• 5.5.6 Decision-Level Comparison between DXA DL and FRAX 166
• 5.4.7 Grad-CAM Visualization 169
• 5.5 Discussion 174
• Chapter 6. Conclusion 177
• 6.1 Overall Summary of Findings 177
• 6.2 Limitations 181
• 6.3 Future Directions 184
• Bibliography 187
• Abstract in Korean 199