Variational Bayes for high-dimensional sparse Cholesky factors

다국어 초록 (Multilingual Abstract)

Understanding the dependence structure in high-dimensional multivariate data is critical for many scientific fields. In Gaussian graphical models, this dependence structure is captured by the precision matrix, whose estimation becomes challenging due to computational complexity and the curse of dimensionality. To overcome these challenges, we propose a scalable variational Bayes approach for estimating sparse precision matrices based on the modified Cholesky decomposition. Our approach employs a spike-and-slab prior, effectively promoting sparsity in the Cholesky factor, and leverages a coordinate ascent variational inference algorithm.
Extensive simulation studies demonstrate that the proposed method achieves comparable or superior accuracy in variable selection and matrix estimation, while significantly outperforming existing Bayesian methods based on Markov chain Monte Carlo in computational efficiency. Furthermore, an application to a high-dimensional gene expression dataset, involving Alzheimer's and Huntington's diseases, illustrates the practical utility of our method in identifying biologically relevant gene networks efficiently.

국문 초록 (Abstract)

고차원 다변량 자료의 의존 구조를 이해하는 일은 다양한 과학 분야에서 핵심적인 과제이다. 가우시안 그래픽 모형(Gaussian graphical model)에서는 이러한 의존 구조가 정밀도 행렬(precision matrix)에 의해 표현되는데, 차원의 저주(curse of dimensionality)와 높은 계산 복잡도 때문에 이 정밀도 행렬의 추정은 매우 어려운 문제로 변한다. 이러한 난점을 극복하기 위해, 본 논문에서는 수정 콜레스키 분해(modified Cholesky decomposition)에 기반하여 희소 정밀도 행렬을 추정하는 확장 가능한 변분 베이즈(variational Bayes) 방법을 제안한다. 제안하는 방법은 콜레스키 인자에 대한 희소성을 효과적으로 유도하기 위해 스파이크-앤드-슬랩(spike-and-slab) 사전분포를 사용하며, 좌표상승 변분추론(coordinate ascent variational inference) 알고리즘을 활용하여 근사 사후분포를 계산한다.
제안 방법의 성능을 검증하기 위한 모의실험 결과, 제안된 방법은 변수 선택 및 행렬 추정 정확도 측면에서 기존 방법들과 동등하거나 더 우수한 성능을 보이는 한편, 마르코프 연쇄 몬테카를로(Markov chain Monte Carlo, MCMC)에 기반한 기존 베이즈 방법들에 비해 계산 효율성에서 현저한 우위를 나타낸다. 더 나아가 알츠하이머병과 헌팅턴병을 포함하는 고차원 유전자 발현 데이터셋에의 적용을 통해, 제안된 방법이 생물학적 관련성이 높은 유전자 네트워크를 효율적으로 식별하는 데 실질적으로 유용한 도구임을 보여준다.

목차 (Table of Contents)

• Chapter 1. Motivation and Background 1
• 1. Introduction 1
• 1.1 Contributions 4
• 1.2 Organization 5
• Chapter 2. Literature Review 6
• 1. Direct Estimation of Precision Matrices 6
• 2. Estimation via Modified Cholesky Decomposition 10
• 2.1 Related Work on Estimation via the MCD Parameterization 11
• 2.2 The Challenge of Ordering Dependence 13
• 3. Research Gap 15
• Chapter 3. Methodological Foundations of Variational Inference 17
• 1. Foundations 17
• 1.1 The computational challenge of Bayesian inference 17
• 1.2 Variational inference as an optimization problem 19
• 1.3 The evidence lower bound (ELBO) 20
• 1.4 Mean-field variational inference and CAVI 22
• 1.5 Theoretical guarantees of variational approximations 25
• 2. Spike-and-Slab Priors for Variable Selection 27
• 2.1 The spike-and-slab framework as a hierarchical mixture model 28
• 2.2 Conceptual foundations: discrete versus continuous formulations 29
• 2.3 Role of the slab: Gaussian versus heavier-tailed priors 30
• 2.4 Prior specification for model parameters and model space 32
• 2.5 Theoretical guarantees: consistency and optimal contraction 34
• Chapter 4. Proposed Method 36
• 1. Introduction and Model Specification 36
• 1.1 Introduction 36
• 1.2 Model specification 36
• 2. Hierarchical priors and posterior formulation 39
• 3. Mean-field variational approximation 42
• 3.1 Rationale for the Laplace Slab Specification 45
• 4. CAVI: Derivation of the Update Equations 48
• 4.1 Update for Error Variance Parameters 48
• 4.2 Update for Cholesky Factor Parameter 50
• 4.3 Update for Inclusion Probability 51
• 5. Algorithm Summary and Implementation 52
• 5.1 Algorithm summary 52
• 5.2 Implementation Details 55
• Chapter 5. Empirical Studies 57
• 1. Introduction 57
• 2. Simulation studies 58
• 2.1 Comparison methods 59
• 2.2 Simulation setup 61
• 2.3 Evaluation metrics 63
• 2.4 Simulation results- Study I 66
• 2.5 Simulation results- Study II 69
• 3. Real-data study 78
• 3.1 GSE33000 dataset and preprocessing 78
• 3.2 Network analysis 79
• Chapter 6. Conclusion 84
• 1. Summary of Contributions 84
• 2. Limitations and Future Directions 86
• 2.1 Mean-Field Approximation and Uncertainty Quantification 86
• 2.2 Permutation Sensitivity and Ordering Uncertainty 87
• References 89
• Appendix 98