딥러닝 생성 모델 기반 H&E?IHC 가상 염색 및 강건한 종양 침윤 림프구 정량화 = Enhancing Translation of H&E to IHC with Robust Tumor- Infiltrating Lymphocytes Quantification using Deep Generative Models

다국어 초록 (Multilingual Abstract)

Breast cancer remains the leading cause of oncological mortality in the female population, making treatment response prediction critical in advanced stages. Tumor-infiltrating lymphocytes (TILs) are a critical prognostic biomarker. However, standard visual quantification of TILs on hematoxylin and eosin (H&E) slides in clinical practice suffers from substantial interobserver variability. While immunohistochemistry (IHC) provides an objective ground truth, it is labor-intensive and costly; H&E- to-IHC virtual stain translation offers a promising solution. In this study, we developed a deep generative framework to synthesize virtual IHC patches from H&E patches and create an automated pipeline for accurate TIL quantification, leveraging IHC as a robust ground truth. To ensure precise H&E–IHC image alignment, H&E slides were destained and subsequently restained with IHC on the same slide, followed by a coarse to fine registration process. To enhance the model for accurate synthesis of TIL-specific chromogen, our framework incorporated two key components: (1) a spatial attention mechanism guided by a cell segmentation map, leveraging the characteristic small size of TILs to direct synthesis specifically toward them, and (2) a chromogen loss function based on color separation to guide accurate diaminobenzidine (DAB) or alkaline phosphatase (AP) synthesis. Furthermore, we developed a novel TIL quantification pipeline capable of accurately counting cells even in overlapping regions. Experimental results demonstrate that integrating the spatial attention mechanism and chromogen loss significantly reduced the Mean Absolute Error (MAE) of TIL quantification per patch. Specifically, the MAE decreased from 12.793 to 9.480 for DAB and from 21.113 to 17.755 for AP (p < 0.001 for both) in the best-performing model. In our reader study, no statistically significant difference was observed between the automated pipeline’s quantification and manual assessments (p = 0.314). Moreover, visual scoring confirmed that the synthesized TIL-specific chromogens were comparable to those of ground truth IHC patches (p = 0.640).

목차 (Table of Contents)

• Abstract ⅰ
• Contents ⅱ
• Contents of Tables ⅲ
• Contents of Figures ⅳ
• Introduction 1
• Materials and Methods 3
• Datasets 3
• Preprocessing 3
• Improving Image-to-Image Translation 6
• Quantitative Evaluations 8
• Reader Study 9
• Statistical Analysis 10
• Experiments and Results 10
• Mutual Information after Registration 10
• Structural Similarity Index (SSIM) 13
• Tumor-infiltrating lymphocyte (TIL) counts 13
• Reader Study 17
• Discussion 18
• Conclusion 20
• References 21
• Abstract (with Korean) 25
• Contents of Tables
• Table 1. Summary of dataset characteristics 5
• Table 2. Computational time required for model training and inference 8
• Table 3. Structural Similarity Index (SSIM) between synthesized and ground truth IHC patches 13
• Table 4. Mean difference and 95% limits of agreement for the Bland–Altman plots in Figure 7 16
• Contents of Figures
• Figure 1. Overview of the two-stage registration and patch extraction pipeline 4
• Figure 2. Quality control procedure 5
• Figure 3. Two typical H&E patches with corresponding cell maps, filtered cell maps (<100 pixels), and ground truth IHC patches 6
• Figure 4. Illustration of chromogen loss between the synthesized and ground truth IHC patches 7
• Figure 5. Overview of the TIL quantification workflow 9
• Figure 6. Distribution of mutual information (MI) and representative H&E–IHC patch pairs for each chromogen 11
• Figure 7. Representative H&E–IHC patch pairs with mutual information (MI) values in the 0.1–0.15 range 12
• Figure 8. Bland–Altman plots showing the agreement between TIL counts from synthesized and ground truth IHC 15
• Figure 9. Visual inspection of outliers identified in the Bland–Altman analysis 17
• Figure 10. Scatter plot comparing manual and automated TIL counts 18