파노라마방사선영상에서 다중작업 딥러닝을 이용한 성별 및 연령 추정에 관한 연구 = Automatic estimation of sex and chronological age from panoramic radiographs using a multi-task deep learning network

국문 초록 (Abstract)

1. 목 적
성별 및 연령 예측은 법의학 수사와 개인의 식별 시 중요한 역할을 한다. 예전부터 성별 및 연령 예측을 위한 수동적인 방법들이 제안되었지만, 이러한 수동적 방법은 노동 집약적이고, 시간이 오래 소요되며, 오류의 발생 가능성이 높다. 본 연구의 목적은 ForensicNet이라는 다중작업 딥러닝 네트워크를 활용하여 파노라마방사선영상에서 성별과 연령을 동시에 자동으로 예측하는 것이다.

2. 연구대상 및 방법
ForensicNet은 백본(Backbone)과 성별 및 연령 어텐션 브랜치(Attention Branch)로 구성되어 있어, 파노라마방사선영상에서 성별 및 연령 관련 해부학적 특징을 스스로 학습하여 성별과 연령의 다중작업 예측을 가능하게 한다. 데이터 분포의 편향적 영향을 최소화하기 위해 15~80세 연령대의 각 성별에 대해 100장의 파노라마방사선영상 총 13,200장의 이미지로 구성된 데이터세트를 구축하였다. 제안된 네트워크인 ForensicNet의 구조는 백본과 성별 및 연령 어텐션 브랜치로 구성되었다. ForensicNet의 백본으로는 VGG16, MobileNet v2, ResNet101, DenseNet121, Vision Transformer, Swin Transformer, TransUNet, 그리고 EfficientNet-B3의 특징 추출 네트워크가 사용되었다. 성별 및 연령에 대한 예측 성능을 평가하기 위해 평균절대오차, 결정계수, 최대편차, 예측성공률, 민감도, 특이도, 정확도를 평가하였다.

3. 결 과
EfficientNet-B3를 적용한 ForensicNet의 예측 성능 결과 연령 예측에 대한 평균절대오차는 2.93±2.61세로 나타났으며, 결정계수가 0.957인 높은 연령 예측 성능을 보여주었다. 또한 성별 예측에 대해서는 정확도, 특이도 및 민감도가 각각 0.992, 0.993 및 0.990으로 나타났다. 이 네트워크에서 제안된 성별 및 연령 어텐션 브랜치와 컨볼루션 블록 어텐션 모듈(Convolutional Block Attention Module)은 특히 50세 이상 고령의 파노라마방사선영상에서 성별과 연령의 예측 성능을 크게 향상시키는 것으로 나타났다.

4. 결 론
본 연구에서 제안한 ForensicNet은 파노라마방사선영상에서 성별 및 나이를 자동으로 예측하고, 개인 식별을 위한 법의학 수사 및 연구에 기여할 수 있을 것으로 기대한다.

주요어 : 성별 추정, 연령 추정, 파노라마영상, 딥러닝, 다중작업 러닝

다국어 초록 (Multilingual Abstract)

1. Objective
Estimating sex and age is critical in forensic investigations and personal identification. While traditional methods for sex and age estimation have been proposed in the past, these manual approaches are labor-intensive, time-consuming, and prone to errors. The objective of this study is to automatically estimate sex and chronological age simultaneously from panoramic images using a multi-task deep learning network called ForensicNet.

2. Material & Methods
ForensicNet consists of a backbone and sex and age attention branches, enabling it to autonomously learn anatomical characteristics related to sex and chronological age from panoramic
radiographs and estimate multiple tasks related to sex and age. To mitigate the biased impact of data distribution, a dataset was created with 13,200 images, using 100 panoramic images per sex across the age range of 15 to 80 years.
The proposed ForensicNet architecture comprises a backbone incorporating feature extraction networks such as VGG16, MobileNet v2, ResNet101, DenseNet121, Vision Transformer, Swin
Transformer, TransUNet, and EfficientNet-B3, along with sex and age attention branches.
The performance of ForensicNet in sex and age estimation was evaluated using metrics including Mean Absolute Error (MAE), Coefficient of Determination (R2), Maximum Deviation (MD), Success Prediction Rate (SER), Sensitivity (SEN), Specificity (SPE), and Accuracy (ACC).

3. Results
ForensicNet using EfficientNet-B3 demonstrated a mean absolute error of 2.93 ± 2.61 years for age estimation, with an excellent coefficient of determination of 0.957, indicating outstanding age prediction performance. For sex estimation, the network achieved accuracy, specificity, and sensitivity values of 0.992, 0.993, and 0.990, respectively. The proposed sex and age
attention branches along with the convolutional block attention module notably improved the estimation performance of sex and age in panoramic images, particularly for elderly patients.

4. Conclusion
ForensicNet, proposed in this study, automatically estimates sex and chronological age from panoramic radiographs, which is expected to contribute to forensic investigations and research for personal identification.

Keywords: Sex Estimation, Age Estimation, Panoramic Radiographs, Deep Learning, Multi-Task Learning

목차 (Table of Contents)

• 제 1 장 서 론 1
• 제 2 장 연구대상 및 방법 6
• A. 연구대상 선택과 데이터 준비 6
• B. 제안된 다중작업 딥러닝 네트워크 9
• C. 가중 다중작업 손실함수 14
• D. 학습 훈련 환경 15
• E. 평가 메트릭스 16
• 제 3 장 결 과 18
• 제 4 장 고 찰 37
• 제 5 장 결 론 50
• 참고문헌 52
• Abstract (in English) 59

참고문헌 (Reference)

1. Age determinations on teeth, Gustafson G., 41:45-54, , 1950

2. Learning from imbalanced data, He H, Garcia EA, 21:1263-84, , 2009

3. Postmortem dental radiography, Chaudhry J, Turner EW, Blankenship JA, Mincer HH, 53:405-7, , 2008

4. A survey on vision transformer, Liu Z et al, Chen H, Chen X, Han K, Guo J, Wang Y, 45:87-110, , 2022

5. The development of permanent teeth, Nolla CM, University of Michigan Ann Arbor, , 1952

6. Densely connected convolutional networks, Liu, Z., Huang, G., Van Der Maaten, L., Weinberger, K. Q., Proc IEEE Int Conf Comput Vis4700–8, , 2017

7. Cbam: Convolutional block attention module, Park, J., Kweon, I. S., Woo, S., Lee, J. Y., Proc European Conf Comput Vis3–19, , 2018

8. Deep residual learning for image recognition, Ren, S., Zhang, X., He, K., Sun, J., Proc IEEE Int Conf Comput Vis770–8, , 2016

9. Ageing dental caries and periodontal diseases, Schwendicke F., Göstemeyer G, López R, Smith PC, 44:S145-S152, , 2017

10. Age estimation of adults from dental radiographs, Kvaal SI, Kolltveit KM, Thomsen IO, Solheim T., 74:175-85, , 1995

11. Sexual dimorphism in modern human permanent teeth, Dean MC, Schwartz GT, Am J Phys Anthropol: The Official Publication of the American Association of Physical Anthropologists128:312-7, , 2005

12. Albumentationsfast and flexible image augmentations, Kalinin AA, Iglovikov VI, Druzhinin M, Buslaev A, Parinov A, Khvedchenya E, Information11:125, , 2020

13. Forensic radiologyAn emerging tool in identification, Athota A, Kumar R, Karumuri SK, Rastogi T, 27:416-22, , 2015

14. An overview of deep learning in the field of dentistry, Hwang JJ, Jung YH, Cho BH, Heo MS., 49:1-7, , 2019

15. Age And Sex Determination Using CT scan Nasal Bone Imaging, Barai J, Dudhbade S, Luharia A, Tivaskar S, 13:1085-9, , 2022

16. New systems for dental maturity based on seven and four teeth, Goldstein H., Demirjian A, 3:411-21, , 1976

17. Dentomaxillofacial imaging with panoramic views and cone beam CT, Suomalainen A, Pakbaznejad Esmaeili E, Robinson S., 6:1-16, , 2015

18. Age estimation in children by measurement of open apices in teeth, Cameriere R, Ferrante L, Cingolani M., 120:49-52, , 2006

19. A review of the most commonly used dental age estimation techniques, Willems G., 19:9- 17, , 2001

20. Factors associated with tooth loss in older adults in rural Colorado, Puma J, Tiwari T, Scarbro S, Bryant LL, 41:476-81, , 2016

21. Deep neural networks for chronological age estimation from OPG images, Balsa-Castro C, Tomas I, Vila-Blanco N, Varas-Quintana P, Carreira MJ, 39:2374-84, , 2020

22. Swin transformerHierarchical vision transformer using shifted windows, Cao Y, Wei Y, Zhang Z et al, Liu Z, Hu H, Lin Y, Proc IEEE Int Conf Comput Vis10012–22, , 2021

23. EfficientnetRethinking model scaling for convolutional neural networks, Le Q., Tan M, Int J Mach Learn Comput PMLR6105–14, , 2019

24. Age estimation in adults by dental imaging assessment systematic review, Marroquin T, Karkhanis S, Kvaal S, Vasudavan S, Kruger E, Tennant M., 275:203- 11, , 2017

25. TransuneTransformers make strong encoders for medical image segmentation, Adeli E, Luo X, Lu Y, Chen J, Wang Y et al, Yu Q, arXiv preprint arXiv:2102.04306, , 2021

26. Mesiodistal crown dimension of the permanent dentition of American Negroes, Malhotra SK, Richardson ER, 68:157-64, , 1975

27. 37 An image is worth 16x16 wordsTransformers for image recognition at scale, Unterthiner T et al, Weissenborn D, Kolesnikov A, Dosovitskiy A, Zhai X, Beyer L, arXiv preprint arXiv:2010.11929, , 2020

28. Age estimation from structural changes of teeth and buccal alveolar bone level, Tan J, Mutalik S, Koh K, Asif MK, Ibrahim N, Nambiar P, 48:15-21, , 2017

29. Epigenetic influences may explain dental differences in monozygotic twin pairs, Townsend G, Schwerdt W., Richards L, Hughes T, Pinkerton S, 50:95-100, , 2005

30. Grad-camVisual explanations from deep networks via gradient-based localization, Cogswell M, Selvaraju RR, Das A, Batra D., Parikh D, Vedantam R, Proc IEEE Int Conf Comput Vis618–26, , 2017

31. A study of various forms of mandibular coronoid process in determination of sex, Jayanthi K., Bajpe R, Subbaramaiah M, Jagannatha S, Indian J Clin Anat Physiol2:199-203, , 2015

32. MobilenetsEfficient convolutional neural networks for mobile vision applications, Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T et al, arXiv preprint arXiv:1704.04861, , 2017

33. Age and gender prediction from face images using attentional convolutional network, Abdolrashidi A Minaei M, Azimi E, Minaee S., arXiv preprint arXiv:2010.03791, , 2020

34. STRBasea short tandem repeat DNA database for the human identity testing community, Reeder DJ, Ruitberg CM, Butler JM, 29:320-2, , 2001

35. Sexual dimorphism of the human mandible and its association with dental development, Halazonetis DJ, Mitteroecker P., Bookstein FL, Weber GW, Coquerelle M, Braga J, Am J Phys Anthropol145:192-202, , 2011

36. Clinical image quality evaluation for panoramic radiography in Korean dental clinics, Huh KH, Yi WJ, Heo MS, Choi SC et al, Choi BR, Choi DH, 42:183-90, , 2012

37. Why 70/30 or 80/20 relation between training and testing setsA pedagogical explanation, Kreinovich V, Kosheleva O, Gholamy A, Int J Intell Technol Appl Stat11:105-11, , 2018

38. 17 Sexual dimorphism in deciduous tooth crown sizeVariability within and between groups, Lukacs JR, 34:e23793, , 2022

39. Forensic age estimation in human skeletal remainscurrent concepts and future directions, Franklin D., Legal Medicine12:1-7, , 2010

40. Semi-supervised automatic dental age and sex estimation using a hybrid transformer model, Ke W, Liu Y, Dai X, Shi L, Fan F, Lin Y et al, 137:721-31, , 2023

41. Age estimation with panoramic radiomorphometric parameters using generalized linear models, Lee YH, An JS, J Oral Med Pain46:21-32, , 2021

42. A look at forensic dentistry–Part 1The role of teeth in the determination of human identity, Pretty I, Sweet D., Br Dent J190:359-66, , 2001

43. Assessment of panoramic radiography as a national oral examination toolreview of the literature, Choi JW, 41:1-6, , 2011

44. Sexual dimorphism in the mandible of indigenous South AfricansA geometric morphometric approach, Franklin D, Oxnard C., O'Higgins P, 104:101-6, , 2008

45. An uncertainty-aware and sex-prior guided biological age estimation from orthopantomogram images, Du S, Guo Yc, Bu W, Zhang D, Yang J, 2023, , 2023

46. Automated estimation of chronological age from panoramic dental X-ray images using deep learning, Milošević D, Vodanović M, Galić I, Subašić M., Expert Systems with Applications189:116038, , 2022

47. Human age estimation from blood using mRNA DNA methylation DNA rearrangement and telomere length, Ijcken W et al, Zubakov D, Kokmeijer I, Choi Y, Meurs J, Liu F, 24:33–43, , 2016

48. Automatic sex estimation using deep convolutional neural network based on orthopantomogram images, Bu Wq, Guo Yx, Zhang D Du Sy, Han Mq, Wu Zx et al, 348:111704, , 2023

49. Age estimation based on pulp chamber volume of first molars from cone-beam computed tomography images, Ge Zp, Ma Rh, Li G, Zhang Jz, Ma Xc, 253:133. e1-e7, , 2015

50. A pilot study of an automated personal identification processapplying machine learning to panoramic radiographs, Ortiz AG, Michel CE, Biazevic MGH, Soares GH, da Rosa GC, 51:187, , 2021

51. Computed tomography based forensic gender determination by measuring the size and volume of the maxillary sinuses, Prabhat K, Prabhat M, Bhatnagar P, Panjwani S., Kaur M, Rai S, 8:40, , 2016

52. Assessment of maturation stages and the accuracy of age estimation methods in a Turkish populationA comparative study, Magat G, Ozcan S., 52:83, , 2022

53. Accurate age classification using manual method and deep convolutional neural network based on orthopantomogram images, Han M, Long H, Guo YC, Chi Y, Zhang D, Yang J et al, 135:1589-97, , 2021

54. Sex determination from lateral cephalometric radiographs using an automated deep learning convolutional neural network, Mollabashi V, Khazaei M, Farhadian M., Khotanlou H, 52:239, , 2022

55. Evaluation of the clinical efficacy of a TW3-based fully automated bone age assessment system using deep neural networks, Lee BI et al, Shin NY, Kim HR, Oh DH, Lee BD, Kang JH, 50:237, , 2020

56. Assessment of paranasal sinus parameters according to ancient skullsgender and age by using cone-beam computed tomography, Demiralp K, Aksoy S, Demir P., Bayrak S, Orhan K, Cakmak SK, 78:344-50, , 2019

57. Volume measurement of mandibular teeth pulp chamber as a prediction tool of gender and ethnicity in a Brazilian population, Gomes JP, Fardim KAC, Rodrigues R, Araújo E, Junior EO, Costa A et al, 24:6-P, , 2021

58. Dental age estimation utilizing third molar developmentA review of principles methods and population studies used in the United States, Senn DR, Lewis JM, 201:79-83, , 2010

59. The World Oral Health Report 2003continuous improvement of oral health in the 21st century–the approach of the WHO Global Oral Health Programme, Petersen PE, Community Dent Oral Epidemiol31:3-24, , 2003