Automatic Diagnosis for Odontogenic Cysts and Tumors of Jaw on Panoramic Radiographs using a Deep Convolutional Neural Network = 파노라마방사선영상에서 딥러닝 신경망을 이용한 치성 낭과 종양의 자동 진단 방법

다국어 초록 (Multilingual Abstract)

Objective: The purpose of this study was to automatically diagnose odontogenic cysts and tumors of the jaw on panoramic radiographs using a deep convolutional neural network. A novel framework method of deep convolutional neural network was proposed with data augmentation for detection and classification of the multiple diseases.

Methods: A deep convolutional neural network modified from YOLOv3 was developed for detecting and classifying odontogenic cysts and tumors of the jaw. Our dataset of 1,282 panoramic radiographs comprised 350 dentigerous cysts, 302 periapical cysts, 300 odontogenic keratocysts, 230 ameloblastomas, and 100 normal jaw with no disease. In addition, the number of radiographs was augmented 12-fold by flip, rotation, and intensity changes. The Intersection over union threshold value of 0.5 was used to obtain performance for detection and classification. The classification performance of the developed convolutional neural network was evaluated by calculating sensitivity, specificity, accuracy, and AUC (Area under the ROC curve) for diseases of the jaw.

Results: The overall classification performance for the diseases improved from 78.2% sensitivity, 93.9% specificity, 91.3% accuracy, and 0.86 AUC using the convolutional neural network with unaugmented dataset to 88.9% sensitivity, 97.2% specificity, 95.6% accuracy, and 0.94 AUC using the convolutional neural network with augmented dataset. Convolutional neural network using augmented dataset had the following sensitivities, specificities, accuracies, and AUC: 91.4%, 99.2%, 97.8%, and 0.96 for dentigerous cysts, 82.8%, 99.2%, 96.2%, and 0.92 for periapical cysts, 98.4%, 92.3%, 94.0%, and 0.97 for odontogenic keratocysts, 71.7%, 100%, 94.3%, and 0.86 for ameloblastomas, and 100.0%, 95.1%, 96.0%, and 0.94 for normal jaw, respectively.
Conclusion: The novel framework convolutional neural network method was developed for automatically diagnosing odontogenic cysts and tumors of the jaw on panoramic radiographs using data augmentation. The proposed convolutional neural network model showed high sensitivity, specificity, accuracy, and AUC despite the limited number of panoramic images involved.

국문 초록 (Abstract)

1. 목 적
구강악안면영역에서 발생하는 낭종 혹은 종양을 조기에 발견하지 못하여 적절한 치료가 이루어지지 못하고 지연되는 경우가 있다. 이러한 문제를 해결하기 위하여 인공신경망을 기반으로 하는 기계학습 기술인 딥러닝신경망(deep convolutional neural network)을 이용하는 컴퓨터 보조진단은 보다 정확하고 빠른 결과를 제공할 수 있다. 따라서 본 연구에서는 파노라마방사선영상에서 딥러닝신경망을 이용하여 구강악안면에서 자주 나타나는 4가지 질환(함치성낭, 치근단당, 치성각화낭, 법랑모세포종)을 자동으로 검출 및 진단하는 딥러닝신경망을 개발하고 그 정확성을 평가하였다.

2. 방 법
본 연구에서는 파노라마방사선영상에서 악골에 발생한 치성 낭과 종양을 검출하고 진단하기 위하여 YoLoV3를 기반으로 한 딥러닝신경망을 구축하였다. 1999년부터 2017년까지 서울대학교치과병원에서 조직병리학적으로 확진된 함치성낭 350례, 치근단낭 302례, 치성각화낭 300례, 법랑모세포종 230례의 환자로부터 획득한 총 1182매 파노라마방사선영상을 분석하였다. 또한 대조군으로 질환이 없는 정상 파노라마방사선영상 100매를 선택하였다. 파노라마방사선영상 데이터는 감마, 보정, 회전, 뒤집기 기법을 통하여 12배 증강되었다. 총 데이터의 60%는 훈련세트, 20%는 검증세트, 20%는 테스트세트로 사용하였다. 개발된 딥러닝신경망은 5배 교차검증(5-fold cross validation)기법을 이용하여 평가하였다. 본 연구에서 개발한 딥러닝신경망의 성능은 정확도(Accuracy), 민감도(sensitivity), 특이도(specificity) 및 ROC분석을 통한 AUC(area under the curve) 지표를 사용하여 측정하였다.

3. 결 과
본 연구에서 개발한 딥러닝신경망은 데이터 증강을 하지 않았을 때 78.2% 민감도, 93.9% 특이도, 91.3% 정확도 및 0.86의 AUC 값을 보였고 데이터 증강을 하였을 때에는 88.9% 민감도, 97.2% 특이도, 95.6% 정확도 및 0.94 AUC의 개선된 성능을 보여주었다. 함치성낭은 91.4% 민감도, 99.2% 특이도, 97.8% 정확도 및 0.96 AUC 값을 보였다. 치근단낭은 82.8% 민감도, 99.2% 특이도, 96.2% 정확도 및 0.92 AUC 값을 나타냈다. 치성각화낭은 98.4% 민감도, 92.3% 특이도, 94.0% 정확도 및 0.97 AUC 결과를 보였다. 법랑모세포종은 71.7% 민감도, 100% 특이도, 94.3% 정확도 및 0.86 AUC의 결과를 보였다. 그리고 정상적인 악골에서는 100% 민감도, 95.1% 특이도, 96.0% 정확도 및 0.97 AUC값을 각각 보였다.

4. 결 론
본 연구에서는 파노라마방사선영상에서 치성 낭과 종양을 자동으로 검출하고 진단하는 딥러닝신경망을 개발하였다. 본 연구는 파노라마방사선영상의 수가 충분하지 않았음에도 불구하고 데이터 증강 기법을 이용하여 우수한 민감도, 특이도 및 정확도 결과를 보였다. 본 연구결과를 통하여 개발된 시스템은 환자의 상기 질환을 조기에 진단하고 적절한 시기에 치료하는데 유용하다.

목차 (Table of Contents)

• Introduction １
• Materials and Methods ５
• Data preparation and augmentation of panoramic radiographs ５
• A deep convolutional neural network model for detection and classification of multiple diseases YOLOv3 ９
• Evaluation of detection and classification performance of the deep convolutional neural network model １３
• Results １５
• Discussion ２８
• Conclusion ３７
• Acknowledgments ３８
• References ３９
• 요약(국문초록) ４８

참고문헌 (Reference)

1. Fergus R., Zeiler MD, Visualizing and understanding convolutional networksEuropean conference on computer vision 818-833, , 2014

2. Basic principles of roc analysis, Metz CE, Semin Nucl Med, , 1978

3. Deep Learning in Neural Networks : An Overview, Schmidhuber, J., Neural Networks, , 2015

4. Erhan D. Deep neural networks for object detection, Szegedy CToshev A, Advances in neural information processing systems 2013:2553-2561,

5. The odontogenic keratocyst : A benign cystic tumor ?, 42.Ahlfors ELarsson ASjggren S, J Oral Maxill Surg42:10 ? 19 . https : //doi.org/10.1016/0278-2391, , 1984

6. A survey on image data augmentation for deep learning, Shorten CKhoshgoftaar TM, Journal of Big Data, , 2019

7. Dentigerous cysts associated with supernumerary teeth, J. LustmannL. Bodner, Int . J Oral Maxillofac Surg17:100 ? 102 . https : //doi.org/10.1016/S0901-5027 ( 88, , 1988

8. Shirasu R. A Clinicopathologic Study of ameloblastoma, Ueno SNakamura SMushimoto K, J Oral Maxill Surg44:361 ? 365 . https : //doi.org/10.1016/S0278-2391 ( 86, , 1986

9. Belongie S. Feature pyramid networks for object detection, Lin TYr PGirshick RHe KHariharan B, Proceedings of the IEEE conference on computer vision and pattern recognition 2017:2117-25,

10. Fiji : an open-source platform for biological-image analysis, Schindelin JArganda-Carreras IFrise EKaynig VLongair MPietzsch T, Nat Methods9:676-682, , 2012

11. 2D to 3D conversion of dental images using deep neural network, Leo LMSimla AJ, Journal of Chemical and Pharmaceutical Sciences10:1432-1436, , 2017

12. Heo MS. An overview of deep learning in the field of dentistry, Hwang JJJung YHCho BH, Imaging Sci Dent, , 2019

13. ImageNet classification with deep convolutional neural networks, Krizhevsky ASutskever IHinton GE, Proceedings of the 25th International Conference on Neural Information Processing Systems1:1097 ? 105, , 2012

14. Lipson H.How transferable are features in deep neural networks ?, Yosinski JClune JBengio Y, Advances in neural information processing systems 3320-3328, , 2014

15. Classification of ct brain images based on deep learning networks, Gao XHWHui RTian ZM, Comput Meth Prog Bio, , 2017

16. A benchmark for comparison of dental radiography analysis algorithms, Wang CWHuang CTLee JHLi CHChang SWSiao MJ, Med Image Anal31 : 63-76, , 2016

17. The evolution and application of dental maxillofacial imaging modalities, White SCPharoah MJ, Dent Clin N Am52 : 689 ? 705, , 2008

18. Oliveira L. Deep instance segmentation of teeth in panoramic x-ray images, Jader GFontineli JRuiz MAbdalla KPithon M, 2018 31st SIBGRAPI Conference on Graphics , Patterns and Images ( SIBGRAPI ) 2018:400-407,

19. Dermatologist-level classification of skin cancer with deep neural networks, Esteva AKuprel BNovoa RAKo JSwetter SMBlau HM, Nature542:115-118, , 2017

20. Efficacy of panoramic radiography in dental diagnosis and treatment planning, Kantor MLSlome BA, J Dent Res68 : 810 ? 812, , 1989

21. A deep learning model integrating FCNNs and CRFs for brain tumor segmentation, Zhao XMWu YHSong GDLi ZYZhang YZFan Y, Med Image Anal43:98-111, , 2018

22. Diagnostic imaging in salivary gland disease . Oral Surg Oral Med Oral Pathol, H.P . van tien Akker, 66 : 625 ? 37, , 1988

23. Classification of teeth in cone-beam CT using deep convolutional neural network, Miki YMuramatsu CHayashi TZhou XRHara TKatsumata A, Comput Biol Med, , 2017

24. Rich feature hierarchies for accurate object detection and semantic segmentation, Girshick RDonahue JDarrell TMalik J, Proceedings of the IEEE conference on computer vision and pattern recognition 2014:580-587,

25. Utilization of computer-aided detection system in diagnosing unilateral maxillary, Ohashi YAriji YKatsumata AFujita HNakayama MFukuda M, , 2016

26. Brox T. Dental X-ray image segmentation using a U-shaped Deep Convolutional network, Ronneberger OFischer P, International Symposium on Biomedical Imaging 2015:1-13,

27. Togashi K. Convolutional neural networks : an overview and application in radiology, Yamashita RNishio MDo RKG, Insights Imaging9 : 611-629, , 2018

28. Detection and classification of dental caries in x-ray images using deep neural networks, Ben ARRidha EMourad Z, Eleventh Int Conf Softw Eng Adv 2016:223-227,

29. Suebnukarn S. Application of convolutional neural network in the diagnosis of jaw tumors, Poedjiastoeti W, Healthc Inform Res, , 2018

30. Cysts and cystic lesions of the mandible : Clinical and radiologic-histopathologic review, Scholl RJKellett HMNeumann DPLurie AG, Radiographics, , 1999

31. radiographic , and histopathologic study . Oral Surg Oral Med Oral Pathol 1988 ; 66:145 ?, Haring JIVan Dis MLOdontogenic keratocystsA clinical, 153 . https : //doi.org/10.1016/0030-4220 ( 88 ) 90082-5,

32. Farhadi A. YOLOv3 : An Incremental Improvement 2018 ; arXiv e-print : [ arXiv:1804.02767 p., Redmon J,

33. Shirasu R. Prognostic evaluation of ameloblastoma based on histologic and radiographic typing, Ueno SMushimoto K, J Oral Maxill Surg47:11 ? 15 . https : //doi.org/10.1016/0278-2391 ( 89 ) 90116-X, , 1989

34. Unsupervised Deep Learning Applied to Breast Density Segmentation and Mammographic Risk Scoring, Kallenberg MPetersen KNielsen MNg AYDiao PFIgel C, IEEE T Med Imaging, , 2016

35. The small dentigerous cyst : A diagnostic dilemma . Oral Surg Oral Med Oral Pathol 1995 ; 17:77 ?, Daley TDWysocki GP, 81 . https : //doi.org/10.1016/S1079-2104 ( 05 ) 80078-2,

36. Kim J. Cephalometric landmark detection in dental x-ray images using convolutional neural networks, Lee HPark M, Medical Imaging10134:101341W, , 2017

37. Strub D. Radiographic characteristics of cystogenic ameloblastoma . Oral Surg Oral Med Oral Pathol 1984, Eversole LRLeider AS, 57:572 ? 577https : //doi.org/10.1016/0030-4220 ( 84 ) 90320-7,

38. Detection and diagnosis of dental caries using a deep learning-based convolutional neural network algorithm, Lee JHKim DHJeong SNChoi SH, Journal of Dentistry, , 2018

39. Leider ASStrub D. Radiographic characteristics of cystogenic ameloblastoma . Oral Surg Oral Med Oral Pathol, Eversole LR, 57:572-577, , 1984

40. Waheed S. Gastrointestinal polyp detection in endoscopic images using an improved feature extraction method, Billah M, Biomed Eng Lett8:69-75, , 2018

41. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network, Hannun AYRajpurkar PHaghpanahi MTison GHBourn CTurakhia MP, Nature Medicine25:530, , 2019

42. Kim C. Boosting proximal dental caries detection via combination of variational methods and convolutional neural network, Choi JEun H, J Signal Process Sys, , 2018

43. Deep-learning classification using convolutional neural network for evaluation of maxillary sinusitis on panoramic radiography, Murata MAriji YOhashi YKawai TFukuda MFunakoshi T, Oral Radiol, , 2018

44. Diagnosis and prediction of periodontally compromised teeth using a deep learning-based convolutional neural network algorithm, Lee JHKim DHJeong SNChoi SH, J Periodontal Implan48:114-123, , 2018

45. A critical evaluation of panoramic radiography as a screening procedure in dental practice . Oral Surg Oral Med Oral Pathol 1984, Barrett APWaters BEGrifiths Cri CJ, 57 : 673-677,

46. Varadarajan S. Detection of Tooth caries in Bitewing Radiographs using Deep Learning 2017 ; arXiv e-print : [ arXiv:1711.07312 p., Srivastava MMKumar PPradhan L,

47. Mahadevan S. An improved method to construct basic probability assignment based on the confusion matrix for classification problem, Deng XYLiu QDeng Y, Inform Sciences340:250-261, , 2016

48. Absence of radiometric differentiation between periapical cysts and granulomas . Oral Surg Oral Med Oral Pathol 1994 ; 78:650 ? 654, White SCSapp JPSeto BGMankovich NI, https : //doi.org/10.1016/0030-4220 ( 94 ) 90180-5,

49. A deep-learning artificial intelligence system for assessment of root morphology of the mandibular first molar on panoramic radiography, Hiraiwa TAriji YFukuda MKise YNakata KKatsumata A, Dentomaxillofac Rad, , 2019

50. Clinical efficacy of dental radiography in the detection of dental caries and periodontal diseases . Oral Surg Oral Med Oral Pathol 1986, Douglass CWValachovic RWWijesinha AChauncey HHKapur KKMcNeil BJ, 62 : 330 ? 339,

51. Okano T. Limitation of panoramic radiography in diagnosing adenomatoid odontogenic tumors . Oral Surg Oral Med Oral Pathol 1994 ; 77 : 662 ?, Dare AYamaguchi AYoshiki S, 668,

52. Takase H. Automatic disease stage classification of glioblastoma multiforme histopathological images using deep convolutional neural network, Yonekura AKawanaka HPrasath VBSAronow BJ, Biomed Eng Lett8 : 321-327, , 2018

53. Differentiation of periapical granulomas and radicular cysts by digital radiometric analysis . Oral Surg Oral Med Oral Pathol 1993 ; 76 : 356 ?, Shrout MKHall JMHiideboit CE, 361 . https : //doi.org/10.1016/0030-4220 ( 93 ) 90268-9,

54. A radiologic analysis of dentigerous cysts and odontogenic keratocysts associated with a mandibular third molar . Oral Surg Oral Med Oral Pathol, Tsukamoto GSasaki AAkiyama TIshikawa TKishimoto KNishiyama A, 91:743-747, , 2001

55. Osteoporosis detection in panoramic radiographs using a deep convolutional neural network-based computer-assisted diagnosis system : a preliminary study, Lee JSAdhikari SLiu LJeong HGKim HYoon SJ, Dentomaxillofac Rad, , 2019

56. Automatic detection and classification of radiolucent lesions in the mandible on panoramic radiographs using a deep learning object detection technique . Oral Surg Oral Med Oral Pathol, Ariji YYanashita YKutsuna SMuramatsu CFukuda MKise Y, 128:424- 430, , 2019