음성 인식을 위한 다중 심층 신경망 병렬 학습 = Parallel training for deep neural network based speech recognizers

다국어 초록 (Multilingual Abstract)

The hybrid deep neural Network (DNN) and hidden Markov model (HMM) have recently achieved great performance in speech recognition. However, the computing hardware was not adequate to learn deep neural networks with more hidden layers from big data sets. Further, despite the powerful performance of a DNN-based acoustic model, the time-consuming learning process has been a problem. This paper proposes a novel DNN-based acoustic modeling framework for speech recognition. The new model adopts parallel training in multiple DNNs. Several hierarchically structured DNNs are trained separately in parallel, using multiple computing units. Weights are averaged after each epoch. The suggested structure separates DNN into 10 and shows approximately 7.5 times faster in training time than baseline hybrid deep neural network. This improvement in average training time is mainly attributed to the use of multiple GPUs and the fact that training is based on only a subset of data in parallel. The WSJ data set was used for proposed parallel DNN performance verification.

목차 (Table of Contents)

• 목 차
• 제 1 장 서론 1
• 제 2 장 관련 연구 4
• 2.1 Multi-layer perceptron(MLP) 4
• 2.2 Error backpropagation 9
• 2.3 Pre-training and fine-tuning 13
• 2.3.1 Deep Belief Network 14
• 제 3 장 DNN의 병렬 학습 방법 22
• 3.1 Model parallelism 23
• 3.2 Data parallelism 27
• 3.3 제안하는 parallel DNN 학습 방법 30
• 제 4 장 실험 및 결과 35
• 4.1 실험 환경 35
• 4.2 실험 결과 37
• 4.2.1 Baseline DNN-HMM 37
• 4.2.2 제안하는 방법 39
• 제 5 장 결론 및 향후 과제 44
• 부록 A. 최적의 epoch 설정 46
• 참고 문헌 49