Improving seq2seq by revising attention mechanism for speech recognition

다국어 초록 (Multilingual Abstract)

Sequence-to-sequence models (seq2seq) have been designed to learn a mapping from arbitrary sized input sequence to an output sequence. Although the models are so versatile that it have been applied to variety of domain successfully, its adapta- tion for speech recognition should be reconsidered in that im- plicit alignment between speech signal and its output sequence is different from other domains. Moreover, speech signal is usu- ally much longer than its corresponding text label sequence.
In this thesis, I modified attention mechanisms of sequence- to-sequence models so that it can perform better for speech recognition. The revised model used double attention mecha- nism instead of conventional single attention mechanism so that it can attend relevant part of input sequence more easily. More- over, I generalized existing hybrid score function and achieved best results with multiplicative score function.
Experimental results on TIMIT dataset showed that pro- posed modifications achieve fast convergence and improved recog- nition performance.

목차 (Table of Contents)

• 1 Introduction 1
• 2 Background 4
• 2.1 Automatic speech recognition 4
• 2.1.1 Acoustic feature vector 5
• 2.1.2 Evaluation metric 7
• 2.2 Sequence-to-sequence model 7
• 2.2.1 Learning 9
• 2.2.2 Decoding 10
• 2.3 Attention mechanism 11
• 2.3.1 Content-based score function 12
• 2.3.2 Hybrid score function 13
• 3 Proposed modification 15
• 3.1 Double attention mechanism 15
• 3.2 Multiplicative hybrid score function 18
• 4 Experiment 20
• 4.1 Data description 20
• 4.2 Training details 20
• 4.3 Results 22
• 5 Conclusion 25
• Appendices 27
• A Alignment examples 28
• Bibliography 32