Speaker selective source localization for non-trivial noise environments

다국어 초록 (Multilingual Abstract)

A speech interface can be useful for various devices, including smartphones, car navigation systems, smart TVs, and humanoid robots. In particular, the ability to control such devices situated at distant positions is one of the most attractive characteristics of speech-based interfaces. However, the performance of speech-based interfaces, including speech recognition and speaker recognition, degrades significantly in real life conditions, where unrelated noises frequently occur. Sound source localization-based speech enhancements can improve the quality of such speech-based interfaces by determining the location of the speaker, and then boosting the signal from the desired location while suppressing the sounds from other locations.
Conventional sound source localization methods, however, cannot provide reliable estimation of a speaker’s location in severe noise conditions. In conventional localization methods, the loudest sound source within a given area is selected as the target location, though this may not necessarily be related to human speech. For speech-based interfaces, the locations with a high correlation to human speech should be given preference. However, in real life applications, speech-like noises, including babble noises, can frequently occur. Therefore, locations showing a high correlation with the target speaker should be given preference. To accomplish this, this paper combines several speech analysis algorithms, including voice activity detection and speaker verification, with a sound source localization algorithm. By incorporating features that are closely correlated with human speech and target speakers, unrelated noise, including speech-like background noise, can be effectively suppressed.
The proposed method was tested under a variety of conditions using both simulation data and real data. Experimental results indicated that the performance of the proposed method was superior to that of a conventional algorithm for various types of noise and signal-to-noise conditions. In particular, the proposed method performed much better in severely degraded noise conditions.

목차 (Table of Contents)

• CHAPTER 1 INTRODUCTION 7
• CHAPTER 2 RELATED WORKS 12
• 2.1 Sound source localization 12
• 2.1.1 Generalized cross-correlation phase transform (GCC-PHAT) 14
• 2.1.2 Steered response power - phase transform (SRP-PHAT) 17
• 2.2 Speech enhancement 20
• 2.2.1 Single channel speech enhancement 20
• 2.2.2 Multi channel speech enhancement 22
• 2.3 Voice activity detection 24
• 2.3.1 Speech features for VAD 26
• 2.3.2 Standard VAD algorithms 28
• 2.4 Speaker recognition 30
• 2.4.1 Speaker verification using GMM-UBM 31
• 2.4.2 Adaptation algorithms for speaker verification 33
• CHAPTER 3 MOTIVATION 34
• 3.1 Sound source localization in noisy environments 34
• 3.2 Enhanced sound source localization using VAD 37
• 3.3 Problem analysis 42
• CHAPTER 4 SPEAKER SELECTIVE SOUND SOURCE LOCALIZATION 45
• 4.1 N-best candidate selection 47
• 4.2 Speech enhancement using GSC 49
• 4.3 Voice similarity calculation using VAD score 50
• 4.4 Speaker likelihood computation using GMM-UBM 52
• 4.5 Candidate rescoring and decision 53
• CHAPTER 5 EXPERIMENTS 55
• 5.1 Experimental set-ups 55
• 5.1.1 Performance metrics and data preparation 55
• 5.1.2 Simulation data configuration 57
• 5.1.3 Real data configuration 59
• 5.2 Speaker verification experiments 61
• 5.2.1 Imposter results 62
• 5.2.2 Non-matching speaker model results 64
• 5.3 Single noise source experiments 66
• 5.3.1 Simulation data results 66
• 5.3.2 Real data results 70
• 5.3.3 Summary 73
• 5.4 Multiple noise source experiments 77
• 5.4.1 Two-noise-source experiments 79
• 5.4.2 Three-noise-source experiments 81
• 5.4.3 Summary 83
• CHAPTER 6 CONCLUSION 85
• CHAPTER 7 REFERENCES 87