음악요약 생성에 관한 연구

김성탁(Sungtak Kim),김상호(Sangho Kim),김회린(Hoirin Kim),최지훈(Ji Hoo Choi),이한규(Hankyu Lee),홍진우(Jinwoo Hong) 한국방송·미디어공학회 2006 방송공학회논문지 Vol.11 No.1

Music summarization means a technique which automatically generates the most importantand representative a part or parts in music content. The techniques of music summarization have been studied with two categories according to summary characteristics. The first one is that the repeated part is provided as music summary and the second provides the combined segments which consist of segments with different characteristics as music summary in music content. In this paper, we propose and evaluate two kinds of music summarization techniques. The algorithm using multi-level vector quantization which provides a repeated part as music summary gives fixed-length music summary oroptimal length music summary. Fixed-length music summary is evaluated by overlapping ratio between hand-made repeated parts and automatically generated summary. As results, the overlapping ratios of conventional methods are 42.2% and 47.4%, but that of proposed method with fixed-length summary is 67.1%. Optimal length music summary is evaluated by the portion of overlapping between summary and repeated part which is different length according to music content and the result shows that automatically-generated summary expresses more effective part than fixed-length summary with optimal length. The cluster-based algorithm using 2-D similarity matrix and k-means algorithm provides the combined segments as music summary. In order to evaluate this algorithm, we use MOS test consisting of two questions (How many similar segments are there in the summarized music?, How many segments are included in same structure?) and the results show good performance.

Automatic Music Recommendation System based on Music Character

김상호(Sangho Kim),김성탁(Sungtak Kim),권석봉(Suk-bong Kwon),지미경(Mikyong Ji),김회린(Hoirin Kim),윤정현(Jeong Hyun Yoon),이한규(Han-kyu Lee) 한국HCI학회 2007 한국HCI학회 학술대회 Vol.2007 No.2

지능형 서비스 로봇을 위한 잡음에 강인한 문맥독립 화자식별 시스템

김성탁(Sungtak Kim),지미경(Mikyoung Ji),김회린(Hoirin Kim),김혜진(Hye-Jin Kim),윤호섭(Ho-Sub Yoon) 한국HCI학회 2008 한국HCI학회 학술대회 Vol.2008 No.2

본 논문은 지능형 서비스 로봇의 여러 기술들 중에서 기본적인 기술인 화자식별 기술에 관한 내용이다. 화자식별 기술은 화자의 음성신호를 이용하여 등록된 화자들 중에서 가장 유사한 화자를 찾아 내는 것이다. 기존의 mel-frequency cepstral coefficient를 이용한 화자식별 시스템은 무잡음 환경에서는 높은 성능을 보장하지만 잡음환경에서는 성능이 급격하게 떨어진다. 이렇게 잡음환경에서 성능이 떨어지는 요인은 등록환경과 식별환경이 다른 불일치문계 때문이다. 본 논문에서는 불일치문계를 해결하기 위해 relative autocorrelation sequence mel-frequencycepstral coefficient를 사용하였다. 또한, 기존의 relative autocorrelation sequence mel-frequency cepstral coefficient 의 제한된 정보문제와 잔여잡음문계를 해결하기 위해 멀티스트리밍 방법과 멀티스트리밍 방법에 특징벡터 재결합 방법을 결합한 하이브리드 방법을 계한 하였다. 실험결과 계한된 방법들이 기존의 특징벡터보다 잡음환경에서 높은 화자식별 성능을 보여주었다. This paper presents a speaker identification technique which is one of the basic techniques of the ubiquitous robot companion. Though the conventional mel-frequency cepstral coefficients guarantee high performance of speaker identification in clean condition, the performance is degraded dramatically in noise condition. To overcome this problem, we employed the relative autocorrelation sequence mel-frequency cepstral coefficient which is one of the noise robust features. However, there are two problems in relative autocorrelation sequence mel-frequency cepstral coefficient! 1) the limited information problem, 2) the residual noise problem. In this paper, to deal with these drawbacks, we propose a multi-streaming method for the limited information problem and a hybrid method for the residual noise problem. To evaluate proposed methods, noisy speech is used in which air conditioner noise, classic music, and vacuum noise are artificially added. Through experiments, proposed methods provide better performance of speaker identification than the conventional methods.

Segmental Two-Dimensional Mel-Frequency Cepstral Coefficients를 이용한 유해 오디오 인식

김명종(Myung Jong Kim),유주홍(Joohong Yoo),김인수(Insoo Kim),김회린(Hoirin Kim) 대한전자공학회 2010 대한전자공학회 학술대회 Vol.2010 No.6

Approximated Posterior Probability for Scoring Speech Recognition Confidence

김규홍,김회린,Kim Kyuhong,Kim Hoirin The Korean Society Of Phonetic Sciences And Speech 2004 말소리 Vol.52 No.-

This paper proposes a new confidence measure for utterance verification with posterior probability approximation. The proposed method approximates probabilistic likelihoods by using Viterbi search characteristics and a clustered phoneme confusion matrix. Our measure consists of the weighted linear combination of acoustic and phonetic confidence scores. The proposed algorithm shows better performance even with the reduced computational complexity than those utilizing conventional confidence measures.

오디오 핑거프린트의 비트에러율을 이용한 자동 음악 요약 기법 및 시스템

김민성(Minseong Kim),박만수(Mansoo Park),김회린(Hoirin Kim) 한국멀티미디어학회 2013 멀티미디어학회논문지 Vol.16 No.4

Universal Background Model 클러스터링 방법을 이용한 고속 화자식별

박주민,서영주,김회린,Park, Jumin,Suh, Youngjoo,Kim, Hoirin 한국음향학회 2014 韓國音響學會誌 Vol.33 No.3

본 논문은 Gaussian Mixture Model (GMM) 기반의 화자식별에서 급격한 계산 복잡도 감소를 위한 새로운 방법을 제안한다. 일반적으로 GMM 기반의 화자식별 시스템은 테스트 발성의 길이, 등록 화자의 수, GMM의 크기 등 크게 세 가지 요인에 비례하는 많은 계산 복잡도를 가진다. 이러한 점은 화자식별 시스템이 다양한 응용분야에 적용되는 것을 막는 큰 요인이기에 계산 복잡도와 식별 성능 사이의 trade-off 관계는 실제 적용을 위해 가장 중요한 고려요소이다. 식별 성능을 거의 그대로 유지하면서 최대한 계산 복잡도를 감소시키기 위해 우리는 Universal Background Model (UBM) 클러스터링 접근 방법을 제시하고, 또한 이 방법은 실시간 구조의 화자식별에 적용할 수 있다는 것을 보여준다. 제안한 방법의 실험을 통해 미미한 정도의 식별 성능 저하에서 speed-up factor 6의 결과를 얻을 수 있었다. In this paper, we propose a new method to drastically reduce computational complexity in Gaussian Mixture Model (GMM)-based Speaker Identification (SI). Generally, GMM-based SI systems have very high computational complexity proportional to the length of the test utterance, the number of enrolled speakers, and the GMM size. These make the SI systems difficult to be used in various real applications in spite of their broad applicability. Thus, a trade-off between computational complexity and identification accuracy is considered as a primary issue for practical applications. In order to reduce computational complexity sharply with a little loss of accuracy, we introduce a method based on the Universal Background Model (UBM) clustering approach and then we show that it can be used successfully in real-time applications. In experiments with the proposed algorithm, we obtained a speed-up factor of 6 with a negligible loss of accuracy.

깊은 신경망 기반의 전이학습을 이용한 사운드 이벤트 분류

임형준,김명종,김회린,Lim, Hyungjun,Kim, Myung Jong,Kim, Hoirin 한국음향학회 2016 韓國音響學會誌 Vol.35 No.2

깊은 신경망은 데이터의 특성을 효과적으로 나타낼 수 있는 방법으로 최근 많은 응용 분야에서 활용되고 있다. 하지만, 제한적인 양의 데이터베이스는 깊은 신경망을 훈련하는 과정에서 과적합 문제를 야기할 수 있다. 본 논문에서는 풍부한 양의 음성 혹은 음악 데이터를 이용한 전이학습을 통해 제한적인 양의 사운드 이벤트에 대한 깊은 신경망을 효과적으로 훈련하는 방법을 제안한다. 일련의 실험을 통해 제안하는 방법이 적은 양의 사운드 이벤트 데이터만으로 훈련된 깊은 신경망에 비해 현저한 성능 향상이 있음을 확인하였다. Deep neural network that effectively capture the characteristics of data has been widely used in various applications. However, the amount of sound database is often insufficient for learning the deep neural network properly, so resulting in overfitting problems. In this paper, we propose a transfer learning framework that can effectively train the deep neural network even with insufficient sound event data by employing rich speech or music data. A series of experimental results verify that proposed method performs significantly better than the baseline deep neural network that was trained only with small sound event data.

문장 독립 화자 검증을 위한 그룹기반 화자 임베딩

정영문,엄영식,이영현,김회린,Jung, Youngmoon,Eom, Youngsik,Lee, Yeonghyeon,Kim, Hoirin 한국음향학회 2021 韓國音響學會誌 Vol.40 No.5

Recently, deep speaker embedding approach has been widely used in text-independent speaker verification, which shows better performance than the traditional i-vector approach. In this work, to improve the deep speaker embedding approach, we propose a novel method called group-based speaker embedding which incorporates group information. We cluster all speakers of the training data into a predefined number of groups in an unsupervised manner, so that a fixed-length group embedding represents the corresponding group. A Group Decision Network (GDN) produces a group weight, and an aggregated group embedding is generated from the weighted sum of the group embeddings and the group weights. Finally, we generate a group-based embedding by adding the aggregated group embedding to the deep speaker embedding. In this way, a speaker embedding can reduce the search space of the speaker identity by incorporating group information, and thereby can flexibly represent a significant number of speakers. We conducted experiments using the VoxCeleb1 database to show that our proposed approach can improve the previous approaches.

L1-norm regularization을 통한 SGMM의 state vector 적응

구자현(Goo, Jahyun),김영관(Kim, Younggwan),김회린(Kim, Hoirin) 한국음성학회 2015 말소리와 음성과학 Vol.7 No.3

In this paper, we propose L1-norm regularization for state vector adaptation of subspace Gaussian mixture model (SGMM). When you design a speaker adaptation system with GMM-HMM acoustic model, MAP is the most typical technique to be considered. However, in MAP adaptation procedure, large number of parameters should be updated simultaneously. We can adopt sparse adaptation such as L1-norm regularization or sparse MAP to cope with that, but the performance of sparse adaptation is not good as MAP adaptation. However, SGMM does not suffer a lot from sparse adaptation as GMM-HMM because each Gaussian mean vector in SGMM is defined as a weighted sum of basis vectors, which is much robust to the fluctuation of parameters. Since there are only a few adaptation techniques appropriate for SGMM, our proposed method could be powerful especially when the number of adaptation data is limited. Experimental results show that error reduction rate of the proposed method is better than the result of MAP adaptation of SGMM, even with small adaptation data.