No-reference quality assessment of dynamic sports videos based on a spatiotemporal motion model

Kim, Hyoung-Gook,Shin, Seung-Su,Kim, Sang-Wook,Lee, Gi Yong Electronics and Telecommunications Research Instit 2021 ETRI Journal Vol.43 No.3

This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.

Enhancing VoIP speech quality using combined playout control and signal reconstruction

Hyoung-Gook Kim,Jin-Ho Lee IEEE 2012 IEEE TRANSACTIONS ON CONSUMER ELECTRONICS - Vol.58 No.2

<P>The quality of real-time Voice over Internet Protocol (VoIP) networks is affected by network impairments such as delays, jitters, and packet loss. To solve this issue, this paper proposes a new receiver-based enhancing method of VoIP speech quality. Our approach is based on the combined playout control and signal reconstruction technique that consists of a set of algorithms that conceal packet loss, reduce buffering delay, detect spike delay, and alleviate packet delay jitter. The proposed fully receiver-based enhancing algorithm is computationally efficient, delivers high-quality voice service, and is suitable for use in any practical mobile VoIP system.</P>

Adaptive Single-Channel Speech Enhancement Method for a Push-To-Talk Enabled Wireless Communication Device

KIM, Hyoung-Gook,KIM, Jin Young 'Institute of Electronics, Information and Communi 2016 IEICE TRANSACTIONS ON COMMUNICATIONS - Vol.eb99 No.8

Dimension-Reduced Audio Spectrum Projection Features for Classifying Video Sound Clips

Kim, Hyoung-Gook The Acoustical Society of Korea 2006 韓國音響學會誌 Vol.25 No.e3

For audio indexing and targeted search of specific audio or corresponding visual contents, the MPEG-7 standard has adopted a sound classification framework, in which dimension-reduced Audio Spectrum Projection (ASP) features are used to train continuous hidden Markov models (HMMs) for classification of various sounds. The MPEG-7 employs Principal Component Analysis (PCA) or Independent Component Analysis (ICA) for the dimensional reduction. Other well-established techniques include Non-negative Matrix Factorization (NMF), Linear Discriminant Analysis (LDA) and Discrete Cosine Transformation (DCT). In this paper we compare the performance of different dimensional reduction methods with Gaussian mixture models (GMMs) and HMMs in the classifying video sound clips.

Music Similarity Search Based on Music Emotion Classification

Kim, Hyoung-Gook,Kim, Jang-Heon The Acoustical Society of Korea 2007 韓國音響學會誌 Vol.26 No.e3

This paper presents an efficient algorithm to retrieve similar music files from a large archive of digital music database. Users are able to navigate and discover new music files which sound similar to a given query music file by searching for the archive. Since most of the methods for finding similar music files from a large database requires on computing the distance between a given query music file and every music file in the database, they are very time-consuming procedures. By measuring the acoustic distance between the pre-classified music files with the same type of emotion, the proposed method significantly speeds up the search process and increases the precision in comparison with the brute-force method.

Automatic Emotion Classification of Music Signals Using MDCT-Driven Timbre and Tempo Features

Kim, Hyoung-Gook,Eom, Ki-Wan The Acoustical Society of Korea 2006 韓國音響學會誌 Vol.25 No.e2

This paper proposes an effective method for classifying emotions of the music from its acoustical signals. Two feature sets, timbre and tempo, are directly extracted from the modified discrete cosine transform coefficients (MDCT), which are the output of partial MP3 (MPEG 1 Layer 3) decoder. Our tempo feature extraction method is based on the long-term modulation spectrum analysis. In order to effectively combine these two feature sets with different time resolution in an integrated system, a classifier with two layers based on AdaBoost algorithm is used. In the first layer the MDCT-driven timbre features are employed. By adding the MDCT-driven tempo feature in the second layer, the classification precision is improved dramatically.

Retrieval of Broadcast News Using Audio Content Analysis

Kim, Hyoung-Gook The Acoustical Society of Korea 2007 韓國音響學會誌 Vol.26 No.e3

In this paper, we report our recent work on a indexing and retrieval system of broadcast news using audio content analysis. Key issues addressed in this work are two major parts of the audio indexing system: anchorperson detection based on audio segmentation, and phone-based spoken document retrieval, developed in the framework of the emerging MPEG-7 standard. Experiments are conducted on a database of Britisch broadcast news videos. We discuss the development of the retrieval system, and the evaluation of each part and the retrieval system.

A New Tempo Feature Extraction Based on Modulation Spectrum Analysis for Music Information Retrieval Tasks

Hyoung-Gook Kim 한국ITS학회 2007 한국ITS학회논문지 Vol.6 No.2

본 논문은 음악 정보검색에 사용되는 효과적인 템포 특징 추출방식을 제안한다. 제안된 템포 정보는 협소 밴드상의 일시적인 변조 성분에 의해 형성된다. 이러한 변조 성분은 시간 축 상의 음악 신호로부터 스펙트럼을 구한 후, 각 스펙트럼 성분에 대한 주파수 영역 분석을 통해 획득된 변조 스펙트럼으로 구성된다. 실제 구현에 있어서는 MP3 음악파일로부터 부분 디코딩에 의해 출력된 변형된 이산 코사인 변환 계수에 퓨리에 변환을 취하여 변조스펙트럼을 구하였다. 획득된 변조 스펙트럼의 진폭으로부터 고속으로 추출된 음악 템포 특징값은 다양한 음악 정보 검색에 적용되었다. 음악 무드 및 장르 분류에서는 로그 변조 주파수 계수를 적용하여 분류 성능을 개선시켰으며, 적응 변조 스펙트럼에서 유도된 비트 벡터는 오디오 핑거프린팅에 적용되어 잡음환경 하에서도 검색 성능을 크게 향상시켰다. This paper proposes an effective tempo feature extraction method for music information retrieval. The tempo information is modeled by the narrow-band temporal modulation components, which are decomposed into a modulation spectrum via joint frequency analysis. In implementation, the tempo feature is directly extracted from the modified discrete cosine transform coefficients, which is the output of partial MP3(MPEG 1 Layer 3) decoder. Then, different features are extracted from the amplitudes of modulation spectrum and applied to different music information retrieval tasks. The logarithmic scale modulation frequency coefficients are employed in automatic music emotion classification and music genre classification. The classification precision in both systems is improved significantly. The bit vectors derived from adaptive modulation spectrum is used in audio fingerprinting task That is proved to be able to achieve high robustness in this application. The experimental results in these tasks validate the effectiveness of the proposed tempo feature.

Robust Music Identification Using Long-Term Dynamic Modulation Spectrum

Kim, Hyoung-Gook,Eom, Ki-Wan The Acoustical Society of Korea 2006 韓國音響學會誌 Vol.25 No.e2

In this paper, we propose a robust music audio fingerprinting system for automatic music retrieval. The fingerprint feature is extracted from the long-term dynamic modulation spectrum (LDMS) estimation in the perceptual compressed domain. The major advantage of this feature is its significant robustness against severe background noise from the street and cars. Further the fast searching is performed by looking up hash table with 32-bit hash values. The hash value bits are quantized from the logarithmic scale modulation frequency coefficients. Experiments illustrate that the LDMS fingerprint has advantages of high scalability, robustness and small fingerprint size. Moreover, the performance is improved remarkably under the severe recording-noise conditions compared with other power spectrum-based robust fingerprints.

Enhanced Timing Recovery Using Active Jitter Estimation for Voice-Over IP Networks

( Hyoung-gook Kim ) 한국인터넷정보학회 2012 KSII Transactions on Internet and Information Syst Vol.6 No.4

Improving the quality of service in IP networks is a major challenge for real-time voice communications. In particular, packet arrival-delay variation, so-called “jitter,” is one of the main factors that degrade the quality of voice in mobile devices with the voice-over Internet protocol (VoIP). To resolve this issue, a receiver-based enhanced timing recovery algorithm combined with active jitter estimation is proposed. The proposed algorithm copes with the effect of transmission jitter by expanding or compressing each packet according to the predicted network delay and variations. Additionally, the active network jitter estimation incorporates rapid detection of delay spikes and reacts to changes in network conditions. Extensive simulations have shown that the proposed algorithm delivers high voice quality by pursuing an optimal trade-off between average buffering delay and packet loss rate.