Multi-view Rate Control based on HEVC for 3D Video Services

임웅,이수연 대한전자공학회 2013 전자공학회논문지 Vol.50 No.8

In this paper, we propose two rate control algorithms for multi-view extension of HEVC with two rate control algorithms adopted in HEVC and analyze the multi-view rate control performance. The proposed multi-view rate controls are designed on HEVC-based multi-view video coding (MV-HEVC) platform with consideration of high-level syntax,inter-view prediction, etc. not only for the base view but also for the extended views using the rate control algorithms based on URQ (Unified Rate-Quantization) and R-lambda model adopted in HEVC. The proposed multi-view rate controls also contain view-wise target bit allocation for providing the compatibility to the base view. By allocating the target bitrates for each view, the proposed multi-view rate control based on URQ model achieved about 1.83% of average bitrate accuracy and 1.73dB of average PSNR degradation. In addition, about 2.97% of average bitrate accuracy and 0.31dB of average PSNR degradation are achieved with the proposed multi-view rate control based on R-lambda model.

Multi-view Rate Control based on HEVC for 3D Video Services

Lim, Woong,Lee, Sooyoun The Institute of Electronics and Information Engin 2013 전자공학회논문지 Vol.50 No.8

In this paper, we propose two rate control algorithms for multi-view extension of HEVC with two rate control algorithms adopted in HEVC and analyze the multi-view rate control performance. The proposed multi-view rate controls are designed on HEVC-based multi-view video coding (MV-HEVC) platform with consideration of high-level syntax, inter-view prediction, etc. not only for the base view but also for the extended views using the rate control algorithms based on URQ (Unified Rate-Quantization) and R-lambda model adopted in HEVC. The proposed multi-view rate controls also contain view-wise target bit allocation for providing the compatibility to the base view. By allocating the target bitrates for each view, the proposed multi-view rate control based on URQ model achieved about 1.83% of average bitrate accuracy and 1.73dB of average PSNR degradation. In addition, about 2.97% of average bitrate accuracy and 0.31dB of average PSNR degradation are achieved with the proposed multi-view rate control based on R-lambda model.

모바일 환경의 이동형 카메라를 이용한 사용자 저작 다시점 동영상의 제안

성보경,고일주,박준형,여지혜 (사)디지털산업정보학회 2012 디지털산업정보학회논문지 Vol.8 No.1

Recently, user-created video shows high increasing in production and consumption. Among these, videos records an identical subject in limited space with multi-view are coming out. Occurring main reason of this kind of video is popularization of portable camera and mobile web environment. Multi-view has studied in visually representation technique fields for point of view. Definition of multi-view has been expanded and applied to various contents authoring lately. To make user-created videos into multi-view contents can be a kind of suggestion as a user experience for new form of video consumption. In this paper,we show the possibility to make user-created videos into multi-view video content through analyzing multi-view video contents even there exist attribute differentiations. To understanding definition and attribution of multi-view classified and analyzed existing multi-view contents. To solve time axis arranging problem occurred in multi-view processing proposed audio matching method. Audio matching method organize feature extracting and comparing. To extract features is proposed MFCC that is most universally used. Comparing is proposed n by n. We proposed multi-view video contents that can consume arranged user-created video by user selection.

Multi-view Rate Control based on HEVC for 3D Video Services

Woong Lim,Sooyoun Lee 대한전자공학회 2013 전자공학회논문지 Vol.50 No.8

In this paper, we propose two rate control algorithms for multi-view extension of HEVC with two rate control algorithms adopted in HEVC and analyze the multi-view rate control performance. The proposed multi-view rate controls are designed on HEVC-based multi-view video coding (MV-HEVC) platform with consideration of high-level syntax, inter-view prediction, etc. not only for the base view but also for the extended views using the rate control algorithms based on URQ (Unified Rate-Quantization) and R-lambda model adopted in HEVC. The proposed multi-view rate controls also contain view-wise target bit allocation for providing the compatibility to the base view. By allocating the target bitrates for each view, the proposed multi-view rate control based on URQ model achieved about 1.83% of average bitrate accuracy and 1.73dB of average PSNR degradation. In addition, about 2.97% of average bitrate accuracy and 0.31dB of average PSNR degradation are achieved with the proposed multi-view rate control based on R-lambda model.

Fast Encoder Design for Multi-view Video

( Fan Zhao ),( Kaiyang Liao ),( Erhu Zhang ),( Fangying Qu ) 한국인터넷정보학회 2014 KSII Transactions on Internet and Information Syst Vol.8 No.7

Multi-view video coding is an international encoding standard that attains good performance by fully utilizing temporal and inter-view correlations. However, it suffers from high computational complexity. This paper presents a fast encoder design to reduce the level of complexity. First, when the temporal correlation of a group of pictures is sufficiently strong, macroblock-based inter-view prediction is not employed for the non-anchor pictures of B-views. Second, when the disparity between two adjacent views is above some threshold, frame-based inter-view prediction is disabled. Third, inter-view prediction is not performed on boundary macroblocks in the auxiliary views, because the references for these blocks may not exist in neighboring views. Fourth, finer partitions of inter-view prediction are cancelled for macroblocks in static image areas. Finally, when estimating the disparity of a macroblock, the search range is adjusted according to the mode size distribution of the neighboring view. Compared with reference software, these techniques produce an average time reduction of 83.65%, while the bit-rate increase and peak signal-to-noise ratio loss are less than 0.54% and 0.05dB, respectively.

멀티 뷰 기법 리뷰: 이해와 응용

배강일,이영섭,임창원 한국통계학회 2019 응용통계연구 Vol.32 No.1

Multi-view learning considers data from various viewpoints as well as attempts to integrate various information from data. Multi-view learning has been studied recently and has showed superior performance to a model learned from only a single view. With the introduction of deep learning techniques to a multi-view learning approach, it has showed good results in various fields such as image, text, voice, and video. In this study, we introduce how multi-view learning methods solve various problems faced in human behavior recognition, medical areas, information retrieval and facial expression recognition. In addition, we review data integration principles of multi-view learning methods by classifying traditional multi-view learning methods into data integration, classifiers integration, and representation integration. Finally, we examine how CNN, RNN, RBM, Autoencoder, and GAN, which are commonly used among various deep learning methods, are applied to multi-view learning algorithms. We categorize CNN and RNN-based learning methods as supervised learning, and RBM, Autoencoder, and GAN-based learning methods as unsupervised learning. 멀티 뷰 기법은 데이터를 다양한 관점에서 보려는 접근 방법이며 데이터의 다양한 정보를 통합하여 사용하려는 시도이다. 최근 많은 연구가 진행되고 있는 멀티 뷰 기법에서는 단일 뷰 만을 이용하여 모형을 학습시켰을 때 보다 좋은 성과를 보인 경우가 많았다. 멀티 뷰 기법에서 딥 러닝 기법의 도입으로 이미지, 텍스트, 음성, 영상 등 다양한 분야에서 좋은 성과를 보였다. 본 연구에서는 멀티 뷰 기법이 인간 행동 인식, 의학, 정보 검색, 표정 인식 분야에서 직면한 여러 가지 문제들을 어떻게 해결하고 있는지 소개하였다. 또한 전통적인 멀티 뷰 기법들을 데이터 차원, 분류기 차원, 표현 간의 통합으로 분류하여 멀티 뷰 기법의 데이터 통합 원리를 리뷰 하였다. 마지막으로 딥 러닝 기법 중 가장 범용적으로 사용되고 있는 CNN, RNN, RBM, Autoencoder, GAN 등이 멀티 뷰 기법에 어떻게 응용되고 있는지를 살펴보았다. 이때 CNN, RNN 기반 학습 모형을 지도학습 기법으로, RBM, Autoencoder, GAN 기반 학습 모형을 비지도 학습 기법으로 분류하여 이 방법들이 대한 이해를 돕고자 하였다.

단계 파이프라인 구조를 갖는 Multi-View 영상 디코더

Bae, Chang-Ho,Yang, Yeong-Yil The Institute of Electronics and Information Engin 2002 電子工學會論文誌-CI (Computer and Information) Vol.39 No.4

본 논문에서는 multi-view 영상 디코딩 알고리듬을 구현하는 디코더의 구조를 제안하였다. 현재까지 multi-view 영상 처리를 위한 하드웨어 구조에 관한 연구는 이루어지지 않았다. 제안한 multi-view) 영상디코더는 3 단계 파이프라인 방식으로 동작하며, 매 클럭마다 디코드된 영상의 화소 값을 추출한다. Multi-view 영상 디코더는 3 부분으로 구성된다. 노드의 값을 반복적으로 전송하는 Node Selector, 4개의 노드 값으로부터 각 화소의 값을 추출하는 Depth Extractor와 주어진 시점과 화소의 깊이 값으로부터 영상평면에 투영되는 위치를 생성하는 Affine Transformer로 구성되어 있다. 제안된 구조는 MAX+PLUS II 설계 툴로 설계되었고 시뮬레이션을 수행하였으며, 동작 주파수는 30㎒이다. 제안된 구조를 갖는 디코더로 영상을 실시간으로 복원할 수 있다. In this paper, we proposed the architecture of the decoder which implements the multi-view images decoding algorithm. The study of the hardware structure of the multi-view image processing has not been accomplished. The proposed multi-view images decoder operates in a three stage pipelined manner and extracts the depth of the pixels of the decoded image every clock. The multi-view images decoder consists of three modules, Node selector which transfers the value of the nodes repeatedly and Depth Extractor which extracts the depth of each pixel from the four values of the nodes and Affine transformer which generates the projecting position on the image plane from the values of the pixels and the specified viewpoint. The proposed architecture is designed and simulated by the Max+plus II design tool and the operating frequency is 30MHz. The image can be constructed in a real time by the decoder with the proposed architecture.

Multi-View video Processing : IVR, Graphics Composition, and Viewer

Junsup Kwon,Won Young Hwang,Chang Yeol Choi,Manbae Kim,Eun-Young Chang,Namho Hur,Jinwoong Kim 한국방송·미디어공학회 2007 방송공학회논문지 Vol.12 No.4

Multi-view video has recently gained much attraction from academic and commercial fields because it can deliver the immersive viewing of natural scenes. This paper presents multi-view video processing being composed of intermediate view reconstruction(IVR), graphics composition, and multi-view video viewer. First we generate virtual views between multi-view cameras using depth and texture images of the input videos. Then we mix graphic objects to the generated view images. The multi-view video viewer is developed to examine the reconstructed images and composite images. As well, it can provide users with some special effects of multi-view video. We present experimental results that validate our proposed method and show that graphic objects could become the inalienable part of the multi-view video.

자유시점 TV를 위한 다시점 비디오의 계층적 깊이 영상 표현과 H.264 부호화

신종홍 (사)디지털산업정보학회 2011 디지털산업정보학회논문지 Vol.7 No.2

Free viewpoint TV can provide multi-angle view point images for viewer needs. In the real world, But all angle view point images can not be captured by camera. Only a few any angle view point images are captured by each camera. Group of the captured images is called multi-view image. Therefore free viewpoint TV wants to production of virtual sub angle view point images form captured any angle view point images. Interpolation methods are known of this problem general solution. To product interpolated view point image of correct angle need to depth image of multi-view image. Unfortunately, multi-view video including depth image is necessary to develop a new compression encoding technique for storage and transmission, because of a huge amount of data. Layered depth image is an efficient representation method of multi-view video data. This method makes a data structure that is synthesis of multi-view color and depth image. This paper proposed enhanced compression method using layered depth image representation and H.264/AVC video coding technology. In experimental results, confirmed high compression performance and good quality reconstructed image.

Integrating Multi-view Stereoscopic Transmission System into MPEG-21 DIA (Digital Item Adaptation)

Seungwon Lee,Ilkwon Park,Manbae Kim,Hyeran Byun 한국방송·미디어공학회 2007 방송공학회논문지 Vol.12 No.4

In general multi-view system, all the view sequences acquired at the server are transmitted to the client. However, this kind of system requires high processing power of the server as well as the client, thus it is posing a difficulty in practical applications. To overcome this problem, a relatively simple method is to transmit only two view-sequences requested by the client in order to deliver a stereoscopic video. In this system, effective communication between the server and the client is one of important aspects. Therefore, we propose an efficient multi-view system that transmits two view-sequences according to user's request. The view selection process is integrated into MPEG-21 DIA (Digital Item Adaptation) so that our system is compatible to MPEG-21 multimedia framework. Furthermore, multi-view descriptors related to multi-view camera and systems are newly introduced. The syntax of the descriptions and their elements is represented in XML (eXtensible Markup Language) schema. Intermediate view reconstruction (IVR) is used to reduce such discomfort with excessive disparity. Furthermore, IVR is useful for smooth transition between two stereoscopic view sequences. Finally, through the implementation of testbed, we can show the valuables and possibilities of our system.