Effects of Depth Map Quantization for Computer-Generated Multiview Images using Depth Image-Based Rendering

( Minyoung Kim ),( Yongjoo Cho ),( Hyon-gon Choo ),( Jinwoong Kim ),( Kyoung Shin Park ) 한국인터넷정보학회 2011 KSII Transactions on Internet and Information Syst Vol.5 No.11

This paper presents the effects of depth map quantization for multiview intermediate image generation using depth image-based rendering (DIBR). DIBR synthesizes multiple virtual views of a 3D scene from a 2D image and its associated depth map. However, it needs precise depth information in order to generate reliable and accurate intermediate view images for use in multiview 3D display systems. Previous work has extensively studied the pre-processing of the depth map, but little is known about depth map quantization. In this paper, we conduct an experiment to estimate the depth map quantization that affords acceptable image quality to generate DIBR-based multiview intermediate images. The experiment uses computer-generated 3D scenes, in which the multiview images captured directly from the scene are compared to the multiview intermediate images constructed by DIBR with a number of quantized depth maps. The results showed that there was no significant effect on depth map quantization from 16-bit to 7-bit (and more specifically 96-scale) on DIBR. Hence, a depth map above 7-bit is needed to maintain sufficient image quality for a DIBR-based multiview 3D system.

경계 기반의 깊이 영상 개선

안양근(Yang-Keun Ahn),홍지만(Ji-Man Hong) 한국컴퓨터정보학회 2012 韓國컴퓨터情報學會論文誌 Vol.17 No.4

최근 깊이영상을 기반으로 한 3D 기술이 3D 공간감지, 3D 영상획득, 3D 인터랙션, 3D 게임 등 다양한 분야에서 응용되고 있다. 깊이영상을 생성하기 위해서는 깊이 카메라를 이용하게 되는데, 이렇게 생성된 깊이영상의 화질을 개선하기 위한 다양한 시도들이 이루어지고 있다. 본 눈문에서는 이러한 깊이 카메라 기반의 3D 응용에 있어서 깊이 영상을 개선하기 위해 영역기반의 에지 검출기를 사용하는 방법을 제안한다. 제안된 방법은 제한된 환경에서의 깊이영상을 획득하는 과정에서 발생 할 수 있는 화질열화를 후처리 또는 전처리를 통해 개선함으로써 보다 향상된 깊이 영상을 제공한다. 다양한 실험결과를 통해서 개선된 깊이영상을 객관적 화질 평가를 위해 가상시점 참조 소프트웨어에 적용하여 비교함으로써 최대 0.42dB의 화질 향상을 확인하였다. 또한 영상의 실제 시청 환경과 가장 유사한 방법인 DSCQS(Double Stimulus Continuous Quality Scale)방법을 통해서 주관적 화질의 객관적 평가를 수행함으로써 개선된 깊이영상의 효용성을 다시 확인하였다. Recently, 3D technology based on depth image is widely used in various fields including 3D space recognition, image acquisition, interaction, and games. Depth camera is used in order to produce depth image, various types of effort are made to improve quality of the depth image. In this paper, we suggests using area-based Canny edge detector to improve depth image in applying 3D technology based on depth camera. The suggested method provides improved depth image with pre-processing and post-processing by fixing image quality deterioration, which may take place in acquiring depth image in a limited environment. For objective image quality evaluation, we have confirmed that the image is improved by 0.42dB at maximum, by applying and comparing improved depth image to virtual view reference software. In addition, with DSCQS(Double Stimulus Continuous Quality Scale) evaluation method, we are reassured of the effectiveness of improved depth image through objective evaluation of subjective quality.

콘크리트 균열 깊이 추정에 유의미한 이미지 특성 변수에 관한 연구

정서영(Seo-Young Jung),손채봉(Chae-Bong Sohn),유정호(Jung-Ho Yu) 한국퍼실리티매니지먼트학회 2021 한국퍼실리티매니지먼트학회지 Vol.16 No.2

Since the depth of concrete cracks is important information for evaluating the safety of facilities, a technology capable of accurately and quickly measuring the depth of cracks is needed. However, in the facility maintenance practice, the crack depth is measured through non-destructive testing using ultrasonic equipment. This method has the disadvantage of poor efficiency. Such limitations of existing technologies can be solved through the development of image-based crack depth measurement technology. This is because if the crack depth measurement is based on an image, restrictions on the timing and location of the inspection can be resolved. In order for such image-based crack depth measurement technology to be developed, identification of image characteristic variables related to crack depth must precede. Therefore, the purpose of this study was to identify image characteristic variables meaningful to the crack depth. In order to achieve the purpose of the study, we collected 100 actual cases of cracking, measured the crack depth, and took an image. In addition, we created a dataset by extracting image characteristic variables such as hue, luminance, and brightness from the captured image through image processing technology. Using the thus-produced dataset, we conducted statistical analysis to identify significant image characteristic variables for crack depth estimation.

TV영상의 화면 깊이와 화질에 따른 시청자 수용연구

이성욱 ( Sungwook Lee ) 부산대학교 영화연구소 2012 아시아영화연구 Vol.5 No.2

본 연구는 텔레비전 영상으로 표현되는 화면의 깊이감과 화질에 따 라 달라질 수 있는 TV시청자들의 수용결과를 바탕으로 영상의 제작 목적에 부합하는 효과적인 영상표현법을 알아보는데 목적이 있다. 화 면으로 표현되는 화면의 깊이감과 화질에 따라 시청자들이 볼 수 있는 정보량의 차이는 장소와 내용에 대한 사실감 표현을 다르게 만들기 때 문이다. 최근 전자기술의 발달로 방송에서도 기존 방송카메라의 이미 지센서보다 약 16배 정도 커진 35mm 사진용 필름 사이즈와 같은 대형 이미지 센서가 사용되면서 그동안 방송 카메라로는 표현이 어려웠던 매우 얕은 피사계심도의 영상표현이 가능해졌다. 기존의 텔레비전 영 상과 차별화되는 영상 표현이 가능해지면서 대형 이미지센서를 활용 한 영상제작은 텔레비전에서 장르를 불문하고 점점 더 많이 사용되고 있는 추세이다. 하지만 단지 새로운 느낌만을 위한, 그리고 이러한 효 과를 더욱 강조하기 위한 극단적인 배경의 생략은 인물과 함께 표현 되어야 할 장소라는 정보의 생략으로 이어질 수밖에 없다. 화면 속 장소 와 공간이 나타내는 정보는 화면 속 인물에 대한 직간접적인 설명인 동시에 장소와 관련한 다양한 정보를 시청자들에게 제공하기도 하는 등 배경이 나타내는 정보는 수용자들에게 내용전달과 이해를 위해 중 요한 역할을 하는 것이 사실이다. 전자기술의 발달로 텔레비전에도 점점 더 다양한 촬영포맷이 사용되고 있다. 현재 방송에서 많이 사용 되 고 있는 세 가지 촬영포맷의 동일조건의 비교실험을 통해 화면의 깊이 감과 화질의 차이에 따라 수용자들이 느끼는 차이를 알아보았다. The study is focused on depth of television-screen and resolution and how it affects the television viewers. The purpose of pictorialimage in depth-of-field scale is to express esthetic and psychological aspects. Process of pictorial-image production will vary the televisionviewers` perception of resolution difference and follow by viewers` recognition of location and realism. Recently, large-sized image sensors have been developed. Due to electronic development, it became possible to express pictorial-images in shoal-depth on television and this application is widely utilized when pictorial-images are broadcasted. Instead of focused on the content and the conveyance of information, television-viewers are biased to new “LOOK.” In pictorial-images with extreme shallowdepth of field, information on the background is out of focus and the main character and/or the main subject in the screen is in focus; omission of background information contacts directly/indirectly with main character/subject can lose its information to be passed on. Viewing pictorial-images in extreme shallow-depth of field will dissent from viewing in reality. The advantage is to have one`s attention while viewing, but in contrary, the images are expressed in different nature and will diminish the realism. The survey concluded the effectiveness of understanding of location was filmed with large-depth of filed image. Visual interest and visual attention was more effective with shallow-depth of field image with high resolution. The outcome was better when filmed with moderatedepth of field image with high resolution to focus on the content; it was better to film foods with shallow-depth of field image with high resolution. As a result, it was effectual to utilize moderate-depth of field image with high resolution to convey information, instead of utilizing too large/too shallow-depth of field image with poor resolution. Visual attraction was more effective utilizing moderatedepth of field image with high resolution as well.

Enhancing Depth Accuracy on the Region of Interest in a Scene for Depth Image Based Rendering

( Yongjoo Cho ),( Kiyoung Seo ),( Kyoung Shin Park ) 한국인터넷정보학회 2014 KSII Transactions on Internet and Information Syst Vol.8 No.7

This research proposed the domain division depth map quantization for multiview intermediate image generation using Depth Image-Based Rendering (DIBR). This technique used per-pixel depth quantization according to the percentage of depth bits assigned in domains of depth range. A comparative experiment was conducted to investigate the potential benefits of the proposed method against the linear depth quantization on DIBR multiview intermediate image generation. The experiment evaluated three quantization methods with computer-generated 3D scenes, which consisted of various scene complexities and backgrounds, under varying the depth resolution. The results showed that the proposed domain division depth quantization method outperformed the linear method on the 7- bit or lower depth map, especially in the scene with the large object.

자유시점 TV를 위한 다시점 비디오의 계층적 깊이 영상 표현과 H.264 부호화

신종홍 (사)디지털산업정보학회 2011 디지털산업정보학회논문지 Vol.7 No.2

Free viewpoint TV can provide multi-angle view point images for viewer needs. In the real world, But all angle view point images can not be captured by camera. Only a few any angle view point images are captured by each camera. Group of the captured images is called multi-view image. Therefore free viewpoint TV wants to production of virtual sub angle view point images form captured any angle view point images. Interpolation methods are known of this problem general solution. To product interpolated view point image of correct angle need to depth image of multi-view image. Unfortunately, multi-view video including depth image is necessary to develop a new compression encoding technique for storage and transmission, because of a huge amount of data. Layered depth image is an efficient representation method of multi-view video data. This method makes a data structure that is synthesis of multi-view color and depth image. This paper proposed enhanced compression method using layered depth image representation and H.264/AVC video coding technology. In experimental results, confirmed high compression performance and good quality reconstructed image.

반전된 Depth 영상을 이용한 실시간 Gaussian Hole-Filling Algorithm

안양근(Yang-Keun Ahn),홍지만(Ji-Man Hong) 한국컴퓨터정보학회 2012 韓國컴퓨터情報學會論文誌 Vol.17 No.7

기존의 Stereoscopy영상의 생성 방법은 2개의 촬영용 렌즈를 일정한 간격으로 띄워놓고 같은 피사체를 촬영하여 좌우 시점의 영상을 생성하는 방법을 이용하게 된다. 하지만 Stereoscopy 카메라를 이용한 3차원 TV 수신의 경우에 좌시점과 우시점에 대한 2개의 영상을 모두 동시에 전송해야 하기 때문에 대역폭의 부담이 크다. 이에 보다 효율적인 여러 방법들에 대한 논의가 이루어지고 있다. 그중 DIBR(Depth Image Based Rendering)은 한장의 영상과 이에 대한 Depth정보를 이용하여 좌시점과 우시점의 영상을 생성하기 때문에 전송 대역폭을 줄일 수 있으며, 이러한 점으로 인하여 기존의 Static Scene에서 DIBR Image 생성에 대한 다양한 Algorithm이 연구되어 왔다. 본 논문에서는 반전된 Depth 영상을 이용하여 자연스럽게 Hole을 채움과 동시에 주변 배경의 왜곡 또한 최소화하는 Gaussian Hole-Filling 방법을 제안하려 한다. 또한 각 Algorithm들의 성능을 비교, 계산하여 각 Algorithm들의 효용성을 분석하였다. Existing method of creating Stereoscopy image, creates viewpoint image from the left and right by shooting one object with 2 lens in certain distance. However, in case of 3-D TV using Stereoscopy camera, the necessity to transmit 2 viewpoint images from the left and right simultaneously, increases the amount of bandwidth. Various and more effective alternatives are under discussion. Among the alternatives, DIBR(Depth Image Based Rendering) creates viewpoint images from the left and right using one image and its Depth information, thus decreasing the amount of transmitted bandwidth. For this reason, there have been various studies on Algorithm to create DIBR Image in existing Static Scene. In this paper, I would like to suggest Gaussian Hole-filling solution, which utilizes reverse-depth image to fill the hole naturally, while minimizing distortion of background. In addition, we have analyzed the effectiveness of each Algorithm by comparing and calculating its functions.

딥러닝 기반 깊이 추정 모델의 정확도 향상을 위한 실시간 데이터세트 증강 알고리즘

손진우(Jin Woo Son),민지현(Ji Hyun Min),김한솔(Han Sol Kim) 한국정보기술학회 2021 Proceedings of KIIT Conference Vol.2021 No.11

본 연구에서는 딥러닝 기반 깊이 추정 모델의 추정 정확도를 향상을 위한 훈련 데이터세트의 실시간 증강기법을 제안한다. 제안하는 기법에서는 동일한 시점을 향하는 단안 카메라와 스테레오 비전을 사용하여 RGB 영상과 그에 해당하는 깊이 정보를 실시간으로 획득한다. 그런 다음 RGB 이미지를 깊이 추정 모델에 입력하여, 추정된 깊이 값을 획득한다. 이후, 해당 이미지에 대한 추정 정확도가 특정 임계값 미만이면 실제 깊이 맵과 RGB 이미지를 훈련 데이터 세트에 추가한다. 마지막으로, 증강시킨 데이터세트를 이용하여 훈련된 모델의 추정 정확도 증가도를 평가하여 제안하는 알고리즘의 유효성을 검증한다. In this study, we propose a real-time augmentation technique of a training dataset to improve the estimation accuracy of a deep learning-based depth estimation model. In the proposed method, RGB images and corresponding depth information are acquired in real-time by using a monocular camera and stereo vision pointing at the same viewpoint. Then, the RGB image is input into the depth estimation model to obtain an estimated depth value. Then, if the estimation accuracy for the image is below a certain threshold, the actual depth map and corresponding RGB image are added to the training dataset. Finally, the validity of the proposed algorithm is verified by evaluating the increase in estimation accuracy of the trained model using the augmented dataset.

적외선 거리 센서 깊이이미지를 이용한 얼굴 인식 알고리즘 평가

권기현(Kwon, Ki-Hyeon) 한국산학기술학회 2012 한국산학기술학회논문지 Vol.13 No.8

적외선 거리 센서를 사용하여 취득한 깊이이미지(depth image)에 대하여 잘 알려진 얼굴인식 알고리즘을 수 행하여 깊이이미지 응용에 적용가능성을 평가한다. 아울러, 기존의 얼굴인식이 정확도 측면에서만 강조를 해온 측면 이 있는데 이렇게 하면 실제 환경에서 적용할 때 문제점을 제대로 평가하기 어렵다. 본 연구에서는 RGB 이미지와 깊이 이미지들에 대해 잘 알려진 얼굴 인식 알고리즘 (PCA, LDA, ICA, SVM)을 적용하여 얼굴인식 정확도뿐만 아니 라 처리 속도, 사용 메모리 그리고 저장 공간에 대한 정보를 구해 이미지 유형과 각 알고리즘에 따른 전반적인 성능 을 구하였다. 처리 결과 깊이이미지와 컬러 색인된 깊이이미지는 컬러이미지에 비해 각각 30% ~ 40% 정도 파일 크 기가 작음에도 전반적인 성능에서 컬러이미지와 마찬가지로 우수한 결과를 보였으며, LDA는 SVM 다음으로 정확도 도 우수하고 훈련시간과 훈련 소요메모리도 양호하고 테스트시간과 테스트 소요 메모리도 낮아 우수한 성능을 보였다. We evaluate the face detection and recognition of depth image that is obtained by infrared range sensor. and Face recognition was usually focused on accuracy aspect but it is not enough to evaluate the performance in testing for real world application. In this paper, we evaluate the overall performance like accuracy, training, test speed and memory use for the well known face recognition algorithm like PCA, LDA, ICA and SVM. This experiment evaluate the good results of depth and colored depth image compatible with the colored image although the file size of depth and colored depth image is 30%~40% less than the colored image. Whereas, LDA got the good accuracy performance next to the SVM and also shows the good performance in speed and the amount of memory.

Evaluation of 3D Data Service Based on Depth Image Based Rendering over T-DMB

Youngjin Oh,Kwanghee Jung,Joong Kyu Kim,Gwangsoon Lee,Hyun Lee,Namho Hur,Jinwoong Kim 대한전자공학회 2008 ITC-CSCC :International Technical Conference on Ci Vol.2008 No.7

3D data service over Terrestrial-Digital Multimedia Broadcasting (T-DMB) is very attractive because the single user environment of T-DMB is suitable to glassless 3D viewing. However, the bit budget for transmission of additional 3D data over T-DMB is very limited with 32Kbps through data service channel. To overcome this limited condition, Depth-Image-Based-Rendering (DIBR) can be applied, because corresponding depth sequence is only additionally needed to current TDMB and this can be compressed effectively. Therefore, in this paper, we evaluate 3D data service based on DIBR over T-DMB. Evaluation mainly consists of two experiments. One is to evaluate the coding efficiency of depth sequences and the other is to measure the subjective quality and percived depth of auto-stereosopic image generated by DIBR with coded depth sequence. Therefore, we evaluate the possibility of transmitted depth image through data service channel that transmission rate is 32Kbps. Evaluation results show that DIBR can efficiently be utilized for 3D data service over T-DMB. However, it is also shown that the development of some techniques such as the depth preprocessing is required for the improvement of image quality.