Hole-filling Based on Disparity Map for DIBR

( Ran Liu ),( Hui Xie ),( Fengchun Tian ),( Yingjian Wu ),( Guoqin Tai ),( Yingchun Tan ),( Weimin Tan ),( Bole Li ),( Hengxin Chen ),( Liang Ge ) 한국인터넷정보학회 2012 KSII Transactions on Internet and Information Syst Vol.6 No.10

Due to sharp depth transition, big holes may be found in the novel view that is synthesized by depth-image-based rendering (DIBR). A hole-filling method based on disparity map is proposed. One important aspect of the method is that the disparity map of destination image is used for hole-filling, instead of the depth image of reference image. Firstly, the big hole detection based on disparity map is conducted, and the start point and the end point of the hole are recorded. Then foreground pixels and background pixels are distinguished for hole-dilating according to disparity map, so that areas with matching errors can be determined and eliminated. In addition, parallaxes of pixels in the area with holes and matching errors are changed to new values. Finally, holes are filled with background pixels from reference image according to these new parallaxes. Experimental results show that the quality of the new view after hole-filling is quite well; and geometric distortions are avoided in destination image, in contrast to the virtual view generated by depth-smoothing methods and image inpainting methods. Moreover, this method is easy for hardware implementation.

Hole-filling Based on Disparity Map and Inpainting for Depth-Image-Based Rendering

Ran Liu,Zekun Deng,Lin Yi,Zhenwei Huang,Donghua Cao,Miao Xu,Ruishuang Jia 보안공학연구지원센터 2016 International Journal of Hybrid Information Techno Vol.9 No.5

Due to the changes in viewpoint, holes may appear in the novel view synthesized by depth-image-based rendering (DIBR). A hole-filling method combining disparity-map-based hole-filling and inpainting is proposed. The method first eliminates matching errors in destination image according to the disparity map. Then, holes are classified into different types according to their sizes. Finally, a disparity-map-based approach and an improved exemplar-based inpainting algorithm are used to fill different types of holes according to the type of hole. Experimental results show that the artifacts at the edge of foreground objects can be reduced in synthesized view since the matching errors are eliminated before hole-filling. In addition, the proposed method can achieve more natural and satisfactory results in filled areas in comparison with the disparity-map-based approach and Gautier’s inpainting algorithm, though it may result in higher time complexity.

수압식 충전의 효율 향상을 위한 선단장치 개발에 관한 연구

유성곤(Sung-Kon Yu),김태혁(Tae-Heok Kim),신동춘(Dong-Chun Shin) 한국암반공학회 2012 터널과지하공간 Vol.22 No.6

최근 국내에서는 폐광산 지반보강공사를 실시함에 있어 수압식 충전법을 적용하는 사례가 증가하고 있으나 효율적인 충전법 적용을 위해 현장 특성을 고려한 다양한 연구가 미흡한 실정이다. 본 연구에서는 수압식 충전공법의 충전효율 개선을 위한 선단장치 개발을 위하여 수조 모형실험 및 현장실험을 실시하였다. 수조 모형 실험을 통해 동일조건 하에서 선단장치 노즐각도 및 노즐형태 변화에 따른 충전효율을 평가하고, 이로부터 고안된 선단장치 모델을 현장실험에 적용한 결과, 노즐각 90°관을 사용한 경우 일반적인 수직관을 사용한 경우에 비해 충전량이 약 18% 증가하는 것을 확인할 수 있었다. 이때 안식각은 30.82°였다. 현장에서는 일반적으로 안식각을 40°로 가정하고 충전공 간격을 5m에서 최대 10m 이내로 설계하였지만, 본 연구 결과에 따라 안식각을 30°~ 35°로 적용하면 충전공 간격을 최소 10m에서 최대 15m까지 넓히는 것이 가능하여 경제적이고 효율적인 채굴적 충전이 가능할 것으로 기대된다. In recent, the using of the hydraulic filling method has increased on the underground reinforcement of the abandoned mine in Korea, however it is the lack of research on the efficient filling method. In this study, tank model tests and field tests were conducted for development of tip device for filling efficiency improvements on the hydraulic filling method. In tank model experiments, the filling efficiency was evaluated according to the form and angle of the nozzle on tip device in the same condition. Then tip device model designed by tank model tests was applied to the field experiment. As a result, the amount of filling of nozzle 90° tube is increased by approximately 18% compared to the common vertical injection pipe. The angle of repose was 30.82°. Filling hole spacing in the field is usually designed from 5m up to 10m assumed to be 40° of the angle of repose. According to the results of this study, it is possible that the filling hole spacing expands at least 10m up to 15m applied to be 30°~35° of the angle of repose. Therefore, it is expected to be economical and efficient mine filling.

Hole-Filling Methods Using Depth and Color Information for Generating Multiview Images

남승우,장경호,반윤지,김혜선,진성일 한국전자통신연구원 2016 ETRI Journal Vol.38 No.5

This paper presents new hole-filling methods for generating multiview images by using depth image based rendering (DIBR). Holes appear in a depth image captured from 3D sensors and in the multiview images rendered by DIBR. The holes are often found around the background regions of the images because the background is prone to occlusions by the foreground objects. Background-oriented priority and gradient-oriented priority are also introduced to find the order of hole-filling after the DIBR process. In addition, to obtain a sample to fill the hole region, we propose the fusing of depth and color information to obtain a weighted sum of two patches for the depth (or rendered depth) images and a new distance measure to find the best-matched patch for the rendered color images. The conventional method produces jagged edges and a blurry phenomenon in the final results, whereas the proposed method can minimize them, which is quite important for high fidelity in stereo imaging. The experimental results show that, by reducing these errors, the proposed methods can significantly improve the hole-filling quality in the multiview images generated.

자유 시점 TV에서 시점 합성을 위한 시공간적 배경 정보 추정 기반 홀 채움 방식

김범수,응웬 띠엔 닷,홍민철,Kim, Beomsu,Nguyen, Tien-Dat,Hong, Min-cheol 한국전기전자학회 2016 전기전자학회논문지 Vol.20 No.1

본 논문에서는 자유 시점 텔레비전에서 시점 합성 영상 획득을 위해 시공간적 배경 정보 추정 기반 홀 채움 방식을 제안한다. 신뢰할 수 있는 시간적 배경 정보를 획득하기 위해 새로운 배경 코드북의 구성 및 갱신하는 과정을 수행한다. 더불어, 공간적인 국부 배경 정보 추정을 위해 홀 영역의 배경 및 전경 영역의 구별 및 갱신 과정을 수행한다. 추정된 시공간 배경 정보를 조합하여 홀 채움 과정을 수행하고, 잔여 홀 채움을 수행하기 위해 깊이 배경 정보를 이용한 우선순위 함수를 결정하여 표본 기반 인페인팅 기법을 적용한다. 실험 결과를 통해 제안 방식은 기존방식들과 비교하여 평균 0.3~0.6dB의 성능 향상이 있음을 확인하였으며, 동영상 특성 및 홀 형태에 관계없이 제안된 방식이 새로운 시점 영상을 효과적으로 합성할 수 있음을 확인할 수 있었다. This paper presents a hole-filling algorithm based on extrapolating spatial-temporal background information used in view synthesis for free-viewpoint television. A new background codebook is constructed and updated in order to extract reliable temporal background information. In addition, an estimation of spatial local background values is conducted to discriminate an adaptive boundary between the background region and the foreground region as well as to update the information about the hole region. The holes then are filled by combining the spatial background information and the temporal background information. In addition, an exemplar-based inpainting technique is used to fill the rest of holes, in which a priority function using background-depth information is defined to determine the order in which the holes are filled. The experimental results demonstrated that the proposed algorithm outperformed the other comparative methods about average 0.3-0.6 dB, and that it synthesized satisfactory views regardless of video characteristics and type of hole region.

반전된 Depth 영상을 이용한 실시간 Gaussian Hole-Filling Algorithm

안양근(Yang-Keun Ahn),홍지만(Ji-Man Hong) 한국컴퓨터정보학회 2012 韓國컴퓨터情報學會論文誌 Vol.17 No.7

기존의 Stereoscopy영상의 생성 방법은 2개의 촬영용 렌즈를 일정한 간격으로 띄워놓고 같은 피사체를 촬영하여 좌우 시점의 영상을 생성하는 방법을 이용하게 된다. 하지만 Stereoscopy 카메라를 이용한 3차원 TV 수신의 경우에 좌시점과 우시점에 대한 2개의 영상을 모두 동시에 전송해야 하기 때문에 대역폭의 부담이 크다. 이에 보다 효율적인 여러 방법들에 대한 논의가 이루어지고 있다. 그중 DIBR(Depth Image Based Rendering)은 한장의 영상과 이에 대한 Depth정보를 이용하여 좌시점과 우시점의 영상을 생성하기 때문에 전송 대역폭을 줄일 수 있으며, 이러한 점으로 인하여 기존의 Static Scene에서 DIBR Image 생성에 대한 다양한 Algorithm이 연구되어 왔다. 본 논문에서는 반전된 Depth 영상을 이용하여 자연스럽게 Hole을 채움과 동시에 주변 배경의 왜곡 또한 최소화하는 Gaussian Hole-Filling 방법을 제안하려 한다. 또한 각 Algorithm들의 성능을 비교, 계산하여 각 Algorithm들의 효용성을 분석하였다. Existing method of creating Stereoscopy image, creates viewpoint image from the left and right by shooting one object with 2 lens in certain distance. However, in case of 3-D TV using Stereoscopy camera, the necessity to transmit 2 viewpoint images from the left and right simultaneously, increases the amount of bandwidth. Various and more effective alternatives are under discussion. Among the alternatives, DIBR(Depth Image Based Rendering) creates viewpoint images from the left and right using one image and its Depth information, thus decreasing the amount of transmitted bandwidth. For this reason, there have been various studies on Algorithm to create DIBR Image in existing Static Scene. In this paper, I would like to suggest Gaussian Hole-filling solution, which utilizes reverse-depth image to fill the hole naturally, while minimizing distortion of background. In addition, we have analyzed the effectiveness of each Algorithm by comparing and calculating its functions.

A Method of Pseudo 3D Video Reconstruction Based on 2D Video Sequences

Yong Yin,Lin Qi,Sen Qi 보안공학연구지원센터 2016 International Journal of Signal Processing, Image Vol.9 No.9

A method of pseudo 3D video reconstruction based on 2D video sequences is proposed, which transforms the conventional 2D video into their stereo version using original image and depth-map image. The red-component map is shifted left to obtain the left-view according to the parallax information. Because the left image has holes, the depth-map preprocessing and novel hole-filling simplified algorithm are used to fill the holes effectively. The experimental results show that the proposed algorithm generates 3D visual view with the same image quality as well as faster rendering speed compared with the reference hole-filling algorithm. The method adapts to the stereo pair synthesis for 3D.

GPU를 이용한 Gaussian Hole-Filling Algorithm 가속

박준호(Jun-Ho Park),한탁돈(Tack-Don Han) 한국컴퓨터정보학회 2012 한국컴퓨터정보학회 학술발표논문집 Vol.20 No.2

3차원 멀티미디어 서비스에 대한 관심이 높아짐에 따라 관련 연구들이 현재 다양하게 논의되고 있다. Stereoscopy영상을 생성하기 위한 기존의 방법으로는 두 대의 촬영용 카메라를 일정한 간격으로 띄워놓고 피사체를 촬영한 후 해당 좌시점과 우시점을 생성하는 방법을 이용하였다. 하지만 이는 영상 대역폭의 부담을 가져오게 된다. 이를 해결하기 위하여 Depth정보와 한 장의 영상을 이용한 DIBR(Depth Image Based Rendering) Algorithm에 대한 연구가 많이 이루어지고 있다. 그중 Gaussian Depth Map을 이용한 Hole-Filling 방법은 DIBR에서 가장 자연스러운 결과를 보여주지만 다른 DIBR Algorithm들에 비해 속도가 현저히 느리다는 단점이 있다. 본 논문에서는 영상 생성의 고속화를 위해 GPU를 이용한 Gaussian Hole-Filling Algorithm의 병렬처리 구조를 제안하고 이를 이용한 DIBR Algorithm 생성과정을 제시한다.

형상 차이 기반 홀 패치의 파라미트릭 블렌딩 기법

박정호,박상훈,윤승현 (사)한국컴퓨터그래픽스학회 2020 컴퓨터그래픽스학회논문지 Vol.26 No.3

In this paper, we propose a new technique for filling holes in triangular mesh. First, arbitrary shaped holes are detected. Second, source and target hole patches are generated through triangulation, refinement, fairing, and smoothing. Finally, the shape difference between the two patches is analyzed and a patch with enhanced features is obtained through blending between patches. The effectiveness of the proposed technique is demonstrated by applying the hole filling technique to the triangular mesh model with various shaped holes. 본 논문에서는 삼각 메쉬의 홀을 채우는 새로운 기법을 제시한다. 첫번째, 임의 모양의 홀을 검출한다. 두번째, 삼각화(triangulation), 세분화(refinement), 공정화(fairing), 스무딩(smoothing) 과정을 통해 소스 및 타겟 홀 패치를 생성한다. 마지막으로, 두 패치 사이의 형상 차이를 분석하고 패치간 블렌딩을 통해 특징이 강조된 홀 패치를 얻는다. 다양한 모양의 홀을 갖는 삼각메쉬 모델에 홀 채움 기법을 적용하여 모델을 복원함으로써 제안된 기법의 효과성을 입증한다.

Virtual View Generation by a New Hole Filling Algorithm

Min Soo Ko,Jisang Yoo 대한전기학회 2014 Journal of Electrical Engineering & Technology Vol.9 No.3

In this paper, performance improved hole-filling algorithm which includes the boundary noise removing pre-process that can be used for an arbitrary virtual view synthesis has been proposed. Boundary noise occurs due to the boundary mismatch between depth and texture images during the 3D warping process and it usually causes unusual defects in a generated virtual view. Common-hole is impossible to recover by using only a given original view as a reference and most of the conventional algorithms generate unnatural views that include constrained parts of the texture. To remove the boundary noise, we first find occlusion regions and expand these regions to the common-hole region in the synthesized view. Then, we fill the common-hole using the spiral weighted average algorithm and the gradient searching algorithm. The spiral weighted average algorithm keeps the boundary of each object well by using depth information and the gradient searching algorithm preserves the details. We tried to combine strong points of both the spiral weighted average algorithm and the gradient searching algorithm. We also tried to reduce the flickering defect that exists around the filled common-hole region by using a probability mask. The experimental results show that the proposed algorithm performs much better than the conventional algorithms.