Visual Fatigue Relaxation for Stereoscopic Video via Nonlinear Disparity Remapping

Changjae Oh,Bumsub Ham,Sunghwan Choi,Kwanghoon Sohn [Institute of Electrical and Electronics Engineers 2015 IEEE transactions on broadcasting Vol.61 No.2

<P>A nonlinear disparity remapping scheme is presented to enhance the visual comfort of stereoscopic videos. The stereoscopic video is analyzed for predicting a degree of fatigue with the viewpoint of three factors: 1) spatial frequency; 2) disparity magnitude; and 3) disparity motion. The degree of fatigue is then estimated in a local manner. It can be visualized as an index map so-called a “visual fatigue map,” and an overall fatigue score is obtained by pooling the visual fatigue map. Based on this information, a nonlinear remapping operator is generated in two phases: 1) disparity range adaptation and 2) operator nonlinearization. First, a disparity range is automatically adjusted according to the determined overall fatigue score. Second, rather than linearly adjusting the disparity range of an original video to the determined disparity range, a nonlinear remapping operator is constructed in a manner that the disparity range of inducible problematic region is compressed, while that of comfortable region is stretched. The proposed scheme is verified via subjective evaluations where visual fatigue and depth sensation are compared among original videos, linearly remapped videos, and nonlinearly remapped videos. Experimental results show that the nonlinearly remapped videos provide more comfort than the linearly remapped videos without losing depth sensation.</P>

OCEAN: Object-centric arranging network for self-supervised visual representations learning

Oh, Changjae,Ham, Bumsub,Kim, Hansung,Hilton, Adrian,Sohn, Kwanghoon Elsevier 2019 expert systems with applications Vol.125 No.-

<P><B>Abstract</B></P> <P>Learning visual representations plays an important role in computer vision and machine learning applications. It facilitates a model to understand and perform high-level tasks intelligently. A common approach for learning visual representations is supervised one which requires a huge amount of human annotations to train the model. This paper presents a self-supervised approach which learns visual representations from input images without human annotations. We learn the correct arrangement of object proposals to represent an image using a convolutional neural network (CNN) without any manual annotations. We hypothesize that the network trained for solving this problem requires the embedding of semantic visual representations. Unlike existing approaches that use uniformly sampled patches, we relate object proposals that contain prominent objects and object parts. More specifically, we discover the representation that considers overlap, inclusion, and exclusion relationship of proposals as well as their relative position. This allows focusing on potential objects and parts rather than on clutter. We demonstrate that our model outperforms existing self-supervised learning methods and can be used as a generic feature extractor by applying it to object detection, classification, action recognition, image retrieval, and semantic matching tasks.</P> <P><B>Highlights</B></P> <P> <UL> <LI> A self-supervised learning which does not require human annotations for training CNN. </LI> <LI> Learning the correct arrangement of object proposals to represent an image by CNN. </LI> <LI> Demonstrating the advantage of our model by applying it to PASCAL VOC datasets. </LI> <LI> Application to other vision tasks including image retrieval and semantic matching. </LI> </UL> </P>

밝기가 다른 이미지에서의 레퍼런스 이미지 결정 방법

오창재(Changjae Oh),함범섭(Bumsub Ham),신형철(Hyungchul Shin),손광훈(Kwanghoon Sohn) 한국방송·미디어공학회 2011 한국방송공학회 학술발표대회 논문집 Vol.2011 No.7

컬러는 영상처리 분야에서 중요한 단서로 사용될 수 있는 정보이다. 하지만 실제로 촬영한 영상의 경우에는 빛과 카메라 특성 등 다양한 요소들의 영향으로 인해 이미지 간 컬러 정보의 불일치가 빈번히 일어난다. 따라서 컬러가 다른 여러 장의 영상을 입력 영상으로 사용하는 경우, 입력 영상간 컬러를 동일하게 맞춰 주어야 한다. 이를 수행함에 있어서, 어떠한 이미지를 레퍼런스 이미지로 결정할 것인가는 매우 중요한 문제이다. 이에 본 논문에서는, 히스토그램 등화(histogram equalization) 기법을 이용하여 입력 이미지들의 비용을 결정해줌으로써, 레퍼런스 이미지를 결정하는 방법을 제시한다. 스테레오 매칭을 통해 다양한 밝기의 입력 영상에서 가장 좋은 결과를 얻을 수 있는 레퍼런스 이미지를 결정할 수 있음을 보였다.

Personness estimation for real-time human detection on mobile devices

Kim, Kyuwon,Oh, Changjae,Sohn, Kwanghoon Elsevier 2017 expert systems with applications Vol.72 No.-

<P><B>Abstract</B></P> <P>One aim of detection proposal methods is to reduce the computational overhead of object detection. However, most of the existing methods have significant computational overhead for real-time detection on mobile devices. A fast and accurate proposal method of human detection called personness estimation is proposed, which facilitates real-time human detection on mobile devices and can be effectively integrated into part-based detection, achieving high detection performance at a low computational cost. Our work is based on two observations: (i) normed gradients, which are designed for generic objectness estimation, effectively generate high-quality detection proposals for the person category; (ii) fusing the normed gradients with color attributes improves the performance of proposal generation for human detection. Thus, the candidate windows generated by the personness estimation will very likely contain human subjects. The human detection is then guided by the candidate windows, offering high detection performance even when the detection task terminates prior to completion. This interruptible detection scheme, called anytime detection, enables real-time human detection on mobile devices. Furthermore, we introduce a new evaluation methodology called time-recall curves to practically evaluate our approach. The applicability of our proposed method is demonstrated in extensive experiments on a publicly available dataset and a real mobile device, facilitating acquisition and enhancement of portrait photographs (e.g. selfie) on widespread mobile platforms.</P> <P><B>Highlights</B></P> <P> <UL> <LI> A fast and accurate detection proposal method for the person category is proposed. </LI> <LI> Detection proposals are used by the part-based human detector in a improved way. </LI> <LI> High effectiveness of the proposed method is demonstrated on a real mobile device. </LI> </UL> </P>

Epitaxial growth and strain-related optical properties of polar magnetic PbVO₃ thin films

Seol Hee Oh,Changjae Roh,Jongseok Lee,William Jo 한국자기학회 2016 한국자기학회 학술연구발표회 논문개요집 Vol.2016 No.11

Structure Selective Depth Superresolution for RGB-D Cameras

Kim, Youngjung,Ham, Bumsub,Oh, Changjae,Sohn, Kwanghoon IEEE 2016 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.25 No.11

<P>This paper describes a method for high-quality depth superresolution. The standard formulations of image-guided depth upsampling, using simple joint filtering or quadratic optimization, lead to texture copying and depth bleeding artifacts. These artifacts are caused by inherent discrepancy of structures in data from different sensors. Although there exists some correlation between depth and intensity discontinuities, they are different in distribution and formation. To tackle this problem, we formulate an optimization model using a nonconvex regularizer. A nonlocal affinity established in a high-dimensional feature space is used to offer precisely localized depth boundaries. We show that the proposed method iteratively handles differences in structure between depth and intensity images. This property enables reducing texture copying and depth bleeding artifacts significantly on a variety of range data sets. We also propose a fast alternating direction method of multipliers algorithm to solve our optimization problem. Our solver shows a noticeable speed up compared with the conventional majorize-minimize algorithm. Extensive experiments with synthetic and real-world data sets demonstrate that the proposed method is superior to the existing methods.</P>

패널티 방법을 이용한 고속 경계 보존 평활화 기법

정형주(Hyeongju Jeong),오창재(Changjae Oh),김영중(Youngjung Kim),김선옥(Sunok Kim),손광훈(Kwanghoon Sohn) 대한전자공학회 2015 대한전자공학회 학술대회 Vol.2015 No.6

In this paper, we propose an edge-preserving smoothing operator based on weighted absolute deviation. This formulation makes the algorithm robust for usage on noisy and textured image. Furthermore, we present an efficient technique to solve the proposed objective function. Specifically, we approximate the solution of nonlinear system, by iteratively solving a sequence of two subproblem. Such decomposition not only reduces the overall computation complexity but also enhances the results, comparing with conventional gradient based method.

Probability-Based Rendering for View Synthesis

Bumsub Ham,Dongbo Min,Changjae Oh,Do, Minh N.,Kwanghoon Sohn IEEE 2014 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.23 No.2

<P>In this paper, a probability-based rendering (PBR) method is described for reconstructing an intermediate view with a steady-state matching probability (SSMP) density function. Conventionally, given multiple reference images, the intermediate view is synthesized via the depth image-based rendering technique in which geometric information (e.g., depth) is explicitly leveraged, thus leading to serious rendering artifacts on the synthesized view even with small depth errors. We address this problem by formulating the rendering process as an image fusion in which the textures of all probable matching points are adaptively blended with the SSMP representing the likelihood that points among the input reference images are matched. The PBR hence becomes more robust against depth estimation errors than existing view synthesis approaches. The MP in the steady-state, SSMP, is inferred for each pixel via the random walk with restart (RWR). The RWR always guarantees visually consistent MP, as opposed to conventional optimization schemes (e.g., diffusion or filtering-based approaches), the accuracy of which heavily depends on parameters used. Experimental results demonstrate the superiority of the PBR over the existing view synthesis approaches both qualitatively and quantitatively. Especially, the PBR is effective in suppressing flicker artifacts of virtual video rendering although no temporal aspect is considered. Moreover, it is shown that the depth map itself calculated from our RWR-based method (by simply choosing the most probable matching point) is also comparable with that of the state-of-the-art local stereo matching methods.</P>

신뢰도 높은 변이추정을 위한 하이브리드 스테레오 정합 알고리듬

김득현(Kim, Deukhyeon),최진욱(Choi, Jinwook),오창재(Oh, Changjae),손광훈(Sohn, Kwanghoon) 한국방송·미디어공학회 2012 한국방송공학회 학술발표대회 논문집 Vol.2012 No.7

본 논문에서는 다양한 변이 추정 방식 중 영역기반(Area-based) 알고리듬과 특정기반(Feature-based) 알고리듬을 결합한 하이브리드(Hybrid) 변이추정 알고리듬을 제안한다. 제안하는 알고리듬은 Features from Accelerated Segment Test(FAST) 코너 점 추출기[2]를 이용하여 좌, 우 영상 각각의 특징 점을 추출한 후, 특징 점들의 정보를 이용한 스테레오 정함을 통해 신뢰도 높은 초기 변이지도(Disparity map)를 생생하게 된다. 그러나 생성된 초기 변이지도는 조밀하지 못하므로, 조밀한 변이 지도를 획득하기 위해 특징점이 추출된 영역에 대해서는 추정된 초기 변이 값을 이웃 픽셀과의 색 유사도를 고려하여 전파시키고 특징 점이 추출되지 않은 영역에 대해서는 이진 윈도우(Binary window)를 활용한 영역기반 변이추정 알고리듬[1]을 이용하여 변이 값을 추정한다. 이를 통해, 제안 알고리듬은 특징 기반 알고리듬에서 발생할 수 있는 보간법 문제를 해결함과 동시에 신뢰도가 높은 초기 변이지도를 사용함으로써, 영역 기반 알고리듬의 정합 오차를 줄여 신뢰도 높은 변이지도를 생생할 수 있다. 실험 결과 추정된 초기 변이지도는 ground truth와 비교 시 약 99%이상의 정확도를 보이며, 특징 점이 추출된 영역에서 기존의 영역기반 알고리듬보다 더 정확한 변이 값이 추정되었음을 확인하였다.