Quality Assessment of Deblocked Images

Changhoon Yim,Bovik, A C IEEE 2011 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.20 No.1

<P>We study the efficiency of deblocking algorithms for improving visual signals degraded by blocking artifacts from compression. Rather than using only the perceptually questionable PSNR, we instead propose a block-sensitive index, named PSNR-B, that produces objective judgments that accord with observations. The PSNR-B modifies PSNR by including a blocking effect factor. We also use the perceptually significant SSIM index, which produces results largely in agreement with PSNR-B. Simulation results show that the PSNR-B results in better performance for quality assessment of deblocked images than PSNR and a well-known blockiness-specific index.</P>

Stereoscopic 3D Visual Discomfort Prediction: A Dynamic Accommodation and Vergence Interaction Model

Heeseok Oh,Sanghoon Lee,Bovik, Alan Conrad IEEE 2016 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.25 No.2

<P>The human visual system perceives 3D depth following sensing via its binocular optical system, a series of massively parallel processing units, and a feedback system that controls the mechanical dynamics of eye movements and the crystalline lens. The process of accommodation (focusing of the crystalline lens) and binocular vergence is controlled simultaneously and symbiotically via cross-coupled communication between the two critical depth computation modalities. The output responses of these two subsystems, which are induced by oculomotor control, are used in the computation of a clear and stable cyclopean 3D image from the input stimuli. These subsystems operate in smooth synchronicity when one is viewing the natural world; however, conflicting responses can occur when viewing stereoscopic 3D (S3D) content on fixed displays, causing physiological discomfort. If such occurrences could be predicted, then they might also be avoided (by modifying the acquisition process) or ameliorated (by changing the relative scene depth). Toward this end, we have developed a dynamic accommodation and vergence interaction (DAVI) model that successfully predicts visual discomfort on S3D images. The DAVI model is based on the phasic and reflex responses of the fast fusional vergence mechanism. Quantitative models of accommodation and vergence mismatches are used to conduct visual discomfort prediction. Other 3D perceptual elements are included in the proposed method, including sharpness limits imposed by the depth of focus and fusion limits implied by Panum's fusional area. The DAVI predictor is created by training a support vector machine on features derived from the proposed model and on recorded subjective assessment results. The experimental results are shown to produce accurate predictions of experienced visual discomfort.</P>

Cross-Layer Optimization for Downlink Wavelet Video Transmission

Hyungkeuk Lee,Sanghoon Lee,Bovik, Alan Conrad IEEE 2011 IEEE transactions on multimedia Vol.13 No.4

<P>Cross-layer optimization for efficient multimedia communications is an important emerging issue towards providing better quality-of-service (QoS) over capacity-limited wireless channels. This paper presents a cross-layer optimization approach that operates between the application and physical layers to achieve high fidelity downlink video transmission by optimizing with respect to a quality criterion termed “visual entropy” using Lagrangian relaxation. By utilizing the natural layered structure of wavelet coding, an optimal level of power allocation is determined, which permits the throughput of visual entropy to be maximized over a multi-cell environment. A theoretical approach to optimization using the Shannon capacity and the Karush-Kuhn-Tucker (KKT) conditions is explored when coupling the application with the physical layers. Simulations show that the throughput gain for cross-layer optimization by visual entropy is increased by nearly 80% at the cell boundary as compared with peak signal-to-noise ratio (PSNR).</P>

Saliency Prediction on Stereoscopic Videos

Kim, Haksub,Sanghoon Lee,Bovik, Alan Conrad IEEE 2014 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.23 No.4

<P>We describe a new 3D saliency prediction model that accounts for diverse low-level luminance, chrominance, motion, and depth attributes of 3D videos as well as high-level classifications of scenes by type. The model also accounts for perceptual factors, such as the nonuniform resolution of the human eye, stereoscopic limits imposed by Panum's fusional area, and the predicted degree of (dis) comfort felt, when viewing the 3D video. The high-level analysis involves classification of each 3D video scene by type with regard to estimated camera motion and the motions of objects in the videos. Decisions regarding the relative saliency of objects or regions are supported by data obtained through a series of eye-tracking experiments. The algorithm developed from the model elements operates by finding and segmenting salient 3D space-time regions in a video, then calculating the saliency strength of each segment using measured attributes of motion, disparity, texture, and the predicted degree of visual discomfort experienced. The saliency energy of both segmented objects and frames are weighted using models of human foveation and Panum's fusional area yielding a single predictor of 3D saliency.</P>

Transfer Function Model of Physiological Mechanisms Underlying Temporal Visual Discomfort Experienced When Viewing Stereoscopic 3D Images

Taewan Kim,Sanghoon Lee,Bovik, Alan Conrad IEEE 2015 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.24 No.11

<P>When viewing 3D images, a sense of visual comfort (or lack of) is developed in the brain over time as a function of binocular disparity and other 3D factors. We have developed a unique temporal visual discomfort model (TVDM) that we use to automatically predict the degree of discomfort felt when viewing stereoscopic 3D (S3D) images. This model is based on physiological mechanisms. In particular, TVDM is defined as a second-order system capturing relevant neuronal elements of the visual pathway from the eyes and through the brain. The experimental results demonstrate that the TVDM transfer function model produces predictions that correlate highly with the subjective visual discomfort scores contained in the large public databases. The transfer function analysis also yields insights into the perceptual processes that yield a stable S3D image.</P>

3D Visual Discomfort Prediction: Vergence, Foveation, and the Physiological Optics of Accommodation

Jincheol Park,Sanghoon Lee,Bovik, Alan Conrad IEEE 2014 IEEE journal of selected topics in signal processi Vol.8 No.3

<P>To achieve clear binocular vision, neural processes that accomplish accommodation and vergence are performed via two collaborative, cross-coupled processes: accommodation-vergence (AV) and vergence-accommodation (VA). However, when people watch stereo images on stereoscopic displays, normal neural functioning may be disturbed owing to anomalies of the cross-link gains. These anomalies are likely the main cause of visual discomfort experienced when viewing stereo images, and are called Accommodation-Vergence Mismatches (AVM). Moreover, the absence of any useful accommodation depth cues when viewing 3D content on a flat panel (planar) display induces anomalous demands on binocular fusion, resulting in possible additional visual discomfort. Most prior efforts in this direction have focused on predicting anomalies in the AV cross-link using measurements on a computed disparity map. We further these contributions by developing a model that accounts for both accommodation and vergence, resulting in a new visual discomfort prediction algorithm dubbed the 3D-AVM Predictor. The 3D-AVM model and algorithm make use of a new concept we call local 3D bandwidth (BW) which is defined in terms of the physiological optics of binocular vision and foveation. The 3D-AVM Predictor accounts for anomalous motor responses of both accommodation and vergence, yielding predictive power that is statistically superior to prior models that rely on a computed disparity distribution only.</P>

Optimal Channel Adaptation of Scalable Video Over a Multicarrier-Based Multicell Environment

Jincheol Park,Hyungkeuk Lee,Sanghoon Lee,Bovik, A.C. IEEE 2009 IEEE transactions on multimedia Vol.11 No.6

<P>To achieve seamless multimedia streaming services over wireless networks, it is important to overcome inter-cell interference (ICI), particularly in cell border regions. In this regard scalable video coding (SVC) has been actively studied due to its advantage of channel adaptation. We explore an optimal solution for maximizing the expected visual entropy over an orthogonal frequency division multiplexing (OFDM)-based broadband network from the perspective of cross-layer optimization. An optimization problem is parameterized by a set of source and channel parameters that are acquired along the user location over a multicell environment. A suboptimal solution is suggested using a greedy algorithm that allocates the radio resources to the scalable bitstreams as a function of their visual importance. The simulation results show that the greedy algorithm effectively resists ICI in the cell border region, while conventional nonscalable coding suffers severely because of ICI.</P>

Video Quality Pooling Adaptive to Perceptual Distortion Severity

Jincheol Park,Seshadrinathan, K.,Sanghoon Lee,Bovik, A. C. IEEE 2013 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.22 No.2

<P>It is generally recognized that severe video distortions that are transient in space and/or time have a large effect on overall perceived video quality. In order to understand this phenomena, we study the distribution of spatio-temporally local quality scores obtained from several video quality assessment (VQA) algorithms on videos suffering from compression and lossy transmission over communication channels. We propose a content adaptive spatial and temporal pooling strategy based on the observed distribution. Our method adaptively emphasizes “worst” scores along both the spatial and temporal dimensions of a video sequence and also considers the perceptual effect of large-area cohesive motion flow such as egomotion. We demonstrate the efficacy of the method by testing it using three different VQA algorithms on the LIVE Video Quality database and the EPFL-PoliMI video quality database.</P>

3D Visual Activity Assessment Based on Natural Scene Statistics

Kwanghyun Lee,Moorthy, Anush Krishna,Sanghoon Lee,Bovik, Alan Conrad IEEE 2014 IEEE TRANSACTIONS ON IMAGE PROCESSING - Vol.23 No.1

<P>One of the most challenging ongoing issues in the field of 3D visual research is how to perceptually quantify object and surface visualizations that are displayed within a virtual 3D space between a human eye and 3D display. To seek an effective method of quantification, it is necessary to measure various elements related to the perception of 3D objects at different depths. We propose a new framework for quantifying 3D visual information that we call 3D visual activity (3DVA), which utilizes natural scene statistics measured over 3D visual coordinates. We account for important aspects of 3D perception by carrying out a 3D coordinate transform reflecting the nonuniform sampling resolution of the eye and the process of stereoscopic fusion. The 3DVA utilizes the empirical distortions of wavelet coefficients to a parametric generalized Gaussian probability distribution model and a set of 3D perceptual weights. We conducted a series of simulations that demonstrate the effectiveness of the 3DVA for quantifying the statistical dynamics of visual 3D space with respect to disparity, motion, texture, and color. A successful example application is also provided, whereby 3DVA is applied to the problem of predicting visual fatigue experienced when viewing 3D displays.</P>

BUCKET: Scheduling of Solar-Powered Sensor Networks via Cross-Layer Optimization

Sungjin Lee,Beom Kwon,Sanghoon Lee,Bovik, Alan Conrad IEEE 2015 IEEE Sensors Journal Vol. No.

<P>Renewable solar energy harvesting systems have received considerable attention as a possible substitute for conventional chemical batteries in sensor networks. However, it is difficult to optimize the use of solar energy based only on empirical power acquisition patterns in sensor networks. We apply acquisition patterns from actual solar energy harvesting systems and build a framework to maximize the utilization of solar energy in general sensor networks. To achieve this goal, we develop a cross-layer optimization-based scheduling scheme called binding optimization of duty cycling and networking through energy tracking (BUCKET), which is formulated in four-stages: 1) prediction of energy harvesting and arriving traffic; 2) internode optimization at the transport and network layers; 3) intranode optimization at the medium access control layer; and 4) flow control of generated communication task sets using a token-bucket algorithm. Monitoring of the structural health of bridges is shown to be a potential application of an energy-harvesting sensor network. The example network deploys five sensor types: 1) temperature; 2) strain gauge; 3) accelerometer; 4) pressure; and 5) humidity. In the simulations, the BUCKET algorithm displays performance enhancements of ~12-15% over those of conventional methods in terms of the average service rate.</P>