Stereo 3D and HDR Imaging: Display Quality Measurement and Enhancement

Karol Myszkowski & Tobias Ritschel

Stereo 3D and HDR Imaging: Display Quality Measurement and Enhancement

Computer models of human perception

We understand human visual perception as a final and mandatory component in a visual processing pipeline. An “as complete and as correct as possible model” of human visual perception is therefore crucial when aiming to improve the visual pipeline as a whole. While psychology provides a range of theoretic models of human perceptual performance, they are often not applicable for two reasons. First, too many simplifying assumptions are made. Second, the models are often only passive descriptions of findings and not algorithms. The challenge in our work is to deliver concrete models and algorithms. These should reliably and efficiently predict human perception of 2D or 3D content, and, at the same time, improve the con- tent using these models.

We have developed an application to predict the perceived difference between two film sequences. Building on perceptual models, it uses existing physiological data to analyze the size, strength, and change of visual patterns in the input sequences and from this produces a third sequence that visualizes the perceived difference of the input sequences. It can be surprising how different the perceived difference is from what naïve numerical differences would suggest. Applications of our approach include video compression and steering image synthesis techniques.

A conventional anaglyph stereo image (top) looks unpleasant, when observed without proper anaglyph glasses, due to color artifacts. Using our backward compatible solution (bottom), we can reduce the artifacts and, using anaglyph glasses, still provide a sense of depth when seeing the image.

In a different track, we have developed an approach to compute the perceived difference between two stereo images. Here, our work goes beyond what is known to the field of visual perception: We first performed studies to acquire a model of human stereo perception that describes how a sinusoidal corrugation with a certain amplitude and frequency is perceived. From this model, we devised an approach that reports the perceived difference of two complex stereo stimuli. This finds applications in the compression and manipulation of stereo images. Further, the display of stereo content on various devices can benefit. Stereo images created for a movie theater but displayed on the screen of a small cell phone are unpleasant and uncomfortable to watch. Our model helps to “re-target” content between different devices.

Beyond the description of human perception, our models can be used to improve the user experience when depicting content. For example, a model of human retinal image integration, eye movement, and tempo-spatial information integration can be used to display a high-resolution image on a low-resolution screen such that the sensation is as close as possible to the original. Along similar lines, our approaches add depth impression to an image by using only minimal cues or by letting colors appear brighter than the monitor can actually display.

Apparent resolution enhancement for moving images: We optimize individual frames (1-3, from bottom left), so that they appear more detailed (ours, frame 4) on a human retina than conventional solutions (bottom right).

The possibility to combine other perceptual models of high contrast (HDR), glare effects, brightness gradients, and colors makes the building of a more and more complete computational model of human perception conceivable. And, this provides us with an improved way of depicting content for all users on all devices.