Vision and Research Strategy
There has been a tremendous increase in quality and number of new output devices, such as stereo and automultiscopic screens, portable and wearable displays, and 3D printers. Some of them have already entered mass production and gained a lot of users’ attention, others will follow this trend promptly. Unfortunately, abilities of these emerging technologies outperform capabilities of methods and tools for creating content. Also, the current level of understanding of how these new technologies inﬂuence user experience is insufficient to fully exploit their advantages. In our work, we concentrate on developing efficient and robust methods for producing input for these new devices. We take a special care that the content is reproduced not in a perfect numerical sense, but that the perceived quality is maximized. To this end, we explore and learn how the human sensory systems work and build efficient computational models that can mimic their response. Such models can be later used in so-called perceptual optimization techniques, which produce content that will provide optimal perceived quality for the human observer.
Research Areas and Achievements
Limited display qualities (e.g., resolution, brightness, color, depth) are insufficient to faithfully reproduce real world experience. Recently emerging new displays devices, such as 3D screens, Oculus Rift, allow us to reproduce new visual cues such as depth, parallax, etc. On one hand, these devices allow us to create more immersive impression than ever before, but on the other hand, they pose many questions and problems both on the computational and quality side. The question we address in our work is: given a device and certain computational resources, how can we exploit the capabilities of the particle output device in a best way? In the past, we have investigated perception in the context of stereoscopic images and videos. In particular, we designed perceptual models for disparity that enabled new ways of manipulating stereoscopic content. Now, we want to expand this work to light ﬁelds. We design perceptual models that will be able to predict the perceived quality of view dependent content. Later, we want to explore applications of such models in the context of multi-view, light ﬁeld, and head-mounted displays. Our group is also concerned with temporal quality of the video content. In one of our current projects, we investigate perception of different framerates and how they impact the viewer experience.
In the context of fabrication (e.g., 3D printing), despite high quality of multi-material printers, reproducing 3D hardcopies of real objects is still a challenging task. One problem are the limitations of current design tools which often do not allow us to fully exploit the capabilities of new fabrication devices. Another problem are the limitations of 3D printers and other manufacturing devices (e.g., resolution, color reproduction, mechanical properties of materials). We believe that taking human perception into account will allow us to overcome these limitations. For example, in our recent project, we addressed the problem of manufacturing compliant objects. We built a computational model for perceived compliance which can predict softness of different materials as perceived by humans. We demonstrated its beneﬁts in the context of creating copies of real materials using 3D printing technology. Similarly, we would like to address reproduction of tactile perception.
Novel Display Designs
The work on perceptual display inspires us to go back, and re-think designs of current display technologies from the perception point of view. In our current work, we are concerned with two new designs. First one addresses the problem of light efficiency. We want to design a screen for movie theaters, which does not loose light, i.e., it reﬂects the light only in the directions where viewers are expected. The second design addresses large-format glass-free 3D screens. We want to overcome one of the major limitation of current automultiscopic displays, i.e., hot-spotting, which hampers adaptation of such screens in movie theaters. We plan to achieve our goal by analyzing layout of the audience in movie theaters and designing custom optical components that would assure that every viewer has exactly the same experience.