Techniques for for Hardware-Accelerated Light Field Rendering

Rendering Vector-quantized Light Fields

In order to generate arbitrary views of a scene image-based rendering combines a number of images taken from different viewpoints. Within Within this context Levoy et al. and Gortler et al. concurrently proposed one technique to store radiance values in such a way that images from arbitrary viewpoints can easily be constructed. Levoy et al. described a 4D representation of the radiance called light field. The radiance samples in the light field are accessed through four-dimensional coordinates which correspond to the spatial position and the direction of each sample. To render the light field at interactive frame rates Gortler et al. presented an algorithm based on hardware supported texture mapping.

The advantage of constructing images from arbitrary viewpoints goes hand in hand with the drawback of increased size and memory consumption. This is why light fields should better be compressed. Levoy suggested to apply vector quantization (VQ) for this task. During the quantization, illumination samples are regularly grouped together. To each group one index value to one vector in a codebook is assigned that represents the initial radiance samples. This index value can be used together with the codebook to reconstruct the original radiance value of each contained sample or at least an approximation of the initial value in the case of quantization errors.

We have developed a technique that decompresses the VQ-encoded light field exactly at the moment of display using texture color tables, a hardware supported OpenGL extension by SGI. They allow to specify a texture of indices which are replaced by the contents of the color table when texturing a geometric primitive. That way it is possible to decompress a light field that is loaded to the texture memory in its compressed form which reduces the needs for this usually scarce resource.

Example pictures for different sizes of the codebook and block sizes:


no compression	8192 entries, block size 2x2x2x2

512 entries, block size: 1x1x2x2	8192 entries, blocksize: 1x1x2x2

Rendering Refractions with Light Fields

We have investigated light field techniques to render scenes with reflecting and refracting objects of arbitrarily complex geometry. The radiance resulting from placing some transmitting object into a scene may of course be captured by a conventional light field, but if the environment or the object is modified the light field must be completely regenerated. Our approach strictly decouples the geometry of the reflecting or refracting object and the illumination. A light field data structure is used to represent geometrical information about light rays after being reflected off the object's surface or rather after passing through the object respecting refractions and inner reflections. For these rays texture coordinates into environment maps or other light fields are computed and stored in the light field data structure instead of radiance values. Rendering with these geometry light fields now consists of two steps: to render the light field with the color coded texture coordinates and to replace the texture coordinates by the radiance values found in the representation of the environment. It is still possible to achieve interactive frame rates since both steps can be accelerated by the use of graphics hardware.

The acquisition of some geometry light field is shown in demonstrated by the following picture. The samples of the light fields are given by the texture coordinates of the leaving ray hitting the illuminating light field:

Here some pictures

Color-coded texture coordinates.

Placing the transparent teapot in a scene.

A bi-convex lens in fron of an array of cylinders which are represented by a second light field. Representing the illuminating environment by a second light field allows to move the refracting object interactively:

Literature

A. Gersho and Gray R.M. Vector Quantization and Signal Compression. Kluwer Academic Press, 1992.

Steven J. Gortler, Radek Grzeszczuk, Richard Szelinski, and Michael F. Cohen. The Lumigraph. In Compter Graphics (SIGGRAPH '96 Proceedings), pages 43-54, August 1996.

Paul Hansen. Introducing pixel textures. In Developer News, pages 23-26. Silicon Graphics Inc., May 1997.

Wolfgang Heidrich, Hendrik Lensch, Michael F. Cohen, and Hans-Peter Seidel. Light Field Techniques for Reflections and Refractions. In Rendering Techniques '99 (Proceedings of Eurographics Rendering Workshop), June 1999.

Marc Levoy and Pat Hanrahan. Light field rendering. In Computer Graphics (SIGGRAPH '96 Proceedings), pages 31-42, August 1996.

Last modified: Thu Dec 9 11:42:53 "MET 1999 by Hendrik Lensch