3D TV

Three-dimensional television is currently experiencing a surge in research activity. Acquisition hardware, computer vision algorithms, and rendering techniques have reached a level of affordability, robustness and sophistication, respectively, that enable building a system to record, reconstruct, and render dynamic scenes in its three-dimensional composition. The development of 3D-TV follows advances recently made in image-based rendering (IBR). In IBR, conventional photographs are used to capture the visual appearance, the light field of a scene. Given enough images from different viewpoints, any view of the scene from outside of the visual hull of the recording positions can be reconstructed. Unfortunately, light field rendering quality depends on the number of images. Very large numbers of images are nessary to attain convincing rendering results. However, if 3D scene structure can be reconstructed from the image data, e.g. per-image depth maps or a complete 3D scene geometry model, hybrid model/image-based rendering methods achieve realistic rendering results from only a relatively small number of images. Furthermore, programable graphics hardware can be used for image-based rendering to accelerate warping and resampling the recorded images.

Until recently, image-based rendering techniques were restricted to static scenes. To record the dynamic light field of a temporally varying scene, the necessary hardware effort is considerably higher. Multiple synchronized video cameras are needed to capture the scene from different viewpoints. We are currently constructing a rendering system for dynamic light fields. An array of multiple synchronized cameras is used to capture an animated scene from different viewpoints. Depth maps are reconstructed from the recorded video streams. Graphics hardware is employed to attain interactive rendering rates of up to 20~frames per second. Our system enables interactively viewing a 3D movie from an arbitrary viewpoint position within the window spawned by the camera positions.

Dynamic Light Field Acquisition

The Stanford Light Field Video Camera consists of numerous CMOS camera heads, arranged in a planar matrix and with aligned optical axes.

Depth Map Estimation

Original image, its correlation-based disparity map and the final processed map using a diffusion process.

Interactive DLF Rendering

Four source images with precomputed depth maps are warped and blended to render the view from a novel viewpoint.

Disparity map with triangle mesh, warped triangle mesh and resulting warped image.