Disparity-compensated Light Field Coding

paper

An efficient codec for light fields must meet two contradicting criteria:

While the first condition demands extensive exploitation of present similarities between images, the second criterion suggests introducing no dependencies between light-field segments.

Two coding schemes have been developed that offer attractive compromises between high compression and fast decoding.

Adaptive Block-based Light Field Coder

paper

Based on techniques originally developed for video coding applications, the Adaptive Block-based Codec exploits light-field specific characteristics of the data structure. The images are subdivided into blocks which are individually coded using one of several block-coding modes. Rate-constrained mode selection is accomplished by employing Lagrangian optimization. Depending on light-field scene characteristics and reconstruction quality, compression ratios up to 1000:1 are achieved.

Original Light Field

Compressed Light Field
17 * 17 images 0.061 bits per pixel at 39.4 dB mean PSNR

MPEG demo Chick (2.1Mb)

Hierarchical Light Field Coder

paper

The second coder is based entirely on disparity compensation. Light-field images are predicted using approximate disparity maps. The image array is coded in hierarchical order to enable progressive refinement of the light field during decoding.

Original Light Field

Compressed Light Field
32 * 32 images 0.037 bits per pixel at 34.1 dB mean PSNR

MPEG demo Dragon (920kb)

Original Light Field

Compressed Light Field
32 * 32 images 0.0175 bits per pixel at 38.3 dB mean PSNR

MPEG demo Buddha (950kb)

Both coders feature compression ratios of up to 1000:1 at acceptable image quality, easing capacity requirements to store light fields and speeding up transmission, e.g., over the internet.

Because of its lower decoding complexity and faster access to arbitrary data segments, the video-based coder is better suited for light-field rendering on machines with limited moemory resources and only modest computation power. At the same decoding level, the disparity-compensating coding requires more local memory/computational capacity during rendering than the video-based coder. In return, the disparity-based coder can generate disparity-compensated intermediate images which are needed to attain photorealistic rendering results from sub-sampled light fields. Also, compression is 20% - 40% higher.