Model-based Coding of Multi-Viewpoint Image Data

Light fields can be can be very efficiently compressed if an approximate 3-D geometry model of the depicted scene is available. Model-aided Coding features improved image prediction using geometry for accurate disparity compensation and occlusion detection. In Model-based Coding, object geometry is employed to convert images into texture maps that are then progressively coded.

Approximate geometry is derived from all images using a volumetric reconstruction algorithm. The voxel model is triangulated, yielding a triangle mesh representation of the surface. The triangle mesh is decimated until mesh accuracy matches voxel model resolution. Embedded Mesh Coding is used to compress the geometry representation. Images are predicted in hierachical order. The residual prediction error is DCT-coded to achieve high reconstruction quality. 3-D scene geometry allows warping recorded images to new viewpoints, augmenting the original light field and improving image-based rendering quality.

---

Model-based Codec

---

paper

For objects of genus 0 (homeomorph to a sphere), an efficient mapping has been found between object surface and texture plane. Using octahedral-based approximations of object geometry, all images are converted into texture maps. The texture maps are progressively coded using a 4-D subband compression scheme. Duing decoding, texture-map reconstruction quality can be continuously traded for decoding time. As the Model-based Coder provides a triangle-mesh representation of object geometry and multiple view-dependent texture maps, graphics hardware can be directly applied to accelerate rendering. Model-based Progressive Decoding: MPEG demo Penguin (550kb)

Model-based Rendering: MPEG demo Mouse (630kb)

Model-based Rendering: MPEG demo Garfield (630kb)