Research Reports

3D hand reconstruction from image data is a widely-studied problem in com-
puter vision and graphics, and has a particularly high relevance for virtual
and augmented reality. Although several 3D hand reconstruction approaches
leverage hand models as a strong prior to resolve ambiguities and achieve a
more robust reconstruction, most existing models account only for the hand
shape and poses and do not model the texture. To fill this gap, in this work
we present the first parametric texture model of human hands. Our model
spans several dimensions of hand appearance variability (e.g., related to gen-
der, ethnicity, or age) and only requires a commodity camera for data acqui-
sition. Experimentally, we demonstrate that our appearance model can be
used to tackle a range of challenging problems such as 3D hand reconstruc-
tion from a single monocular image. Furthermore, our appearance model
can be used to define a neural rendering layer that enables training with a
self-supervised photometric loss. We make our model publicly available.

We present a novel real-time approach for user-guided intrinsic decomposition

of static scenes captured by an RGB-D sensor. In the

first step, we acquire a three-dimensional representation of the scene

using a dense volumetric reconstruction framework. The obtained

reconstruction serves as a proxy to densely fuse reflectance estimates

and to store user-provided constraints in three-dimensional space.

User constraints, in the form of constant shading and reflectance

strokes, can be placed directly on the real-world geometry using

an intuitive touch-based interaction metaphor, or using interactive

mouse strokes. Fusing the decomposition results and constraints in

three-dimensional space allows for robust propagation of this information

to novel views by re-projection.We leverage this information

to improve on the decomposition quality of existing intrinsic video

decomposition techniques by further constraining the ill-posed decomposition

problem. In addition to improved decomposition quality,

we show a variety of live augmented reality applications such as

recoloring of objects, relighting of scenes and editing of material

appearance.

This paper contributes a novel sensing approach to support on- and above-skin finger input for interaction on the move. WatchSense uses a depth sensor embedded in a wearable device to expand the input space to neighboring areas of skin and the space above it. Our approach addresses challenging camera-based tracking conditions, such as oblique viewing angles and occlusions. It can accurately detect fingertips, their locations, and whether they are touching the skin or hovering above it. It extends previous work that supported either mid-air or multitouch input by simultaneously supporting both. We demonstrate feasibility with a compact, wearable prototype attached to a user's forearm (simulating an integrated depth sensor). Our prototype---which runs in real-time on consumer mobile devices---enables a 3D input space on the back of the hand. We evaluated the accuracy and robustness of the approach in a user study. We also show how WatchSense increases the expressiveness of input by interweaving mid-air and multitouch for several interactive applications.

Real-time simultaneous tracking of hands manipulating and interacting with

external objects has many potential applications in augmented reality, tangible

computing, and wearable computing. However, due to dicult occlusions,

fast motions, and uniform hand appearance, jointly tracking hand and object

pose is more challenging than tracking either of the two separately. Many

previous approaches resort to complex multi-camera setups to remedy the occlusion

problem and often employ expensive segmentation and optimization

steps which makes real-time tracking impossible. In this paper, we propose

a real-time solution that uses a single commodity RGB-D camera. The core

of our approach is a 3D articulated Gaussian mixture alignment strategy tailored

to hand-object tracking that allows fast pose optimization. The alignment

energy uses novel regularizers to address occlusions and hand-object

contacts. For added robustness, we guide the optimization with discriminative

part classication of the hand and segmentation of the object. We

conducted extensive experiments on several existing datasets and introduce

a new annotated hand-object dataset. Quantitative and qualitative results

show the key advantages of our method: speed, accuracy, and robustness.

This paper advances a novel markerless hand tracking method for interactive applications. FullHand uses input from RGB and depth cameras in a desktop setting. It combines, in a voting scheme, a discriminative, part-based pose retrieval with a generative pose estimation method based on local optimization. We develop this approach to enable: (1) capturing hand articulations with high number of degrees of freedom, including the motion of all fingers, (2) sufficient precision, shown in a dataset of user-generated gestures, and (3) a high framerate of 50 fps for one hand. We discuss the design of free-hand interactions with the tracker and present several demonstrations ranging from simple (few DOFs) to complex (finger individuation plus global hand motion), including mouse operation, a first-person shooter and virtual globe navigation. A user study on the latter shows that free-hand interactions implemented for the tracker can equal mouse-based interactions in user performance.

In this report we present a novel method for detecting partial symmetries

in very large point clouds of 3D city scans. Unlike previous work, which

was limited to data sets of a few hundred megabytes maximum, our method

scales to very large scenes. We map the detection problem to a nearestneighbor

search in a low-dimensional feature space, followed by a cascade of

tests for geometric clustering of potential matches. Our algorithm robustly

handles noisy real-world scanner data, obtaining a recognition performance

comparable to state-of-the-art methods. In practice, it scales linearly with

the scene size and achieves a high absolute throughput, processing half a

terabyte of raw scanner data over night on a dual socket commodity PC.

Many computer vision and computational photography applications

essentially solve an image enhancement problem. The image has been

deteriorated by a specific noise process, such as aberrations from camera

optics and compression artifacts, that we would like to remove. We

describe a framework for learning-based image enhancement. At the core of

our algorithm lies a generic regularization framework that comprises a

prior on natural images, as well as an application-specific conditional

model based on Gaussian processes. In contrast to prior learning-based

approaches, our algorithm can instantly learn task-specific degradation

models from sample images which enables users to easily adapt the

algorithm to a specific problem and data set of interest. This is

facilitated by our efficient approximation scheme of large-scale Gaussian

processes. We demonstrate the efficiency and effectiveness of our approach

by applying it to example enhancement applications including single-image

super-resolution, as well as artifact removal in JPEG- and JPEG

2000-encoded images.

Removing dynamic objects from videos is an extremely challenging problem that

even visual effects professionals often solve with time-consuming manual

frame-by-frame editing.

We propose a new approach to video completion that can deal with complex scenes

containing dynamic background and non-periodical moving objects.

We build upon the idea that the spatio-temporal hole left by a removed object

can be filled with data available on other regions of the video where the

occluded objects were visible.

Video completion is performed by solving a large combinatorial problem that

searches for an optimal pattern of pixel offsets from occluded to unoccluded

regions.

Our contribution includes an energy functional that generalizes well over

different scenes with stable parameters, and that has the desirable convergence

properties for a graph-cut-based optimization.

We provide an interface to guide the completion process that both reduces

computation time and allows for efficient correction of small errors in the

result.

We demonstrate that our approach can effectively complete complex,

high-resolution occlusions that are greater in difficulty than what existing

methods have shown.

We introduce morphable part models for smart shape manipulation using an assembly

of deformable parts with appropriate boundary conditions. In an analysis

phase, we characterize the continuous allowable variations both for the individual

parts and their interconnections using Gaussian shape models with low

rank covariance. The discrete aspect of how parts can be assembled is captured

using a shape grammar. The parts and their interconnection rules are learned

semi-automatically from symmetries within a single object or from semantically

corresponding parts across a larger set of example models. The learned discrete

and continuous structure is encoded as a graph. In the interaction phase, we

obtain an interactive yet intuitive shape deformation framework producing realistic

deformations on classes of objects that are difficult to edit using existing

structure-aware deformation techniques. Unlike previous techniques, our method

uses self-similarities from a single model as training input and allows the user

to reassemble the identified parts in new configurations, thus exploiting both the

discrete and continuous learned variations while ensuring appropriate boundary

conditions across part boundaries.

In this paper, we investigate the problem of statistical reconstruction of

piecewise linear manifold topology. Given a noisy, probably undersampled point

cloud from a one- or two-manifold, the algorithm reconstructs an approximated

most likely mesh in a Bayesian sense from which the sample might have been

taken. We incorporate statistical priors on the object geometry to improve the

reconstruction quality if additional knowledge about the class of original

shapes is available. The priors can be formulated analytically or learned from

example geometry with known manifold tessellation. The statistical objective

function is approximated by a linear programming / integer programming problem,

for which a globally optimal solution is found. We apply the algorithm to a set

of 2D and 3D reconstruction examples, demon-strating that a statistics-based

manifold reconstruction is feasible, and still yields plausible results in

situations where sampling conditions are violated.

In this report we describe the MPI Informatics building

model that provides the data of the Max-Planck-Institut

f\"{u}r Informatik (MPII) building. We present our

motivation for this work and its relationship to

reproducibility of a scientific research. We describe the

dataset acquisition and creation including geometry,

luminaires, surface reflectances, reference photographs etc.

needed to use this model in testing of algorithms. The

created dataset can be used in computer graphics and beyond,

in particular in global illumination algorithms with focus

on realistic and predictive image synthesis. Outside of

computer graphics, it can be used as general source of real

world geometry with an existing counterpart and hence also

suitable for computer vision.

Structure tensors are a common tool for orientation estimation in

image processing and computer vision. We present a generalization of

the traditional second-order model to a higher-order structure

tensor (HOST), which is able to model more than one significant

orientation, as found in corners, junctions, and multi-channel images. We

provide a theoretical analysis and a number of mathematical tools

that facilitate practical use of the HOST, visualize it using a

novel glyph for higher-order tensors, and demonstrate how it can be

applied in an improved integrated edge, corner, and junction

In fluorescent materials, energy from a certain band of incident wavelengths is

reflected or reradiated at larger wavelengths, i.e. with lower energy per

photon. While fluorescent materials are common in everyday life, they have

received little attention in computer graphics. Especially, no bidirectional

reflectance measurements of fluorescent materials have been available so far. In

this paper, we develop the concept of a bispectral BRDF, which extends the

well-known concept of the bidirectional reflectance distribution function (BRDF)

to account for energy transfer between wavelengths. Using a bidirectional and

bispectral measurement setup, we acquire reflectance data of a variety of

fluorescent materials, including vehicle paints, paper and fabric. We show

bispectral renderings of the measured data and compare them with reduced

versions of the bispectral BRDF, including the traditional RGB vector valued

BRDF. Principal component analysis of the measured data reveals that for some

materials the fluorescent reradiation spectrum changes considerably over the

range of directions. We further show that bispectral BRDFs can be efficiently

acquired using an acquisition strategy based on principal components.

In this paper, we address the problem of detecting partial symmetries in

3D objects. In contrast to previous work, our algorithm is able to match

deformed symmetric parts: We first develop an algorithm for the case of

approximately isometric deformations, based on matching graphs of

surface feature lines that are annotated with intrinsic geometric

properties. The sensitivity to non-isometry is controlled by tolerance

parameters for each such annotation. Using large tolerance values for

some of these annotations and a robust matching of the graph topology

yields a more general symmetry detection algorithm that can detect

similarities in structures that have undergone strong deformations. This

approach for the first time allows for detecting partial intrinsic as

well as more general, non-isometric symmetries. We evaluate the

recognition performance of our technique for a number synthetic and

real-world scanner data sets.

We present an approach to automatically compute the complexity of a

given 3D shape. Previous approaches have made use of geometric

and/or topological properties of the 3D shape to compute

complexity. Our approach is based on shape appearance and estimates

the complexity of a given 3D shape according to how 2D views of the

shape diverge from each other. We use similarity among views of the

3D shape as the basis for our complexity computation. Hence our

approach uses claims from psychology that humans mentally represent

3D shapes as organizations of 2D views and, therefore, mimics how

humans gauge shape complexity. Experimental results show that our

approach produces results that are more in agreement with the human

notion of shape complexity than those obtained using previous

approaches.

Crease surfaces are two-dimensional manifolds along which a scalar

field assumes a local maximum (ridge) or a local minimum (valley) in

a constrained space. Unlike isosurfaces, they are able to capture

extremal structures in the data. Creases have a long tradition in

image processing and computer vision, and have recently become a

popular tool for visualization. When extracting crease surfaces,

degeneracies of the Hessian (i.e., lines along which two eigenvalues

are equal), have so far been ignored. We show that these loci,

however, have two important consequences for the topology of crease

surfaces: First, creases are bounded not only by a side constraint

on eigenvalue sign, but also by Hessian degeneracies. Second, crease

surfaces are not in general orientable. We describe an efficient

algorithm for the extraction of crease surfaces which takes these

insights into account and demonstrate that it produces more accurate

results than previous approaches. Finally, we show that DT-MRI

streamsurfaces, which were previously used for the analysis of

planar regions in diffusion tensor MRI data, are mathematically

ill-defined. As an example application of our method, creases in a

measure of planarity are presented as a viable substitute.

Tracking of human motion in video is usually tackled either

by local optimization or filtering approaches. While

local optimization offers accurate estimates but often looses

track due to local optima, particle filtering can recover from

errors at the expense of a poor accuracy due to overestimation

of noise. In this paper, we propose to embed global

stochastic optimization in a tracking framework. This new

optimization technique exhibits both the robustness of filtering

strategies and a remarkable accuracy. We apply the

optimization to an energy function that relies on silhouettes

and color, as well as some prior information on physical

constraints. This framework provides a general solution to

markerless human motion capture since neither excessive

preprocessing nor strong assumptions except of a 3D model

are required. The optimization provides initialization and

accurate tracking even in case of low contrast and challenging

illumination. Our experimental evaluation demonstrates

the large improvements obtained with this technique.

It comprises a quantitative error analysis comparing the

approach with local optimization, particle filtering, and a

heuristic based on particle filtering.

This paper presents a representation of visemes that defines a measure

of similarity between different visemes, and a system of viseme

categories. The representation is derived from a statistical data

analysis of feature points on 3D scans, using Locally Linear

Embedding (LLE). The similarity measure determines which available

viseme and triphones to use to synthesize 3D face animation for a

novel audio file. From a corpus of dynamic recorded 3D mouth

articulation data, our system is able to find the best suited sequence

of triphones over which to interpolate while reusing the

coarticulation information to obtain correct mouth movements over

time. Due to the similarity measure, the system can deal with

relatively small triphone databases and find the most appropriate

candidates. With the selected sequence of database triphones, we can

finally morph along the successive triphones to produce the final

articulation animation.

In an entirely data-driven approach, our automated procedure for

defining viseme categories reproduces the groups of related visemes

that are defined in the phonetics literature.

We present a novel approach to real-time shape editing that produces

physically plausible deformations using an efficient and

easy-to-implement volumetric approach. Our algorithm alternates between

a linear tetrahedral Laplacian deformation step and a differential

update in which rotational transformations are approximated. By means of

this iterative process we can achieve non-linear deformation results

while having to solve only linear equation systems. The differential

update step relies on estimating the rotational component of the

deformation relative to the rest pose. This makes the method very stable

as the shape can be reverted to its rest pose even after extreme

deformations. Only a few point handles or area handles imposing

an orientation are needed to achieve high quality deformations, which

makes the approach intuitive to use. We show that our technique is well

suited for interactive shape manipulation and also provides an elegant

way to animate models with captured motion data.

We present an implementation approach to high-speed Marching

Cubes, running entirely on the Graphics Processing Unit of Shader Model

3.0 and 4.0 graphics hardware. Our approach is based on the interpretation

of Marching Cubes as a stream compaction and expansion process, and is

implemented using the HistoPyramid, a hierarchical data structure

previously only used in GPU data compaction. We extend the HistoPyramid

structure to allow for stream expansion, which provides an efficient

method for generating geometry directly on the GPU, even on Shader Model

3.0 hardware. Currently, our algorithm outperforms all other known

GPU-based iso-surface extraction algorithms. We describe our

implementation and present a performance analysis on several generations

of graphics hardware.

We present an approach for estimating the 3D position and in case of

articulated objects also the joint configuration from segmented 2D

images. The pose estimation without initial information is a challenging

optimization problem in a high dimensional space and is essential for

texture acquisition and initialization of model-based tracking

algorithms. Our method is able to recognize the correct object in the

case of multiple objects and estimates its pose with a high accuracy.

The key component is a particle-based global optimization method that

converges to the global minimum similar to simulated annealing. After

detecting potential bounded subsets of the search space, the particles

are divided into clusters and migrate to the most attractive cluster as

the time increases. The performance of our approach is verified by means

of real scenes and a quantative error analysis for image distortions.

Our experiments include rigid bodies and full human bodies.

We present a novel framework for efficiently computing the indirect

illumination in diffuse and moderately glossy scenes using density estimation

techniques.

A vast majority of existing global illumination approaches either quickly

computes an approximate solution, which may not be adequate for previews, or

performs a much more time-consuming computation to obtain high-quality results

for the indirect illumination. Our method improves photon density estimation,

which is an approximate solution, and leads to significantly better visual

quality in particular for complex geometry, while only slightly increasing the

computation time. We perform direct splatting of photon rays, which allows us

to use simpler search data structures. Our novel lighting computation is

derived from basic radiometric theory and requires only small changes to

existing photon splatting approaches.

Since our density estimation is carried out in ray space rather than on

surfaces, as in the commonly used photon mapping algorithm, the results are

more robust against geometrically incurred sources of bias. This holds also in

combination with final gathering where photon mapping often overestimates the

illumination near concave geometric features. In addition, we show that our

splatting technique can be extended to handle moderately glossy surfaces and

can be combined with traditional irradiance caching for sparse sampling and

filtering in image space.

Interacting and annealing are two powerful strategies that are applied

in different areas of stochastic modelling and data analysis.

Interacting particle systems approximate a distribution of interest by a

finite number of particles where the particles interact between the time

steps. In computer vision, they are commonly known as particle filters.

Simulated annealing, on the other hand, is a global optimization method

derived from statistical mechanics. A recent heuristic approach to fuse

these two techniques for motion capturing has become known as annealed

particle filter. In order to analyze these techniques, we rigorously

derive in this paper two algorithms with annealing properties based on

the mathematical theory of interacting particle systems. Convergence

results and sufficient parameter restrictions enable us to point out

limitations of the annealed particle filter. Moreover, we evaluate the

impact of the parameters on the performance in various experiments,

including the tracking of articulated bodies from noisy measurements.

Our results provide a general guidance on suitable parameter choices for

different applications.

Bernstein polynomials are a classical tool in Computer Aided Design to

create smooth maps

with a high degree of local control.

They are used for the construction of B\'ezier surfaces, free-form

deformations, and many other applications.

However, classical Bernstein polynomials are only defined for simplices

and parallelepipeds.

These can in general not directly capture the shape of arbitrary

objects. Instead,

a tessellation of the desired domain has to be done first.

We construct smooth maps on arbitrary sets of polytopes

such that the restriction to each of the polytopes is a Bernstein

polynomial in mean value coordinates

(or any other generalized barycentric coordinates).

In particular, we show how smooth transitions between different

domain polytopes can be ensured.

Since their introduction, mean value coordinates enjoy ever increasing

popularity in computer graphics and computational mathematics

because they exhibit a variety of good properties. Most importantly,

they are defined in the whole plane which allows interpolation and

extrapolation without restrictions. Recently, mean value coordinates

were generalized to spheres and to $\mathbb{R}^{3}$. We show that these

spherical and 3D mean value coordinates are well-defined on the whole

sphere and the whole space $\mathbb{R}^{3}$, respectively.

Animated characters that move and gesticulate appropriately with spoken

text are useful in a wide range of applications. Unfortunately, they are

very difficult to generate, even more so when a unique, individual

movement style is required. We present a system that is capable of

producing full-body gesture animation for given input text in the style of

a particular performer. Our process starts with video of a performer whose

gesturing style we wish to animate. A tool-assisted annotation process is

first performed on the video, from which a statistical model of the

person.s particular gesturing style is built. Using this model and tagged

input text, our generation algorithm creates a gesture script appropriate

for the given text. As opposed to isolated singleton gestures, our gesture

script specifies a stream of continuous gestures coordinated with speech.

This script is passed to an animation system, which enhances the gesture

description with more detail and prepares a refined description of the

motion. An animation subengine can then generate either kinematic or

physically simulated motion based on this description. The system is

capable of creating animation that replicates a particular performance in

the video corpus, generating new animation for the spoken text that is

consistent with the given performer.s style and creating performances of a

given text sample in the style of different performers.

Free-Viewpoint Video of Human Actors allows photo-

realistic rendering of real-world people under novel viewing

conditions. Dynamic Reflectometry extends the concept of free-view point

video

and allows rendering in addition under novel lighting

conditions. In this work, we present an enhanced method for capturing

human shape and motion as well as dynamic surface reflectance

properties from a sparse set of input video streams.

We augment our initial method for model-based relightable

free-viewpoint video in several ways. Firstly, a single-skin

mesh is introduced for the continuous appearance of the model.

Moreover an algorithm to detect and

compensate lateral shifting of textiles in order to improve

temporal texture registration is presented. Finally, a

structured resampling approach is introduced which enables

reliable estimation of spatially varying surface reflectance

despite a static recording setup.

The new algorithm ingredients along with the Relightable 3D

Video framework enables us to realistically reproduce the

appearance of animated virtual actors under different

lighting conditions, as well as to interchange surface

attributes among different people, e.g. for virtual

dressing. Our contribution can be used to create 3D

renditions of real-world people under arbitrary novel

lighting conditions on standard graphics hardware.

In this report we describe the details of the backward compatible high

dynamic range (HDR) video compression algorithm. The algorithm is

designed to facilitate a smooth transition from standard low dynamic

range (LDR) video to high fidelity high dynamic range content. The HDR

and the corresponding LDR video frames are decorrelated and then

compressed into a single MPEG stream, which can be played on both

existing DVD players and HDR-enabled devices.

In this report, a new free-form shape deformation approach is proposed.

We combine a skeleton-driven mesh deformation technique with discrete

differential coordinates in order to create natural-looking global shape

deformations. Given a triangle mesh, we first extract a skeletal mesh, a

two-sided

Voronoi-based approximation of the medial axis. Next the skeletal mesh

is modified by free-form deformations. Then a desired global shape

deformation is obtained by reconstructing the shape corresponding to the

deformed skeletal mesh. The reconstruction is based on using discrete

differential coordinates.

Our method preserves fine geometric details and original shape

thickness because of using discrete differential coordinates and

skeleton-driven deformations. We also develop a new mesh evolution

technique which allow us to eliminate possible global and local

self-intersections of the deformed mesh while preserving fine geometric

details. Finally, we present a multiresolution version of our approach

in order to simplify and accelerate the deformation process.

We present a novel versatile, fast and simple framework to generate

highquality animations of scanned human characters from input motion data.

Our method is purely mesh-based and, in contrast to skeleton-based

animation, requires only a minimum of manual interaction. The only manual

step that is required to create moving virtual people is the placement of

a sparse set of correspondences between triangles of an input mesh and

triangles of the mesh to be animated. The proposed algorithm implicitly

generates realistic body deformations, and can easily transfer motions

between human erent shape and proportions. erent types of input data, e.g.

other animated meshes and motion capture les, in just the same way.

Finally, and most importantly, it creates animations at interactive frame

rates. We feature two working prototype systems that demonstrate that our

method can generate lifelike character animations from both marker-based

and marker-less optical motion capture data.

Top-k query processing is an important building block for ranked retrieval,

with applications ranging from text and data integration to distributed

aggregation of network logs and sensor data.

Top-k queries operate on index lists for a query's elementary conditions

and aggregate scores for result candidates. One of the best implementation

methods in this setting is the family of threshold algorithms, which aim

to terminate the index scans as early as possible based on lower and upper

bounds for the final scores of result candidates. This procedure

performs sequential disk accesses for sorted index scans, but also has the option

of performing random accesses to resolve score uncertainty. This entails

scheduling for the two kinds of accesses: 1) the prioritization of different

index lists in the sequential accesses, and 2) the decision on when to perform

random accesses and for which candidates.

The prior literature has studied some of these scheduling issues, but only for each of the two access types in isolation.

The current paper takes an integrated view of the scheduling issues and develops

novel strategies that outperform prior proposals by a large margin.

Our main contributions are new, principled, scheduling methods based on a Knapsack-related

optimization for sequential accesses and a cost model for random accesses.

The methods can be further boosted by harnessing probabilistic estimators for scores,

selectivities, and index list correlations.

We also discuss efficient implementation techniques for the

underlying data structures.

In performance experiments with three different datasets (TREC Terabyte, HTTP server logs, and IMDB),

our methods achieved significant performance gains compared to the best previously known methods:

a factor of up to 3 in terms of execution costs, and a factor of 5

in terms of absolute run-times of our implementation.

Our best techniques are close to a lower bound for the execution cost of the considered class

of threshold algorithms.

We present a novel algorithm for accurately denoising static and

time-varying range data. Our approach is inspired by

similarity-based non-local image filtering. We show that our

proposed method is easy to implement and outperforms recent

state-of-the-art filtering approaches. Furthermore, it preserves fine

shape features and produces an accurate smoothing result in the

spatial and along the time domain.

Image Pyramids are frequently used in porting non-local algorithms to graphics

hardware. A Histogram pyramid (short: HistoPyramid), a special version

of image pyramid, sums up the number of active entries in a 2D

image hierarchically. We show how a HistoPyramid can be utilized as an implicit indexing data

structure, allowing us to convert a sparse matrix into a coordinate list

of active cell entries (a point list) on graphics hardware . The algorithm

reduces a highly sparse matrix with N elements to a list of its M active

entries in O(N) + M (log N) steps, despite the restricted graphics

hardware architecture. Applications are numerous, including feature

detection, pixel classification and binning, conversion of 3D volumes to

particle clouds and sparse matrix compression.

In this paper we address the problem of fast construction of spatial

hierarchies for ray tracing with applications in animated environments

including non-rigid animations. We discuss properties of currently

used techniques with $O(N \log N)$ construction time for kd-trees and

bounding volume hierarchies. Further, we propose a hybrid data

structure blending between a spatial kd-tree and bounding volume

primitives. We keep our novel hierarchical data structures

algorithmically efficient and comparable with kd-trees by the use of a

cost model based on surface area heuristics. Although the time

complexity $O(N \log N)$ is a lower bound required for construction of

any spatial hierarchy that corresponds to sorting based on

comparisons, using approximate method based on discretization we

propose a new hierarchical data structures with expected $O(N \log\log N)$ time complexity. We also discuss constants behind the construction algorithms of spatial hierarchies that are important in practice. We document the performance of our algorithms by results obtained from the implementation tested on nine different scenes.

We introduce an empirical model for multiple scattering in heterogeneous

translucent objects for which classical approximations such as the

dipole approximation to the di usion equation are no longer valid.

Motivated by the exponential fall-o of scattered intensity with

distance, di use subsurface scattering is represented as a sum of

exponentials per surface point plus a modulation texture. Modeling

quality can be improved by using an anisotropic model where exponential

parameters are determined per surface location and scattering direction.

We validate the scattering model for a set of planar object samples

which were recorded under controlled conditions and quantify the

modeling error. Furthermore, several translucent objects with complex

geometry are captured and compared to the real object under similar

illumination conditions.

Accurate estimations of geometric properties of a surface (a curve) from

its discrete approximation are important for many computer graphics and

computer vision applications.

To assess and improve the quality of such an approximation we assume

that the

smooth surface (curve) is known in general form. Then we can represent the

surface (curve) by a Taylor series expansion

and compare its geometric properties with the corresponding discrete

approximations. In turn

we can either prove convergence of these approximations towards the true

properties

as the edge lengths tend to zero, or we can get hints how

to eliminate the error.

In this report we propose and study discrete schemes for estimating

the curvature and torsion of a smooth 3D curve approximated by a polyline.

Thereby we make some interesting findings about connections between

(smooth) classical curves

and certain estimation schemes for polylines.

Furthermore, we consider several popular schemes for estimating the

surface normal

of a dense triangle mesh interpolating a smooth surface,

and analyze their asymptotic properties.

Special attention is paid to the mean curvature vector, that

approximates both,

normal direction and mean curvature. We evaluate a common discrete

approximation and

show how asymptotic analysis can be used to improve it.

It turns out that the integral formulation of the mean curvature

\begin{equation*}

H = \frac{1}{2 \pi} \int_0^{2 \pi} \kappa(\phi) d\phi,

\end{equation*}

can be computed by an exact quadrature formula.

The same is true for the integral formulations of Gaussian curvature and

the Taubin tensor.

The exact quadratures are then used to obtain reliable estimates

of the curvature tensor of a smooth surface approximated by a dense triangle

mesh. The proposed method is fast and often demonstrates a better

performance

than conventional curvature tensor estimation approaches. We also show

that the curvature tensor approximated by

our approach converges towards the true curvature tensor as the edge

lengths tend to zero.

In this paper, we develop a method for generating

a high-quality approximation of a noisy set of points sampled

from a smooth surface by a sparse triangle mesh. The main

idea of the method consists of defining an appropriate set

of approximation centers and use them as the vertices

of a mesh approximating given scattered data.

To choose the approximation centers, a clustering

procedure is used. With every point of the input data

we associate a local uncertainty

measure which is used to estimate the importance of

the point contribution to the reconstructed surface.

Then a global uncertainty measure is constructed from local ones.

The approximation centers are chosen as the points where

the global uncertainty measure attains its local minima.

It allows us to achieve a high-quality approximation of uncertain and

noisy point data by a sparse mesh. An interesting feature of our

approach

is that the uncertainty measures take into account the normal

directions

estimated at the scattered points.

In particular it results in accurate reconstruction of high-curvature

regions.

\begin{abstract}

Passive optical motion capture is able to provide

authentically animated, photo-realistically and view-dependently

textured models of real people.

To import real-world characters into virtual environments, however,

also surface reflectance properties must be known.

We describe a video-based modeling approach that captures human

motion as well as reflectance characteristics from a handful of

synchronized video recordings.

The presented method is able to recover spatially varying

reflectance properties of clothes % dynamic objects ?

by exploiting the time-varying orientation of each surface point

with respect to camera and light direction.

The resulting model description enables us to match animated subject

appearance to different lighting conditions, as well as to

interchange surface attributes among different people, e.g. for

virtual dressing.

Our contribution allows populating virtual worlds with correctly relit,

real-world people.\\

\end{abstract}

In this paper, we present an image-based framework that acquires the

reflectance properties of a human face. A range scan of the face is not

required.

Based on a morphable face model, the system estimates the 3D

shape, and establishes point-to-point correspondence across images taken

from different viewpoints, and across different individuals' faces.

This provides a common parameterization of all reconstructed surfaces

that can be used to compare and transfer BRDF data between different

faces. Shape estimation from images compensates deformations of the face

during the measurement process, such as facial expressions.

In the common parameterization, regions of homogeneous materials on the

face surface can be defined a-priori. We apply analytical BRDF models to

express the reflectance properties of each region, and we estimate their

parameters in a least-squares fit from the image data. For each of the

surface points, the diffuse component of the BRDF is locally refined,

which provides high detail.

We present results for multiple analytical BRDF models, rendered at

novelorientations and lighting conditions.

We discuss FaceSketch, an interface for 2D facial human-like cartoon

sketching. The basic paradigm in FaceSketch is to offer a 2D interaction

style and feel while employing 3D techniques to facilitate various

tasks involved in drawing and redrawing faces from different views.

The system works by accepting freeform strokes denoting head, eyes, nose,

and other facial features, constructing an internal 3D model that

conforms to the input silhouettes, and redisplaying the result in

simple sketchy style from any user-specified viewing direction. In a

manner similar to conventional 2D drawing process, the displayed shape may

be changed by oversketching silhouettes, and hatches and strokes may be

added within its boundary. Implementation-wise, we demonstrate the

feasibility of using simple point primitive as a fundamental 3D

modeling primitive in a sketch-based system. We discuss relatively

simple but robust and efficient point-based algorithms for shape

inflation, modification and display in 3D view. We discuss the

feasibility of our ideas using a number of example interactions and

facial sketches.

We present a new Monte Carlo method for solving the global illumination

problem in environments with general geometry descriptions and

light emission and scattering properties. Current

Monte Carlo global illumination algorithms are based

on generic density estimation techniques that do not take into account any

knowledge about the nature of the data points --- light and potential

particle hit points --- from which a global illumination solution is to be

reconstructed. We propose a novel estimator, especially designed

for solving linear integral equations such as the rendering equation.

The resulting single-pass global illumination algorithm promises to

combine the flexibility and robustness of bi-directional

path tracing with the efficiency of algorithms such as photon mapping.

We develop a new approach to reconstruct non-discrete models from gridded

volume samples. As a model, we use quadratic, trivariate super splines on

a uniform tetrahedral partition $\Delta$. The approximating splines are

determined in a natural and completely symmetric way by averaging local

data samples such that appropriate smoothness conditions are automatically

satisfied. On each tetrahedron of $\Delta$ , the spline is a polynomial of

total degree two which provides several advantages including the e cient

computation, evaluation and visualization of the model. We apply

Bernstein-B{\´e}zier techniques wellknown in Computer Aided Geometric

Design to compute and evaluate the trivariate spline and its gradient.

With this approach the volume data can be visualized e ciently e.g. with

isosurface ray-casting. Along an arbitrary ray the splines are univariate,

piecewise quadratics and thus the exact intersection for a prescribed

isovalue can be easily determined in an analytic and exact way. Our

results confirm the e ciency of the method and demonstrate a high visual

quality for rendered isosurfaces.

Angle Based Flattening is a robust parameterization method that finds a

quasi-conformal mapping by solving a non-linear optimization problem. We

take advantage of a characterization of convex planar drawings of

triconnected graphs to introduce new boundary constraints. This prevents

boundary intersections and avoids post-processing of the parameterized

mesh. We present a simple transformation to e ectively relax the

constrained minimization problem, which improves the convergence of the

optimization method. As a natural extension, we discuss the construction

of Delaunay flat meshes. This may further enhance the quality of the

resulting parameterization.

In recent years, the convergence of Computer Vision and Computer Graphics has put forth

new research areas that work on scene reconstruction from and analysis of multi-view video

footage. In free-viewpoint video, for example, new views of a scene are generated from an arbitrary viewpoint

in real-time from a set of real multi-view input video streams.

The analysis of real-world scenes from multi-view video

to extract motion information or reflection models is another field of research that

greatly benefits from high-quality input data.

Building a recording setup for multi-view video involves a great effort on the hardware

as well as the software side. The amount of image data to be processed is huge,

a decent lighting and camera setup is essential for a naturalistic scene appearance and

robust background subtraction, and the computing infrastructure has to enable

real-time processing of the recorded material.

This paper describes the recording setup for multi-view video acquisition that enables the

synchronized recording

of dynamic scenes from multiple camera positions under controlled conditions. The requirements

to the room and their implementation in the separate components of the studio are described in detail.

The efficiency and flexibility of the room is demonstrated on the basis of the results

that we obtain with a real-time 3D scene reconstruction system, a system for non-intrusive optical

motion capture and a model-based free-viewpoint video system for human actors.

~

Objects with mirroring optical characteristics are left out of the

scope of most 3D scanning methods. We present here a new automatic

acquisition approach, shape-from-distortion, that focuses on that

category of objects, requires only a still camera and a color

monitor, and produces range scans (plus a normal and a reflectance

map) of the target.

Our technique consists of two steps: first, an improved

environment matte is captured for the mirroring object, using the

interference of patterns with different frequencies in order to

obtain sub-pixel accuracy. Then, the matte is converted into a

normal and a depth map by exploiting the self coherence of a

surface when integrating the normal map along different paths.

The results show very high accuracy, capturing even smallest

surface details. The acquired depth maps can be further processed

using standard techniques to produce a complete 3D mesh of the

object.

We consider the linear space of piecewise polynomials in three variables

which are globally smooth, i.e., trivariate $C^1$ splines. The splines are

defined on a uniform tetrahedral partition $\Delta$, which is a natural

generalization of the four-directional mesh. By using Bernstein-B{\´e}zier

techniques, we establish formulae for the dimension of the $C^1$ splines

of arbitrary degree.

This tutorial highlights some recent results on the acquisition and

interactive display of high quality 3D models. For further use in

photorealistic rendering or interactive display, a high quality

representation must capture two different things: the shape of the

model represented as a geometric description of its surface and on the

other hand the physical properties of the object. The physics of the

material which an object is made of determine its appearance, e.g. the

object's color, texture, deformation or reflection properties.

The tutorial shows how computer vision and computer graphics

techniques can be seamlessly integrated into a single framework for

the acquisition, processing, and interactive display of high quality

3D models.

Seven tone mapping methods currently available to display high

dynamic range images were submitted to perceptual evaluation in

order to find the attributes most predictive of the success of

a robust all-around tone mapping algorithm.

The two most salient Stimulus Space dimensions underlying the perception of

a set of images produced by six of the tone mappings were revealed

using INdividual Differences SCALing (INDSCAL) analysis;

and an ideal preference point

within the INDSCAL-derived Stimulus Space was determined for a group of

11 observers using PREFerence MAPping (PREFMAP) analysis.

Interpretation of the INDSCAL results was aided by

pairwise comparisons of images that led to an ordering of the images according

to which were more or less natural looking.

The measurement of accurate material properties is an important step

towards photorealistic rendering. Many real-world objects are composed

of a number of materials that often show subtle changes even within a

single material. Thus, for photorealistic rendering both the general

surface properties as well as the spatially varying effects of the

object are needed.

We present an image-based measuring method that robustly detects the

different materials of real objects and fits an average bidirectional

reflectance distribution function (BRDF) to each of them. In order to

model the local changes as well, we project the measured data for each

surface point into a basis formed by the recovered BRDFs leading to a

truly spatially varying BRDF representation.

A high quality model of a real object can be generated with relatively

few input data. The generated model allows for rendering under

arbitrary viewing and lighting conditions and realistically reproduces

the appearance of the original object.

Visibility computations are the most time-consuming part of global

illumination algorithms. The cost is amplified by the fact that

quite often identical or similar information is recomputed multiple

times. In particular this is the case when multiple images of the

same scene are to be generated under varying lighting conditions

and/or viewpoints. But even for a single image with static

illumination, the computations could be accelerated by reusing

visibility information for many different light paths.

In this report we describe a general method of precomputing, storing,

and reusing visibility information for light transport in a number

of different types of scenes. In particular, we consider general

parametric surfaces, triangle meshes without a global

parameterization, and participating media.

We also reorder the light transport in such a way that the

visibility information is accessed in structured memory access

patterns. This yields a method that is well suited for SIMD-style

parallelization of the light transport, and can efficiently be

implemented both in software and using graphics hardware. We finally

demonstrate applications of the method to highly efficient

precomputation of BRDFs, bidirectional texture functions, light

fields, as well as near-interactive volume lighting.

Digital documents often require highly detailed representations of

real world objects. This is especially true for advanced e-commerce

applications and other multimedia data bases like online

encyclopaedias or virtual museums. Their further success will

strongly depend on advances in the field of high quality object

representation, distribution and rendering.

This tutorial highlights some recent results on the acquisition and

interactive display of high quality 3D models and shows how these

results can be seamlessly integrated with previous work into a

single framework for the acquisition, processing, transmission, and

interactive display of high quality 3D models on the Web.

Medial axis transform (MAT) is very sensitive to the noise,

in the sense that, even if a shape is perturbed only slightly,

the Hausdorff distance between the MATs of the original shape and

the perturbed one may be large. But it turns out that MAT is stable,

if we view this phenomenon with the one-sided Hausdorff distance,

rather than with the two-sided Hausdorff distance. In this paper,

we show that, if the original domain is weakly injective,

which means that the MAT of the domain has no end point which

is the center of an inscribed circle osculating the boundary at

only one point, the one-sided Hausdorff distance of the original

domain's MAT with respect to that of the perturbed one is bounded

linearly with the Hausdorff distance of the perturbation.

We also show by example that the linearity of this bound cannot be

achieved for the domains which are not weakly injective. In particular,

these results apply to the domains with the sharp corners, which

were excluded in the past. One consequence of these results is that

we can clarify theoretically the notion of extracting ``the essential

part of the MAT'', which is the heart of the existing pruning methods.

This tutorial highlights some recent results on the acquisition and

interactive display of high quality 3D models. For further use in

photorealistic rendering or object recognition, a high quality

representation must capture two different things: the shape of the

model represented as a geometric description of its surface and on the

other hand the appearance of the material or materials it is made of,

e.g. the object's color, texture, or reflection properties.

The tutorial shows how computer vision and computer graphics

techniques can be seamlessly integrated into a single framework for

the acquisition, processing, and interactive display of high quality

3D models.

Although the Hausdorff distance is a popular device

to measure the differences between sets,

it is not natural for some specific classes of sets,

especially for the medial axis transform

which is defined as the set of all pairs

of the centers and the radii of the maximal balls

contained in another set.

In spite of its many advantages and possible applications,

the medial axis transform has one great weakness,

namely its instability under the Hausdorff distance

when the boundary of the original set is perturbed.

Though many attempts have been made for the resolution of this phenomenon,

most of them are heuristic in nature

and lack precise error analysis.

While traditional computer aided design (CAD) is mainly based on

piecewise polynomial surface representations, the recent advances in

the efficient handling of polygonal meshes have made available a set

of powerful techniques which enable sophisticated modeling operations

on freeform shapes. In this tutorial we are going to give a detailed

introduction into the various techniques that have been proposed over

the last years. Those techniques address important issues such as

surface generation from discrete samples (e.g. laser scans) or from

control meshes (ab initio design); complexity control by adjusting the

level of detail of a given 3D-model to the current application or to

the available hardware resources; advanced mesh optimization

techniques that are based on the numerical simulation of physical

material (e.g. membranes or thin plates) and finally the generation

and modification of hierarchical representations which enable

sophisticated multiresolution modeling functionality.

In this research report we present a framework for evaluating and comparing the

quality of various lossy image compression techniques based on a

multiresolution decomposition of the image data. In contrast to many other

publications, much attention is paid to the interdependencies of the individual

steps of such compression techniques. In our result section we are able to

show that it is quite worthwile to fine-tune the parameters of every step to

obtain an optimal interplay among them, which in turn leads to a higher

reconstruction quality.