max planck institut
informatik
informatik
Using global illumination algorithms significantly improves the realism of synthetic images. Both the hardware and algorithms at hand are now mature enough to consider lighting simulations even for animated environments feasible. We have proposed several new algorithmic solutions which aim at either providing a fast feedback to the user in interactive scenarios, in order to improve the usability of lighting simulation, or at shortening production time for high quality movie sequences.
A vast majority of global illumination techniques is based on some form of ray shooting therefore it is instrumental to make the ray computation as efficient as possible. In Section 0.1.1 we describe our real time ray tracing solution and its applications to the global illumination computation. For many practical applications the scene geometry remains mostly static and the main issue is dynamic lighting simulation. In Section 0.1.2 we discuss our research on Precomputed Radiance Transfer techniques and we present their application in a virtual reality system. In Sections 0.1.3 and 0.1.4 we overview our off-line global illumination and rendering techniques which are aimed at the high quality rendering. In Section 0.1.3 we discuss some extensions for core path tracing algorithms which improve the error estimation and result in a more efficient sample selection. In Section 0.1.4 we present our animation rendering solutions which exploit temporal coherence between subsequent animation frames to improve the rendering efficiency and reduce temporal aliasing.
Investigators: Ingo Wald, Johannes Günther, in conjunction with the Computer Graphics Group at Saarland University
One of the key ingredients to any kind of global illumination algorithm is ray tracing. Thus, realtime ray tracing offers a great potential for eventually computing global illumination at interactive rates. For this reason, we are also pursuing the topic of realtime ray tracing, both in core ray tracing technology, as well as in its application to fast and interactive global illumination.
The research on realtime ray tracing is performed mostly within the RTRT/OpenRT realtime ray tracing project [Wal04,Web]. This project aims at achieving realtime ray tracing performance on commodity desktop PCs and small PC clusters. It was originally founded at the neighboring Saarland University [WSBW01,WSB01], but is now also being actively worked on at the MPII.
While core ray tracing technology - i.e., the ability to quickly intersect rays with geometry - is not automatically related to global illumination, it bears a huge potential of accelerating global illumination as well. Therefore, we are also pursuing core ray tracing research without direct application to global illumination.
Since 2003, we have been working mostly on extending the OpenRT core to support more and different kinds of geometry. In particular, the OpenRT system has been extended to now also support free form geometry [BWS04], point-based models, and isosurface data sets [MKF+04,WFM+]. For all these extensions, we have taken special care to achieve a clean integration in which all existing modules continue to work as before. In particular the handling of dynamic scenes [WBS03a] and the interactive global illumination framework [WKB+02] (see below) work for these new kinds of geometry just as well as on triangles.
Additionally, we have been working extensively on extending the OpenRT system to handle massively complex data sets. This not only comprises large volumetric data sets like the Visible Female and the 8 Gigabyte Lawrence-Livermore data set [WFM+] (see Figure 0.1), but in particular the handling of massively complex triangular scenes.
For this purpose, the OpenRT system has been extended by an interactive out-of-core (OOC) ray tracing framework [WDS04,DWS04,Wal04]. In contrast to earlier approaches towards ray tracing massively complex models [WSB01], the new framework is significantly more powerful. Even for the complete ``Boeing 777'' model - which consists of 350 million triangles and roughly 40GB scene data on disk - we achieve interactive performance on a single dual-Opteron desktop PC [WDS04]. This is achieved by explicitly managing the scene data in an operating system friendly way, avoiding page-faults by killing off potentially page-faulting rays, and by loading the missing data asynchronously from the disk. As not all missing data can be loaded within the same frame, we use some precomputed, lightfield-like proxy data for faulting rays [WDS04]. Once all data is loaded into memory, the complete model is rendered without any kind of simplifications or approximations at all. Of course, all usual effects like shadows, reflections, etc. are fully supported even for the out-of-core ray tracing module.
Since its original publication in 2004, the system has been extended to also run on SGI Altix shared memory systems, and is currently being evaluated by Boeing Corp. for integration into the industrial workflow.
|
|
| (a) | (b) |
|
|
| (c) | (d) |
Ray tracing is an essential tool for virtually all of todays global illumination algorithms. By itself however, it does not automatically compute global effects like smooth shadows or indirect illumination. In order to leverage the availability of realtime ray tracing for achieving interactive global illumination as well, the Instant Global Illumination Method [WKB+02,BWS03,Wal04] is particularly designed to fit the requirements of a distributed ray tracing engine. Instant Global Illumination - which was developed in 2002 at Saarland University - shoots lights particles into the scene, and places ``Virtual Point Lights'' (VPLs) where these light particles hit the scene. The scene is then ray traced, and illumination is computed by shooting shadow rays towards these VPLs. As the VPLs represent both direct and indirect (diffuse) illumination, all of the most important kinds of illumination can be simulated: Hard as well as smooth shadows, direct and indirect illumination, reflections, refraction, and - in combination with Photon Mapping [Jen97,Jen96] - even some simple form of caustics (see Figure 0.2a-c).
In 2003, this method was extended to also handle massively complex scenes with millions of polygons and thousands of light sources [WBS03b,WPS+03,Wal04]. This is achieved via a two-step algorithm: In a first step, a path tracer gathers information on the importance of the different light sources. This information is then used in the second step to concentrate all computations to the actually visible light sources, thereby achieving reasonable rendering quality even under interactive performance (see Figure 0.2d).
|
|
| (a) | (b) |
|
|
| (c) | (d) |
In the way discussed above, Instant Global Illumination is a powerful technique for simulating global illumination. However, it cannot capture all kinds of light transport equally well. In particular caustics (i.e., the patterns caused by light being focussed through reflection or refraction) cannot be captured easily with this method - whereas the original method [WKB+02] still supported some simple form of photon mapping. Unfortunately, due to their often bright and visually important patterns, caustics are a desirable effect in global illumination, and their absence often results in a dull, unrealistic appearance.
For this reason, we have developped a new method [GWS04] that allows for interactively simulating caustics via a variant of Photon Mapping as originally introduced by Jensen et al. [Jen97,Jen96]. Our method builds on the existing OpenRT Realtime Ray Tracing framework [Wal04], but is particularly designed for interactive, distributed Photon Mapping. In particular, this required solving three problems: reducing the photon generation time, reducing the photon query time, and fitting the whole process into the constraints imposed by a distributed realtime ray tracing system.
Virtually all literature on Photon Mapping proposes to store the photons in a balanced kd-tree, which allows for asymptotically optimal O(log N) query times, and additionally does not require any additional memory at all. However, the balancing can be detrimental on performance, in particular if the photons are irregularly distributed. Therefore, we have proposed a new heuristic for accelerating photon mapping, the Voxel Volume Heuristic (VVH) [WGS04]. The VVH explicitly unbalances the kd-tree used for storing the photons, in a way such that the expected value of the number of touched photons during photon query gets minimized. For typical scenes, the VVH can reduce the number of touched photons by significant factors, translating to photon map queries that are 1.3 to 5.8 times faster [WGS04] than a balanced kd-tree.
Typically, photon mapping spends several seconds per frame in photon generation, which obviously is not possible for an interactive application. Therefore, we have developped new methods for significantly reducing the photon generation time. First, we have adapted our data structures and algorithms to allow for temporal accumulation of photon maps during runtime, thereby allowing for switching to a coarser, cheaper photon map during scene modifications. After scene modification, the photon map resolution is gradually and progressively improved, eventually reaching a very high-quality photon map after only few frames. Additionally, we use a variant of Selected Photon Tracing [DBMS02] to trace only those photon paths that actually contribute to caustics, reducing the number of computed trajectories by a significant factor.
In order to achieve interactive performance, our framework builds on the existing OpenRT realtime ray tracing framework [Wal04]. To exploit its full performance, our system had to be specially designed to allow for distributed execution across a cluster of PCs [GWS04]. To this end, we combine ideas from Instant Global Illumination [WKB+02] with a static load balancing scheme and a framework for distributed Photon Mapping. In particular, different clients work on only a subset of all photons using a different, coarser photon map on each client. The individual clients' results are then combined in a filtering stage on the server.
Using our distributed Photon Mapping framework, we achieve realtime photon mapping performance of up to 22 frames per second on non-trivial scenes, while still allowing for interactively updating all aspects of the scene, including lighting, material properties, and geometry (see Figure 0.3).
|
|
| (a) | (b) |
|
|
| (c) | (d) |
Investigators: Thomas Annen, Kirill Dmitriev, Grzegorz Krawczyk, Karol Myszkowski, and Michael Goesele
Precomputed Radiance Transfer (PRT) [SKS02] has become a standard base method for realtime global illumination rendering of static scenes. Computationally expensive parts of the simulation such as material reflectance computation, light integration, and interreflection/occlusion determination are done offline in a precomputation phase. Incident lighting as well as the light transport within the scene are expressed in low order Spherical Harmonics which allows compact storage and fast reconstruction, hence attaining realtime framerates.
However, one drawback of this class of algorithms is that they assume that the incident light emitted by low-frequency but distant lighting is sampled only once at the center of the object/scene. According to this assumption it is possible to reuse the precomputed incident light vector along all scene vertices, which gives a significant speedup. For interaction with nearby emitters, computing the incident radiance at the center of an object only is not sufficient. The previous mentioned techniques [SKS02] then require expensive sampling of the incident radiance field at many points distributed over the object. Therefore, we continued research along this path (in collaboration with Jan Kautz and Fredo Durand from the MIT) and developed a new technique [AKDS04] that alleviates this costly requirement by using a first-order Taylor expansion of the spherical-harmonic lighting coefficients around a point. We propose an interpolation scheme based on these gradients requiring far fewer samples (one is often sufficient). We show that the gradient of the incident-radiance spherical harmonics can be computed for little additional cost compared to the coefficients alone. We introduce a semi-analytical formula to calculate this gradient at run-time and describe how a simple vertex shader can interpolate the shading. The interpolated representation of the incident radiance can be used with any low-frequency light-transfer technique using Spherical Harmonics.
The difference in quality is depicted in Figure 0.4 which shows a hand model shaded using various means and lit by a green and red local area light. Figure 0.4(a) shows the standard method of using only one sample at the center (the method of Sloan et al. [SKS02]). Figure 0.4(b) illustrates the same model shaded using one sample and the analytical gradient, as we propose. The last image Figure 0.4(c) shows a reference rendering where the incident radiance field is sampled per-vertex. As our method achieves quite good results with only little additional cost it can easily be used to enhance the level of realism in future games or entertainment application. ATIs new X850 Demo ``Ruby: Dangerous Curves'' (http://www.ati.com/developer/demos/rx850.html) incorporated our Gradient method and demonstrates its applicability for real applications.
|
In another project, we investigated the application of the PRT in a VR system developed by our group for the DaimlerChrysler company [DAK+04]. The system is tailored for the CAVE environment with five projection screens. The goal of the system is the global illumination modeling in the car interior under free driving conditions (refer to Figure 0.5a). We illuminate the car using dynamically changing High Dynamic Range (HDR) environment maps and use the PRT technique for the global illumination computation. We leverage the PRT method to handle scenes with non-trivial topology represented by complex meshes (refer to Figures 0.5b and c). Also, we propose a hybrid of PRT and final gathering approach for high-quality rendering of objects with complex Bi-directional Reflectance Distribution Function (BRDF). We use this method for predictive rendering of the navigation LCD panel based on its measured BRDF. Since the global illumination computation leads to HDR images we propose a tone mapping algorithm tailored specifically for the CAVE. We employ head tracking to identify the observed screen region and derive for it proper luminance adaptation conditions, which are then used for tone mapping on all walls in the CAVE. We distribute our global illumination and tone mapping computation on all CPUs and GPUs available in the CAVE, which enables us to achieve interactive performance even for the costly final gathering approach.
|
In previous work, we introduced an digitization approach for complex light sources [GGHS03a]. Continuing the collaboration with Xavier Granier and Wolfgang Heidrich, we developed a hardware-accelerated rendering system that allows to render a scene illuminated with a captured light source at interactive frame rates [GGHS03b].
Investigator: Kirill Dmitriev
Path tracing is known as a powerful and robust global illumination technique, which incurs very high computation costs. One of the methods to increase the path tracing performance is to use adaptive sampling algorithms that choose the number of samples in each pixel as a function of the illumination complexity.
Adaptive sampling techniques typically applied in path tracing are not progressive because all pixel samples are needed for a robust error estimation. Thus progressive computation would require storing all the samples for all the pixels, which is too expensive. The lack of progressivity is a big disadvantage of adaptive path tracing algorithms because a user may become aware of some unwanted effects on the image only after a quite significant time. In [DS04] we propose a new estimate of local error in path tracing. The new technique is lightweight in terms of both the memory and execution time and is suitable for progressive rendering. Also, even though a perceptual error metric is used, it allows for changes of any tone mapping parameter in the course of the computation. In this case none of the previous effort is lost, the error distribution is immediately updated and used for refining the solution.
The comparison of estimated and real error (computed in respect to the reference solution) shows that our algorithm produces a very robust and precise error estimate. Figure 0.6(a) shows a sample image generated using our technique and the corresponding sample point density (Figure 0.6(b)).
|
The solution of global illumination problems usually involves the evaluation of high dimensional integrals. This is why Monte Carlo methods, whose convergence rate does not depend on the integral dimensionality, are the usual choice in the global illumination domain. Nevertheless, modern computer graphics studies are focused on quasi-random sampling whose efficiency depends on the dimensionality of the integration space, but under certain conditions it enables to achieve even higher convergence rates than the stochastic methods. In [VGDK04] we propose an approach to the bidirectional ray tracing algorithm that reduces the overheads of the quasi-Monte Carlo approach caused by a high dimensionality of the integral and discontinuities in the integrand. The pseudo-random and quasi-random integration methods are compared using the scene examples with known analytic solutions.
Investigators: Takehiro Tawara, Vlastimil Havran, Kirill Dmitriev, Cyrille Damez, and Karol Myszkowski
Producing high quality animations featuring rich object appearance and compelling lighting effects is very time consuming using traditional frame-by-frame rendering systems. In this section we present a number of our global illumination and rendering solutions that exploit temporal coherence in lighting distribution for subsequent frames to improve the computation performance and overall animation quality [TMD+04]. Our strategy relies on extending into temporal domain well-known global illumination techniques such as density estimation photon tracing, photon mapping, and bi-directional path tracing, which were originally designed to handle static scenes only. For a more complete overview of research in this field refer to our survey paper [DDM03].
Photon tracing and density estimation are well established techniques in the global illumination computation and rendering of high-quality animation sequences. Using traditional density estimation techniques it is difficult to remove stochastic noise inherent for photon-based methods while avoiding overblurring lighting details. To overcome those problems we investigated the use of bilateral filtering for lighting reconstruction based on the local density of photon hit points [WMM+04]. Bilateral filtering is applied in spatio-temporal domain and provides control over the level-of-details in reconstructed lighting. All changes of lighting below this level are treated as stochastic noise and are suppressed. Bilateral filtering proves to be efficient in preserving sharp features in lighting which is in particular important for high-quality caustic reconstruction. Also, flickering between subsequent animation frames is substantially reduced due to extending bilateral filtering into temporal domain.
In high-quality animation rendering costly ``final gathering'' technique is commonly used. We extend this technique into temporal domain by storing previously computed incoming radiance samples and refreshing them evenly in space and time using some aging criteria [TMS04b]. The approach is based upon a two-pass photon mapping algorithm with irradiance cache, but it can be applied also in other gathering methods. Since we noticed that many artifacts in animation is caused by strong secondary emitters we extend the photon mapping algorithm to handle them efficiently [TMS04a]. We introduce an additional photon map as an implicit representation of such light sources. At the rendering stage this map is used for the explicit sampling of strong indirect lighting in a similar way as it is usually performed for primary light sources.
In all solutions introduced so far temporal coherence was exploited in the object space. In [HDMS03] we present a rendering architecture for computing multiple frames at once by re-using global illumination samples in the image space. For each sample representing a given point in the scene we update its view-dependent components for each frame and add its contribution to pixels identified through the compensation of camera and object motion. This leads naturally to a high quality motion blur and significantly reduces the cost of illumination computations. The required visibility information is provided using a custom ray tracing acceleration data structure for multiple frames simultaneously. We demonstrate that precise and costly global illumination techniques such as bidirectional path tracing become affordable in this rendering architecture, achieving speedup by an order of magnitude (refer to sample frames in Figure 0.7 obtained using this technique). The reuse of information in the computationally intensive shaders was further studied in [KHS04].
| ||
|
