Publications

%0 Journal Article %A Arabadzhiyska, Elena %A Tursun, Cara %A Seidel, Hans-Peter %A Didyk, Piotr %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Practical Saccade Prediction for Head-Mounted Displays: Towards a Comprehensive Model : %G eng %U http://hdl.handle.net/21.11116/0000-000C-B76B-E %R 10.1145/3568311 %7 2023 %D 2023 %J ACM Transactions on Applied Perception %V 20 %N 1 %& 1 %P 1 - 23 %Z sequence number: 2 %I ACM %C New York, NY %@ false

%0 Journal Article %A Çoğalan, Uğur %A Bemana, Mojtaba %A Seidel, Hans-Peter %A Myszkowski, Karol %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors : %G eng %U http://hdl.handle.net/21.11116/0000-000C-F953-E %R 10.1111/cgf.14748 %D 2023 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 42 %N 2 %I Blackwell-Wiley %C Oxford %@ false %B The European Association for Computer Graphics 43rdAnnual Conference %O EUROGRAPHICS 2023 EG 2023 Saarbrücken, Germany, May 8-12, 2023

%0 Journal Article %A Habermann, Marc %A Xu, Weipeng %A Zollh&#246;fer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T A Deeper Look into DeepCap : (Invited Paper) %G eng %U http://hdl.handle.net/21.11116/0000-0009-8C33-0 %R 10.1109/TPAMI.2021.3093553 %7 2021 %D 2023 %X Human performance capture is a highly important computer vision problem with<br>many applications in movie production and virtual/augmented reality. Many<br>previous performance capture approaches either required expensive multi-view<br>setups or did not recover dense space-time coherent geometry with<br>frame-to-frame correspondences. We propose a novel deep learning approach for<br>monocular dense human performance capture. Our method is trained in a weakly<br>supervised manner based on multi-view supervision completely removing the need<br>for training data with 3D ground truth annotations. The network architecture is<br>based on two separate networks that disentangle the task into a pose estimation<br>and a non-rigid surface deformation step. Extensive qualitative and<br>quantitative evaluations show that our approach outperforms the state of the<br>art in terms of quality and robustness. This work is an extended version of<br>DeepCap where we provide more detailed explanations, comparisons and results as<br>well as applications.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV %J IEEE Transactions on Pattern Analysis and Machine Intelligence %O IEEE Trans. Pattern Anal. Mach. Intell. %V 45 %N 4 %& 4009 %P 4009 - 4002 %I IEEE %C Piscataway, NJ %@ false

%0 Conference Proceedings %A Haynes, Alice %A Reed, Courtney N. %A Nordmoen, Charlotte %A Skach, Sophie %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Being Meaningful: Weaving Soma-Reflective Technological Mediations into the Fabric of Daily Life : %G eng %U http://hdl.handle.net/21.11116/0000-000C-BDEB-7 %R 10.1145/3569009.3571844 %D 2023 %B Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction %Z date of event: 2023-02-26 - 2023-03-01 %C Warsaw, Poland %B TEI '23 %P 1 - 5 %Z sequence number: 68 %I ACM %@ 978-1-4503-9977-7

%0 Journal Article %A Jambon, Cl&#233;ment %A Kerbl, Bernhard %A Kopanas, Georgios %A Diolatzis, Stavros %A Leimk&#252;hler, Thomas %A Drettakis, George %+ External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T NeRFshop: Interactive Editing of Neural Radiance Fields : %G eng %U http://hdl.handle.net/21.11116/0000-000C-BEE0-1 %D 2023 %J Proceedings of the ACM on Computer Graphics and Interactive Techniques %V 6 %N 1 %I ACM %C New York, NY %@ false %U http://www-sop.inria.fr/reves/Basilic/2023/JKKDLD23

%0 Conference Proceedings %A Liao, Kang %A Tricard, Thibault %A Piovar&#269;i, Michal %A Seidel, Hans-Peter %A Babaei, Vahid %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Learning Deposition Policies for Fused Multi-Material 3D Printing : %G eng %U http://hdl.handle.net/21.11116/0000-000C-44C2-C %D 2023 %B IEEE International Conference on Robotics and Automation %Z date of event: 2023-05-29 - 2023-06-02 %C London, UK %B IEEE International Conference on Robotics and Automation %I IEEE

%0 Conference Proceedings %A Reed, Courtney N. %A Strohmeier, Paul %A McPherson, Andrew %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Negotiating Experience and Communicating Information Through Abstract Metaphor : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A035-3 %R 10.1145/3544548.3580700 %D 2023 %B CHI Conference on Human Factors in Computing Systems %Z date of event: 2023-04-23 - 2023-04-28 %C Hamburg, Germany %B CHI '23 %I ACM

%0 Conference Proceedings %A Reed, Courtney N. %A McPherson, Andrew P. %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T The Body as Sound: Unpacking Vocal Embodiment through Auditory Biofeedback : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A02B-F %R 10.1145/3569009.3572738 %D 2023 %B Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction %Z date of event: 2023-02-26 - 2023-03-01 %C Warsaw, Poland %B TEI '23 %P 1 - 15 %Z sequence number: 7 %I ACM %@ 978-1-4503-9977-7

%0 Conference Proceedings %A Reed, Courtney N. %+ Computer Graphics, MPI for Informatics, Max Planck Society %T As the Luthiers Do: Designing with a Living, Growing, Changing Body-Material : %G eng %U http://hdl.handle.net/21.11116/0000-000C-BDD8-C %D 2023 %B ACM CHI Workshop on Body X Materials %Z date of event: 2023-04-23 - 2023-04-23 %C Hamburg, Germany %B CHI&#8217; 23 Workshop - Body x Materials

%0 Report %A Ruan, Lingyan %A Bemana, Mojtaba %A Seidel, Hans-Peter %A Myszkowski, Karol %A Chen, Bin %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Revisiting Image Deblurring with an Efficient ConvNet : %G eng %U http://hdl.handle.net/21.11116/0000-000C-C7B9-3 %U https://arxiv.org/abs/2302.02234 %D 2023 %X Image deblurring aims to recover the latent sharp image from its blurry<br>counterpart and has a wide range of applications in computer vision. The<br>Convolution Neural Networks (CNNs) have performed well in this domain for many<br>years, and until recently an alternative network architecture, namely<br>Transformer, has demonstrated even stronger performance. One can attribute its<br>superiority to the multi-head self-attention (MHSA) mechanism, which offers a<br>larger receptive field and better input content adaptability than CNNs.<br>However, as MHSA demands high computational costs that grow quadratically with<br>respect to the input resolution, it becomes impractical for high-resolution<br>image deblurring tasks. In this work, we propose a unified lightweight CNN<br>network that features a large effective receptive field (ERF) and demonstrates<br>comparable or even better performance than Transformers while bearing less<br>computational costs. Our key design is an efficient CNN block dubbed LaKD,<br>equipped with a large kernel depth-wise convolution and spatial-channel mixing<br>structure, attaining comparable or larger ERF than Transformers but with a<br>smaller parameter scale. Specifically, we achieve +0.17dB / +0.43dB PSNR over<br>the state-of-the-art Restormer on defocus / motion deblurring benchmark<br>datasets with 32% fewer parameters and 39% fewer MACs. Extensive experiments<br>demonstrate the superior performance of our network and the effectiveness of<br>each module. Furthermore, we propose a compact and intuitive ERFMeter metric<br>that quantitatively characterizes ERF, and shows a high correlation to the<br>network performance. We hope this work can inspire the research community to<br>further explore the pros and cons of CNN and Transformer architectures beyond<br>image deblurring tasks.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Sabnis, Nihar %A Wittchen, Dennis %A Reed, Courtney N. %A Pourjafarian, Narjes %A Steimle, J&#252;rgen %A Strohmeier, Paul %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Haptic Servos: Self-Contained Vibrotactile Rendering System for Creating or Augmenting Material Experiences : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A03C-C %R 10.1145/3544548.3580716 %D 2023 %B CHI Conference on Human Factors in Computing Systems %Z date of event: 2023-04-23 - 2023-04-28 %C Hamburg, Germany %B CHI '23 %I ACM

%0 Conference Proceedings %A Sabnis, Nihar %A Wittchen, Dennis %A Vega, Gabriela %A Reed, Courtney N. %A Strohmeier, Paul %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Tactile Symbols with Continuous and Motion-Coupled Vibration: An Exploration of using Embodied Experiences for Hermeneutic Design : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A042-4 %R 10.1145/3544548.3581356 %D 2023 %B CHI Conference on Human Factors in Computing Systems %Z date of event: 2023-04-23 - 2023-04-28 %C Hamburg, Germany %B CHI '23 %I ACM

%0 Journal Article %A Surace, Luca %A Wernikowski, Marek %A Tursun, Cara %A Myszkowski, Karol %A Mantiuk, Rados&#322;aw %A Didyk, Piotr %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Learning GAN-based Foveated Reconstruction to Recover Perceptually Important Image Features : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A00D-1 %R 10.1145/3583072 %7 2023 %D 2023 %J ACM Transactions on Applied Perception %I ACM %C New York, NY %@ false

%0 Journal Article %A Weinrauch, Alexander %A Seidel, Hans-Peter %A Mlakar, Daniel %A Steinberger, Markus %A Zayer, Rhaleb %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and Reeb Graph Construction on Surfaces : %G eng %U http://hdl.handle.net/21.11116/0000-000C-B851-9 %7 2023 %D 2023 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 42 %N 2 %I Blackwell-Wiley %C Oxford %@ false

%0 Conference Proceedings %A Wittchen, Dennis %A Marinez-Missir, Valenin %A Mavali, Sina %A Sabnis, Nihar %A Reed, Courtney N. %A Strohmeier, Paul %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Designing Interactive Shoes for Tactile Augmented Reality : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A04D-9 %R 10.1145/3582700.3582728 %D 2023 %B Augmented Humans International Conference %Z date of event: 2023-03-12 - 2023-03-14 %C Glasgow, UK %B AHs '23 %P 1 - 14 %I ACM %@ 978-1-4503-9984-5

%0 Journal Article %A Ansari, Navid %A Seidel, Hans-Peter %A Babaei, Vahid %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Mixed Integer Neural Inverse Design : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1678-5 %R 10.1145/3528223.3530083 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 4 %& 1 %P 1 - 14 %Z sequence number: 151 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2022 %O ACM SIGGRAPH 2022

%0 Conference Proceedings %A Ansari, Navid %A Seidel, Hans-Peter %A Vahidi Ferdowsi, Nima %A Babaei, Vahid %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Autoinverse: Uncertainty Aware Inversion of Neural Networks : %G eng %U http://hdl.handle.net/21.11116/0000-000C-16F6-6 %D 2022 %B 36th Conference on Neural Information Processing Systems %Z date of event: 2022-11-28 - 2022-12-09 %C New Orleans, LA, USA %B Advances in Neural Information Processing Systems 35 %E Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; Oh, A. %P 8675 - 8686 %I Curran Associates, Inc %U https://openreview.net/pdf?id=dNyCj1AbOb

%0 Report %A Arabadzhiyska, Elena %A Tursun, Cara %A Seidel, Hans-Peter %A Didyk, Piotr %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Practical Saccade Prediction for Head-Mounted Displays: Towards a Comprehensive Model : %G eng %U http://hdl.handle.net/21.11116/0000-000C-16E3-B %U https://arxiv.org/abs/2205.01624 %D 2022 %X Eye-tracking technology is an integral component of new display devices such<br>as virtual and augmented reality headsets. Applications of gaze information<br>range from new interaction techniques exploiting eye patterns to<br>gaze-contingent digital content creation. However, system latency is still a<br>significant issue in many of these applications because it breaks the<br>synchronization between the current and measured gaze positions. Consequently,<br>it may lead to unwanted visual artifacts and degradation of user experience. In<br>this work, we focus on foveated rendering applications where the quality of an<br>image is reduced towards the periphery for computational savings. In foveated<br>rendering, the presence of latency leads to delayed updates to the rendered<br>frame, making the quality degradation visible to the user. To address this<br>issue and to combat system latency, recent work proposes to use saccade landing<br>position prediction to extrapolate the gaze information from delayed<br>eye-tracking samples. While the benefits of such a strategy have already been<br>demonstrated, the solutions range from simple and efficient ones, which make<br>several assumptions about the saccadic eye movements, to more complex and<br>costly ones, which use machine learning techniques. Yet, it is unclear to what<br>extent the prediction can benefit from accounting for additional factors. This<br>paper presents a series of experiments investigating the importance of<br>different factors for saccades prediction in common virtual and augmented<br>reality applications. In particular, we investigate the effects of saccade<br>orientation in 3D space and smooth pursuit eye-motion (SPEM) and how their<br>influence compares to the variability across users. We also present a simple<br>yet efficient correction method that adapts the existing saccade prediction<br>methods to handle these factors without performing extensive data collection.<br> %K Computer Science, Human-Computer Interaction, cs.HC,Computer Science, Graphics, cs.GR

%0 Conference Proceedings %A Bemana, Mojtaba %A Myszkowski, Karol %A Frisvad, Jeppe Revall %A Seidel, Hans-Peter %A Ritschel, Tobias %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Eikonal Fields for Refractive Novel-View Synthesis : %G eng %U http://hdl.handle.net/21.11116/0000-000A-BA61-7 %R 10.1145/3528233.3530706 %D 2022 %B ACM SIGGRAPH %Z date of event: 2022-08-07 - 2022-08-11 %C Vancouver, Canada %B Proceedings SIGGRAPH 2022 Conference Papers Proceedings %E Nandigjav, Munkhtsetseg; Mitra, Niloy J.; Hertzmann, Aaron %P 1 - 9 %Z sequence number: 39 %I ACM %@ 978-1-4503-9337-9

%0 Conference Proceedings %A Chen, Bin %A Piovar&#269;i, Michal %A Wang, Chao %A Seidel, Hans-Peter %A Didyk, Piotr %A Myszkowski, Karol %A Serrano, Ana %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Gloss Management for Consistent Reproduction of Real and Virtual Objects : %G eng %U http://hdl.handle.net/21.11116/0000-000C-167F-E %R 10.1145/3550469.3555406 %D 2022 %B Proceedings SIGGRAPH Asia 2022 %E Jung, Soon Ki; Lee, Jehee; Bargteil, Adam %P 1 - 9 %Z sequence number: 35 %I ACM %@ 978-1-4503-9470-3

%0 Journal Article %A Chizhov, Vassillen %A Georgiev, Iliyan %A Myszkowski, Karol %A Singh, Gurprit %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Perceptual Error Optimization for Monte Carlo Rendering : %G eng %U http://hdl.handle.net/21.11116/0000-000A-BA49-3 %R 10.1145/3504002 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 3 %& 1 %P 1 - 17 %Z sequence number: 26 %I ACM %C New York, NY %@ false

%0 Journal Article %A Chu, Mengyu %A Liu, Lingjie %A Zheng, Quan %A Franz, Erik %A Seidel, Hans-Peter %A Theobalt, Christian %A Zayer, Rhaleb %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data : %G eng %U http://hdl.handle.net/21.11116/0000-000B-6561-6 %R 10.1145/3528223.3530169 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 4 %& 1 %P 1 - 14 %Z sequence number: 119 %I ACM %C New York, NY %@ false %U https://people.mpi-inf.mpg.de/~mchu/projects/PI-NeRF/

%0 Journal Article %A &#199;o&#287;alan, U&#287;ur %A Bemana, Mojtaba %A Myszkowski, Karol %A Seidel, Hans-Peter %A Ritschel, Tobias %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Learning HDR Video Reconstruction for Dual-Exposure Sensors with Temporally-alternating Exposures : %G eng %U http://hdl.handle.net/21.11116/0000-000A-9D95-D %R 10.1016/j.cag.2022.04.008 %7 2022 %D 2022 %J Computers and Graphics %V 105 %& 57 %P 57 - 72 %I Elsevier %C Amsterdam %@ false

%0 Report %A &#199;o&#287;alan, U&#287;ur %A Bemana, Mojtaba %A Seidel, Hans-Peter %A Myszkowski, Karol %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors : %G eng %U http://hdl.handle.net/21.11116/0000-000C-16E8-6 %U https://arxiv.org/abs/2206.09485 %D 2022 %X Video frame interpolation (VFI) enables many important applications that<br>might involve the temporal domain, such as slow motion playback, or the spatial<br>domain, such as stop motion sequences. We are focusing on the former task,<br>where one of the key challenges is handling high dynamic range (HDR) scenes in<br>the presence of complex motion. To this end, we explore possible advantages of<br>dual-exposure sensors that readily provide sharp short and blurry long<br>exposures that are spatially registered and whose ends are temporally aligned.<br>This way, motion blur registers temporally continuous information on the scene<br>motion that, combined with the sharp reference, enables more precise motion<br>sampling within a single camera shot. We demonstrate that this facilitates a<br>more complex motion reconstruction in the VFI task, as well as HDR frame<br>reconstruction that so far has been considered only for the originally captured<br>frames, not in-between interpolated frames. We design a neural network trained<br>in these tasks that clearly outperforms existing solutions. We also propose a<br>metric for scene motion complexity that provides important insights into the<br>performance of VFI methods at the test time.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Journal Article %A Hladk&#253;, Jozef %A Stengel, Michael %A Vining, Nicholas %A Kerbl, Bernhard %A Seidel, Hans-Peter %A Steinberger, Markus %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T QuadStream: A Quad-Based Scene Streaming Architecture for Novel Viewpoint Reconstruction : %G eng %U http://hdl.handle.net/21.11116/0000-000C-208B-3 %R 10.1145/3550454.3555524 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 6 %& 1 %P 1 - 13 %Z sequence number: 233 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2022 %O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

%0 Journal Article %A Huang, Xingchang %A Memari, Pooran %A Seidel, Hans-Peter %A Singh, Gurprit %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Point-Pattern Synthesis using Gabor and Random Filters : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1675-8 %R 10.1111/cgf.14596 %7 2022 %D 2022 %J Computer Graphics Forum %V 41 %N 4 %& 169 %P 169 - 179 %I Wiley-Blackwell %C Oxford %@ false %B Eurographics Symposium on Rendering 2022 %O Eurographics Symposium on Rendering 2022 EGSR 2022 Prague, Czech Republic & Virtual ; 4 - 6 July 2022 %U https://onlinelibrary.wiley.com/share/X44DPUPXHCYNCUKSEBEE?target=10.1111/cgf.14596

%0 Journal Article %A Kopanas, Georgios %A Leimk&#252;hler, Thomas %A Rainer, Gilles %A Jambon, Cl&#233;ment %A Drettakis, George %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations %T Neural Point Catacaustics for Novel-View Synthesis of Reflections : %G eng %U http://hdl.handle.net/21.11116/0000-000C-209B-1 %R 10.1145/3550454.3555497 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 6 %& 1 %P 1 - 15 %Z sequence number: 201 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2022 %O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

%0 Report %A Li, Jizhou %A Chen, Bin %A Zan, Guibin %A Qian, Guannan %A Pianetta, Piero %A Liu, Yijin %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T Subspace Modeling for Fast and High-sensitivity X-ray Chemical Imaging : %G eng %U http://hdl.handle.net/21.11116/0000-000C-C7BE-E %U https://arxiv.org/abs/2201.00259 %D 2022 %X Resolving morphological chemical phase transformations at the nanoscale is of<br>vital importance to many scientific and industrial applications across various<br>disciplines. The TXM-XANES imaging technique, by combining full field<br>transmission X-ray microscopy (TXM) and X-ray absorption near edge structure<br>(XANES), has been an emerging tool which operates by acquiring a series of<br>microscopy images with multi-energy X-rays and fitting to obtain the chemical<br>map. Its capability, however, is limited by the poor signal-to-noise ratios due<br>to the system errors and low exposure illuminations for fast acquisition. In<br>this work, by exploiting the intrinsic properties and subspace modeling of the<br>TXM-XANES imaging data, we introduce a simple and robust denoising approach to<br>improve the image quality, which enables fast and high-sensitivity chemical<br>imaging. Extensive experiments on both synthetic and real datasets demonstrate<br>the superior performance of the proposed method.<br> %K eess.IV,Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Multimedia, cs.MM

%0 Journal Article %A Mueller, Florian %A Semertzidis, Nathan %A Andres, Josh %A Weigel, Martin %A Nanayakkara, Suranga %A Patibanda, Rakesh %A Li, Zhuying %A Strohmeier, Paul %A Knibbe, Jarrod %A Greuter, Stefan %A Obrist, Marianna %A Maes, Pattie %A Wang, Dakuo %A Wolf, Katrin %A Gerber, Liz %A Marshall, Joe %A Kunze, Kai %A Grudin, Jonathan %A Reiterer, Harald %A Byrne, Richard %+ External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations %T Human&#8211;Computer Integration: Towards Integrating the Human Body with the Computational Machine : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1734-0 %R 10.1561/1100000086 %@ 978-1-63828-068-2 %7 2022 %D 2022 %J Foundations and Trends in Human-Computer Interaction %O Foundations and Trends&#174; in Human-Computer Interaction %V 16 %N 1 %& 1 %P 1 - 64 %@ false

%0 Journal Article %A Panetta, Julian %A Mohammadian, Haleh %A Luci, Emiliano %A Babaei, Vahid %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Shape from Release: Inverse Design and Fabrication of Controlled Release Structures : %G eng %U http://hdl.handle.net/21.11116/0000-000B-5E7D-1 %R 10.1145/3550454.3555518 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 6 %& 1 %P 1 - 14 %Z sequence number: 274 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2022 %O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

%0 Journal Article %A Piovar&#269;i, Michal %A Foshey, Michael %A Xu, Jie %A Erps, Timmothy %A Babaei, Vahid %A Didyk, Piotr %A Rusinkiewicz, Szymon %A Matusik, Wojciech %A Bickel, Bernd %+ External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T Closed-Loop Control of Direct Ink Writing via Reinforcement Learning : %G eng %U http://hdl.handle.net/21.11116/0000-000C-178D-C %R 10.1145/3528223.3530144 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 4 %& 1 %P 1 - 10 %Z sequence number: 112 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2022 %O ACM SIGGRAPH 2022

%0 Conference Proceedings %A Pourjafarian, Narjes %A Koelle, Marion %A Mjaku, Fjolla %A Strohmeier, Paul %A Steimle, J&#252;rgen %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Print-A-Sketch: A Handheld Printer for Physical Sketching of Circuits and Sensors on Everyday Surfaces : %G eng %U http://hdl.handle.net/21.11116/0000-000A-215D-9 %R 10.1145/3491102.3502074 %D 2022 %B CHI Conference on Human Factors in Computing Systems %Z date of event: 2022-04-29 - 2022-05-05 %C New Orleans, LA, USA %B CHI '22 %P 1 - 17 %Z sequence number: 270 %I ACM %@ 9781450391573

%0 Conference Proceedings %A Rao, Sukrut %A B&#246;hle, Moritz %A Schiele, Bernt %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Towards Better Understanding Attribution Methods : %G eng %U http://hdl.handle.net/21.11116/0000-000A-6F91-6 %R 10.1109/CVPR52688.2022.00998 %D 2022 %B 35th IEEE/CVF Conference on Computer Vision and Pattern Recognition %Z date of event: 2022-06-19 - 2022-06-24 %C New Orleans, LA, USA %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 10213 - 10222 %I IEEE %@ 978-1-6654-6946-3

%0 Conference Proceedings %A Reed, Courtney N. %A Nordmoen, Charlotte %A Martelloni, Andrea %A Lepri, Giacomo %A Robson, Nicole %A Zayas-Garin, Eevee %A Cotton, Kelsey %A Mice, Lia %A McPherson, Andrew %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations %T Exploring Experiences with New Musical Instruments through Micro-phenomenology : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A023-7 %R 10.21428/92fbeb44.b304e4b1 %D 2022 %B International Conference on New Interfaces for Musical Expression %Z date of event: 2022-06-28 - 2022-07-01 %C Auckland, New Zealand %B NIME 2022 %I PubPub

%0 Conference Proceedings %A Reed, Courtney N. %A Skach, Sophie %A Strohmeier, Paul %A McPherson, Andrew P. %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Singing Knit: Soft Knit Biosensing for Augmenting Vocal Performances : %G eng %U http://hdl.handle.net/21.11116/0000-000A-2178-A %R 10.1145/3519391.3519412 %D 2022 %B Augmented Humans International Conference %Z date of event: 2022-03-13 - 2022-03-15 %C Munich, Germany (Hybrid) %B AHs '22 %P 170 - 183 %I ACM

%0 Conference Proceedings %A Ruan, Lingyan %A Chen, Bin %A Li, Jizhou %A Lam, Miuling %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Learning to Deblur using Light Field Generated and Real Defocus Images : %G eng %U http://hdl.handle.net/21.11116/0000-000C-4354-A %R 10.1109/CVPR52688.2022.01582 %D 2022 %B 35th IEEE/CVF Conference on Computer Vision and Pattern Recognition %Z date of event: 2022-06-19 - 2022-06-24 %C New Orleans, LA, USA %X Although considerable progress has been made in semantic scene understanding<br>under clear weather, it is still a tough problem under adverse weather<br>conditions, such as dense fog, due to the uncertainty caused by imperfect<br>observations. Besides, difficulties in collecting and labeling foggy images<br>hinder the progress of this field. Considering the success in semantic scene<br>understanding under clear weather, we think it is reasonable to transfer<br>knowledge learned from clear images to the foggy domain. As such, the problem<br>becomes to bridge the domain gap between clear images and foggy images. Unlike<br>previous methods that mainly focus on closing the domain gap caused by fog --<br>defogging the foggy images or fogging the clear images, we propose to alleviate<br>the domain gap by considering fog influence and style variation simultaneously.<br>The motivation is based on our finding that the style-related gap and the<br>fog-related gap can be divided and closed respectively, by adding an<br>intermediate domain. Thus, we propose a new pipeline to cumulatively adapt<br>style, fog and the dual-factor (style and fog). Specifically, we devise a<br>unified framework to disentangle the style factor and the fog factor<br>separately, and then the dual-factor from images in different domains.<br>Furthermore, we collaborate the disentanglement of three factors with a novel<br>cumulative loss to thoroughly disentangle these three factors. Our method<br>achieves the state-of-the-art performance on three benchmarks and shows<br>generalization ability in rainy and snowy scenes.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 16283 - 16292 %I IEEE %@ 978-1-6654-6946-3

%0 Journal Article %A Sala&#252;n, Corentin %A Georgiev, Iliyan %A Seidel, Hans-Peter %A Singh, Gurprit %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Scalable Multi-Class Sampling via Filtered Sliced Optimal Transport : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1716-2 %R 10.1145/3550454.3555484 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 6 %& 1 %P 1 - 14 %Z sequence number: 261 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2022 %O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

%0 Journal Article %A Sala&#252;n, Corentin %A Gruson, Adrien %A Hua, Binh-Son %A Hachisuka, Toshiya %A Singh, Gurprit %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Regression-based Monte Carlo integration : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1793-4 %R 10.1145/3528223.3530095 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 4 %& 1 %P 1 - 14 %Z sequence number: 79 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2022 %O ACM SIGGRAPH 2022

%0 Conference Proceedings %A Schneider, Oliver %A Fruchard, Bruno %A Wittchen, Dennis %A Joshi, Bibhushan Raj %A Freitag, Georg %A Degraen, Donald %A Strohmeier, Paul %+ External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Sustainable Haptic Design: Improving Collaboration, Sharing, and Reuse in Haptic Design Research : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1748-A %R 10.1145/3491101.3503734 %D 2022 %B CHI Conference on Human Factors in Computing Systems %Z date of event: 2022-04-30 - 2022-05-05 %C New Orleans, LA, USA %B CHI '22 %E Barbosa, Simone; Lampe, Cliff; Appert, Caroline; Shamma, David A. %P 1 - 5 %Z sequence number: 79 %I ACM %@ 9781450391566

%0 Conference Proceedings %A Shen, Zhijie %A Lin, Chunyu %A Liao, Kang %A Nie, Lang %A Zheng, Zishuo %A Zhao, Yao %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations %T PanoFormer: Panorama Transformer for Indoor 360&#176; Depth Estimation : %G eng %U http://hdl.handle.net/21.11116/0000-000C-9585-5 %R 10.1007/978-3-031-19769-7_12 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni Maria; Hassner, Tal %P 195 - 211 %I Springer %@ 978-3-031-19768-0 %B Lecture Notes in Computer Science %N 13661 %U https://rdcu.be/c5Ays

%0 Conference Proceedings %A Shimada, Soshi %A Golyanik, Vladislav %A Li, Zhi %A P&#233;rez, Patrick %A Xu, Weipeng %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T HULC: 3D HUman Motion Capture with Pose Manifold SampLing and Dense Contact Guidance : %G eng %U http://hdl.handle.net/21.11116/0000-000B-7918-3 %R 10.1007/978-3-031-20047-2_30 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni Maria; Hassner, Tal %P 516 - 533 %I Springer %@ 978-3-031-20046-5 %B Lecture Notes in Computer Science %N 13682 %U https://rdcu.be/c0aoZ

%0 Book Section %A Strohmeier, Paul %A Mottelson, Aske %A Pohl, Henning %A McIntosh, Jess %A Knibbe, Jarrod %A Bergstr&#246;m, Joanna %A Jansen, Yvonne %A Hornb&#230;k, Kasper %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations %T Body-based user interfaces : %G eng %U http://hdl.handle.net/21.11116/0000-000C-A013-9 %D 2022 %B The Routledge Handbook of Bodily Awareness %E Alsmith, Adrian J. T.; Longo, Matthew R. %P 478 - 502 %I Routledge %C London %@ 9780429321542

%0 Report %A Wang, Chao %A Serrano, Ana %A Pan, X. %A Chen, Bin %A Seidel, Hans-Peter %A Theobalt, Christian %A Myszkowski, Karol %A Leimk&#252;hler, Thomas %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild : %G eng %U http://hdl.handle.net/21.11116/0000-000B-9D08-C %U https://arxiv.org/abs/2211.12352 %D 2022 %X Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving<br>as a partial observation of the High Dynamic Range (HDR) visual world. Despite<br>limited dynamic range, these LDR images are often captured with different<br>exposures, implicitly containing information about the underlying HDR image<br>distribution. Inspired by this intuition, in this work we present, to the best<br>of our knowledge, the first method for learning a generative model of HDR<br>images from in-the-wild LDR image collections in a fully unsupervised manner.<br>The key idea is to train a generative adversarial network (GAN) to generate HDR<br>images which, when projected to LDR under various exposures, are<br>indistinguishable from real LDR images. The projection from HDR to LDR is<br>achieved via a camera model that captures the stochasticity in exposure and<br>camera response function. Experiments show that our method GlowGAN can<br>synthesize photorealistic HDR images in many challenging cases such as<br>landscapes, lightning, or windows, where previous supervised generative models<br>produce overexposed images. We further demonstrate the new application of<br>unsupervised inverse tone mapping (ITM) enabled by GlowGAN. Our ITM method does<br>not need HDR images or paired multi-exposure images for training, yet it<br>reconstructs more plausible information for overexposed regions than<br>state-of-the-art supervised learning models trained on such data.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

%0 Journal Article %A Wang, Chao %A Chen, Bin %A Seidel, Hans-Peter %A Myszkowski, Karol %A Serrano, Ana %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Learning a self-supervised tone mapping operator via feature contrast masking loss : %G eng %U http://hdl.handle.net/21.11116/0000-000A-BA09-B %R 10.1111/cgf.14459 %7 2022 %D 2022 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 41 %N 2 %& 71 %P 71 - 84 %I Blackwell-Wiley %C Oxford %@ false %B The European Association for Computer Graphics 43rdAnnual Conference %O EUROGRAPHICS 2022 EG 2022 Reims, France, April 25 - 29, 2022

%0 Conference Proceedings %A Wang, Yidong %A Chen, Hao %A Fan, Yue %A Sun, Wang %A Tao, Ran %A Hou, Wenxin %A Wang, Renjie %A Yang, Linyi %A Zhou, Zhi %A Guo, Lan-Zhe %A Qi, Heli %A Wu, Zhen %A Li, Yu-Feng %A Nakamura, Satoshi %A Ye, Wei %A Savvides, Marios %A Raj, Bhiksha %A Shinozaki, Takahiro %A Schiele, Bernt %A Wang, Jindong %A Xie, Xing %A Zhang, Yue %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T USB: A Unified Semi-supervised Learning Benchmark for Classification : %G eng %U http://hdl.handle.net/21.11116/0000-000C-184A-7 %D 2022 %B 36th Conference on Neural Information Processing Systems %Z date of event: 2022-11-28 - 2022-12-09 %C New Orleans, LA, USA %B Advances in Neural Information Processing Systems 35 %E Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; Oh, A. %P 3938 - 3961 %I Curran Associates, Inc. %U https://openreview.net/pdf?id=QeuwINa96C

%0 Conference Proceedings %A Wittchen, Dennis %A Spiel, Katta %A Fruchard, Bruno %A Degraen, Donald %A Schneider, Oliver %A Freitag, Georg %A Strohmeier, Paul %+ External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T TactJam: An End-to-End Prototyping Suite for Collaborative Design of On-Body Vibrotactile Feedback : %G eng %U http://hdl.handle.net/21.11116/0000-000A-20B9-1 %R 10.1145/3490149.3501307 %D 2022 %B Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction %Z date of event: 2022-02-13 - 2022-02-16 %C Daejeon, Republic of Korea (Online) %B TEI '22 %P 1 - 13 %Z sequence number: 1 %I ACM %@ 978-1-4503-9147-4

%0 Journal Article %A Wolski, Krzysztof %A Trutoiu, Laura %A Dong, Zhao %A Shen, Zhengyang %A Mackenzie, Kevin %A Chapiro, Alexandre %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations External Organizations %T Geo-Metric: A Perceptual Dataset of Distortions on Faces : %G eng %U http://hdl.handle.net/21.11116/0000-000C-209F-D %R 10.1145/3550454.3555475 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 6 %& 1 %P 1 - 13 %Z sequence number: 215 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2022 %O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

%0 Journal Article %A Wolski, Krzysztof %A Zhong, Fangcheng %A Myszkowski, Karol %A Mantiuk, Rafa&#322; K. %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Dark Stereo: Improving Depth Perception Under Low Luminance : %G eng %U http://hdl.handle.net/21.11116/0000-000A-BA6D-B %R 10.1145/3528223.3530136 %7 2022 %D 2022 %J ACM Transactions on Graphics %V 41 %N 4 %& 1 %P 1 - 12 %Z sequence number: 146 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2022 %O ACM SIGGRAPH 2022

%0 Conference Proceedings %A Ali, Junaid %A Lahoti, Preethi %A Gummadi, Krishna P. %+ Computer Graphics, MPI for Informatics, Max Planck Society Databases and Information Systems, MPI for Informatics, Max Planck Society External Organizations %T Accounting for Model Uncertainty in Algorithmic Discrimination : %G eng %U http://hdl.handle.net/21.11116/0000-0008-72E3-7 %R 10.1145/3461702.3462630 %D 2021 %B Fourth AAAI/ACM Conference on Artificial Intelligence, Ethics and Society %Z date of event: 2021-05-19 - 2021-05-21 %C Virtual Conference %B AIES '21 %E Fourcade, Marion; Kuipers, Benjamin; Lazar, Seth; Mulligan, Deirdre %P 336 - 345 %I ACM %@ 978-1-4503-8473-5

%0 Report %A Ansari, Navid %A Seidel, Hans-Peter %A Babaei, Vahid %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Mixed Integer Neural Inverse Design : %G eng %U http://hdl.handle.net/21.11116/0000-0009-7104-3 %U https://arxiv.org/abs/2109.12888 %D 2021 %X In computational design and fabrication, neural networks are becoming<br>important surrogates for bulky forward simulations. A long-standing,<br>intertwined question is that of inverse design: how to compute a design that<br>satisfies a desired target performance? Here, we show that the piecewise linear<br>property, very common in everyday neural networks, allows for an inverse design<br>formulation based on mixed-integer linear programming. Our mixed-integer<br>inverse design uncovers globally optimal or near optimal solutions in a<br>principled manner. Furthermore, our method significantly facilitates emerging,<br>but challenging, combinatorial inverse design tasks, such as material<br>selection. For problems where finding the optimal solution is not desirable or<br>tractable, we develop an efficient yet near-optimal hybrid optimization.<br>Eventually, our method is able to find solutions provably robust to possible<br>fabrication perturbations among multiple designs with similar performances.<br> %K Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

%0 Journal Article %A Chen, Bin %A Wang, Chao %A Piovar&#269;i, Michal %A Seidel, Hans-Peter %A Didyk, Piotr %A Myszkowski, Karol %A Serrano, Ana %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T The Effect of Geometry and Illumination on Appearance Perception of Different Material Categories : %G eng %U http://hdl.handle.net/21.11116/0000-0008-F05C-2 %R 10.1007/s00371-021-02227-x %7 2021 %D 2021 %J The Visual Computer %V 37 %& 2975 %P 2975 - 2987 %I Springer %C Berlin %@ false

%0 Journal Article %A Chu, Mengyu %A Thuerey, Nils %A Seidel, Hans-Peter %A Theobalt, Christian %A Zayer, Rhaleb %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Learning Meaningful Controls for Fluids : %G eng %U http://hdl.handle.net/21.11116/0000-0009-4B91-F %R 10.1145/3450626.3459845 %7 2021 %D 2021 %J ACM Transactions on Graphics %V 40 %N 4 %& 1 %P 1 - 13 %Z sequence number: 100 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2021 %O ACM SIGGRAPH 2021

%0 Journal Article %A Delanoy, Johanna %A Serrano, Ana %A Masia, Belen %A Gutierrez, Diego %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Perception of Material Appearance: A Comparison between Painted and Rendered Images : %G eng %U http://hdl.handle.net/21.11116/0000-0008-A6CC-7 %R 10.1167/jov.21.5.16 %2 PMC8131993 %7 2021 %D 2021 %J Journal of Vision %V 21 %N 5 %Z sequence number: 16 %I Scholar One, Inc. %C Charlottesville, VA %@ false

%0 Conference Proceedings %A Doosti, Nikan %A Panetta, Julian %A Babaei, Vahid %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Topology Optimization via Frequency Tuning of Neural Design Representations : %G eng %U http://hdl.handle.net/21.11116/0000-000C-C159-6 %R 10.1145/3485114.3485124 %D 2021 %B ACM Symposium on Computational Fabrication %Z date of event: 2021-10-28 - 2021-10-29 %C Virtual Event %B Proceedings SCF 2021 %E Whiting, Emily; Hart, John; Sung, Cynthia; McCann, James; Peek, Nadya %P 1 - 9 %Z sequence number: 1 %I ACM %@ 978-1-4503-9090-3

%0 Journal Article %A Elek, Oskar %A Zhang, Ran %A Sumin, Denis %A Myszkowski, Karol %A Bickel, Bernd %A Wilkie, Alexander %A Krivanek, Jaroslav %A Weyrich, Tim %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T Robust and Practical Measurement of Volume Transport Parameters in Solid Photo-polymer Materials for 3D Printing : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E013-6 %R 10.1364/OE.406095 %7 2021 %D 2021 %J Optics Express %O Opt. Express %V 29 %N 5 %& 7568 %P 7568 - 7588 %I Optical Society of America %C Washington, DC %@ false

%0 Conference Proceedings %A Fox, Gereon %A Liu, Wentao %A Kim, Hyeongwoo %A Seidel, Hans-Peter %A Elgharib, Mohamed %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T VideoForensicsHQ: Detecting High-quality Manipulated Face Videos : %G eng %U http://hdl.handle.net/21.11116/0000-0008-88DF-4 %R 10.1109/ICME51207.2021.9428101 %D 2021 %B 22nd IEEE International Conference on Multimedia and Expo %Z date of event: 2021-07-05 - 2021-07-07 %C Shenzhen, China (Virtual) %B IEEE International Conference on Multimedia and Expo %P 1 - 6 %I IEEE %@ 978-1-6654-3864-3 %U http://gvv.mpi-inf.mpg.de/projects/VForensicsHQ/

%0 Thesis %A Habermann, Marc %Y Theobalt, Christian %A referee: Seidel, Hans-Peter %A referee: Hilton, Adrian %+ Computer Graphics, MPI for Informatics, Max Planck Society International Max Planck Research School, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Real-time human performance capture and synthesis : %G eng %U http://hdl.handle.net/21.11116/0000-0009-7D87-3 %R 10.22028/D291-34961 %U nbn:de:bsz:291--ds-349617 %F OTHER: hdl:20.500.11880/31986 %I Universit&#228;t des Saarlandes %C Saarbr&#252;cken %D 2021 %P 153 p. %V phd %9 phd %X Most of the images one finds in the media, such as on the Internet or in textbooks and magazines, contain humans as the main point of attention. Thus, there is an inherent necessity for industry, society, and private persons to be able to thoroughly analyze and synthesize the human-related content in these images. One aspect of this analysis and subject of this thesis is to infer the 3D pose and surface deformation, using only visual information, which is also known as human performance capture. Human performance capture enables the tracking of virtual characters from real-world observations, and this is key for visual effects, games, VR, and AR, to name just a few application areas. However, traditional capture methods usually rely on expensive multi-view (marker-based) systems that are prohibitively expensive for the vast majority of people, or they use depth sensors, which are still not as common as single color cameras. Recently, some approaches have attempted to solve the task by assuming only a single RGB image is given. Nonetheless, they can either not track the dense deforming geometry of the human, such as the clothing layers, or they are far from real time, which is indispensable for many applications. To overcome these shortcomings, this thesis proposes two monocular human performance capture methods, which for the first time allow the real-time capture of the dense deforming geometry as well as an unseen 3D accuracy for pose and surface deformations. At the technical core, this work introduces novel GPU-based and data-parallel optimization strategies in conjunction with other algorithmic design choices that are all geared towards real-time performance at high accuracy. Moreover, this thesis presents a new weakly supervised multiview training strategy combined with a fully differentiable character representation that shows superior 3D accuracy. However, there is more to human-related Computer Vision than only the analysis of people in images. It is equally important to synthesize new images of humans in unseen poses and also from camera viewpoints that have not been observed in the real world. Such tools are essential for the movie industry because they, for example, allow the synthesis of photo-realistic virtual worlds with real-looking humans or of contents that are too dangerous for actors to perform on set. But also video conferencing and telepresence applications can benefit from photo-real 3D characters, as they can enhance the immersive experience of these applications. Here, the traditional Computer Graphics pipeline for rendering photo-realistic images involves many tedious and time-consuming steps that require expert knowledge and are far from real time. Traditional rendering involves character rigging and skinning, the modeling of the surface appearance properties, and physically based ray tracing. Recent learning-based methods attempt to simplify the traditional rendering pipeline and instead learn the rendering function from data resulting in methods that are easier accessible to non-experts. However, most of them model the synthesis task entirely in image space such that 3D consistency cannot be achieved, and/or they fail to model motion- and view-dependent appearance effects. To this end, this thesis presents a method and ongoing work on character synthesis, which allow the synthesis of controllable photoreal characters that achieve motion- and view-dependent appearance effects as well as 3D consistency and which run in real time. This is technically achieved by a novel coarse-to-fine geometric character representation for efficient synthesis, which can be solely supervised on multi-view imagery. Furthermore, this work shows how such a geometric representation can be combined with an implicit surface representation to boost synthesis and geometric quality. %U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/31986

%0 Report %A Habibie, Ikhsanul %A Xu, Weipeng %A Mehta, Dushyant %A Liu, Lingjie %A Seidel, Hans-Peter %A Pons-Moll, Gerard %A Elgharib, Mohamed %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Learning Speech-driven 3D Conversational Gestures from Video : %G eng %U http://hdl.handle.net/21.11116/0000-0009-70C7-8 %U https://arxiv.org/abs/2102.06837 %D 2021 %X We propose the first approach to automatically and jointly synthesize both<br>the synchronous 3D conversational body and hand gestures, as well as 3D face<br>and head animations, of a virtual character from speech input. Our algorithm<br>uses a CNN architecture that leverages the inherent correlation between facial<br>expression and hand gestures. Synthesis of conversational body gestures is a<br>multi-modal problem since many similar gestures can plausibly accompany the<br>same input speech. To synthesize plausible body gestures in this setting, we<br>train a Generative Adversarial Network (GAN) based model that measures the<br>plausibility of the generated sequences of 3D body motion when paired with the<br>input audio features. We also contribute a new way to create a large corpus of<br>more than 33 hours of annotated body, hand, and face data from in-the-wild<br>videos of talking people. To this end, we apply state-of-the-art monocular<br>approaches for 3D body and hand pose estimation as well as dense 3D face<br>performance capture to the video corpus. In this way, we can train on orders of<br>magnitude more data than previous algorithms that resort to complex in-studio<br>motion capture solutions, and thereby train more expressive synthesis<br>algorithms. Our experiments and user study show the state-of-the-art quality of<br>our speech-synthesized full 3D character animations.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Habibie, Ikhsanul %A Xu, Weipeng %A Mehta, Dushyant %A Liu, Lingjie %A Seidel, Hans-Peter %A Pons-Moll, Gerard %A Elgharib, Mohamed %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Learning Speech-driven 3D Conversational Gestures from Video : %G eng %U http://hdl.handle.net/21.11116/0000-0009-4D19-6 %R 10.1145/3472306.3478335 %D 2021 %B 21st ACM International Conference on Intelligent Virtual Agents %Z date of event: 2021-09-14 - 2021-09-17 %C Virtual Event, Japan %B Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents %P 101 - 108 %I ACM %@ 9781450386197

%0 Journal Article %A Hladk&#253;, Jozef %A Seidel, Hans-Peter %A Steinberger, Markus %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T SnakeBinning: Efficient Temporally Coherent Triangle Packing for Shading Streaming : %G eng %U http://hdl.handle.net/21.11116/0000-0008-7AFD-3 %R 10.1111/cgf.142648 %7 2021 %D 2021 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 40 %N 2 %& 475 %P 475 - 488 %I Blackwell-Wiley %C Oxford %@ false %B 42nd Annual Conference of the European Association for Computer Graphics %O EUROGRAPHICS 2021 EG 2021

%0 Journal Article %A Jiang, Caigui %A Tang, Chengcheng %A Seidel, Hans-Peter %A Chen, Renjie %A Wonka, Peter %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Computational Design of Lightweight Trusses : %U http://hdl.handle.net/21.11116/0000-0009-70C2-D %R 10.1016/j.cad.2021.103076 %7 2021 %D 2021 %J Computer-Aided Design %V 141 %Z sequence number: 103076 %I Elsevier %C Amsterdam %@ false

%0 Journal Article %A Jindal, Akshay %A Wolski, Krzysztof %A Mantiuk, Rafa&#322; K. %A Myszkowski, Karol %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Perceptual Model for Adaptive Local Shading and Refresh Rate : %G eng %U http://hdl.handle.net/21.11116/0000-0009-9B45-B %R 10.1145/3478513.3480514 %7 2021 %D 2021 %J ACM Transactions on Graphics %V 40 %N 6 %& 1 %P 1 - 18 %Z sequence number: 281 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2021 %O ACM SIGGRAPH Asia 2021 SA '21 SA 2021

%0 Conference Proceedings %A Kappel, Moritz %A Golyanik, Vladislav %A Elgharib, Mohamed %A Henningson, Jann-Ole %A Seidel, Hans-Peter %A Castillo, Susana %A Theobalt, Christian %A Magnor, Marcus A. %+ External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations %T High-Fidelity Neural Human Motion Transfer from Monocular Video Computer Vision and Pattern Recognition : %G eng %U http://hdl.handle.net/21.11116/0000-0008-8947-E %R 10.1109/CVPR46437.2021.00159 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Virtual Conference %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 1541 - 1550 %I IEEE %@ 978-1-6654-4509-2 %U https://gvv.mpi-inf.mpg.de/projects/NHMT/

%0 Conference Proceedings %A Knibbe, Jarrod %A Freire, Rachel %A Koelle, Marion %A Strohmeier, Paul %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Skill-Sleeves: Designing Electrode Garments for Wearability : %G eng %U http://hdl.handle.net/21.11116/0000-000C-BDF0-0 %R 10.1145/3430524.3440652 %D 2021 %B Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction %Z date of event: 2021-02-14 - 2021-02-17 %C Salzburg, Austria %B TEI '21 %P 1 - 16 %Z sequence number: 33 %I ACM %@ 978-1-4503-8213-7

%0 Journal Article %A Lagunas, Manuel %A Serrano, Ana %A Gutierrez, Diego %A Masia, Belen %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T The Joint Role of Geometry and Illumination on Material Recognition : %G eng %U http://hdl.handle.net/21.11116/0000-0007-EAF9-9 %R 10.1167/jov.21.2.2 %7 2021 %D 2021 %J Journal of Vision %V 21 %N 2 %& 1 %P 1 - 18 %I Scholar One, Inc. %C Charlottesville, VA %@ false

%0 Journal Article %A Leimk&#252;hler, Thomas %A Drettakis, George %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold : %G eng %U http://hdl.handle.net/21.11116/0000-000C-AFF0-0 %R 10.1145/3478513.3480538 %7 2021 %D 2021 %J ACM Transactions on Graphics %V 40 %N 6 %& 1 %P 1 - 15 %Z sequence number: 224 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2021 %O ACM SIGGRAPH Asia 2021 SA '21 SA 2021

%0 Journal Article %A Liu, Lingjie %A Xu, Weipeng %A Habermann, Marc %A Zollh&#246;fer, Michael %A Bernard, Florian %A Kim, Hyeongwoo %A Wang, Wenping %A Theobalt, Christian %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation : %G eng %U http://hdl.handle.net/21.11116/0000-0008-0369-F %R 10.1109/TVCG.2020.2996594 %7 2020 %D 2021 %J IEEE Transactions on Visualization and Computer Graphics %V 27 %N 10 %& 4009 %P 4009 - 4022 %I IEEE %C Piscataway, NJ %@ false

%0 Conference Proceedings %A Li, Yue %A Habermann, Marc %A Thomaszewski,, Bernhard %A Coros, Stelian %A Beeler, Thabo %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E9D0-7 %R 10.1109/3DV53792.2021.00047 %D 2021 %B International Conference on 3D Vision %Z date of event: 2021-12-01 - 2021-12-03 %C Virtual Conference %B 2021 International Conference on 3D Vision %P 373 - 384 %I IEEE %@ 978-1-6654-2688-6

%0 Conference Proceedings %A Mallikarjun B R, %A Tewari, Ayush %A Seidel, Hans-Peter %A Elgharib, Mohamed %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Learning Complete 3D Morphable Face Models from Images and Videos : %G eng %U http://hdl.handle.net/21.11116/0000-0008-8926-3 %R 10.1109/CVPR46437.2021.00337 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Virtual Conference %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 3360 - 3370 %I IEEE %@ 978-1-6654-4509-2 %U https://gvv.mpi-inf.mpg.de/projects/LeMoMo/

%0 Conference Proceedings %A Mallikarjun B R, %A Tewari, Ayush %A Oh, Tae-Hyun %A Weyrich, Tim %A Bickel, Bernd %A Seidel, Hans-Peter %A Pfister, Hanspeter %A Matusik, Wojciech %A Elgharib, Mohamed %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Monocular Reconstruction of Neural Face Reflectance Fields : %G eng %U http://hdl.handle.net/21.11116/0000-0008-88FB-4 %R 10.1109/CVPR46437.2021.00476 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Virtual Conference %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 4789 - 4798 %I IEEE %@ 978-1-6654-4509-2 %U https://gvv.mpi-inf.mpg.de/projects/FaceReflectanceFields/

%0 Journal Article %A Mallikarjun B R, %A Tewari, Ayush %A Dib, Abdallah %A Weyrich, Tim %A Bickel, Bernd %A Seidel, Hans-Peter %A Pfister, Hanspeter %A Matusik, Wojciech %A Chevallier, Louis %A Elgharib, Mohamed %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T PhotoApp: Photorealistic Appearance Editing of Head Portraits : %G eng %U http://hdl.handle.net/21.11116/0000-0009-2A9B-A %R 10.1145/3450626.3459765 %7 2021 %D 2021 %J ACM Transactions on Graphics %V 40 %N 4 %& 1 %P 1 - 16 %Z sequence number: 44 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2021 %O ACM SIGGRAPH 2021

%0 Report %A Martin, Daniel %A Malpica, Sandra %A Gutierrez, Diego %A Masia, Belen %A Serrano, Ana %+ External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Multimodality in VR: A Survey : %G eng %U http://hdl.handle.net/21.11116/0000-0007-EB00-0 %U https://arxiv.org/abs/2101.07906 %D 2021 %X Virtual reality has the potential to change the way we create and consume<br>content in our everyday life. Entertainment, training, design and<br>manufacturing, communication, or advertising are all applications that already<br>benefit from this new medium reaching consumer level. VR is inherently<br>different from traditional media: it offers a more immersive experience, and<br>has the ability to elicit a sense of presence through the place and<br>plausibility illusions. It also gives the user unprecedented capabilities to<br>explore their environment, in contrast with traditional media. In VR, like in<br>the real world, users integrate the multimodal sensory information they receive<br>to create a unified perception of the virtual world. Therefore, the sensory<br>cues that are available in a virtual environment can be leveraged to enhance<br>the final experience. This may include increasing realism, or the sense of<br>presence; predicting or guiding the attention of the user through the<br>experience; or increasing their performance if the experience involves the<br>completion of certain tasks. In this state-of-the-art report, we survey the<br>body of work addressing multimodality in virtual reality, its role and benefits<br>in the final user experience. The works here reviewed thus encompass several<br>fields of research, including computer graphics, human computer interaction, or<br>psychology and perception. Additionally, we give an overview of different<br>applications that leverage multimodal input in areas such as medicine, training<br>and education, or entertainment; we include works in which the integration of<br>multiple sensory information yields significant improvements, demonstrating how<br>multimodality can play a fundamental role in the way VR systems are designed,<br>and VR experiences created and consumed.<br> %K Computer Science, Human-Computer Interaction, cs.HC,Computer Science, Graphics, cs.GR

%0 Journal Article %A Masia, Belen %A Camon, Javier %A Gutierrez,, Diego %A Serrano, Ana %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Influence of Directional Sound Cues on Users' Exploration Across 360&#176; Movie Cuts : %G eng %U http://hdl.handle.net/21.11116/0000-0008-F077-3 %R 10.1109/MCG.2021.3064688 %7 2021 %D 2021 %J IEEE Computer Graphics and Applications %V 41 %N 4 %& 64 %P 64 - 75 %I IEEE %C Piscataway, NJ %@ false

%0 Journal Article %A Meka, Abhimitra %A Shafiei, Mohammad %A Zollh&#246;fer, Michael %A Richardt, Christian %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Real-time Global Illumination Decomposition of Videos : %G eng %U http://hdl.handle.net/21.11116/0000-0007-EE07-6 %R 10.1145/3374753 %7 2021 %D 2021 %J ACM Transactions on Graphics %V 40 %N 3 %& 1 %P 1 - 16 %Z sequence number: 22 %I ACM %C New York, NY %@ false %U http://gvv.mpi-inf.mpg.de/projects/LiveIlluminationDecomposition/

%0 Conference Proceedings %A Nehvi, Jalees %A Golyanik, Vladislav %A Mueller, Franziska %A Seidel, Hans-Peter %A Elgharib, Mohamed %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Differentiable Event Stream Simulator for Non-Rigid 3D Tracking : %G eng %U http://hdl.handle.net/21.11116/0000-0008-8957-C %R 10.1109/CVPRW53098.2021.00143 %D 2021 %B Third International Workshop on Event-Based Vision %Z date of event: 2021-06-19 - 2021-06-19 %C Virtual Workshop %B Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops %P 1302 - 1311 %I IEEE %@ 978-1-6654-4899-4 %U https://gvv.mpi-inf.mpg.de/projects/Event-based_Non-rigid_3D_Tracking/

%0 Conference Proceedings %A Rao, Sukrut %A Stutz, David %A Schiele, Bernt %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Adversarial Training Against Location-Optimized Adversarial Patches : %G eng %U http://hdl.handle.net/21.11116/0000-0008-1662-1 %R 10.1007/978-3-030-68238-5_32 %D 2021 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV Workshops 2020 %E Bartoli, Adrian; Fusiello, Andrea %P 429 - 448 %I Springer %@ 978-3-030-68237-8 %B Lecture Notes in Computer Science %N 12539

%0 Journal Article %A Rittig, Tobias %A Sumin, Denis %A Babaei, Vahid %A Didyk, Piotr %A Voloboy, Alexei %A Wilkie, Alexander %A Bickel, Bernd %A Myszkowski, Karol %A Weyrich, Tim %A Krivanek, Jaroslav %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Neural Acceleration of Scattering-Aware Color 3D Printing : %G eng %U http://hdl.handle.net/21.11116/0000-0007-F073-8 %R 10.1111/cgf.142626 %7 2021 %D 2021 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 40 %N 2 %& 205 %P 205 - 219 %I Blackwell-Wiley %C Oxford %@ false %B 42nd Annual Conference of the European Association for Computer Graphics %O EUROGRAPHICS 2021 EG 2021

%0 Journal Article %A Ruan, Lingyan %A Chen, Bin %A Li, Jizhou %A Lam, Miu-Ling %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T AIFNet: All-in-Focus Image Restoration Network Using a Light Field-Based Dataset : %G eng %U http://hdl.handle.net/21.11116/0000-0009-4795-F %R 10.1109/TCI.2021.3092891 %7 2021 %D 2021 %J IEEE Transactions on Computational Imaging %V 7 %& 675 %P 675 - 688 %I IEEE %C Piscataway, NJ %@ false

%0 Conference Proceedings %A Rudnev, Viktor %A Golyanik, Vladislav %A Wang, Jiayi %A Seidel, Hans-Peter %A Mueller, Franziska %A Elgharib, Mohamed %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B709-1 %R 10.1109/ICCV48922.2021.01216 %D 2021 %B IEEE/CVF International Conference on Computer Vision %Z date of event: 2021-10-10 - 2021-10-17 %C Virtual Event %B ICCV 2021 %P 12365 - 12375 %I IEEE %@ 978-1-6654-2812-5

%0 Report %A Sarkar, Kripasindhu %A Mehta, Dushyant %A Xu, Weipeng %A Golyanik, Vladislav %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Neural Re-Rendering of Humans from a Single Image : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CF05-B %U https://arxiv.org/abs/2101.04104 %D 2021 %X Human re-rendering from a single image is a starkly under-constrained<br>problem, and state-of-the-art algorithms often exhibit undesired artefacts,<br>such as over-smoothing, unrealistic distortions of the body parts and garments,<br>or implausible changes of the texture. To address these challenges, we propose<br>a new method for neural re-rendering of a human under a novel user-defined pose<br>and viewpoint, given one input image. Our algorithm represents body pose and<br>shape as a parametric mesh which can be reconstructed from a single image and<br>easily reposed. Instead of a colour-based UV texture map, our approach further<br>employs a learned high-dimensional UV feature map to encode appearance. This<br>rich implicit representation captures detailed appearance variation across<br>poses, viewpoints, person identities and clothing styles better than learned<br>colour texture maps. The body model with the rendered feature maps is fed<br>through a neural image-translation network that creates the final rendered<br>colour image. The above components are combined in an end-to-end-trained neural<br>network architecture that takes as input a source person image, and images of<br>the parametric body model in the source pose and desired target pose.<br>Experimental evaluation demonstrates that our approach produces higher quality<br>single image re-rendering results than existing methods.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Journal Article %A Serrano, Ana %A Chen, Bin %A Wang, Chao %A Piovar&#269;i, Michal %A Seidel, Hans-Peter %A Didyk, Piotr %A Myszkowski, Karol %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T The Effect of Shape and Illumination on Material Perception : Model and Applications %G eng %U http://hdl.handle.net/21.11116/0000-0009-0565-0 %R 10.1145/3450626.3459813 %7 2021 %D 2021 %J ACM Transactions on Graphics %V 40 %N 4 %& 1 %P 1 - 16 %Z sequence number: 125 %I Association for Computing Machinery %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2021 %O ACM SIGGRAPH 2021

%0 Report %A Surace, Luca %A Wernikowski, Marek %A Tursun, Okan Tarhan %A Myszkowski, Karol %A Mantiuk, Rados&#322;aw %A Didyk, Piotr %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Learning Foveated Reconstruction to Preserve Perceived Image Statistics : %G eng %U http://hdl.handle.net/21.11116/0000-0009-73D9-1 %U https://arxiv.org/abs/2108.03499 %D 2021 %X Foveated image reconstruction recovers full image from a sparse set of<br>samples distributed according to the human visual system's retinal sensitivity<br>that rapidly drops with eccentricity. Recently, the use of Generative<br>Adversarial Networks was shown to be a promising solution for such a task as<br>they can successfully hallucinate missing image information. Like for other<br>supervised learning approaches, also for this one, the definition of the loss<br>function and training strategy heavily influences the output quality. In this<br>work, we pose the question of how to efficiently guide the training of foveated<br>reconstruction techniques such that they are fully aware of the human visual<br>system's capabilities and limitations, and therefore, reconstruct visually<br>important image features. Due to the nature of GAN-based solutions, we<br>concentrate on the human's sensitivity to hallucination for different input<br>sample densities. We present new psychophysical experiments, a dataset, and a<br>procedure for training foveated image reconstruction. The strategy provides<br>flexibility to the generator network by penalizing only perceptually important<br>deviations in the output. As a result, the method aims to preserve perceived<br>image statistics rather than natural image statistics. We evaluate our strategy<br>and compare it to alternative solutions using a newly trained objective metric<br>and user experiments.<br> %K Computer Science, Graphics, cs.GR,Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Thesis %A Tewari, Ayush %Y Theobalt, Christian %A referee: Zollh&#246;fer, Michael %A referee: Wonka, Peter %+ Computer Graphics, MPI for Informatics, Max Planck Society International Max Planck Research School, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Self-supervised reconstruction and synthesis of faces : %G eng %U http://hdl.handle.net/21.11116/0000-0009-9CD2-A %R 10.22028/D291-34598 %U nbn:de:bsz:291--ds-345982 %F OTHER: hdl:20.500.11880/31754 %I Universit&#228;t des Saarlandes %C Saarbr&#252;cken %D 2021 %P 173 p. %V phd %9 phd %U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/31754

%0 Report %A van Onzenoodt, Christian %A Singh, Gurprit %A Ropinski, Timo %A Ritschel, Tobias %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Blue Noise Plots : %G eng %U http://hdl.handle.net/21.11116/0000-0008-01ED-C %U https://arxiv.org/abs/2102.04072 %D 2021 %X We propose Blue Noise Plots, two-dimensional dot plots that depict data<br>points of univariate data sets. While often one-dimensional strip plots are<br>used to depict such data, one of their main problems is visual clutter which<br>results from overlap. To reduce this overlap, jitter plots were introduced,<br>whereby an additional, non-encoding plot dimension is introduced, along which<br>the data point representing dots are randomly perturbed. Unfortunately, this<br>randomness can suggest non-existent clusters, and often leads to visually<br>unappealing plots, in which overlap might still occur. To overcome these<br>shortcomings, we introduce BlueNoise Plots where random jitter along the<br>non-encoding plot dimension is replaced by optimizing all dots to keep a<br>minimum distance in 2D i. e., Blue Noise. We evaluate the effectiveness as well<br>as the aesthetics of Blue Noise Plots through both, a quantitative and a<br>qualitative user study.<br> %K Computer Science, Graphics, cs.GR

%0 Journal Article %A van Onzenoodt, Christian %A Singh, Gurprit %A Ropinski, Timo %A Ritschel, Tobias %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Blue Noise Plots : %G eng %U http://hdl.handle.net/21.11116/0000-0008-010F-7 %R 10.1111/cgf.142644 %7 2021 %D 2021 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 40 %N 2 %& 425 %P 425 - 433 %I Blackwell-Wiley %C Oxford %@ false %B 42nd Annual Conference of the European Association for Computer Graphics %O EUROGRAPHICS 2021 EG 2021

%0 Report %A Wang, Chao %A Chen, Bin %A Seidel, Hans-Peter %A Myszkowski, Karol %A Serrano, Ana %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Learning a self-supervised tone mapping operator via feature contrast masking loss : %G eng %U http://hdl.handle.net/21.11116/0000-0009-710E-9 %U https://arxiv.org/abs/2110.09866 %D 2021 %X High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid<br>development of capture technologies. Nevertheless, the dynamic range of common<br>display devices is still limited, therefore tone mapping (TM) remains a key<br>challenge for image visualization. Recent work has demonstrated that neural<br>networks can achieve remarkable performance in this task when compared to<br>traditional methods, however, the quality of the results of these<br>learning-based methods is limited by the training data. Most existing works use<br>as training set a curated selection of best-performing results from existing<br>traditional tone mapping operators (often guided by a quality metric),<br>therefore, the quality of newly generated results is fundamentally limited by<br>the performance of such operators. This quality might be even further limited<br>by the pool of HDR content that is used for training. In this work we propose a<br>learning-based self-supervised tone mapping operator that is trained at test<br>time specifically for each HDR image and does not need any data labeling. The<br>key novelty of our approach is a carefully designed loss function built upon<br>fundamental knowledge on contrast perception that allows for directly comparing<br>the content in the HDR and tone mapped images. We achieve this goal by<br>reformulating classic VGG feature maps into feature contrast maps that<br>normalize local feature differences by their average magnitude in a local<br>neighborhood, allowing our loss to account for contrast masking effects. We<br>perform extensive ablation studies and exploration of parameters and<br>demonstrate that our solution outperforms existing approaches with a single set<br>of fixed parameters, as confirmed by both objective and subjective metrics.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

%0 Report %A Weinrauch, Alexander %A Seidel, Hans-Peter %A Mlakar, Daniel %A Steinberger, Markus %A Zayer, Rhaleb %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and Reeb Graph Construction on Surfaces : %G eng %U http://hdl.handle.net/21.11116/0000-0009-70EE-D %U https://arxiv.org/abs/2105.13168 %D 2021 %X The humble loop shrinking property played a central role in the inception of<br>modern topology but it has been eclipsed by more abstract algebraic formalism.<br>This is particularly true in the context of detecting relevant non-contractible<br>loops on surfaces where elaborate homological and/or graph theoretical<br>constructs are favored in algorithmic solutions. In this work, we devise a<br>variational analogy to the loop shrinking property and show that it yields a<br>simple, intuitive, yet powerful solution allowing a streamlined treatment of<br>the problem of handle and tunnel loop detection. Our formalization tracks the<br>evolution of a diffusion front randomly initiated on a single location on the<br>surface. Capitalizing on a diffuse interface representation combined with a set<br>of rules for concurrent front interactions, we develop a dynamic data structure<br>for tracking the evolution on the surface encoded as a sparse matrix which<br>serves for performing both diffusion numerics and loop detection and acts as<br>the workhorse of our fully parallel implementation. The substantiated results<br>suggest our approach outperforms state of the art and robustly copes with<br>highly detailed geometric models. As a byproduct, our approach can be used to<br>construct Reeb graphs by diffusion thus avoiding commonly encountered issues<br>when using Morse functions.<br> %K Computer Science, Graphics, cs.GR,Computer Science, Computational Geometry, cs.CG,Mathematics, Algebraic Topology, math.AT

%0 Conference Proceedings %A Yenamandra, Tarun %A Tewari, Ayush %A Bernard, Florian %A Seidel, Hans-Peter %A Elgharib, Mohamed %A Cremers, Daniel %A Theobalt, Christian %+ External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T i3DMM: Deep Implicit 3D Morphable Model of Human Heads : %G eng %U http://hdl.handle.net/21.11116/0000-0008-8966-B %R 10.1109/CVPR46437.2021.01261 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Virtual Conference %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 12798 - 12808 %I IEEE %@ 978-1-6654-4509-2 %U https://gvv.mpi-inf.mpg.de/projects/i3DMM/

%0 Conference Proceedings %A Zheng, Quan %A Singh, Gurprit %A Seidel, Hans-Peter %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Neural Relightable Participating Media Rendering : %G eng %U http://hdl.handle.net/21.11116/0000-0009-7117-E %D 2021 %B 35th Conference on Neural Information Processing Systems %Z date of event: 2021-12-06 - 2021-12-14 %C Virtual %B Advances in Neural Information Processing Systems 34 %E Ranzato, M.; Beygelzimer, A.; Liang, P. S.; Vaughan, J. W.; Dauphin, Y. %P 15203 - 15215 %I Curran Associates, Inc. %@ 9781713845393

%0 Report %A Ali, Sk Aziz %A Kahraman, Kerem %A Theobalt, Christian %A Stricker, Didier %A Golyanik, Vladislav %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E8FA-A %U https://arxiv.org/abs/2009.14005 %D 2020 %X This article introduces a new physics-based method for rigid point set<br>alignment called Fast Gravitational Approach (FGA). In FGA, the source and<br>target point sets are interpreted as rigid particle swarms with masses<br>interacting in a globally multiply-linked manner while moving in a simulated<br>gravitational force field. The optimal alignment is obtained by explicit<br>modeling of forces acting on the particles as well as their velocities and<br>displacements with second-order ordinary differential equations of motion.<br>Additional alignment cues (point-based or geometric features, and other<br>boundary conditions) can be integrated into FGA through particle masses. We<br>propose a smooth-particle mass function for point mass initialization, which<br>improves robustness to noise and structural discontinuities. To avoid<br>prohibitive quadratic complexity of all-to-all point interactions, we adapt a<br>Barnes-Hut tree for accelerated force computation and achieve quasilinear<br>computational complexity. We show that the new method class has characteristics<br>not found in previous alignment methods such as efficient handling of partial<br>overlaps, inhomogeneous point sampling densities, and coping with large point<br>clouds with reduced runtime compared to the state of the art. Experiments show<br>that our method performs on par with or outperforms all compared competing<br>non-deep-learning-based and general-purpose techniques (which do not assume the<br>availability of training data and a scene prior) in resolving transformations<br>for LiDAR data and gains state-of-the-art accuracy and speed when coping with<br>different types of data disturbances.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

%0 Journal Article %A Ansari, Navid %A Alizadeh-Mousavi, Omid %A Seidel, Hans-Peter %A Babaei, Vahid %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Mixed Integer Ink Selection for Spectral Reproduction : %G eng %U http://hdl.handle.net/21.11116/0000-0007-9B23-3 %R 10.1145/3414685.3417761 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 255 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Journal Article %A Bemana, Mojtaba %A Myszkowski, Karol %A Seidel, Hans-Peter %A Ritschel, Tobias %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation : %G eng %U http://hdl.handle.net/21.11116/0000-0006-FBF0-0 %R 10.1145/3414685.3417827 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 257 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Report %A Bemana, Mojtaba %A Myszkowski, Karol %A Seidel, Hans-Peter %A Ritschel, Tobias %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B6EC-2 %U https://arxiv.org/abs/2010.00450 %D 2020 %X We suggest to represent an X-Field -a set of 2D images taken across different<br>view, time or illumination conditions, i.e., video, light field, reflectance<br>fields or combinations thereof-by learning a neural network (NN) to map their<br>view, time or light coordinates to 2D images. Executing this NN at new<br>coordinates results in joint view, time or light interpolation. The key idea to<br>make this workable is a NN that already knows the "basic tricks" of graphics<br>(lighting, 3D projection, occlusion) in a hard-coded and differentiable form.<br>The NN represents the input to that rendering as an implicit map, that for any<br>view, time, or light coordinate and for any pixel can quantify how it will move<br>if view, time or light coordinates change (Jacobian of pixel position with<br>respect to view, time, illumination, etc.). Our X-Field representation is<br>trained for one scene within minutes, leading to a compact set of trainable<br>parameters and hence real-time navigation in view, time and illumination.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

%0 Conference Proceedings %A Bernard, Florian %A Suri, Zeeshan Khan %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D059-A %R 10.1109/CVPR42600.2020.01384 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 13823 - 13832 %I IEEE %@ 978-1-7281-7168-5

%0 Report %A Bernard, Florian %A Suri, Zeeshan Khan %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E00C-F %U https://arxiv.org/abs/2002.12623 %D 2020 %X We present a convex mixed-integer programming formulation for non-rigid shape<br>matching. To this end, we propose a novel shape deformation model based on an<br>efficient low-dimensional discrete model, so that finding a globally optimal<br>solution is tractable in (most) practical cases. Our approach combines several<br>favourable properties: it is independent of the initialisation, it is much more<br>efficient to solve to global optimality compared to analogous quadratic<br>assignment problem formulations, and it is highly flexible in terms of the<br>variants of matching problems it can handle. Experimentally we demonstrate that<br>our approach outperforms existing methods for sparse shape matching, that it<br>can be used for initialising dense shape matching methods, and we showcase its<br>flexibility on several examples.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG,Mathematics, Optimization and Control, math.OC

%0 Report %A Bhatnagar, Bharat Lal %A Sminchisescu, Cristian %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E91C-4 %U https://arxiv.org/abs/2010.12447 %D 2020 %X We address the problem of fitting 3D human models to 3D scans of dressed<br>humans. Classical methods optimize both the data-to-model correspondences and<br>the human model parameters (pose and shape), but are reliable only when<br>initialized close to the solution. Some methods initialize the optimization<br>based on fully supervised correspondence predictors, which is not<br>differentiable end-to-end, and can only process a single scan at a time. Our<br>main contribution is LoopReg, an end-to-end learning framework to register a<br>corpus of scans to a common 3D human model. The key idea is to create a<br>self-supervised loop. A backward map, parameterized by a Neural Network,<br>predicts the correspondence from every scan point to the surface of the human<br>model. A forward map, parameterized by a human model, transforms the<br>corresponding points back to the scan based on the model parameters (pose and<br>shape), thus closing the loop. Formulating this closed loop is not<br>straightforward because it is not trivial to force the output of the NN to be<br>on the surface of the human model - outside this surface the human model is not<br>even defined. To this end, we propose two key innovations. First, we define the<br>canonical surface implicitly as the zero level set of a distance field in R3,<br>which in contrast to morecommon UV parameterizations, does not require cutting<br>the surface, does not have discontinuities, and does not induce distortion.<br>Second, we diffuse the human model to the 3D domain R3. This allows to map the<br>NN predictions forward,even when they slightly deviate from the zero level set.<br>Results demonstrate that we can train LoopRegmainly self-supervised - following<br>a supervised warm-start, the model becomes increasingly more accurate as<br>additional unlabelled raw scans are processed. Our code and pre-trained models<br>can be downloaded for research.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Bhatnagar, Bharat Lal %A Sminchisescu, Cristian %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration : %G eng %U http://hdl.handle.net/21.11116/0000-0007-6FD1-1 %D 2020 %B 34th Conference on Neural Information Processing Systems %Z date of event: 2020-12-06 - 2020-12-12 %C Virtual Event %B Advances in Neural Information Processing Systems 33 %E Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.; Lin, H. %P 12909 - 12922 %I Curran Associates, Inc. %@ 9781713829546 %U https://papers.nips.cc/paper/2020/file/970af30e481057c48f87e101b61e6994-Paper.pdf

%0 Conference Proceedings %A Bhatnagar, Bharat Lal %A Sminchisescu, Cristian %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction : %G eng %U http://hdl.handle.net/21.11116/0000-0006-E89E-3 %R 10.1007/978-3-030-58536-5_19 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 311 - 329 %I Springer %@ 978-3-030-58535-8 %B Lecture Notes in Computer Science %N 12347

%0 Report %A Bhatnagar, Bharat Lal %A Sminchisescu, Cristian %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E8A0-E %U https://arxiv.org/abs/2007.11432 %D 2020 %X Implicit functions represented as deep learning approximations are powerful<br>for reconstructing 3D surfaces. However, they can only produce static surfaces<br>that are not controllable, which provides limited ability to modify the<br>resulting model by editing its pose or shape parameters. Nevertheless, such<br>features are essential in building flexible models for both computer graphics<br>and computer vision. In this work, we present methodology that combines<br>detail-rich implicit functions and parametric representations in order to<br>reconstruct 3D models of people that remain controllable and accurate even in<br>the presence of clothing. Given sparse 3D point clouds sampled on the surface<br>of a dressed person, we use an Implicit Part Network (IP-Net)to jointly predict<br>the outer 3D surface of the dressed person, the and inner body surface, and the<br>semantic correspondences to a parametric body model. We subsequently use<br>correspondences to fit the body model to our inner surface and then non-rigidly<br>deform it (under a parametric body + displacement model) to the outer surface<br>in order to capture garment, face and hair detail. In quantitative and<br>qualitative experiments with both full body data and hand scans we show that<br>the proposed methodology generalizes, and is effective even given incomplete<br>point clouds collected from single-view depth images. Our models and code can<br>be downloaded from http://virtualhumans.mpi-inf.mpg.de/ipnet.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Bo&#382;i&#269;, Alja&#382; %A Zollh&#246;fer, Michael %A Theobalt, Christian %A Nie&#223;ner, Matthias %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T DeepDeform: Learning Non-Rigid RGB-D Reconstruction With Semi-Supervised Data : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CF63-1 %R 10.1109/CVPR42600.2020.00703 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 7000 - 7010 %I IEEE %@ 978-1-7281-7168-5

%0 Report %A Chizhov, Vassillen %A Georgiev, Iliyan %A Myszkowski, Karol %A Singh, Gurprit %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Perceptual Error Optimization for Monte Carlo Rendering : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CEB7-3 %U https://arxiv.org/abs/2012.02344 %D 2020 %X Realistic image synthesis involves computing high-dimensional light transport<br>integrals which in practice are numerically estimated using Monte Carlo<br>integration. The error of this estimation manifests itself in the image as<br>visually displeasing aliasing or noise. To ameliorate this, we develop a<br>theoretical framework for optimizing screen-space error distribution. Our model<br>is flexible and works for arbitrary target error power spectra. We focus on<br>perceptual error optimization by leveraging models of the human visual system's<br>(HVS) point spread function (PSF) from halftoning literature. This results in a<br>specific optimization problem whose solution distributes the error as visually<br>pleasing blue noise in image space. We develop a set of algorithms that provide<br>a trade-off between quality and speed, showing substantial improvements over<br>prior state of the art. We perform evaluations using both quantitative and<br>perceptual error metrics to support our analysis, and provide extensive<br>supplemental material to help evaluate the perceptual improvements achieved by<br>our methods.<br> %K Computer Science, Graphics, cs.GR

%0 Journal Article %A &#199;o&#287;alan, U&#287;ur %A Aky&#252;z , Ahmet O&#287;uz %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Deep Joint Deinterlacing and Denoising for Single Shot Dual-ISO HDR Reconstruction : %G eng %U http://hdl.handle.net/21.11116/0000-0006-DCA7-6 %R 10.1109/TIP.2020.3004014 %7 2020 %D 2020 %J IEEE Transactions on Image Processing %V 29 %& 7511 %P 7511 - 7524 %I IEEE %C Piscataway, NJ %@ false

%0 Report %A &#199;o&#287;alan, U&#287;ur %A Bemana, Mojtaba %A Myszkowski, Karol %A Seidel, Hans-Peter %A Ritschel, Tobias %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B721-5 %U https://arxiv.org/abs/2012.12009 %D 2020 %X We seek to reconstruct sharp and noise-free high-dynamic range (HDR) video<br>from a dual-exposure sensor that records different low-dynamic range (LDR)<br>information in different pixel columns: Odd columns provide low-exposure,<br>sharp, but noisy information; even columns complement this with less noisy,<br>high-exposure, but motion-blurred data. Previous LDR work learns to deblur and<br>denoise (DISTORTED->CLEAN) supervised by pairs of CLEAN and DISTORTED images.<br>Regrettably, capturing DISTORTED sensor readings is time-consuming; as well,<br>there is a lack of CLEAN HDR videos. We suggest a method to overcome those two<br>limitations. First, we learn a different function instead: CLEAN->DISTORTED,<br>which generates samples containing correlated pixel noise, and row and column<br>noise, as well as motion blur from a low number of CLEAN sensor readings.<br>Second, as there is not enough CLEAN HDR video available, we devise a method to<br>learn from LDR video in-stead. Our approach compares favorably to several<br>strong baselines, and can boost existing methods when they are re-trained on<br>our data. Combined with spatial and temporal super-resolution, it enables<br>applications such as re-lighting with low noise or blur.<br> %K eess.IV,Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Journal Article %A Cucerca, Sebastian %A Didyk, Piotr %A Seidel, Hans-Peter %A Babaei, Vahid %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Computational Image Marking on Metals via Laser Induced Heating : %G eng %U http://hdl.handle.net/21.11116/0000-0007-9664-F %R 10.1145/3386569.3392423 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 4 %Z sequence number: 70 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2020 %O ACM SIGGRAPH 2020 Virtual Conference ; 2020, 17-28 August

%0 Conference Proceedings %A Dunn, David %A Tursun, Okan %A Yu, Hyeonseung %A Didyk, Piotr %A Myszkowski, Karol %A Fuchs, Henry %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Stimulating the Human Visual System Beyond Real World Performance in Future Augmented Reality Displays : %G eng %U http://hdl.handle.net/21.11116/0000-0006-FBDF-5 %R 10.1109/ISMAR50242.2020.00029 %D 2020 %B International Symposium on Mixed and Augmented Reality %Z date of event: 2020-11-09 - 2020-11-13 %C Recife/Porto de Galinhas, Brazil (Virtual Conference) %B IEEE International Symposium on Mixed and Augmented Reality %P 90 - 100 %I IEEE %@ 978-1-7281-8508-8

%0 Journal Article %A Egger, Bernhard %A Smith, William A. P. %A Tewari, Ayush %A Wuhrer, Stefanie %A Zollh&#246;fer, Michael %A Beeler, Thabo %A Bernard, Florian %A Bolkart, Timo %A Kortylewski, Adam %A Romdhani, Sami %A Theobalt, Christian %A Blanz, Volker %A Vetter, Thomas %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T 3D Morphable Face Models -Past, Present and Future : %G eng %U http://hdl.handle.net/21.11116/0000-0007-1CF5-6 %R 10.1145/3395208 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 5 %Z sequence number: 157 %I ACM %C New York, NY %@ false

%0 Journal Article %A Elgharib, Mohamed %A Mendiratta, Mohit %A Thies, Justus %A Nie&#223;ner, Matthias %A Seidel, Hans-Peter %A Tewari, Ayush %A Golyanik, Vladislav %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Egocentric Videoconferencing : %G eng %U http://hdl.handle.net/21.11116/0000-0007-9B36-E %R 10.1145/3414685.3417808 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 268 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Report %A Fox, Gereon %A Liu, Wentao %A Kim, Hyeongwoo %A Seidel, Hans-Peter %A Elgharib, Mohamed %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T VideoForensicsHQ: Detecting High-quality Manipulated Face Videos : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B109-7 %U https://arxiv.org/abs/2005.10360 %D 2020 %X New approaches to synthesize and manipulate face videos at very high quality<br>have paved the way for new applications in computer animation, virtual and<br>augmented reality, or face video analysis. However, there are concerns that<br>they may be used in a malicious way, e.g. to manipulate videos of public<br>figures, politicians or reporters, to spread false information. The research<br>community therefore developed techniques for automated detection of modified<br>imagery, and assembled benchmark datasets showing manipulatons by<br>state-of-the-art techniques. In this paper, we contribute to this initiative in<br>two ways: First, we present a new audio-visual benchmark dataset. It shows some<br>of the highest quality visual manipulations available today. Human observers<br>find them significantly harder to identify as forged than videos from other<br>benchmarks. Furthermore we propose new family of deep-learning-based fake<br>detectors, demonstrating that existing detectors are not well-suited for<br>detecting fakes of a quality as high as presented in our dataset. Our detectors<br>examine spatial and temporal features. This allows them to outperform existing<br>approaches both in terms of high detection accuracy and generalization to<br>unseen fake generation methods and unseen identities.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Golyanik, Vladislav %A Shimada, Soshi %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Fast Simultaneous Gravitational Alignment of Multiple Point Sets : %G eng %U http://hdl.handle.net/21.11116/0000-0008-0358-2 %R 10.1109/3DV50981.2020.00019 %D 2020 %B International Conference on 3D Vision %Z date of event: 2020-11-25 - 2020-11-28 %C Fukuoka, Japan (Virtual Event) %B International Conference on 3D Vision %P 91 - 100 %I IEEE %@ 978-1-7281-8128-8

%0 Conference Proceedings %A Golyanik, Vladislav %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T A Quantum Computational Approach to Correspondence Problems on Point Sets : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D053-0 %R 10.1109/CVPR42600.2020.00920 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 9179 - 9188 %I IEEE %@ 978-1-7281-7168-5

%0 Conference Proceedings %A Golyanik, Vladislav %A Jonas, Andr&#233; %A Stricker, Didier %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Intrinsic Dynamic Shape Prior for Dense Non-Rigid Structure from Motion : %G eng %U http://hdl.handle.net/21.11116/0000-0008-0364-4 %R 10.1109/3DV50981.2020.00079 %D 2020 %B International Conference on 3D Vision %Z date of event: 2020-11-25 - 2020-11-28 %C Fukuoka, Japan (Virtual Event) %B International Conference on 3D Vision %P 692 - 701 %I IEEE %@ 978-1-7281-8128-8

%0 Journal Article %A G&#252;nther, Felix %A Jiang, Caigui %A Pottmann, Helmut %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Smooth Polyhedral Surfaces : %G eng %U http://hdl.handle.net/21.11116/0000-0006-9760-3 %U http://arxiv.org/abs/1703.05318 %R 10.1016/j.aim.2020.107004 %D 2020 %* Review method: peer-reviewed %X Polyhedral surfaces are fundamental objects in architectural geometry and industrial design. Whereas closeness of a given mesh to a smooth reference surface and its suitability for numerical simulations were already studied extensively, the aim of our work is to find and to discuss suitable assessments of smoothness of polyhedral surfaces that only take the geometry of the polyhedral surface itself into account. Motivated by analogies to classical differential geometry, we propose a theory of smoothness of polyhedral surfaces including suitable notions of normal vectors, tangent planes, asymptotic directions, and parabolic curves that are invariant under projective transformations. It is remarkable that seemingly mild conditions significantly limit the shapes of faces of a smooth polyhedral surface. Besides being of theoretical interest, we believe that smoothness of polyhedral surfaces is of interest in the architectural context, where vertices and edges of polyhedral surfaces are highly visible.<br><br><br> %K Mathematics, Metric Geometry, Mathematics, Differential Geometry %J Advances in Mathematics %O Adv. Math. %V 363 %Z sequence number: 107004 %I Elsevier

%0 Report %A Habermann, Marc %A Xu, Weipeng %A Zollh&#246;fer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T DeepCap: Monocular Human Performance Capture Using Weak Supervision : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E010-9 %U https://arxiv.org/abs/2003.08325 %D 2020 %X Human performance capture is a highly important computer vision problem with<br>many applications in movie production and virtual/augmented reality. Many<br>previous performance capture approaches either required expensive multi-view<br>setups or did not recover dense space-time coherent geometry with<br>frame-to-frame correspondences. We propose a novel deep learning approach for<br>monocular dense human performance capture. Our method is trained in a weakly<br>supervised manner based on multi-view supervision completely removing the need<br>for training data with 3D ground truth annotations. The network architecture is<br>based on two separate networks that disentangle the task into a pose estimation<br>and a non-rigid surface deformation step. Extensive qualitative and<br>quantitative evaluations show that our approach outperforms the state of the<br>art in terms of quality and robustness.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Habermann, Marc %A Xu, Weipeng %A Zollh&#246;fer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T DeepCap: Monocular Human Performance Capture Using Weak Supervision : %G eng %U http://hdl.handle.net/21.11116/0000-0006-A895-4 %R 10.1109/CVPR42600.2020.00510 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 5051 - 5062 %I IEEE %@ 978-1-7281-7168-5

%0 Conference Proceedings %A Huang, Lifeng %A Gao, Chengying %A Zhou, Yuyin %A Xie, Cihang %A Yuille, Alan %A Zou, Changqing %A Liu, Ning %+ External Organizations External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Universal Physical Camou&#64258;age Attacks on Object Detectors : %G eng %U http://hdl.handle.net/21.11116/0000-0006-09F0-1 %R 10.1109/CVPR42600.2020.00080 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 717 - 726 %I IEEE %@ 978-1-7281-7168-5

%0 Report %A Kappel, Moritz %A Golyanik, Vladislav %A Elgharib, Mohamed %A Henningson, Jann-Ole %A Seidel, Hans-Peter %A Castillo, Susana %A Theobalt, Christian %A Magnor, Marcus A. %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T High-Fidelity Neural Human Motion Transfer from Monocular Video : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B715-3 %U https://arxiv.org/abs/2012.10974 %D 2020 %X Video-based human motion transfer creates video animations of humans<br>following a source motion. Current methods show remarkable results for<br>tightly-clad subjects. However, the lack of temporally consistent handling of<br>plausible clothing dynamics, including fine and high-frequency details,<br>significantly limits the attainable visual quality. We address these<br>limitations for the first time in the literature and present a new framework<br>which performs high-fidelity and temporally-consistent human motion transfer<br>with natural pose-dependent non-rigid deformations, for several types of loose<br>garments. In contrast to the previous techniques, we perform image generation<br>in three subsequent stages, synthesizing human shape, structure, and<br>appearance. Given a monocular RGB video of an actor, we train a stack of<br>recurrent deep neural networks that generate these intermediate representations<br>from 2D poses and their temporal derivatives. Splitting the difficult motion<br>transfer problem into subtasks that are aware of the temporal motion context<br>helps us to synthesize results with plausible dynamics and pose-dependent<br>detail. It also allows artistic control of results by manipulation of<br>individual framework stages. In the experimental results, we significantly<br>outperform the state-of-the-art in terms of video realism. Our code and data<br>will be made publicly available.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

%0 Conference Proceedings %A Liu, Lingjie %A Gu, Jiatao %A Lin, Kyaw Zaw %A Chua, Tat-Seng %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Neural Sparse Voxel Fields : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D437-C %D 2020 %B 34th Conference on Neural Information Processing Systems %Z date of event: 2020-12-06 - 2020-12-12 %C Virtual Event %B Advances in Neural Information Processing Systems 33 %E Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.; Lin, H. %P 15651 - 15663 %I Curran Associates, Inc. %@ 9781713829546 %U https://proceedings.neurips.cc/paper/2020/file/b4b758962f17808746e9bb832a6fa4b8-Paper.pdf

%0 Journal Article %A Liu, Lingjie %A Xu, Weipeng %A Habermann, Marc %A Zollh&#246;fer, Michael %A Bernard, Florian %A Kim, Hyeongwoo %A Wang, Wenping %A Theobalt, Christian %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Learning Dynamic Textures for Neural Rendering of Human Actors : %G eng %U http://hdl.handle.net/21.11116/0000-0009-4C96-9 %R 10.1109/TVCG.2020.2996594 %7 2020 %D 2020 %J IEEE Transactions on Visualization and Computer Graphics %V 27 %N 10 %& 4009 %P 4009 - 4022 %I IEEE %C Piscataway, NJ %@ false

%0 Report %A Liu, Lingjie %A Gu, Jiatao %A Lin, Kyaw Zaw %A Chua, Tat-Seng %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Neural Sparse Voxel Fields : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E8B2-A %U https://arxiv.org/abs/2007.11571 %D 2020 %X Photo-realistic free-viewpoint rendering of real-world scenes using classical<br>computer graphics techniques is challenging, because it requires the difficult<br>step of capturing detailed appearance and geometry models. Recent studies have<br>demonstrated promising results by learning scene representations that<br>implicitly encode both geometry and appearance without 3D supervision. However,<br>existing approaches in practice often show blurry renderings caused by the<br>limited network capacity or the difficulty in finding accurate intersections of<br>camera rays with the scene geometry. Synthesizing high-resolution imagery from<br>these representations often requires time-consuming optical ray marching. In<br>this work, we introduce Neural Sparse Voxel Fields (NSVF), a new neural scene<br>representation for fast and high-quality free-viewpoint rendering. NSVF defines<br>a set of voxel-bounded implicit fields organized in a sparse voxel octree to<br>model local properties in each cell. We progressively learn the underlying<br>voxel structures with a differentiable ray-marching operation from only a set<br>of posed RGB images. With the sparse voxel octree structure, rendering novel<br>views can be accelerated by skipping the voxels containing no relevant scene<br>content. Our method is typically over 10 times faster than the state-of-the-art<br>(namely, NeRF(Mildenhall et al., 2020)) at inference time while achieving<br>higher quality results. Furthermore, by utilizing an explicit sparse voxel<br>representation, our method can easily be applied to scene editing and scene<br>composition. We also demonstrate several challenging tasks, including<br>multi-scene learning, free-viewpoint rendering of a moving human, and<br>large-scale scene rendering. Code and data are available at our website:<br>https://github.com/facebookresearch/NSVF.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

%0 Report %A Long, Xiaoxiao %A Liu, Lingjie %A Li, Wei %A Theobalt, Christian %A Wang, Wenping %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E9EA-B %U https://arxiv.org/abs/2011.13118 %D 2020 %X We present a novel method for multi-view depth estimation from a single<br>video, which is a critical task in various applications, such as perception,<br>reconstruction and robot navigation. Although previous learning-based methods<br>have demonstrated compelling results, most works estimate depth maps of<br>individual video frames independently, without taking into consideration the<br>strong geometric and temporal coherence among the frames. Moreover, current<br>state-of-the-art (SOTA) models mostly adopt a fully 3D convolution network for<br>cost regularization and therefore require high computational cost, thus<br>limiting their deployment in real-world applications. Our method achieves<br>temporally coherent depth estimation results by using a novel Epipolar<br>Spatio-Temporal (EST) transformer to explicitly associate geometric and<br>temporal correlation with multiple estimated depth maps. Furthermore, to reduce<br>the computational cost, inspired by recent Mixture-of-Experts models, we design<br>a compact hybrid network consisting of a 2D context-aware network and a 3D<br>matching network which learn 2D context information and 3D disparity cues<br>separately. Extensive experiments demonstrate that our method achieves higher<br>accuracy in depth estimation and significant speedup than the SOTA methods.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Report %A Long, Xiaoxiao %A Liu, Lingjie %A Theobalt, Christian %A Wang, Wenping %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Occlusion-Aware Depth Estimation with Adaptive Normal Constraints : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E0E9-5 %U https://arxiv.org/abs/2004.00845 %D 2020 %X We present a new learning-based method for multi-frame depth estimation from<br>a color video, which is a fundamental problem in scene understanding, robot<br>navigation or handheld 3D reconstruction. While recent learning-based methods<br>estimate depth at high accuracy, 3D point clouds exported from their depth maps<br>often fail to preserve important geometric feature (e.g., corners, edges,<br>planes) of man-made scenes. Widely-used pixel-wise depth errors do not<br>specifically penalize inconsistency on these features. These inaccuracies are<br>particularly severe when subsequent depth reconstructions are accumulated in an<br>attempt to scan a full environment with man-made objects with this kind of<br>features. Our depth estimation algorithm therefore introduces a Combined Normal<br>Map (CNM) constraint, which is designed to better preserve high-curvature<br>features and global planar regions. In order to further improve the depth<br>estimation accuracy, we introduce a new occlusion-aware strategy that<br>aggregates initial depth predictions from multiple adjacent views into one<br>final depth map and one occlusion probability map for the current reference<br>view. Our method outperforms the state-of-the-art in terms of depth estimation<br>accuracy, and preserves essential geometric features of man-made indoor scenes<br>much better than other algorithms.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV %J ECCV 2020. Lecture Notes in Computer Science, vol 12354. Springer, Cham

%0 Conference Proceedings %A Long, Xiaoxiao %A Liu, Lingjie %A Theobalt, Christian %A Wang, Wenping %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Occlusion-Aware Depth Estimation with Adaptive Normal Constraints : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D42B-A %R 10.1007/978-3-030-58545-7_37 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 640 - 657 %I Springer %@ 978-3-030-58544-0 %B Lecture Notes in Computer Science %N 12354

%0 Report %A Malik, Jameel %A Abdelaziz, Ibrahim %A Elhayek, Ahmed %A Shimada, Soshi %A Ali, Sk Aziz %A Golyanik, Vladislav %A Theobalt, Christian %A Stricker, Didier %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E0FF-D %U https://arxiv.org/abs/2004.01588 %D 2020 %X 3D hand shape and pose estimation from a single depth map is a new and<br>challenging computer vision problem with many applications. The<br>state-of-the-art methods directly regress 3D hand meshes from 2D depth images<br>via 2D convolutional neural networks, which leads to artefacts in the<br>estimations due to perspective distortions in the images. In contrast, we<br>propose a novel architecture with 3D convolutions trained in a<br>weakly-supervised manner. The input to our method is a 3D voxelized depth map,<br>and we rely on two hand shape representations. The first one is the 3D<br>voxelized grid of the shape which is accurate but does not preserve the mesh<br>topology and the number of mesh vertices. The second representation is the 3D<br>hand surface which is less accurate but does not suffer from the limitations of<br>the first representation. We combine the advantages of these two<br>representations by registering the hand surface to the voxelized hand shape. In<br>the extensive experiments, the proposed approach improves over the state of the<br>art by 47.8% on the SynHand5M dataset. Moreover, our augmentation policy for<br>voxelized depth maps further enhances the accuracy of 3D hand pose estimation<br>on real data. Our method produces visually more reasonable and realistic hand<br>shapes on NYU and BigHand2.2M datasets compared to the existing approaches.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Malik, Jameel %A Abdelaziz, Ibrahim %A Elhayek, Ahmed %A Shimada, Soshi %A Ali, Sk Aziz %A Golyanik, Vladislav %A Theobalt, Christian %A Stricker, Didier %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation From a Single Depth Map : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CFCA-D %R 10.1109/CVPR42600.2020.00714 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 7111 - 7120 %I IEEE %@ 978-1-7281-7168-5

%0 Report %A Mallikarjun B R, %A Tewari, Ayush %A Seidel, Hans-Peter %A Elgharib, Mohamed %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Learning Complete 3D Morphable Face Models from Images and Videos : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B6FB-1 %U https://arxiv.org/abs/2010.01679 %D 2020 %X Most 3D face reconstruction methods rely on 3D morphable models, which<br>disentangle the space of facial deformations into identity geometry,<br>expressions and skin reflectance. These models are typically learned from a<br>limited number of 3D scans and thus do not generalize well across different<br>identities and expressions. We present the first approach to learn complete 3D<br>models of face identity geometry, albedo and expression just from images and<br>videos. The virtually endless collection of such data, in combination with our<br>self-supervised learning-based approach allows for learning face models that<br>generalize beyond the span of existing approaches. Our network design and loss<br>functions ensure a disentangled parameterization of not only identity and<br>albedo, but also, for the first time, an expression basis. Our method also<br>allows for in-the-wild monocular reconstruction at test time. We show that our<br>learned models better generalize and lead to higher quality image-based<br>reconstructions than existing approaches.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG,Computer Science, Multimedia, cs.MM

%0 Report %A Mallikarjun B R, %A Tewari, Ayush %A Oh, Tae-Hyun %A Weyrich, Tim %A Bickel, Bernd %A Seidel, Hans-Peter %A Pfister, Hanspeter %A Matusik, Wojciech %A Elgharib, Mohamed %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Monocular Reconstruction of Neural Face Reflectance Fields : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B110-E %U https://arxiv.org/abs/2008.10247 %D 2020 %X The reflectance field of a face describes the reflectance properties<br>responsible for complex lighting effects including diffuse, specular,<br>inter-reflection and self shadowing. Most existing methods for estimating the<br>face reflectance from a monocular image assume faces to be diffuse with very<br>few approaches adding a specular component. This still leaves out important<br>perceptual aspects of reflectance as higher-order global illumination effects<br>and self-shadowing are not modeled. We present a new neural representation for<br>face reflectance where we can estimate all components of the reflectance<br>responsible for the final appearance from a single monocular image. Instead of<br>modeling each component of the reflectance separately using parametric models,<br>our neural representation allows us to generate a basis set of faces in a<br>geometric deformation-invariant space, parameterized by the input light<br>direction, viewpoint and face geometry. We learn to reconstruct this<br>reflectance field of a face just from a monocular image, which can be used to<br>render the face from any viewpoint in any light condition. Our method is<br>trained on a light-stage training dataset, which captures 300 people<br>illuminated with 150 light conditions from 8 viewpoints. We show that our<br>method outperforms existing monocular reflectance reconstruction methods, in<br>terms of photorealism due to better capturing of physical premitives, such as<br>sub-surface scattering, specularities, self-shadows and other higher-order<br>effects.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

%0 Journal Article %A Mehta, Dushyant %A Sotnychenko, Oleksandr %A Mueller, Franziska %A Xu, Weipeng %A Elgharib, Mohamed %A Fua, Pascal %A Seidel, Hans-Peter %A Rhodin, Helge %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0007-832D-3 %R 10.1145/3386569.3392410 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 4 %Z sequence number: 82 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2020 %O ACM SIGGRAPH 2020 Virtual Conference ; 2020, 17-28 August

%0 Journal Article %A Meka, Abhimitra %A Pandey, Rohit %A H&#228;ne, Christian %A Orts-Escolano, Sergio %A Barnum, Peter %A David-Son, Philip %A Erickson, Daniel %A Zhang, Yinda %A Taylor, Jonathan %A Bouaziz, Sofien %A Legendre, Chloe %A Ma, Wan-Chun %A Overbeck, Ryan %A Beeler, Thabo %A Debevec, Paul %A Izadi, Shahram %A Theobalt, Christian %A Rhemann, Christoph %A Fanello, Sean %+ External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Deep Relightable Textures Volumetric Performance Capture with Neural Rendering : %G eng %U http://hdl.handle.net/21.11116/0000-0007-A6FA-4 %R 10.1145/3414685.3417814 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 259 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020 %U https://dl.acm.org/doi/pdf/10.1145/3414685.3417814

%0 Thesis %A Meka, Abhimitra %Y Theobalt, Christian %A referee: Drettakis, George %+ Computer Graphics, MPI for Informatics, Max Planck Society International Max Planck Research School, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Live inverse rendering : %G eng %U http://hdl.handle.net/21.11116/0000-0007-715A-5 %R http://dx.doi.org/10.22028/D291-30206 %U urn:nbn:de:bsz:291--ds-302066 %F OTHER: hdl:20.500.11880/28721 %I Universit&#228;t des Saarlandes %C Saarbr&#252;cken %D 2020 %P 189 p. %V phd %9 phd %U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/28721

%0 Conference Proceedings %A Meng, Xiaoxu %A Zheng, Quan %A Varshney, Amitabh %A Singh, Gurprit %A Zwicker, Matthias %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Real-time Monte Carlo Denoising with the Neural Bilateral Grid : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CEC2-6 %R 10.2312/sr.20201133 %U https://diglib.eg.org:443/handle/10.2312/sr20201133 %D 2020 %B 31st Eurographics Symposium on Rendering %Z date of event: 2020-06-29 - 2020-07-02 %C London, UK %B Rendering 2020 - DL-only Track %E Dachsbacher, Carsten; Pharr, Matt %P 1 - 12 %I The Eurographics Association %@ 978-3-03868-117-5

%0 Journal Article %A Mlakar, Daniel %A Winter, M. %A Stadlbauer, Pascal %A Seidel, Hans-Peter %A Steinberger, Markus %A Zayer, Rhaleb %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Subdivision-Specialized Linear Algebra Kernels for Static and Dynamic Mesh Connectivity on the GPU : %G eng %U http://hdl.handle.net/21.11116/0000-0006-DB80-2 %R 10.1111/cgf.13934 %7 2020 %D 2020 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 39 %N 2 %& 335 %P 335 - 349 %I Blackwell-Wiley %C Oxford %@ false %B The European Association for Computer Graphics 41st Annual Conference %O EUROGRAPHICS 2020 EG 2020 The European Association for Computer Graphics 41st Annual Conference ; Norrk&#246;ping, Sweden, May 25 &#8211; 29, 2020

%0 Thesis %A Mueller, Franziska %Y Theobalt, Christian %A referee: Seidel, Hans-Peter %A referee: Izadi, Shahram %+ Computer Graphics, MPI for Informatics, Max Planck Society International Max Planck Research School, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Real-time 3D Hand Reconstruction in Challenging Scenes from a Single Color or Depth Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D8C7-5 %R 10.22028/D291-32846 %F OTHER: hdl:20.500.11880/30313 %U urn:nbn:de:bsz:291--ds-328467 %I Universit&#228;t des Saarlandes %C Saarbr&#252;cken %D 2020 %P 155 p. %V phd %9 phd %U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/30313

%0 Journal Article %A Piovar&#269;i, Michal %A Foshey, Michael %A Babaei, Vahid %A Rusinkiewicz, Szymon %A Matusik, Wojciech %A Didyk, Piotr %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations %T Towards Spatially Varying Gloss Reproduction for 3D Printing : %G eng %U http://hdl.handle.net/21.11116/0000-0007-A6FE-0 %R 10.1145/3414685.3417850 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 206 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Conference Proceedings %A Qian, Neng %A Wang, Jiayi %A Mueller, Franziska %A Bernard, Florian %A Golyanik, Vladislav %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D062-F %R 10.1007/978-3-030-58621-8_4 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 54 - 71 %I Springer %@ 978-3-030-58621-8 %B Lecture Notes in Computer Science %N 12356

%0 Report %A Qian, Neng %A Wang, Jiayi %A Mueller, Franziska %A Bernard, Florian %A Golyanik, Vladislav %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization : %G eng %U http://hdl.handle.net/21.11116/0000-0006-9128-9 %Y Max-Planck-Institut f&#252;r Informatik %C Saarbr&#252;cken %D 2020 %P 37 p. %X 3D hand reconstruction from image data is a widely-studied problem in com-<br>puter vision and graphics, and has a particularly high relevance for virtual<br>and augmented reality. Although several 3D hand reconstruction approaches<br>leverage hand models as a strong prior to resolve ambiguities and achieve a<br>more robust reconstruction, most existing models account only for the hand<br>shape and poses and do not model the texture. To &#64257;ll this gap, in this work<br>we present the &#64257;rst parametric texture model of human hands. Our model<br>spans several dimensions of hand appearance variability (e.g., related to gen-<br>der, ethnicity, or age) and only requires a commodity camera for data acqui-<br>sition. Experimentally, we demonstrate that our appearance model can be<br>used to tackle a range of challenging problems such as 3D hand reconstruc-<br>tion from a single monocular image. Furthermore, our appearance model<br>can be used to de&#64257;ne a neural rendering layer that enables training with a<br>self-supervised photometric loss. We make our model publicly available. %K hand texture model, appearance modeling, hand tracking, 3D hand recon- struction %B Research Report %@ false

%0 Report %A Rao, Sukrut %A Stutz, David %A Schiele, Bernt %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Adversarial Training against Location-Optimized Adversarial Patches : %G eng %U http://hdl.handle.net/21.11116/0000-0007-80D0-C %U https://arxiv.org/abs/2005.02313 %D 2020 %X Deep neural networks have been shown to be susceptible to adversarial<br>examples -- small, imperceptible changes constructed to cause<br>mis-classification in otherwise highly accurate image classifiers. As a<br>practical alternative, recent work proposed so-called adversarial patches:<br>clearly visible, but adversarially crafted rectangular patches in images. These<br>patches can easily be printed and applied in the physical world. While defenses<br>against imperceptible adversarial examples have been studied extensively,<br>robustness against adversarial patches is poorly understood. In this work, we<br>first devise a practical approach to obtain adversarial patches while actively<br>optimizing their location within the image. Then, we apply adversarial training<br>on these location-optimized adversarial patches and demonstrate significantly<br>improved robustness on CIFAR10 and GTSRB. Additionally, in contrast to<br>adversarial training on imperceptible adversarial examples, our adversarial<br>patch training does not reduce accuracy.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Cryptography and Security, cs.CR,Computer Science, Learning, cs.LG,Statistics, Machine Learning, stat.ML

%0 Journal Article %A Saberpour, Artin %A Hersch, Roger D. %A Fang, Jiajing %A Zayer, Rhaleb %A Seidel, Hans-Peter %A Babaei, Vahid %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Fabrication of Moir&#233; on Curved Surfaces : %G eng %U http://hdl.handle.net/21.11116/0000-0006-D39D-B %R 10.1364/OE.393843 %7 2020 %D 2020 %J Optics Express %O Opt. Express %V 28 %N 13 %& 19413 %P 19413 - 19427 %I Optical Society of America %C Washington, DC %@ false

%0 Conference Proceedings %A Sarkar, Kripasindhu %A Mehta, Dushyant %A Xu, Weipeng %A Golyanik, Vladislav %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Neural Re-rendering of Humans from a Single Image : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D0A4-4 %R 10.1007/978-3-030-58621-8_35 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 596 - 613 %I Springer %@ 978-3-030-58621-8 %B Lecture Notes in Computer Science %N 12356

%0 Conference Proceedings %A Seelbach Benkner, Marcel %A Golyanik, Vladislav %A Theobalt, Christian %A Moeller, Michael %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Adiabatic Quantum Graph Matching with Permutation Matrix Constraints : %G eng %U http://hdl.handle.net/21.11116/0000-0008-0362-6 %R 10.1109/3DV50981.2020.00068 %D 2020 %B International Conference on 3D Vision %Z date of event: 2020-11-25 - 2020-11-28 %C Fukuoka, Japan (Virtual Event) %B International Conference on 3D Vision %P 583 - 592 %I IEEE %@ 978-1-7281-8128-8

%0 Journal Article %A Serrano, Ana %A Martin, Daniel %A Gutierrez, Diego %A Myszkowski, Karol %A Masia, Belen %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Imperceptible Manipulation of Lateral Camera Motion for Improved Virtual Reality Applications : %G eng %U http://hdl.handle.net/21.11116/0000-0006-FBE8-A %R 10.1145/3414685.3417773 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 268 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Journal Article %A Shahmirzadi, Azadeh Asadi %A Babaei, Vahid %A Seidel, Hans-Peter %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T A Multispectral Dataset of Oil and Watercolor Paints : %G eng %U http://hdl.handle.net/21.11116/0000-0007-F064-9 %R 10.2352/ISSN.2470-1173.2020.5.MAAP-107 %7 2020 %D 2020 %J Electronic Imaging %& 1 %P 1 - 4 %Z sequence number: 107 %I IS&T %C Springfield, VA

%0 Journal Article %A Shimada, Soshi %A Golyanik, Vladislav %A Xu, Weipeng %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time : %G eng %U http://hdl.handle.net/21.11116/0000-0007-A709-3 %R 10.1145/3414685.3417877 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 235 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Report %A Shimada, Soshi %A Golyanik, Vladislav %A Xu, Weipeng %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E8F3-1 %U https://arxiv.org/abs/2008.08880 %D 2020 %X Marker-less 3D human motion capture from a single colour camera has seen<br>significant progress. However, it is a very challenging and severely ill-posed<br>problem. In consequence, even the most accurate state-of-the-art approaches<br>have significant limitations. Purely kinematic formulations on the basis of<br>individual joints or skeletons, and the frequent frame-wise reconstruction in<br>state-of-the-art methods greatly limit 3D accuracy and temporal stability<br>compared to multi-view or marker-based motion capture. Further, captured 3D<br>poses are often physically incorrect and biomechanically implausible, or<br>exhibit implausible environment interactions (floor penetration, foot skating,<br>unnatural body leaning and strong shifting in depth), which is problematic for<br>any use case in computer graphics. We, therefore, present PhysCap, the first<br>algorithm for physically plausible, real-time and marker-less human 3D motion<br>capture with a single colour camera at 25 fps. Our algorithm first captures 3D<br>human poses purely kinematically. To this end, a CNN infers 2D and 3D joint<br>positions, and subsequently, an inverse kinematics step finds space-time<br>coherent joint angles and global 3D pose. Next, these kinematic reconstructions<br>are used as constraints in a real-time physics-based pose optimiser that<br>accounts for environment constraints (e.g., collision handling and floor<br>placement), gravity, and biophysical plausibility of human postures. Our<br>approach employs a combination of ground reaction force and residual force for<br>plausible root control, and uses a trained neural network to detect foot<br>contact events in images. Our method captures physically plausible and<br>temporally stable global 3D human motion, without physically implausible<br>postures, floor penetrations or foot skating, from video in real time and in<br>general scenes. The video is available at<br>http://gvv.mpi-inf.mpg.de/projects/PhysCap<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

%0 Conference Proceedings %A Sidhu, Vikramjit %A Tretschk, Edgar %A Golyanik, Vladislav %A Agudo, Antonio %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D080-C %R 10.1007/978-3-030-58517-4_13 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 204 - 222 %I Springer %@ 978-3-030-58516-7 %B Lecture Notes in Computer Science %N 12361

%0 Journal Article %A Singh, Gurprit %A Subr, Kartic %A Coeurjolly, David %A Ostromoukhov, Victor %A Jarosz, Wojciech %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T Fourier Analysis of Correlated Monte Carlo Importance Sampling : %G eng %U http://hdl.handle.net/21.11116/0000-0006-978D-1 %R 10.1111/cgf.13613 %7 2020 %D 2020 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 39 %N 1 %& 7 %P 7 - 19 %I Blackwell-Wiley %C Oxford %@ false

%0 Journal Article %A Stadlbauer, Pascal %A Mlakar, Daniel %A Seidel, Hans-Peter %A Steinberger, Markus %A Zayer, Rhaleb %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Interactive Modeling of Cellular Structures on Surfaces with Application to Additive Manufacturing : %G eng %U http://hdl.handle.net/21.11116/0000-0006-DB8A-8 %R 10.1111/cgf.13929 %7 2020 %D 2020 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 39 %N 2 %& 277 %P 277 - 289 %I Blackwell-Wiley %C Oxford %@ false %B The European Association for Computer Graphics 41st Annual Conference %O EUROGRAPHICS 2020 EG 2020 The European Association for Computer Graphics 41st Annual Conference ; Norrk&#246;ping, Sweden, May 25 &#8211; 29, 2020

%0 Journal Article %A Sultan, Ahmed S. %A Elgharib, Mohamed %A Tavares, Tiffany %A Jessri, Maryam %A Basile, John R. %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations %T The Use of Artificial Intelligence, Machine Learning and Deep Learning in Oncologic Histopathology : %G eng %U http://hdl.handle.net/21.11116/0000-0006-A2C9-0 %R 10.1111/jop.13042 %7 2020 %D 2020 %J Journal of Oral Pathology & Medicine %V 49 %N 9 %& 849 %P 849 - 856 %I Wiley-Blackwell %C Oxford %@ false

%0 Conference Proceedings %A Tewari, Ayush %A Elgharib, Mohamed %A Bharaj, Gaurav %A Bernard, Florian %A Seidel, Hans-Peter %A P&#233;rez, Patrick %A Zollh&#246;fer, Michael %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T StyleRig: Rigging StyleGAN for 3D Control Over Portrait Images : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B0E7-D %R 10.1109/CVPR42600.2020.00618 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 6141 - 6150 %I IEEE %@ 978-1-7281-7168-5

%0 Report %A Tewari, Ayush %A Fried, Ohad %A Thies, Justus %A Sitzmann, Vincent %A Lombardi, Stephen %A Sunkavalli, Kalyan %A Martin-Brualla, Ricardo %A Simon, Tomas %A Saragih, Jason %A Nie&#223;ner, Matthias %A Pandey, Rohit %A Fanello, Sean %A Wetzstein, Gordon %A Zhu, Jun-Yan %A Theobalt, Christian %A Agrawala, Maneesh %A Shechtman, Eli %A Goldman, Dan B %A Zollh&#246;fer, Michael %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T State of the Art on Neural Rendering : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E114-4 %U https://arxiv.org/abs/2004.03805 %D 2020 %X Efficient rendering of photo-realistic virtual worlds is a long standing<br>effort of computer graphics. Modern graphics techniques have succeeded in<br>synthesizing photo-realistic images from hand-crafted scene representations.<br>However, the automatic generation of shape, materials, lighting, and other<br>aspects of scenes remains a challenging problem that, if solved, would make<br>photo-realistic computer graphics more widely accessible. Concurrently,<br>progress in computer vision and machine learning have given rise to a new<br>approach to image synthesis and editing, namely deep generative models. Neural<br>rendering is a new and rapidly emerging field that combines generative machine<br>learning techniques with physical knowledge from computer graphics, e.g., by<br>the integration of differentiable rendering into network training. With a<br>plethora of applications in computer graphics and vision, neural rendering is<br>poised to become a new area in the graphics community, yet no survey of this<br>emerging field exists. This state-of-the-art report summarizes the recent<br>trends and applications of neural rendering. We focus on approaches that<br>combine classic computer graphics techniques with deep generative models to<br>obtain controllable and photo-realistic outputs. Starting with an overview of<br>the underlying computer graphics and machine learning concepts, we discuss<br>critical aspects of neural rendering approaches. This state-of-the-art report<br>is focused on the many important use cases for the described algorithms such as<br>novel view synthesis, semantic photo manipulation, facial and body reenactment,<br>relighting, free-viewpoint video, and the creation of photo-realistic avatars<br>for virtual and augmented reality telepresence. Finally, we conclude with a<br>discussion of the social implications of such technology and investigate open<br>research problems.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

%0 Report %A Tewari, Ayush %A Elgharib, Mohamed %A Bharaj, Gaurav %A Bernard, Florian %A Seidel, Hans-Peter %A P&#233;rez, Patrick %A Zollh&#246;fer, Michael %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T StyleRig: Rigging StyleGAN for 3D Control over Portrait Images : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B0FC-6 %U https://arxiv.org/abs/2004.00121 %D 2020 %X StyleGAN generates photorealistic portrait images of faces with eyes, teeth,<br>hair and context (neck, shoulders, background), but lacks a rig-like control<br>over semantic face parameters that are interpretable in 3D, such as face pose,<br>expressions, and scene illumination. Three-dimensional morphable face models<br>(3DMMs) on the other hand offer control over the semantic parameters, but lack<br>photorealism when rendered and only model the face interior, not other parts of<br>a portrait image (hair, mouth interior, background). We present the first<br>method to provide a face rig-like control over a pretrained and fixed StyleGAN<br>via a 3DMM. A new rigging network, RigNet is trained between the 3DMM's<br>semantic parameters and StyleGAN's input. The network is trained in a<br>self-supervised manner, without the need for manual annotations. At test time,<br>our method generates portrait images with the photorealism of StyleGAN and<br>provides explicit control over the 3D semantic parameters of the face.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

%0 Journal Article %A Tewari, Ayush %A Zollh&#246;fer, Michael %A Bernard, Florian %A Garrido, Pablo %A Kim, Hyeongwoo %A P&#233;rez, Patrick %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T High-Fidelity Monocular Face Reconstruction based on an Unsupervised Model-based Face Autoencoder : %G eng %U http://hdl.handle.net/21.11116/0000-0002-EF5B-C %R 10.1109/TPAMI.2018.2876842 %7 2018 %D 2020 %J IEEE Transactions on Pattern Analysis and Machine Intelligence %O IEEE Trans. Pattern Anal. Mach. Intell. %V 42 %N 2 %& 357 %P 357 - 370 %I IEEE %C Piscataway, NJ %@ false

%0 Journal Article %A Tewari, Ayush %A Fried, Ohad %A Thies, Justus %A Sitzmann, Vincent %A Lombardi, Stephen %A Sunkavalli, Kalyan %A Martin-Brualla, Ricardo %A Simon, Tomas %A Saragih, Jason %A Nie&#223;ner, Matthias %A Pandey, Rohit %A Fanello, Sean %A Wetzstein, Gordon %A Zhu, Jun-Yan %A Theobalt, Christian %A Agrawala, Maneesh %A Shechtman, Eli %A Goldman, Dan B. %A Zollh&#246;fer, Michael %+ Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T State of the Art on Neural Rendering : %G eng %U http://hdl.handle.net/21.11116/0000-0006-DB93-D %R 10.1111/cgf.14022 %7 2020 %D 2020 %J Computer Graphics Forum %O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum %V 39 %N 2 %& 701 %P 701 - 727 %I Blackwell-Wiley %C Oxford %@ false %B The European Association for Computer Graphics 41st Annual Conference %O EUROGRAPHICS 2020 EG 2020 The European Association for Computer Graphics 41st Annual Conference ; Norrk&#246;ping, Sweden, May 25 &#8211; 29, 2020

%0 Journal Article %A Tewari, Ayush %A Elgharib, Mohamed %A Mallikarjun B R, %A Bernard, Florian %A Seidel, Hans-Peter %A P&#233;rez, Patrick %A Zollh&#246;fer, Michael %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T PIE: Portrait Image Embedding for Semantic Control : %G eng %U http://hdl.handle.net/21.11116/0000-0007-9B0C-E %R 10.1145/3414685.3417803 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 223 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Report %A Tewari, Ayush %A Elgharib, Mohamed %A Mallikarjun B R, %A Bernard, Florian %A Seidel, Hans-Peter %A P&#233;rez, Patrick %A Zollh&#246;fer, Michael %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T PIE: Portrait Image Embedding for Semantic Control : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B117-7 %U https://arxiv.org/abs/2009.09485 %D 2020 %X Editing of portrait images is a very popular and important research topic<br>with a large variety of applications. For ease of use, control should be<br>provided via a semantically meaningful parameterization that is akin to<br>computer animation controls. The vast majority of existing techniques do not<br>provide such intuitive and fine-grained control, or only enable coarse editing<br>of a single isolated control parameter. Very recently, high-quality<br>semantically controlled editing has been demonstrated, however only on<br>synthetically created StyleGAN images. We present the first approach for<br>embedding real portrait images in the latent space of StyleGAN, which allows<br>for intuitive editing of the head pose, facial expression, and scene<br>illumination in the image. Semantic editing in parameter space is achieved<br>based on StyleRig, a pretrained neural network that maps the control space of a<br>3D morphable face model to the latent space of the GAN. We design a novel<br>hierarchical non-linear optimization problem to obtain the embedding. An<br>identity preservation energy term allows spatially coherent edits while<br>maintaining facial integrity. Our approach runs at interactive frame rates and<br>thus allows the user to explore the space of possible edits. We evaluate our<br>approach on a wide set of portrait photos, compare it to the current state of<br>the art, and validate the effectiveness of its components in an ablation study.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

%0 Report %A Thies, Justus %A Zollh&#246;fer, Michael %A Stamminger, Marc %A Theobalt, Christian %A Nie&#223;ner, Matthias %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Face2Face: Real-time Face Capture and Reenactment of RGB Videos : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E8E9-D %U https://arxiv.org/abs/2007.14808 %D 2020 %X We present Face2Face, a novel approach for real-time facial reenactment of a<br>monocular target video sequence (e.g., Youtube video). The source sequence is<br>also a monocular video stream, captured live with a commodity webcam. Our goal<br>is to animate the facial expressions of the target video by a source actor and<br>re-render the manipulated output video in a photo-realistic fashion. To this<br>end, we first address the under-constrained problem of facial identity recovery<br>from monocular video by non-rigid model-based bundling. At run time, we track<br>facial expressions of both source and target video using a dense photometric<br>consistency measure. Reenactment is then achieved by fast and efficient<br>deformation transfer between source and target. The mouth interior that best<br>matches the re-targeted expression is retrieved from the target sequence and<br>warped to produce an accurate fit. Finally, we convincingly re-render the<br>synthesized target face on top of the corresponding video stream such that it<br>seamlessly blends with the real-world illumination. We demonstrate our method<br>in a live setup, where Youtube videos are reenacted in real time.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Thies, Justus %A Elgharib, Mohamed %A Tewari, Ayush %A Theobalt, Christian %A Nie&#223;ner, Matthias %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Neural Voice Puppetry: Audio-Driven Facial Reenactment : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D42F-6 %R 10.1007/978-3-030-58517-4_42 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 716 - 731 %I Springer %@ 978-3-030-58516-7 %B Lecture Notes in Computer Science %N 12361

%0 Conference Proceedings %A Thies, Justus %A Zollh&#246;fer, Michael %A Theobalt, Christian %A Stamminger, Marc %A Nie&#223;ner, Matthias %+ External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Image-guided Neural Object Rendering : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D431-2 %U https://openreview.net/forum?id=Hyg9anEFPS %D 2020 %B 8th International Conference on Learning Representations %Z date of event: 2020-04-26 - 2020-04-30 %C Addis Ababa, Ethopia %B International Conference on Learning Representations %I OpenReview.net %U https://openreview.net/forum?id=Hyg9anEFPS

%0 Journal Article %A Tong, Xin %A Myszkowski, Karol %A Huang, Jin %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Foreword to the Special Section on the International Conference on Computer-Aided Design and Computer Graphics (CAD/Graphics) 2019 : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CEAF-D %R 10.1016/j.cag.2019.12.002 %7 2019 %D 2020 %J Computers and Graphics %V 86 %& A5 %P A5 - A6 %I Elsevier %C Amsterdam %@ false

%0 Report %A Tretschk, Edgar %A Tewari, Ayush %A Golyanik, Vladislav %A Zollh&#246;fer, Michael %A Stoll, Carsten %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E8ED-9 %U https://arxiv.org/abs/2008.01639 %D 2020 %X Implicit surface representations, such as signed-distance functions, combined<br>with deep learning have led to impressive models which can represent detailed<br>shapes of objects with arbitrary topology. Since a continuous function is<br>learned, the reconstructions can also be extracted at any arbitrary resolution.<br>However, large datasets such as ShapeNet are required to train such models. In<br>this paper, we present a new mid-level patch-based surface representation. At<br>the level of patches, objects across different categories share similarities,<br>which leads to more generalizable models. We then introduce a novel method to<br>learn this patch-based representation in a canonical space, such that it is as<br>object-agnostic as possible. We show that our representation trained on one<br>category of objects from ShapeNet can also well represent detailed shapes from<br>any other category. In addition, it can be trained using much fewer shapes,<br>compared to existing approaches. We show several applications of our new<br>representation, including shape interpolation and partial point cloud<br>completion. Due to explicit control over positions, orientations and scales of<br>patches, our representation is also more controllable compared to object-level<br>representations, which enables us to deform encoded shapes non-rigidly.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

%0 Conference Proceedings %A Tretschk, Edgar %A Tewari, Ayush %A Golyanik, Vladislav %A Zollh&#246;fer, Michael %A Stoll, Carsten %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D095-5 %R 10.1007/978-3-030-58517-4_18 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 293 - 309 %I Springer %@ 978-3-030-58516-7 %B Lecture Notes in Computer Science %N 12361

%0 Report %A Tretschk, Edgar %A Tewari, Ayush %A Golyanik, Vladislav %A Zollh&#246;fer, Michael %A Lassner, Christoph %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video : %G eng %U http://hdl.handle.net/21.11116/0000-0007-EA00-1 %U https://arxiv.org/abs/2012.12247 %D 2020 %X In this tech report, we present the current state of our ongoing work on<br>reconstructing Neural Radiance Fields (NERF) of general non-rigid scenes via<br>ray bending. Non-rigid NeRF (NR-NeRF) takes RGB images of a deforming object<br>(e.g., from a monocular video) as input and then learns a geometry and<br>appearance representation that not only allows to reconstruct the input<br>sequence but also to re-render any time step into novel camera views with high<br>fidelity. In particular, we show that a consumer-grade camera is sufficient to<br>synthesize convincing bullet-time videos of short and simple scenes. In<br>addition, the resulting representation enables correspondence estimation across<br>views and time, and provides rigidity scores for each point in the scene. We<br>urge the reader to watch the supplemental videos for qualitative results. We<br>will release our code.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

%0 Conference Proceedings %A Tretschk, Edgar %A Tewari, Ayush %A Zollh&#246;fer, Michael %A Golyanik, Vladislav %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T DEMEA: Deep Mesh Autoencoders for Non-rigidly Deforming Objects : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D425-0 %R 10.1007/978-3-030-58548-8_35 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 601 - 617 %I Springer %@ 978-3-030-58516-7 %B Lecture Notes in Computer Science %N 12349

%0 Conference Proceedings %A Wang, Jiayi %A Mueller, Franziska %A Bernard, Florian %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation : %G eng %U http://hdl.handle.net/21.11116/0000-0008-1687-7 %R 10.1109/FG47880.2020.00013. %D 2020 %B 15th IEEE International Conference on Automatic Face and Gesture Recognition %Z date of event: 2020-11-16 - 2020-11-20 %C Buenos Aires, Argentina %B 15th IEEE International Conference on Automatic Face and Gesture Recognition %E &#352;truc, Vitomir; G&#243;mez-Fern&#225;ndez, Francisco %P 101 - 108 %I IEEE %@ 978-1-7281-3079-8

%0 Journal Article %A Wang, Jiayi %A Mueller, Franziska %A Bernard, Florian %A Sorli, Suzanne %A Sotnychenko, Oleksandr %A Qian, Neng %A Otaduy, Miguel A. %A Casas, Dan %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CF20-C %R 10.1145/3414685.3417852 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 218 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2020 %O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Report %A Wang, Jiayi %A Mueller, Franziska %A Bernard, Florian %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E89A-6 %U https://arxiv.org/abs/2007.03073 %D 2020 %X We propose to use a model-based generative loss for training hand pose<br>estimators on depth images based on a volumetric hand model. This additional<br>loss allows training of a hand pose estimator that accurately infers the entire<br>set of 21 hand keypoints while only using supervision for 6 easy-to-annotate<br>keypoints (fingertips and wrist). We show that our partially-supervised method<br>achieves results that are comparable to those of fully-supervised methods which<br>enforce articulation consistency. Moreover, for the first time we demonstrate<br>that such an approach can be used to train on datasets that have erroneous<br>annotations, i.e. "ground truth" with notable measurement errors, while<br>obtaining predictions that explain the depth images better than the given<br>"ground truth".<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Journal Article %A Wang, Peng %A Liu, Lingjie %A Chen, Nenglun %A Chu, Hung-Kuo %A Theobalt, Christian %A Wang, Wenping %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video : %G eng %U http://hdl.handle.net/21.11116/0000-0007-9A74-9 %R 10.1145/3386569.3392476 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 4 %Z sequence number: 132 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2020 %O ACM SIGGRAPH 2020 Virtual Conference ; 2020, 17-28 August

%0 Report %A Wang, Peng %A Liu, Lingjie %A Chen, Nenglun %A Chu, Hung-Kuo %A Theobalt, Christian %A Wang, Wenping %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E122-4 %U https://arxiv.org/abs/2005.03372 %D 2020 %X Thin structures, such as wire-frame sculptures, fences, cables, power lines,<br>and tree branches, are common in the real world. It is extremely challenging to<br>acquire their 3D digital models using traditional image-based or depth-based<br>reconstruction methods because thin structures often lack distinct point<br>features and have severe self-occlusion. We propose the first approach that<br>simultaneously estimates camera motion and reconstructs the geometry of complex<br>3D thin structures in high quality from a color video captured by a handheld<br>camera. Specifically, we present a new curve-based approach to estimate<br>accurate camera poses by establishing correspondences between featureless thin<br>objects in the foreground in consecutive video frames, without requiring visual<br>texture in the background scene to lock on. Enabled by this effective<br>curve-based camera pose estimation strategy, we develop an iterative<br>optimization method with tailored measures on geometry, topology as well as<br>self-occlusion handling for reconstructing 3D thin structures. Extensive<br>validations on a variety of thin structures show that our method achieves<br>accurate camera pose estimation and faithful reconstruction of 3D thin<br>structures with complex shape and topology at a level that has not been<br>attained by other existing reconstruction methods.<br> %K Computer Science, Graphics, cs.GR,Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

%0 Conference Proceedings %A Xu, Lan %A Xu, Weipeng %A Golyanik, Vladislav %A Habermann, Marc %A Fang, Lu %A Theobalt, Christian %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T EventCap: Monocular 3D Capture of High-Speed Human Motions Using an Event Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CF57-F %R 10.1109/CVPR42600.2020.00502 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 4967 - 4977 %I IEEE %@ 978-1-7281-7168-5

%0 Conference Proceedings %A Xu, Yifan %A Fan, Tianqi %A Yuan, Yi %A Singh, Gurprit %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CEBE-C %R 10.1007/978-3-030-58452-8_15 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 248 - 263 %I Springer %@ 978-3-030-58451-1 %B Lecture Notes in Computer Science %N 12346

%0 Report %A Xu, Yifan %A Fan, Tianqi %A Yuan, Yi %A Singh, Gurprit %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry : %G eng %U http://hdl.handle.net/21.11116/0000-0007-CECA-E %U https://arxiv.org/abs/2007.13393 %D 2020 %X Deep implicit field regression methods are effective for 3D reconstruction<br>from single-view images. However, the impact of different sampling patterns on<br>the reconstruction quality is not well-understood. In this work, we first study<br>the effect of point set discrepancy on the network training. Based on Farthest<br>Point Sampling algorithm, we propose a sampling scheme that theoretically<br>encourages better generalization performance, and results in fast convergence<br>for SGD-based optimization algorithms. Secondly, based on the reflective<br>symmetry of an object, we propose a feature fusion method that alleviates<br>issues due to self-occlusions which makes it difficult to utilize local image<br>features. Our proposed system Ladybird is able to create high quality 3D object<br>reconstructions from a single input image. We evaluate Ladybird on a large<br>scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms<br>of Chamfer distance, Earth Mover's distance and Intersection Over Union (IoU).<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Report %A Yenamandra, Tarun %A Tewari, Ayush %A Bernard, Florian %A Seidel, Hans-Peter %A Elgharib, Mohamed %A Cremers, Daniel %A Theobalt, Christian %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T i3DMM: Deep Implicit 3D Morphable Model of Human Heads : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B702-8 %U https://arxiv.org/abs/2011.14143 %D 2020 %X We present the first deep implicit 3D morphable model (i3DMM) of full heads.<br>Unlike earlier morphable face models it not only captures identity-specific<br>geometry, texture, and expressions of the frontal face, but also models the<br>entire head, including hair. We collect a new dataset consisting of 64 people<br>with different expressions and hairstyles to train i3DMM. Our approach has the<br>following favorable properties: (i) It is the first full head morphable model<br>that includes hair. (ii) In contrast to mesh-based models it can be trained on<br>merely rigidly aligned scans, without requiring difficult non-rigid<br>registration. (iii) We design a novel architecture to decouple the shape model<br>into an implicit reference shape and a deformation of this reference shape.<br>With that, dense correspondences between shapes can be learned implicitly. (iv)<br>This architecture allows us to semantically disentangle the geometry and color<br>components, as color is learned in the reference space. Geometry is further<br>disentangled as identity, expressions, and hairstyle, while color is<br>disentangled as identity and hairstyle components. We show the merits of i3DMM<br>using ablation studies, comparisons to state-of-the-art models, and<br>applications such as semantic head editing and texture transfer. We will make<br>our model publicly available.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

%0 Report %A Yoon, Jae Shin %A Liu, Lingjie %A Golyanik, Vladislav %A Sarkar, Kripasindhu %A Park, Hyun Soo %A Theobalt, Christian %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Pose-Guided Human Animation from a Single Image in the Wild : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E9F3-0 %U https://arxiv.org/abs/2012.03796 %D 2020 %X We present a new pose transfer method for synthesizing a human animation from<br>a single image of a person controlled by a sequence of body poses. Existing<br>pose transfer methods exhibit significant visual artifacts when applying to a<br>novel scene, resulting in temporal inconsistency and failures in preserving the<br>identity and textures of the person. To address these limitations, we design a<br>compositional neural network that predicts the silhouette, garment labels, and<br>textures. Each modular network is explicitly dedicated to a subtask that can be<br>learned from the synthetic data. At the inference time, we utilize the trained<br>network to produce a unified representation of appearance and its labels in UV<br>coordinates, which remains constant across poses. The unified representation<br>provides an incomplete yet strong guidance to generating the appearance in<br>response to the pose change. We use the trained network to complete the<br>appearance and render it with the background. With these strategies, we are<br>able to synthesize human animations that can preserve the identity and<br>appearance of the person in a temporally coherent way without any fine-tuning<br>of the network on the testing scene. Experiments show that our method<br>outperforms the state-of-the-arts in terms of synthesis quality, temporal<br>coherence, and generalization ability.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Yu, Ye %A Meka, Abhimitra %A Elgharib, Mohamed %A Seidel, Hans-Peter %A Theobalt, Christian %A Smith, William A. P. %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Self-supervised Outdoor Scene Relighting : %G eng %U http://hdl.handle.net/21.11116/0000-0007-B0F6-C %R 10.1007/978-3-030-58542-6_6 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 84 - 101 %I Springer %@ 978-3-030-58541-9 %B Lecture Notes in Computer Science %N 12367

%0 Journal Article %A Zheng, Quan %A Babaei, Vahid %A Wetzstein, Gordon %A Seidel, Hans-Peter %A Zwicker, Matthias %A Singh, Gurprit %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society %T Neural Light Field 3D Printing : %U http://hdl.handle.net/21.11116/0000-0007-9AA8-E %R 10.1145/3414685.3417879 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 6 %Z sequence number: 207 %I ACM %C New York, NY %@ false %B Proceedings of the SIGGRAPH Asia 2020 %O SIGGRAPH Asia 2020 SA'20 SA 2020

%0 Report %A Zhou, Yuxiao %A Habermann, Marc %A Habibie, Ikhsanul %A Tewari, Ayush %A Theobalt, Christian %A Xu, Feng %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Monocular Real-time Full Body Capture with Inter-part Correlations : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E9FB-8 %U https://arxiv.org/abs/2012.06087 %D 2020 %X We present the first method for real-time full body capture that estimates<br>shape and motion of body and hands together with a dynamic 3D face model from a<br>single color image. Our approach uses a new neural network architecture that<br>exploits correlations between body and hands at high computational efficiency.<br>Unlike previous works, our approach is jointly trained on multiple datasets<br>focusing on hand, body or face separately, without requiring data where all the<br>parts are annotated at the same time, which is much more difficult to create at<br>sufficient variety. The possibility of such multi-dataset training enables<br>superior generalization ability. In contrast to earlier monocular full body<br>methods, our approach captures more expressive 3D face geometry and color by<br>estimating the shape, expression, albedo and illumination parameters of a<br>statistical face model. Our method achieves competitive accuracy on public<br>benchmarks, while being significantly faster and providing more complete face<br>reconstructions.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

%0 Conference Proceedings %A Zhou, Yuxiao %A Habermann, Marc %A Xu, Weipeng %A Habibie, Ikhsanul %A Theobalt, Christian %A Xu, Feng %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data : %G eng %U http://hdl.handle.net/21.11116/0000-0006-A89E-B %R 10.1109/CVPR42600.2020.00539 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 5345 - 5354 %I IEEE %@ 978-1-7281-7168-5

%0 Report %A Zhou, Yuxiao %A Habermann, Marc %A Xu, Weipeng %A Habibie, Ikhsanul %A Theobalt, Christian %A Xu, Feng %+ External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data : %G eng %U http://hdl.handle.net/21.11116/0000-0007-E0D3-D %U https://arxiv.org/abs/2003.09572 %D 2020 %X We present a novel method for monocular hand shape and pose estimation at<br>unprecedented runtime performance of 100fps and at state-of-the-art accuracy.<br>This is enabled by a new learning based architecture designed such that it can<br>make use of all the sources of available hand training data: image data with<br>either 2D or 3D annotations, as well as stand-alone 3D animations without<br>corresponding image data. It features a 3D hand joint detection module and an<br>inverse kinematics module which regresses not only 3D joint positions but also<br>maps them to joint rotations in a single feed-forward pass. This output makes<br>the method more directly usable for applications in computer vision and<br>graphics compared to only regressing 3D joint positions. We demonstrate that<br>our architectural design leads to a significant quantitative and qualitative<br>improvement over the state of the art on several challenging benchmarks. Our<br>model is publicly available for future research.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV