Publications

2026

Article

M. Bálint, C. Salaün, H.-P. Seidel, and K. Myszkowski

“Forget Superresolution, Sample Adaptively (when Path Tracing),” ACM Transactions on Graphics (Proc. SIGGRAPH 2026), vol. 45, no. 4, 2026.

@article{Balint2602.08642,
TITLE = {Forget Superresolution, Sample Adaptively (when Path Tracing)},
AUTHOR = {B{\'a}lint, Martin and Sala{\"u}n, Corentin and Seidel, Hans-Peter and Myszkowski, Karol},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3811377},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2026},
DATE = {2026},
JOURNAL = {ACM Transactions on Graphics (Proc. SIGGRAPH)},
VOLUME = {45},
NUMBER = {4},
PAGES = {1--19},
EID = {90},
BOOKTITLE = {ACM SIGGRAPH Conference 2026},
}

Endnote

%0 Journal Article
%A B&#225;lint, Martin
%A Sala&#252;n, Corentin
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Forget Superresolution, Sample Adaptively (when Path Tracing) : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-49BB-7
%R 10.1145/3811377
%7 2026-07-03
%D 2026
%J ACM Transactions on Graphics
%V 45
%N 4
%& 1
%P 1 - 19
%Z sequence number: 90
%I ACM
%C New York, NY
%@ false
%B ACM SIGGRAPH Conference 2026
%O SIGGRAPH 2026 Los Angeles, California, 19-23 July 2026

Conference paper

B. Fruchard, D. Wittchen, N. Sabnis, P. Strohmeier, and D. Degraen

“How are Vibrotactile Experiences Visually Represented? A Taxonomy of Illustration Characteristics,” in CHI ’26, CHI Conference on Human Factors in Computing Systems, Barcelona, Spain, 2026.

@inproceedings{Fruchard_CHI26,
TITLE = {How are Vibrotactile Experiences Visually Represented? {A} Taxonomy of Illustration Characteristics},
AUTHOR = {Fruchard, Bruno and Wittchen, Dennis and Sabnis, Nihar and Strohmeier, Paul and Degraen, Donald},
LANGUAGE = {eng},
ISBN = {979-8-4007-2278-3},
DOI = {10.1145/3772318.3790598},
PUBLISHER = {ACM},
YEAR = {2026},
DATE = {2026},
BOOKTITLE = {CHI '26, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Oliver, Nuria and Shamma, A. and Candello, Heloisa and Cesar, Pablo and Lopes, Pedro and Bozzon, Allesandro and Kosch, Thomas and Liao, Vera and Ma, Xiaojuan and Artizzu, Valentino and Draxler, Fiona and L{\'o}pez, Gustavo and Reinschluessel, Anke V. and Tong, Xin and Toups Dugas, Phoebe O.},
PAGES = {1--24},
EID = {1251},
ADDRESS = {Barcelona, Spain},
}

Endnote

%0 Conference Proceedings
%A Fruchard, Bruno
%A Wittchen, Dennis
%A Sabnis, Nihar
%A Strohmeier, Paul
%A Degraen, Donald
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T How are Vibrotactile Experiences Visually Represented? A Taxonomy of Illustration Characteristics : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-EF76-B
%R 10.1145/3772318.3790598
%D 2026
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2026-04-13 - 2026-04-17
%C Barcelona, Spain
%B CHI '26
%E Oliver, Nuria; Shamma, A.; Candello, Heloisa; Cesar, Pablo; Lopes, Pedro; Bozzon, Allesandro; Kosch, Thomas; Liao, Vera; Ma, Xiaojuan; Artizzu, Valentino; Draxler, Fiona; L&#243;pez, Gustavo; Reinschluessel, Anke V.; Tong, Xin; Toups Dugas, Phoebe O.
%P 1 - 24
%Z sequence number: 1251
%I ACM
%@ 979-8-4007-2278-3

Paper

S. Holl, G. Singh, and H.-P. Seidel

“Diffusion Restore: Real-Time Markov Chain Monte Carlo Light Transport,” 2026. [Online]. Available: https://arxiv.org/abs/2605.08916.

Abstract

We present Diffusion Restore, a real-time framework for diffusion-based MCMC light transport. MCMC methods are highly suitable for sampling from complex high-dimensional distributions and for approximating integrals over them. In practice, they are often the only viable solution when direct sampling is not possible and alternative methods are either inefficient or cannot be applied due to the structure of the target distribution. However, controlling the exploration of the target distribution in MCMC methods remains challenging. Efficient exploration requires a balance between local exploration and global discovery, and local dynamics must rapidly explore individual modes without getting stuck or exhibiting excessive backtracking. The problem of global discovery has recently been addressed by the introduction of the Restore framework. In this work, we build on this framework and focus on improving local exploration. We show how to choose diffusion-based local dynamics within the Restore framework while completely avoiding Metropolis-adjustment, which is known to slow down convergence. Furthermore, we model these dynamics as nonreversible, introducing momentum in the drift and thereby enabling more directed exploration of the target distribution compared to reversible, random-walk-like dynamics. We provide a theoretical justification for the validity of our choice of local dynamics. Empirically, we demonstrate across diverse scenes that Diffusion Restore outperforms all existing MCMC light transport methods and establishes a new state of the art. In addition, we present a GPU implementation in ray tracing and compute shaders and achieve real-time frame rates. This demonstrates that Diffusion Restore is not only superior in offline rendering, but also outperforms traditional Path Tracing methods in real-time rendering settings, such as interactive applications and games.

BibTeX

@online{Holl2605.08916,
TITLE = {Diffusion Restore: Real-Time Markov Chain Monte Carlo Light Transport},
AUTHOR = {Holl, Sascha and Singh, Gurprit and Seidel, Hans-Peter},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2605.08916},
EPRINT = {2605.08916},
EPRINTTYPE = {arXiv},
YEAR = {2026},
ABSTRACT = {We present Diffusion Restore, a real-time framework for diffusion-based MCMC light transport. MCMC methods are highly suitable for sampling from complex high-dimensional distributions and for approximating integrals over them. In practice, they are often the only viable solution when direct sampling is not possible and alternative methods are either inefficient or cannot be applied due to the structure of the target distribution. However, controlling the exploration of the target distribution in MCMC methods remains challenging. Efficient exploration requires a balance between local exploration and global discovery, and local dynamics must rapidly explore individual modes without getting stuck or exhibiting excessive backtracking. The problem of global discovery has recently been addressed by the introduction of the Restore framework. In this work, we build on this framework and focus on improving local exploration. We show how to choose diffusion-based local dynamics within the Restore framework while completely avoiding Metropolis-adjustment, which is known to slow down convergence. Furthermore, we model these dynamics as nonreversible, introducing momentum in the drift and thereby enabling more directed exploration of the target distribution compared to reversible, random-walk-like dynamics. We provide a theoretical justification for the validity of our choice of local dynamics. Empirically, we demonstrate across diverse scenes that Diffusion Restore outperforms all existing MCMC light transport methods and establishes a new state of the art. In addition, we present a GPU implementation in ray tracing and compute shaders and achieve real-time frame rates. This demonstrates that Diffusion Restore is not only superior in offline rendering, but also outperforms traditional Path Tracing methods in real-time rendering settings, such as interactive applications and games.},
}

Endnote

%0 Report
%A Holl, Sascha
%A Singh, Gurprit
%A Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Diffusion Restore: Real-Time Markov Chain Monte Carlo Light Transport :
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-49D6-8
%U https://arxiv.org/abs/2605.08916
%D 2026
%X We present Diffusion Restore, a real-time framework for diffusion-based MCMC light transport. MCMC methods are highly suitable for sampling from complex high-dimensional distributions and for approximating integrals over them. In practice, they are often the only viable solution when direct sampling is not possible and alternative methods are either inefficient or cannot be applied due to the structure of the target distribution. However, controlling the exploration of the target distribution in MCMC methods remains challenging. Efficient exploration requires a balance between local exploration and global discovery, and local dynamics must rapidly explore individual modes without getting stuck or exhibiting excessive backtracking. The problem of global discovery has recently been addressed by the introduction of the Restore framework. In this work, we build on this framework and focus on improving local exploration. We show how to choose diffusion-based local dynamics within the Restore framework while completely avoiding Metropolis-adjustment, which is known to slow down convergence. Furthermore, we model these dynamics as nonreversible, introducing momentum in the drift and thereby enabling more directed exploration of the target distribution compared to reversible, random-walk-like dynamics. We provide a theoretical justification for the validity of our choice of local dynamics. Empirically, we demonstrate across diverse scenes that Diffusion Restore outperforms all existing MCMC light transport methods and establishes a new state of the art. In addition, we present a GPU implementation in ray tracing and compute shaders and achieve real-time frame rates. This demonstrates that Diffusion Restore is not only superior in offline rendering, but also outperforms traditional Path Tracing methods in real-time rendering settings, such as interactive applications and games.
%K Computer Science, Computational Engineering, Finance, and Science, cs.CE,Computer Science, Numerical Analysis, cs.NA,Mathematics, Numerical Analysis, math.NA,Mathematics, Probability, math.PR

Paper

S. Holl, J. Vandersanden, G. Singh, and H.-P. Seidel

“Score-Based Generative Modeling through Anisotropic Stochastic Partial Differential Equations,” 2026. [Online]. Available: https://arxiv.org/abs/2605.08976.

Abstract

Score-based generative modeling (SBGM) has achieved state-of-the-art performance in image generation, with the quality of generated images being highly dependent on the design of the forward (diffusion) process. Among these, models based on stochastic differential equations (SDEs) have proven particularly effective. While traditional methods aim to progressively destroy all image information to enable reconstruction from pure noise, we propose a class of anisotropic stochastic partial differential equations (SPDEs) that preserve the geometric structure of the data over longer time scales throughout the transformation. These SPDEs consist of a drift term that enforces deterministic destruction via structured smoothing, and a diffusion coefficient that enables random destruction through noise injection. Both components are governed by anisotropy coefficients, enabling controlled, direction-dependent information degradation. This framework provides the theoretical foundation for a novel anisotropic score-based generative model. By retaining geometric structure for longer time scales, the backward generative process can exploit residual geometric cues, leading to improved reconstruction fidelity. We empirically validate this improvement in a proof-of-concept implementation on unconditional image generation, showing that anisotropic diffusion can achieve superior image quality metrics. We demonstrate consistent improvements in both pixel and latent space experiments over the SDE-driven baseline as well as over the state-of-the-art Flow Matching approach. Finally, we demonstrate the effectiveness of the introduced anisotropy in a conditional stroke-to-image generation task.

BibTeX

@online{Holl2605.08976,
TITLE = {Score-Based Generative Modeling through Anisotropic Stochastic Partial Differential Equations},
AUTHOR = {Holl, Sascha and Vandersanden, Jente and Singh, Gurprit and Seidel, Hans-Peter},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2605.08976},
EPRINT = {2605.08976},
EPRINTTYPE = {arXiv},
YEAR = {2026},
ABSTRACT = {Score-based generative modeling (SBGM) has achieved state-of-the-art performance in image generation, with the quality of generated images being highly dependent on the design of the forward (diffusion) process. Among these, models based on stochastic differential equations (SDEs) have proven particularly effective. While traditional methods aim to progressively destroy all image information to enable reconstruction from pure noise, we propose a class of anisotropic stochastic partial differential equations (SPDEs) that preserve the geometric structure of the data over longer time scales throughout the transformation. These SPDEs consist of a drift term that enforces deterministic destruction via structured smoothing, and a diffusion coefficient that enables random destruction through noise injection. Both components are governed by anisotropy coefficients, enabling controlled, direction-dependent information degradation. This framework provides the theoretical foundation for a novel anisotropic score-based generative model. By retaining geometric structure for longer time scales, the backward generative process can exploit residual geometric cues, leading to improved reconstruction fidelity. We empirically validate this improvement in a proof-of-concept implementation on unconditional image generation, showing that anisotropic diffusion can achieve superior image quality metrics. We demonstrate consistent improvements in both pixel and latent space experiments over the SDE-driven baseline as well as over the state-of-the-art Flow Matching approach. Finally, we demonstrate the effectiveness of the introduced anisotropy in a conditional stroke-to-image generation task.},
}

Endnote

%0 Report
%A Holl, Sascha
%A Vandersanden, Jente
%A Singh, Gurprit
%A Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Score-Based Generative Modeling through Anisotropic Stochastic Partial Differential Equations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-49D9-5
%U https://arxiv.org/abs/2605.08976
%D 2026
%X Score-based generative modeling (SBGM) has achieved state-of-the-art performance in image generation, with the quality of generated images being highly dependent on the design of the forward (diffusion) process. Among these, models based on stochastic differential equations (SDEs) have proven particularly effective. While traditional methods aim to progressively destroy all image information to enable reconstruction from pure noise, we propose a class of anisotropic stochastic partial differential equations (SPDEs) that preserve the geometric structure of the data over longer time scales throughout the transformation. These SPDEs consist of a drift term that enforces deterministic destruction via structured smoothing, and a diffusion coefficient that enables random destruction through noise injection. Both components are governed by anisotropy coefficients, enabling controlled, direction-dependent information degradation. This framework provides the theoretical foundation for a novel anisotropic score-based generative model. By retaining geometric structure for longer time scales, the backward generative process can exploit residual geometric cues, leading to improved reconstruction fidelity. We empirically validate this improvement in a proof-of-concept implementation on unconditional image generation, showing that anisotropic diffusion can achieve superior image quality metrics. We demonstrate consistent improvements in both pixel and latent space experiments over the SDE-driven baseline as well as over the state-of-the-art Flow Matching approach. Finally, we demonstrate the effectiveness of the introduced anisotropy in a conditional stroke-to-image generation task.
%K Computer Science, Computational Engineering, Finance, and Science, cs.CE

Article

D. Jiménez Navarro, C. Groth, X. Peng, J. Pina, Q. Sun, and P. Chakravarthula

“Overdriving Visual Depth Perception via Sound Modulation in VR,” IEEE Transactions on Visualization and Computer Graphics, vol. 32, no. 5, 2026.

@article{Jimenez-Navarro26,
TITLE = {Overdriving Visual Depth Perception via Sound Modulation in {VR}},
AUTHOR = {Jim{\'e}nez Navarro, Daniel and Groth, Colin and Peng, Xi and Pina, Jorge and Sun, Qi and Chakravarthula, Praneeth},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2026.3679918},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2026},
DATE = {2026},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics},
VOLUME = {32},
NUMBER = {5},
PAGES = {3422--3432},
}

Endnote

%0 Journal Article
%A Jim&#233;nez Navarro, Daniel
%A Groth, Colin
%A Peng, Xi
%A Pina, Jorge
%A Sun, Qi
%A Chakravarthula, Praneeth
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Overdriving Visual Depth Perception via Sound Modulation in VR : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-1F97-F
%R 10.1109/TVCG.2026.3679918
%7 2026-04-02
%D 2026
%J IEEE Transactions on Visualization and Computer Graphics
%V 32
%N 5
%& 3422
%P 3422 - 3432
%I IEEE
%C Piscataway, NJ
%@ false

Conference paper

A. Jingu, E. AliAbbasi, S. Safae, P. Strohmeier, and J. Steimle

“Scene2Hap: Generating Scene-Wide Haptics for VR from Scene Context with Multimodal LLMs,” in CHI ’26, CHI Conference on Human Factors in Computing Systems, Barcelona, Spain, 2026.

@inproceedings{Jingu_CHI26,
TITLE = {{Scene2Hap}: {G}enerating Scene-Wide Haptics for {VR} from Scene Context with Multimodal {LLMs}},
AUTHOR = {Jingu, Arata and AliAbbasi, Easa and Safae, Sara and Strohmeier, Paul and Steimle, J{\"u}rgen},
LANGUAGE = {eng},
ISBN = {979-8-4007-2278-3},
DOI = {10.1145/3772318.3791297},
PUBLISHER = {ACM},
YEAR = {2026},
DATE = {2026},
BOOKTITLE = {CHI '26, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Oliver, Nuria and Shamma, A. and Candello, Heloisa and Cesar, Pablo and Lopes, Pedro and Bozzon, Allesandro and Kosch, Thomas and Liao, Vera and Ma, Xiaojuan and Artizzu, Valentino and Draxler, Fiona and L{\'o}pez, Gustavo and Reinschluessel, Anke V. and Tong, Xin and Toups Dugas, Phoebe O.},
PAGES = {1--21},
EID = {745},
ADDRESS = {Barcelona, Spain},
}

Endnote

%0 Conference Proceedings
%A Jingu, Arata
%A AliAbbasi, Easa
%A Safae, Sara
%A Strohmeier, Paul
%A Steimle, J&#252;rgen
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Scene2Hap: Generating Scene-Wide Haptics for VR from Scene Context with Multimodal LLMs : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-4D77-2
%R 10.1145/3772318.3791297
%D 2026
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2026-04-13 - 2026-04-17
%C Barcelona, Spain
%K Computer Science, Human-Computer Interaction, cs.HC
%B CHI '26
%E Oliver, Nuria; Shamma, A.; Candello, Heloisa; Cesar, Pablo; Lopes, Pedro; Bozzon, Allesandro; Kosch, Thomas; Liao, Vera; Ma, Xiaojuan; Artizzu, Valentino; Draxler, Fiona; L&#243;pez, Gustavo; Reinschluessel, Anke V.; Tong, Xin; Toups Dugas, Phoebe O.
%P 1 - 21
%Z sequence number: 745
%I ACM
%@ 979-8-4007-2278-3

Thesis

N. Sabnis

“Re-envisioning Touch: Designing Embodied material and Force Experiences with Motion-coupled Vibrotactile Feedback,” Universität des Saarlandes, Saarbrücken, 2026.

@phdthesis{ThesisPhDSabnis26,
TITLE = {{Re-envisioning} touch: {Designing} embodied material and force experiences with motion-coupled vibrotactile feedback},
AUTHOR = {Sabnis, Nihar},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-473346},
DOI = {10.22028/D291-47334},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2026},
DATE = {2026},
}

Endnote

%0 Thesis
%A Sabnis, Nihar
%Y Strohmeier, Paul
%Y Seidel, Hans-Peter
%A referee: Jimenez, Ana-Tajdura
%A referee: Withana, Anusha
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Re-envisioning Touch: Designing Embodied material and Force Experiences with Motion-coupled Vibrotactile Feedback : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-1905-A
%R 10.22028/D291-47334
%U urn:nbn:de:bsz:291--ds-473346
%F OTHER: hdl:20.500.11880/41500
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2026
%P XXIII, 252 p.
%V phd
%9 phd
%U http://dx.doi.org/10.22028/D291-47334

Article

J. Vandersanden, S. Holl, X. Huang, and G. Singh

“Edge-preserving Noise for Diffusion Models,” Compuer Graphics Forum, 2026.

@article{Ponton_EG26,
TITLE = {Edge-preserving Noise for Diffusion Models},
AUTHOR = {Vandersanden, Jente and Holl, Sascha and Huang, Xingchang and Singh, Gurprit},
LANGUAGE = {eng},
ISSN = {1467-8659},
DOI = {10.1111/cgf.70405},
PUBLISHER = {Eurographics Association},
YEAR = {2026},
JOURNAL = {Compuer Graphics Forum},
EID = {e70383},
BOOKTITLE = {Eurographics 2026},
EDITOR = {Masia, B. and Thies, J.},
}

Endnote

%0 Journal Article
%A Vandersanden, Jente
%A Holl, Sascha
%A Huang, Xingchang
%A Singh, Gurprit
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Edge-preserving Noise for Diffusion Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-13C9-3
%R 10.1111/cgf.70405
%7 2026-04-30
%D 2026
%8 30.04.2026
%J Compuer Graphics Forum
%Z sequence number: e70383
%I Eurographics Association
%@ false
%B Eurographics 2026
%O EG 2026

2025

Article

J. Ahmad, E. AliAbbasi

Mr. A. Sormoli, and C. Basdogan, “Effect of Finger Orientation on Contact Stiffness and Area During Sliding,” IEEE Transactions on Haptics, vol. 18, no. 1, 2025.

@article{Ahmad2025,
TITLE = {Effect of Finger Orientation on Contact Stiffness and Area During Sliding},
AUTHOR = {Ahmad, Jahangier and AliAbbasi, Easa and Sormoli, MReza Alipour and Basdogan, Cagatay},
LANGUAGE = {eng},
ISSN = {2329-4051},
DOI = {10.1109/TOH.2024.3509219},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {IEEE Transactions on Haptics},
VOLUME = {18},
NUMBER = {1},
PAGES = {175--187},
}

Endnote

%0 Journal Article
%A Ahmad, Jahangier
%A AliAbbasi, Easa
%A Sormoli, MReza Alipour
%A Basdogan, Cagatay
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Effect of Finger Orientation on Contact Stiffness and Area During Sliding : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D3AB-F
%R 10.1109/TOH.2024.3509219
%7 2024-12-05
%D 2025
%J IEEE Transactions on Haptics
%V 18
%N 1
%& 175
%P 175 - 187
%I IEEE
%C Piscataway, NJ
%@ false

Conference paper

E. AliAbbasi, G. Vega, D. Wittchen, and P. Strohmeier

“Physical Compliance and the Compliance Illusion: The Importance of Action for Perception,” in IEEE World Haptics Conference 2025 (WHC 2025), Suwon, Korea, 2025.

@inproceedings{AliAbbasiWHC25,
TITLE = {Physical Compliance and the Compliance Illusion: {The} Importance of Action for Perception},
AUTHOR = {AliAbbasi, Easa and Vega, Gabriela and Wittchen, Dennis and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-3315-3353-3},
DOI = {10.1109/WHC64065.2025.11123193},
PUBLISHER = {IEEE},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {IEEE World Haptics Conference 2025 (WHC 2025)},
PAGES = {349--356},
ADDRESS = {Suwon, Korea},
}

Endnote

%0 Conference Proceedings
%A AliAbbasi, Easa
%A Vega, Gabriela
%A Wittchen, Dennis
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Physical Compliance and the Compliance Illusion: The Importance of Action for Perception : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-5C77-0
%R 10.1109/WHC64065.2025.11123193
%D 2025
%B IEEE World Haptics Conference
%Z date of event: 2025-07-08 - 2025-07-11
%C Suwon, Korea
%B IEEE World Haptics Conference 2025
%P 349 - 356
%I IEEE
%@ 979-8-3315-3353-3

Conference paper

E. AliAbbasi, N. Sabnis, Y. Ding, N. Wagener, and P. Strohmeier

“Haptic Redirection: Modulating Hand Movement Speed with Vibrotactile Feedback,” in MuC ’25, Conference on Mensch und Computer, Chemnitz, Germany, 2025.

@inproceedings{AliAbbasiMuC25,
TITLE = {Haptic Redirection: {M}odulating Hand Movement Speed with Vibrotactile Feedback},
AUTHOR = {AliAbbasi, Easa and Sabnis, Nihar and Ding, Yuran and Wagener, Nadine and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-1582-2},
DOI = {10.1145/3743049.37485},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {MuC '25, Conference on Mensch und Computer},
EDITOR = {Chuang, Lewis and Eibl, Maximilian and Gaedke, Martin and Niess, Jasmin},
PAGES = {631--636},
ADDRESS = {Chemnitz, Germany},
}

Endnote

%0 Conference Proceedings
%A AliAbbasi, Easa
%A Sabnis, Nihar
%A Ding, Yuran
%A Wagener, Nadine
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Haptic Redirection: Modulating Hand Movement Speed with Vibrotactile Feedback : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-0A06-B
%R 10.1145/3743049.37485
%D 2025
%B Conference on Mensch und Computer
%Z date of event: 2025-08-31 - 2025-09-03
%C Chemnitz, Germany
%B MuC '25
%E Chuang, Lewis; Eibl, Maximilian; Gaedke, Martin; Niess, Jasmin
%P 631 - 636
%I ACM
%@ 979-8-4007-1582-2

Conference paper

N. Ansari, H.-P. Seidel, and V. Babaei

“Uncertainty Separation via Ensemble Quantile Regression,” in 4th Annual AAAI Workshop on AI to Accelerate Science and Engineering (AI2ASE 2025), Philadelphia, PA, USA, 2025.

@inproceedings{Ansari_AAAI25,
TITLE = {Uncertainty Separation via Ensemble Quantile Regression},
AUTHOR = {Ansari, Navid and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
PUBLISHER = {AAAI},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {4th Annual AAAI Workshop on AI to Accelerate Science and Engineering (AI2ASE 2025)},
ADDRESS = {Philadelphia, PA, USA},
}

Endnote

%0 Conference Proceedings
%A Ansari, Navid
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Uncertainty Separation via Ensemble Quantile Regression : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-C013-F
%D 2025
%B 4th Annual AAAI Workshop on AI to Accelerate Science and Engineering 
%Z date of event: 2025-03-03 - 2025-03-03
%C Philadelphia, PA, USA
%B 4th Annual AAAI Workshop on AI to Accelerate Science and Engineering 
%I AAAI
%U https://ai-2-ase.github.io/papers/53_Uncertainty_separation_via_ensemble_quantile_regression_workshop_AAAI_2025.pdf

Conference paper

N. Ansari, H.-P. Seidel, and V. Babaei

“Accelerated Gamut Discovery via Massive Parallelization,” in SIGGRAPH 2025 Conference Papers, Vancouver, Canada, 2025.

@inproceedings{Ansari_SIGGRAPH25,
TITLE = {Accelerated Gamut Discovery via Massive Parallelization},
AUTHOR = {Ansari, Navid and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
ISBN = {979-8-4007-1540-2},
DOI = {10.1145/3721238.3730712},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {SIGGRAPH 2025 Conference Papers},
EDITOR = {Alford, Ginger and Zhang, Hao Richard and Schulz, Adriana},
PAGES = {1--10},
EID = {110},
ADDRESS = {Vancouver, Canada},
}

Endnote

%0 Conference Proceedings
%A Ansari, Navid
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Accelerated Gamut Discovery via Massive Parallelization : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-6E8F-1
%R 10.1145/3721238.3730712
%D 2025
%B ACM SIGGRAPH Conference
%Z date of event: 2025-08-10 - 2025-08-14
%C Vancouver, Canada
%B SIGGRAPH 2025 Conference Papers
%E Alford, Ginger; Zhang, Hao Richard; Schulz, Adriana
%P 1 - 10
%Z sequence number: 110
%I ACM
%@ 979-8-4007-1540-2

Conference paper

L. Badr and T. Juldo

“ModalPlayground: Collaborative Spatial Design of Multi-modal Haptic Feedback,” in UIST ’25, 38th Annual ACM Symposium on User Interface Software and Technology, Busan, Korea, 2025.

@inproceedings{Badr_UIST25,
TITLE = {{ModalPlayground}: {C}ollaborative Spatial Design of Multi-modal Haptic Feedback},
AUTHOR = {Badr, Louis and Juldo, Thomas},
LANGUAGE = {eng},
ISBN = {979-8-4007-2036-9},
DOI = {10.1145/3746058.3758977},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {UIST '25, 38th Annual ACM Symposium on User Interface Software and Technology},
EDITOR = {Bianchi, Andrea and Glassman, Elena and Mackay, Wendy E. and Zhao, Shengdong and Kim, Jeeeun and Oakley, Ian},
PAGES = {1--3},
EID = {62},
ADDRESS = {Busan, Korea},
}

Endnote

%0 Conference Proceedings
%A Badr, Louis
%A Juldo, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T ModalPlayground: Collaborative Spatial Design of Multi-modal Haptic Feedback : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-6E9F-F
%R 10.1145/3746058.3758977
%D 2025
%B 38th Annual ACM Symposium on User Interface Software and Technology
%Z date of event: 2025-09-28 - 2025-10-01
%C Busan, Korea 
%B UIST '25
%E Bianchi, Andrea; Glassman, Elena; Mackay, Wendy E.; Zhao, Shengdong; Kim, Jeeeun; Oakley, Ian
%P 1 - 3
%Z sequence number: 62
%I ACM
%@ 979-8-4007-2036-9

Article

M. Bemana, T. Leimkühler, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“Bracket Diffusion: HDR Image Generation by Consistent LDR Denoising,” Computer Graphics Forum (Proc. EUROGRAPHICS 2025), vol. 44, no. 2, 2025.

@article{mbemana2025bracket,
TITLE = {Bracket Diffusion: {HDR} Image Generation by Consistent {LDR} Denoising},
AUTHOR = {Bemana, Mojtaba and Leimk{\"u}hler, Thomas and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.70086},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {44},
NUMBER = {2},
PAGES = {1--13},
EID = {e70086},
BOOKTITLE = {EUROGRAPHICS 2025},
EDITOR = {Dai, Angela and Bousseau, Adrien},
}

Endnote

%0 Journal Article
%A Bemana, Mojtaba
%A Leimk&#252;hler, Thomas
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Bracket Diffusion: HDR Image Generation by Consistent LDR 
Denoising
 : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-E93C-5
%R 10.1111/cgf.70086
%7 2025-04-16
%D 2025
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 44
%N 2
%& 1
%P 1 - 13
%Z sequence number: e70086
%I Blackwell-Wiley
%C Oxford
%@ false
%B EUROGRAPHICS 2025
%O EUROGRAPHICS 2025 EG 2025 46th Annual Conference of the European Association for Computer Graphics ; 12th&#8211;16th May 2025 in London

Article

U. Çoğalan, M. Bemana, K. Myszkowski, H.-P. Seidel, and C. Groth

“MILO: A Lightweight Perceptual Quality Metric for Image and Latent-Space Optimization,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2025), vol. 44, no. 6, 2025.

@article{CogalanSIGGRAPHAsia25,
TITLE = {{MILO}: A Lightweight Perceptual Quality Metric for Image and Latent-Space Optimization},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur and Bemana, Mojtaba and Myszkowski, Karol and Seidel, Hans-Peter and Groth, Colin},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3763340},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {44},
NUMBER = {6},
PAGES = {1--11},
EID = {214},
BOOKTITLE = {SIGGRAPH Asia 2025 (ACM SIGGRAPH Asia 2025)},
}

Endnote

%0 Journal Article
%A &#199;o&#287;alan, U&#287;ur
%A Bemana, Mojtaba
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Groth, Colin
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T MILO: A Lightweight Perceptual Quality Metric for Image and Latent-Space Optimization : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-131E-6
%R 10.1145/3763340
%7 2025
%D 2025
%J ACM Transactions on Graphics
%V 44
%N 6
%& 1
%P 1 - 11
%Z sequence number: 214
%I ACM
%C New York, NY
%@ false
%B SIGGRAPH Asia 2025
%O ACM SIGGRAPH Asia 2025

Conference paper

D. Dietz, S. B. Rogers, J. Rasch, S. Sakel, N. Wagener, A. M. Butz, and M. Hoppe

“The 2x2 of Being Me and You: How the Combination of Self and Other Avatars and Movements Alters How We Reflect on Ourselves in VR,” in VRST ’25, 31st ACM Symposium on Virtual Reality Software and Technology, Montreal, Canada, 2025.

Abstract

Effective self-reflection is crucial for motor skill acquisition, yet it is challenging to facilitate in single-user VR training environments. We investigate this through a method where users are embodied as a virtual trainer and prompted to actively evaluate a recorded performance. In an empirical study, we systematically varied the trainee’s appearance and their movements. Our mixed-methods analysis reveals that confronting one’s own performance triggers a fundamental role conflict between the user’s identity as the performer and their new role as the evaluator. Most importantly, this conflict challenges a binary view of embodiment. Participants experienced a multi-faceted sense of self, oscillating between identifying with the trainee and detaching as the trainer. Our work contributes a novel characterization of embodied self-evaluation, revealing a psychological duality at its core and offering clear design implications for VR systems that foster self-insight in training and therapy.

BibTeX

@inproceedings{Dietz_VRST25,
TITLE = {The 2x2 of Being Me and You: {H}ow the Combination of Self and Other Avatars and Movements Alters How We Reflect on Ourselves in {VR}},
AUTHOR = {Dietz, Dennis and Rogers, Samuel Benjamin and Rasch, Julian and Sakel, Sophia and Wagener, Nadine and Butz, Andreas Martin and Hoppe, Matthias},
LANGUAGE = {eng},
ISBN = {979-8-4007-2118-2},
DOI = {10.1145/3756884.3765986},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
ABSTRACT = {Effective self-reflection is crucial for motor skill acquisition, yet it is challenging to facilitate in single-user VR training environments. We investigate this through a method where users are embodied as a virtual trainer and prompted to actively evaluate a recorded performance. In an empirical study, we systematically varied the trainee{\textquoteright}s appearance and their movements. Our mixed-methods analysis reveals that confronting one{\textquoteright}s own performance triggers a fundamental role conflict between the user{\textquoteright}s identity as the performer and their new role as the evaluator. Most importantly, this conflict challenges a binary view of embodiment. Participants experienced a multi-faceted sense of self, oscillating between identifying with the trainee and detaching as the trainer. Our work contributes a novel characterization of embodied self-evaluation, revealing a psychological duality at its core and offering clear design implications for VR systems that foster self-insight in training and therapy.},
BOOKTITLE = {VRST '25, 31st ACM Symposium on Virtual Reality Software and Technology},
EDITOR = {Batmaz, Anil and Barrera Machuca, Mayra and Ortega, Francisco and Zielasko, Daniel and Kim, Kangsoo and Skarbez, Rick and Plabst, Lucas and Bani{\'c}, Amy and Stroppa, Mine},
PAGES = {1--11},
EID = {72},
ADDRESS = {Montreal, Canada},
}

Endnote

%0 Conference Proceedings
%A Dietz, Dennis
%A Rogers, Samuel Benjamin
%A Rasch, Julian
%A Sakel, Sophia
%A Wagener, Nadine
%A Butz, Andreas Martin
%A Hoppe, Matthias
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T The 2x2 of Being Me and You: How the Combination of Self and Other Avatars and Movements Alters How We Reflect on Ourselves in VR : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-AD8D-B
%R 10.1145/3756884.3765986
%D 2025
%B 31st ACM Symposium on Virtual Reality Software and Technology
%Z date of event: 2025-11-12 - 2025-11-14
%C Montreal, Canada
%X Effective self-reflection is crucial for motor skill acquisition, yet it is challenging to facilitate in single-user VR training environments. We investigate this through a method where users are embodied as a virtual trainer and prompted to actively evaluate a recorded performance. In an empirical study, we systematically varied the trainee&#8217;s appearance and their movements. Our mixed-methods analysis reveals that confronting one&#8217;s own performance triggers a fundamental role conflict between the user&#8217;s identity as the performer and their new role as the evaluator. Most importantly, this conflict challenges a binary view of embodiment. Participants experienced a multi-faceted sense of self, oscillating between identifying with the trainee and detaching as the trainer. Our work contributes a novel characterization of embodied self-evaluation, revealing a psychological duality at its core and offering clear design implications for VR systems that foster self-insight in training and therapy.
%K Virtual Reality, Motor Skill Acquisition, Embodiment, Self-Confrontation, Role Conflict
%B VRST '25
%E Batmaz, Anil; Barrera Machuca, Mayra; Ortega, Francisco; Zielasko, Daniel; Kim, Kangsoo; Skarbez, Rick; Plabst, Lucas; Bani&#263;, Amy; Stroppa, Mine
%P 1 - 11
%Z sequence number: 72
%I ACM
%@ 979-8-4007-2118-2

Article

A. Djeacoumar, F. Mujkanovic, H.-P. Seidel, and T. Leimkühler

“Learning Image Fractals Using Chaotic Differentiable Point Splatting,” Computer Graphics Forum (Proc. EUROGRAPHICS 2025), vol. 44, no. 2, 2025.

@article{DjeacoumarEG25,
TITLE = {Learning Image Fractals Using Chaotic Differentiable Point Splatting},
AUTHOR = {Djeacoumar, Adarsh and Mujkanovic, Felix and Seidel, Hans-Peter and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.70084},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {44},
NUMBER = {2},
EID = {e70084},
BOOKTITLE = {EUROGRAPHICS 2025},
}

Endnote

%0 Journal Article
%A Djeacoumar, Adarsh
%A Mujkanovic, Felix
%A Seidel, Hans-Peter
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Learning Image Fractals Using Chaotic Differentiable Point Splatting : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-E94A-5
%R 10.1111/cgf.70084
%7 2025-04-18
%D 2025
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 44
%N 2
%Z sequence number: e70084
%I Blackwell-Wiley
%C Oxford
%@ false
%B EUROGRAPHICS 2025
%O EUROGRAPHICS 2025 EG 2025 46th Annual Conference of the European Association for Computer Graphics ; 12th&#8211;16th May 2025 in London

Article

S. Holl, H.-P. Seidel, and G. Singh

“Jump Restore Light Transport,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2025), vol. 44, no. 6, 2025.

@article{HollSIGGRAPHAsia25,
TITLE = {Jump Restore Light Transport},
AUTHOR = {Holl, Sascha and Seidel, Hans-Peter and Singh, Gurprit},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3763286},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {44},
NUMBER = {6},
PAGES = {1--13},
EID = {251},
BOOKTITLE = {SIGGRAPH Asia 2025 (ACM SIGGRAPH Asia 2025)},
}

Endnote

%0 Journal Article
%A Holl, Sascha
%A Seidel, Hans-Peter
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Jump Restore Light Transport : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-81EC-2
%R 10.1145/3763286
%7 2025-12-04
%D 2025
%J ACM Transactions on Graphics
%V 44
%N 6
%& 1
%P 1 - 13
%Z sequence number: 251
%I ACM
%C New York, NY
%@ false
%B SIGGRAPH Asia 2025
%O ACM SIGGRAPH Asia 2025

Paper

D4D6

X. Huang, A. K. Singh, F. Dubost, C. N. Vasconcelos, S. Khattar, L. Shi, C. Theobalt, C. Oztireli, and G. Singh

“Restereo: Diffusion Stereo Video Generation and Restoration,” 2025. [Online]. Available: https://arxiv.org/abs/2506.06023.

Abstract

Stereo video generation has been gaining increasing attention with recent advancements in video diffusion models. However, most existing methods focus on generating 3D stereoscopic videos from monocular 2D videos. These approaches typically assume that the input monocular video is of high quality, making the task primarily about inpainting occluded regions in the warped video while preserving disoccluded areas. In this paper, we introduce a new pipeline that not only generates stereo videos but also enhances both left-view and right-view videos consistently with a single model. Our approach achieves this by fine-tuning the model on degraded data for restoration, as well as conditioning the model on warped masks for consistent stereo generation. As a result, our method can be fine-tuned on a relatively small synthetic stereo video datasets and applied to low-quality real-world videos, performing both stereo video generation and restoration. Experiments demonstrate that our method outperforms existing approaches both qualitatively and quantitatively in stereo video generation from low-resolution inputs.

BibTeX

@online{Huang2506.06023,
TITLE = {Restereo: Diffusion Stereo Video Generation and Restoration},
AUTHOR = {Huang, Xingchang and Singh, Ashish Kumar and Dubost, Florian and Vasconcelos, Cristina Nader and Khattar, Sakar and Shi, Liang and Theobalt, Christian and Oztireli, Cengiz and Singh, Gurprit},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2506.06023},
EPRINT = {2506.06023},
EPRINTTYPE = {arXiv},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Stereo video generation has been gaining increasing attention with recent advancements in video diffusion models. However, most existing methods focus on generating 3D stereoscopic videos from monocular 2D videos. These approaches typically assume that the input monocular video is of high quality, making the task primarily about inpainting occluded regions in the warped video while preserving disoccluded areas. In this paper, we introduce a new pipeline that not only generates stereo videos but also enhances both left-view and right-view videos consistently with a single model. Our approach achieves this by fine-tuning the model on degraded data for restoration, as well as conditioning the model on warped masks for consistent stereo generation. As a result, our method can be fine-tuned on a relatively small synthetic stereo video datasets and applied to low-quality real-world videos, performing both stereo video generation and restoration. Experiments demonstrate that our method outperforms existing approaches both qualitatively and quantitatively in stereo video generation from low-resolution inputs.},
}

Endnote

%0 Report
%A Huang, Xingchang
%A Singh, Ashish Kumar
%A Dubost, Florian
%A Vasconcelos, Cristina Nader
%A Khattar, Sakar
%A Shi, Liang
%A Theobalt, Christian
%A Oztireli, Cengiz
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Restereo: Diffusion Stereo Video Generation and Restoration : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-8D11-B
%U https://arxiv.org/abs/2506.06023
%D 2025
%X Stereo video generation has been gaining increasing attention with recent advancements in video diffusion models. However, most existing methods focus on generating 3D stereoscopic videos from monocular 2D videos. These approaches typically assume that the input monocular video is of high quality, making the task primarily about inpainting occluded regions in the warped video while preserving disoccluded areas. In this paper, we introduce a new pipeline that not only generates stereo videos but also enhances both left-view and right-view videos consistently with a single model. Our approach achieves this by fine-tuning the model on degraded data for restoration, as well as conditioning the model on warped masks for consistent stereo generation. As a result, our method can be fine-tuned on a relatively small synthetic stereo video datasets and applied to low-quality real-world videos, performing both stereo video generation and restoration. Experiments demonstrate that our method outperforms existing approaches both qualitatively and quantitatively in stereo video generation from low-resolution inputs.
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

C. Jewitt, S. Price, L. Seminara, N. Sabnis, J. Steimle, N. Bianchi-Berthouze, K. Minamizawa, D. Forster, and G. J. Otsuki

“Digital Communication Moving Beyond Human-centric Replication,” in CHI ’25, CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 2025.

@inproceedings{jewitt2025beyond,
TITLE = {Digital Communication Moving Beyond Human-centric Replication},
AUTHOR = {Jewitt, Carey and Price, Sara and Seminara, Lucia and Sabnis, Nihar and Steimle, J{\"u}rgen and Bianchi-Berthouze, Nadia and Minamizawa, Kouta and Forster, Desiree and Otsuki, Grant Jun},
LANGUAGE = {eng},
ISBN = {979-8-4007-1395-8},
DOI = {10.1145/3706599.3707415},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {CHI '25, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Yamashita, Naomi and Evers, Vanessa and Ding, Xianghua Sharon},
PAGES = {1--5},
EID = {776},
ADDRESS = {Yokohama, Japan},
}

Endnote

%0 Conference Proceedings
%A Jewitt, Carey
%A Price, Sara
%A Seminara, Lucia
%A Sabnis, Nihar
%A Steimle, J&#252;rgen
%A Bianchi-Berthouze, Nadia
%A Minamizawa, Kouta
%A Forster, Desiree
%A Otsuki, Grant Jun
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Digital Communication Moving Beyond Human-centric Replication : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D379-8
%R 10.1145/3706599.3707415
%D 2025
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2025-04-26 - 2025-05-01
%C Yokohama, Japan
%B CHI '25
%E Yamashita, Naomi; Evers, Vanessa; Ding, Xianghua Sharon
%P 1 - 5
%Z sequence number: 776
%I ACM
%@ 979-8-4007-1395-8

Article

D. Jiménez Navarro, A. Serrano, and S. Malpica

“Minimally Disruptive Auditory Cues: Their Impact on Visual Performance in Virtual Reality,” The Visual Computer, vol. 41, 2025.

@article{JimenezNavarro24,
TITLE = {Minimally Disruptive Auditory Cues: Their Impact on Visual Performance in Virtual Reality},
AUTHOR = {Jim{\'e}nez Navarro, Daniel and Serrano, Ana and Malpica, Sandra},
LANGUAGE = {eng},
ISSN = {0178-2789},
DOI = {10.1007/s00371-024-03707-6},
PUBLISHER = {Springer International},
ADDRESS = {Berlin},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {The Visual Computer},
VOLUME = {41},
PAGES = {5059--5073},
}

Endnote

%0 Journal Article
%A Jim&#233;nez Navarro, Daniel
%A Serrano, Ana
%A Malpica, Sandra
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Minimally Disruptive Auditory Cues: Their Impact on Visual Performance in Virtual Reality : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-6BE3-6
%R 10.1007/s00371-024-03707-6
%7 2024-12-28
%D 2025
%J The Visual Computer
%V 41
%& 5059
%P 5059 - 5073
%I Springer International
%C Berlin
%@ false

Article

J. Kudnick, M. Weier, C. Groth, B. Fu, and R. Horst

“WarpVision: Using Spatial Curvature to Guide Attention in Virtual Reality,” IEEE Transactions on Visualization and Computer Graphics, vol. 31, no. 11, 2025.

@article{Kudnick25,
TITLE = {{WarpVision}: {U}sing Spatial Curvature to Guide Attention in Virtual Reality},
AUTHOR = {Kudnick, J{\'e}r{\^o}me and Weier, Martin and Groth, Colin and Fu, Biying and Horst, Robin},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2025.3616806},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics},
VOLUME = {31},
NUMBER = {11},
PAGES = {9741--9749},
}

Endnote

%0 Journal Article
%A Kudnick, J&#233;r&#244;me
%A Weier, Martin
%A Groth, Colin
%A Fu, Biying
%A Horst, Robin
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T WarpVision: Using Spatial Curvature to Guide Attention in Virtual Reality : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-29D5-E
%R 10.1109/TVCG.2025.3616806
%7 2025-10-02
%D 2025
%J IEEE Transactions on Visualization and Computer Graphics
%V 31
%N 11
%& 9741
%P 9741 - 9749
%I IEEE
%C Piscataway, NJ
%@ false

Article

E. Luci, K. T. Wijaya, and V. Babaei

“Differentiable Search Based Halftoning,” Computer Graphics Forum (Proc. Eurographics Symposium on Rendering 2025), vol. 44, no. 4, 2025.

@article{LuciRendering25,
TITLE = {Differentiable Search Based Halftoning},
AUTHOR = {Luci, Emiliano and Wijaya, Kevin Tirta and Babaei, Vahid},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.70173},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Computer Graphics Forum (Proc. Eurographics Symposium on Rendering)},
VOLUME = {44},
NUMBER = {4},
EID = {e70173},
BOOKTITLE = {Eurographics Symposium on Rendering 2025},
EDITOR = {Wang, Beibei and Wilkie, Alexander},
}

Endnote

%0 Journal Article
%A Luci, Emiliano
%A Wijaya, Kevin Tirta
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Differentiable Search Based Halftoning : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-951F-3
%R 10.1111/cgf.70173
%7 2025
%D 2025
%J Computer Graphics Forum
%V 44
%N 4
%Z sequence number: e70173
%I Wiley-Blackwell
%C Oxford
%@ false
%B Eurographics Symposium on Rendering 2025
%O Eurographics Symposium on Rendering 2025 EGSR 2025 Copenhagen, Denmark, 25 &#8211; 27 June 2025

Article

H. Lu, Y. Belhe, G. Singh, T.-M. Li, and T. Hachisuka

“Gaussian Integral Linear Operators for Precomputed Graphics,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2025), vol. 44, no. 6, 2025.

@article{LuSIGGRAPHAsia25,
TITLE = {Gaussian Integral Linear Operators for Precomputed Graphics},
AUTHOR = {Lu, Haolin and Belhe, Yash and Singh, Gurprit and Li, Tzu-Mao and Hachisuka, Toshiya},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3763321},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {44},
NUMBER = {6},
PAGES = {1--15},
EID = {252},
BOOKTITLE = {SIGGRAPH Asia 2025 (ACM SIGGRAPH Asia 2025)},
}

Endnote

%0 Journal Article
%A Lu, Haolin
%A Belhe, Yash
%A Singh, Gurprit
%A Li, Tzu-Mao
%A Hachisuka, Toshiya
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Gaussian Integral Linear Operators for Precomputed Graphics : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-5D1D-5
%R 10.1145/3763321
%7 2025-12-04
%D 2025
%J ACM Transactions on Graphics
%V 44
%N 6
%& 1
%P 1 - 15
%Z sequence number: 252
%I ACM
%C New York, NY
%@ false
%B SIGGRAPH Asia 2025
%O ACM SIGGRAPH Asia 2025

Article

H. Lu, D. Vicini, W. Chang, and T.-M. Li

“Vector-Valued Monte Carlo Integration Using Ratio Control Variates,” ACM Transactions on Graphics (Proc. SIGGRAPH 2025), vol. 44, no. 4, 2025.

@article{Lu_SIGGRAPH25,
TITLE = {Vector-Valued {Monte Carlo} Integration Using Ratio Control Variates},
AUTHOR = {Lu, Haolin and Vicini, Delio and Chang, Wesley and Li, Tzu-Mao},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3731175},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {ACM Transactions on Graphics (Proc. SIGGRAPH)},
VOLUME = {44},
NUMBER = {4},
PAGES = {1--16},
EID = {100},
BOOKTITLE = {ACM SIGGRAPH Conference},
}

Endnote

%0 Journal Article
%A Lu, Haolin
%A Vicini, Delio
%A Chang, Wesley
%A Li, Tzu-Mao
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T Vector-Valued Monte Carlo Integration Using Ratio Control Variates  : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-9EE7-7
%R 10.1145/3731175
%7 2025-07-27
%D 2025
%J ACM Transactions on Graphics
%V 44
%N 4
%& 1
%P 1 - 16
%Z sequence number: 100
%I ACM
%C New York, NY
%@ false
%B ACM SIGGRAPH Conference
%O SIGGRAPH 2025 Vancouver, Canada, August 10-14, 2025

Article

F. Nassajian Mojarrad

“A New Approximation Method for Solving Stochastic Differential Equations,” Iranian Journal of Science, vol. 49, 2025.

@article{Nassajian25,
TITLE = {A New Approximation Method for Solving Stochastic Differential Equations},
AUTHOR = {Nassajian Mojarrad, Faezeh},
LANGUAGE = {eng},
DOI = {10.1007/s40995-025-01778-1},
PUBLISHER = {Springer},
ADDRESS = {New York, NY},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Iranian Journal of Science},
VOLUME = {49},
PAGES = {979--986},
}

Endnote

%0 Journal Article
%A Nassajian Mojarrad, Faezeh
%+ Computer Graphics, MPI for Informatics, Max Planck Society
%T A New Approximation Method for Solving Stochastic Differential Equations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-BF3A-7
%R 10.1007/s40995-025-01778-1
%7 2025-02-04
%D 2025
%J Iranian Journal of Science
%V 49
%&  979
%P  979 - 986
%I Springer
%C New York, NY
%U https://rdcu.be/d9PLh

Paper

F. Rubab, N. E. Nsampi, M. Bálint, F. Mujkanovic, H.-P. Seidel, T. Ritschel, and T. Leimkühler

“Learning Neural Antiderivatives,” 2025. [Online]. Available: https://arxiv.org/abs/2509.17755.

Abstract

Neural fields offer continuous, learnable representations that extend beyond traditional discrete formats in visual computing. We study the problem of learning neural representations of repeated antiderivatives directly from a function, a continuous analogue of summed-area tables. Although widely used in discrete domains, such cumulative schemes rely on grids, which prevents their applicability in continuous neural contexts. We introduce and analyze a range of neural methods for repeated integration, including both adaptations of prior work and novel designs. Our evaluation spans multiple input dimensionalities and integration orders, assessing both reconstruction quality and performance in downstream tasks such as filtering and rendering. These results enable integrating classical cumulative operators into modern neural systems and offer insights into learning tasks involving differential and integral operators.

BibTeX

@online{Rubab25,
TITLE = {Learning Neural Antiderivatives},
AUTHOR = {Rubab, Fizza and Nsampi, Ntumba Elie and B{\'a}lint, Martin and Mujkanovic, Felix and Seidel, Hans-Peter and Ritschel, Tobias and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2509.17755},
EPRINT = {2509.17755},
EPRINTTYPE = {arXiv},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Neural fields offer continuous, learnable representations that extend beyond traditional discrete formats in visual computing. We study the problem of learning neural representations of repeated antiderivatives directly from a function, a continuous analogue of summed-area tables. Although widely used in discrete domains, such cumulative schemes rely on grids, which prevents their applicability in continuous neural contexts. We introduce and analyze a range of neural methods for repeated integration, including both adaptations of prior work and novel designs. Our evaluation spans multiple input dimensionalities and integration orders, assessing both reconstruction quality and performance in downstream tasks such as filtering and rendering. These results enable integrating classical cumulative operators into modern neural systems and offer insights into learning tasks involving differential and integral operators.},
}

Endnote

%0 Report
%A Rubab, Fizza
%A Nsampi, Ntumba Elie
%A B&#225;lint, Martin
%A Mujkanovic, Felix
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Learning Neural Antiderivatives : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-1E9A-E
%U https://arxiv.org/abs/2509.17755
%D 2025
%X Neural fields offer continuous, learnable representations that extend beyond traditional discrete formats in visual computing. We study the problem of learning neural representations of repeated antiderivatives directly from a function, a continuous analogue of summed-area tables. Although widely used in discrete domains, such cumulative schemes rely on grids, which prevents their applicability in continuous neural contexts. We introduce and analyze a range of neural methods for repeated integration, including both adaptations of prior work and novel designs. Our evaluation spans multiple input dimensionalities and integration orders, assessing both reconstruction quality and performance in downstream tasks such as filtering and rendering. These results enable integrating classical cumulative operators into modern neural systems and offer insights into learning tasks involving differential and integral operators.
%K Computer Science, Learning, cs.LG,Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Conference paper

N. Sabnis, A. Otaran, D. Wittchen, J. K. Didion, J. Steimle, and P. Strohmeier

“Foot Pedal Control: The Role of Vibrotactile Feedback in Performance and Perceived Control,” in Proceedings of the Nineteenth International Conference on Tangible, Embedded and Embodied Interaction (TEI 2025), Bordeaux, France, 2025.

@inproceedings{SabnisTEI25,
TITLE = {Foot Pedal Control: {T}he Role of Vibrotactile Feedback in Performance and Perceived Control},
AUTHOR = {Sabnis, Nihar and Otaran, Ata and Wittchen, Dennis and Didion, Johanna Katharina and Steimle, J{\"u}rgen and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-1197-8},
DOI = {10.1145/3689050.3704937},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {Proceedings of the Nineteenth International Conference on Tangible, Embedded and Embodied Interaction (TEI 2025)},
PAGES = {1--15},
EID = {25},
ADDRESS = {Bordeaux, France},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Otaran, Ata
%A Wittchen, Dennis
%A Didion, Johanna Katharina
%A Steimle, J&#252;rgen
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Foot Pedal Control: The Role of Vibrotactile Feedback in Performance and Perceived Control : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D07C-8
%R 10.1145/3689050.3704937
%D 2025
%B 19th International Conference on Tangible, Embedded and Embodied
Interaction
%Z date of event: 2025-03-04 - 2025-03-07
%C Bordeaux, France
%B Proceedings of the Nineteenth International Conference on Tangible, Embedded and Embodied
Interaction
%P 1 - 15
%Z sequence number: 25
%I ACM
%@ 979-8-4007-1197-8

Conference paper

N. Sabnis, D. Wittchen, A. Otaran, F. Badmos, S. Liu, A. Saini, G. Regal, V. van Rheden, F. F. Müller, S. Nanayakkara, D. J. C. Matthies, T. Nachtigall, P. Strohmeier, L. Turmo Vidal, and D. S. Elvitigala

“Walking the Future: Bridging Foot Augmentation into Next Steps of Human Augmentation,” in CHI ’25, CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 2025.

@inproceedings{Sabnis2025WalinkingTheFuture,
TITLE = {Walking the Future: {B}ridging Foot Augmentation into Next Steps of Human Augmentation},
AUTHOR = {Sabnis, Nihar and Wittchen, Dennis and Otaran, Ata and Badmos, Fatima and Liu, Siyi and Saini, Aryan and Regal, Georg and van Rheden, Vincent and M{\"u}ller, Florian Floyd and Nanayakkara, Suranga1 and Matthies, Denys J. C. and Nachtigall, Troy and Strohmeier, Paul and Turmo Vidal, Laia and Elvitigala, Don Samitha},
LANGUAGE = {eng},
ISBN = {979-8-4007-1395-8},
DOI = {10.1145/3706599.3706742},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {CHI '25, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Yamashita, Naomi and Evers, Vanessa and Yatani, Koji and Ding, Xianghua Sharon},
PAGES = {1--7},
EID = {805},
ADDRESS = {Yokohama, Japan},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Wittchen, Dennis
%A Otaran, Ata
%A Badmos, Fatima
%A Liu, Siyi
%A Saini, Aryan
%A Regal, Georg
%A van Rheden, Vincent
%A M&#252;ller, Florian Floyd
%A Nanayakkara, Suranga1
%A Matthies, Denys J. C.
%A Nachtigall, Troy
%A Strohmeier, Paul
%A Turmo Vidal, Laia
%A Elvitigala, Don Samitha
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Walking the Future: Bridging Foot Augmentation into Next Steps of Human Augmentation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D361-2
%R 10.1145/3706599.3706742
%D 2025
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2025-04-26 - 2025-05-01
%C Yokohama, Japan
%B CHI '25
%E Yamashita, Naomi; Evers, Vanessa; Yatani, Koji; Ding, Xianghua Sharon
%P 1 - 7
%Z sequence number: 805
%I ACM
%@ 979-8-4007-1395-8

Conference paper

N. Sabnis

“Designing Vibrotactile Feedback for rendering Material and Force Experiences,” in CHI ’25, CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 2025.

@inproceedings{sabnis2025DC,
TITLE = {Designing Vibrotactile Feedback for rendering Material and Force Experiences},
AUTHOR = {Sabnis, Nihar},
LANGUAGE = {eng},
ISBN = {979-8-4007-1395-8},
DOI = {10.1145/3706599.3707598},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {CHI '25, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Yamashita, Naomi and Evers, Vanessa and Ding, Xianghua Sharon},
PAGES = {1--6},
EID = {833},
ADDRESS = {Yokohama, Japan},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%+ Computer Graphics, MPI for Informatics, Max Planck Society
%T Designing Vibrotactile Feedback for rendering Material and Force Experiences
 : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D37B-6
%R 10.1145/3706599.3707598
%D 2025
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2025-04-26 - 2025-05-01
%C Yokohama, Japan
%B CHI '25
%E Yamashita, Naomi; Evers, Vanessa; Ding, Xianghua Sharon
%P 1 - 6
%Z sequence number: 833
%I ACM
%@ 979-8-4007-1395-8

Conference paper

N. Sabnis, M. Roche, D. Wittchen, D. Degraen, and P. Strohmeier

“Motion-Coupled Asymmetric Vibration for Pseudo Force Rendering in Virtual Reality,” in CHI ’25, CHI Conference on Human Factors in Computing Sytems, Yokohama, Japan, 2025.

@inproceedings{Sabnis2025MCAV,
TITLE = {Motion-Coupled Asymmetric Vibration for Pseudo Force Rendering in Virtual Reality},
AUTHOR = {Sabnis, Nihar and Roche, Maelle and Wittchen, Dennis and Degraen, Donald and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-1394-1},
DOI = {10.1145/3706598.3713358},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {CHI '25, CHI Conference on Human Factors in Computing Sytems},
EDITOR = {Yamashita, Naomi and Evers, Vanessa and Yatani, Koji and Ding, Xianghua Sharon and Lee, Bongshin and Chetty, Marshini and Toups-Dugas, Phoebe},
PAGES = {1--22},
EID = {1134},
ADDRESS = {Yokohama, Japan},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Roche, Maelle
%A Wittchen, Dennis
%A Degraen, Donald
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Motion-Coupled Asymmetric Vibration for Pseudo Force Rendering in Virtual Reality : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D395-7
%R 10.1145/3706598.3713358
%D 2025
%B CHI Conference on Human Factors in Computing Sytems
%Z date of event: 2025-04-26 - 2025-05-01
%C Yokohama, Japan
%B CHI '25
%E Yamashita, Naomi; Evers, Vanessa; Yatani, Koji; Ding, Xianghua Sharon; Lee, Bongshin; Chetty, Marshini; Toups-Dugas, Phoebe
%P 1 - 22
%Z sequence number: 1134
%I ACM
%@ 979-8-4007-1394-1

Conference paper

N. Sabnis, G. Vega, V. Martinez-Missir, and P. Strohmeier

“VibRacket: Designing and Experiencing Embodied Vibrotactile Feedback on a Badminton Racket,” in SportsHCI 2025, First Annual Conference on Human-Computer Interaction and Sports, Enschede The Netherlands, 2025.

@inproceedings{Sabnis_SportsHCI25,
TITLE = {{VibRacket}: {D}esigning and Experiencing Embodied Vibrotactile Feedback on a Badminton Racket},
AUTHOR = {Sabnis, Nihar and Vega, Gabriela and Martinez-Missir, Valentin and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-1428-3},
DOI = {10.1145/3749385.3749417},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {SportsHCI 2025, First Annual Conference on Human-Computer Interaction and Sports},
EDITOR = {Jones, Michael D. and Lallemand, Carine},
PAGES = {1--5},
EID = {28},
ADDRESS = {Enschede The Netherlands},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Vega, Gabriela
%A Martinez-Missir, Valentin
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T VibRacket: Designing and Experiencing Embodied Vibrotactile Feedback on a Badminton Racket : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-A6CE-9
%R 10.1145/3749385.3749417
%D 2025
%B First Annual Conference on Human-Computer Interaction and Sports
%Z date of event: 2025-11-17 - 2025-11-19
%C Enschede The Netherlands
%B SportsHCI 2025
%E Jones, Michael D.; Lallemand, Carine
%P 1 - 5
%Z sequence number: 28
%I ACM
%@ 979-8-4007-1428-3

Conference paper

C. Salaün, M. Bálint, L. Belcour, E. Heitz, G. Singh, and K. Myszkowski

“Histogram Stratification for Spatio-Temporal Reservoir Sampling,” in SIGGRAPH 2025 Conference Papers, Vancouver, Canada, 2025.

@inproceedings{Salaun_SIGGRAPH25,
TITLE = {Histogram Stratification for Spatio-Temporal Reservoir Sampling},
AUTHOR = {Sala{\"u}n, Corentin and B{\'a}lint, Martin and Belcour, Laurent and Heitz, Eric and Singh, Gurprit and Myszkowski, Karol},
LANGUAGE = {eng},
ISBN = {979-8-4007-1540-2},
DOI = {10.1145/3721238.3730723},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {SIGGRAPH 2025 Conference Papers},
EDITOR = {Alford, Ginger and Zhang, Hao Richard and Schulz, Adriana},
PAGES = {1--10},
EID = {83},
ADDRESS = {Vancouver, Canada},
}

Endnote

%0 Conference Proceedings
%A Sala&#252;n, Corentin
%A B&#225;lint, Martin
%A Belcour, Laurent
%A Heitz, Eric
%A Singh, Gurprit
%A Myszkowski, Karol
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Histogram Stratification for Spatio-Temporal Reservoir Sampling : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-6ED6-0
%R 10.1145/3721238.3730723
%D 2025
%B ACM SIGGRAPH Conference
%Z date of event: 2025-08-10 - 2025-08-14
%C Vancouver, Canada
%B SIGGRAPH 2025 Conference Papers
%E Alford, Ginger; Zhang, Hao Richard; Schulz, Adriana
%P 1 - 10
%Z sequence number: 83
%I ACM
%@ 979-8-4007-1540-2

Thesis

C. Salaün

“Toward Improving Monte Carlo Estimator for Rendering and Machine Learning,” Universität des Saarlandes, Saarbrücken, 2025.

@phdthesis{ThesisPhDSalaun25,
TITLE = {Toward Improving Monte Carlo Estimator for Rendering and Machine Learning},
AUTHOR = {Sala{\"u}n, Corentin},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-466596},
DOI = {10.22028/D291-46659},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
}

Endnote

%0 Thesis
%A Sala&#252;n, Corentin
%Y Singh, Gurprit
%A referee: Seidel, Hans-Peter
%A referee: Bonneel, Nicolas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Toward Improving Monte Carlo Estimator for Rendering and Machine Learning : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-95ED-9
%R 10.22028/D291-46659
%U urn:nbn:de:bsz:291--ds-466596
%F OTHER: hdl:20.500.11880/40923
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2025
%P XII, 158 p.
%V phd
%9 phd
%U http://dx.doi.org/10.22028/D291-46659

Conference paper

C. Salaün, X. Huang, I. Georgiev, N. J. Mitra, and G. Singh

“Multiple Importance Sampling for Stochastic Gradient Estimation,” in Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Vol. 1 (ICPRAM 2025), Porto, Portugal, 2025.

@inproceedings{Salaun2407.15525,
TITLE = {Multiple Importance Sampling for Stochastic Gradient Estimation},
AUTHOR = {Sala{\"u}n, Corentin and Huang, Xingchang and Georgiev, Iliyan and Mitra, Niloy J. and Singh, Gurprit},
LANGUAGE = {eng},
ISBN = {978-989-758-730-6},
DOI = {10.5220/0013311200003905},
PUBLISHER = {SciTePress},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods -- Vol. 1 (ICPRAM 2025)},
PAGES = {401--409},
ADDRESS = {Porto, Portugal},
}

Endnote

%0 Conference Proceedings
%A Sala&#252;n, Corentin
%A Huang, Xingchang
%A Georgiev, Iliyan
%A Mitra, Niloy J.
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Multiple Importance Sampling for Stochastic Gradient Estimation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8ADC-B
%R 10.5220/0013311200003905
%D 2025
%B 14th International Conference on Pattern Recognition Applications and Methods
%Z date of event: 2025-02-23 - 2025-02-25
%C Porto, Portugal
%B Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Vol. 1
%P 401 - 409
%I SciTePress
%@ 978-989-758-730-6

Conference paper

C. Salaün, X. Huang, I. Georgiev, N. J. Mitra, and G. Singh

“Online Importance Sampling for Stochastic Gradient Optimization,” in Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods. - Vol. 1 (ICPRAM 202), Porto, Portugal, 2025.

@inproceedings{Salaun2311.14468,
TITLE = {Online Importance Sampling for Stochastic Gradient Optimization},
AUTHOR = {Sala{\"u}n, Corentin and Huang, Xingchang and Georgiev, Iliyan and Mitra, Niloy J. and Singh, G.},
LANGUAGE = {eng},
ISBN = {978-989-758-730-6},
DOI = {10.5220/0013311100003905},
PUBLISHER = {SciTePress},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods. -- Vol. 1 (ICPRAM 202)},
PAGES = {130--140},
ADDRESS = {Porto, Portugal},
}

Endnote

%0 Conference Proceedings
%A Sala&#252;n, Corentin
%A Huang, Xingchang
%A Georgiev, Iliyan
%A Mitra, Niloy J.
%A Singh, G.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Online Importance Sampling for Stochastic Gradient Optimization : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8B07-A
%R 10.5220/0013311100003905
%D 2025
%B 14th International Conference on Pattern Recognition Applications and Methods
%Z date of event: 2025-02-23 - 2025-02-25
%C Porto, Portugal
%B Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods. - Vol. 1
%P 130 - 140
%I SciTePress
%@ 978-989-758-730-6

Conference paper

G. Singh, X. Huang, J. Vandersanden, C. Oztireli, and N. Mitra

“Demystifying Noise: The Role of Randomness in Generative AI,” in Eurographics 2025 - Tutorials, London, UK, 2025, vol. 44, no. 2.

@inproceedings{SinghEG25,
TITLE = {Demystifying noise: {T}he role of randomness in generative {AI}},
AUTHOR = {Singh, Gurprit and Huang, Xingchang and Vandersanden, Jente and Oztireli, Cengiz and Mitra, Niloy},
LANGUAGE = {eng},
ISBN = {978-3-03868-267-7{\textbraceright}},
DOI = {10.2312/egt.20251004},
PUBLISHER = {Eurographics Association},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {Eurographics 2025 -- Tutorials},
VOLUME = {44},
ISSUE = {2},
ADDRESS = {London, UK},
}

Endnote

%0 Conference Proceedings
%A Singh, Gurprit
%A Huang, Xingchang
%A Vandersanden, Jente
%A Oztireli, Cengiz
%A Mitra, Niloy
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Demystifying Noise: The Role of Randomness in Generative AI : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-242D-1
%R 10.2312/egt.20251004
%D 2025
%B 46th Annual Conference of the European Association for Computer Graphics 
%Z date of event: 2025-05-12 - 2025-05-16
%C London, UK
%B Eurographics 2025 - Tutorials
%V 44
%N 2
%I Eurographics Association
%@ 978-3-03868-267-7}

Conference paper

G. Singh, X. Huang, J. Vandersanden, C. Oztireli, and N. Mitra

“Demystifying Noise: Role of Randomness in Generative AI,” in Proceedings SIGGRAPH 2025 Courses, Vancouver, Canada, 2025.

@inproceedings{Singh_SIGGRAPH25Courses,
TITLE = {Demystifying noise: {R}ole of randomness in generative {AI}},
AUTHOR = {Singh, Gurprit and Huang, Xingchang and Vandersanden, Jente and Oztireli, Cengiz and Mitra, Niloy},
LANGUAGE = {eng},
ISBN = {979-8-4007-1543-3},
DOI = {10.1145/3721241.3734002},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {Proceedings SIGGRAPH 2025 Courses},
EDITOR = {Alford, Ginger and Borkiewicz, Kalina},
PAGES = {1--7},
EID = {18},
ADDRESS = {Vancouver, Canada},
}

Endnote

%0 Conference Proceedings
%A Singh, Gurprit
%A Huang, Xingchang
%A Vandersanden, Jente
%A Oztireli, Cengiz
%A Mitra, Niloy
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Demystifying Noise: Role of Randomness in Generative AI : Demystifying randomness in generative AI
%G eng
%U http://hdl.handle.net/21.11116/0000-0013-2930-7
%R 10.1145/3721241.3734002
%D 2025
%B ACM SIGGRAPH Conference
%Z date of event: 2025-08-10 - 2025-08-14
%C Vancouver, Canada
%B Proceedings SIGGRAPH 2025 Courses
%E Alford, Ginger; Borkiewicz, Kalina
%P 1 - 7
%Z sequence number: 18
%I ACM
%@ 979-8-4007-1543-3

Article

E. Stefanidi, N. Wagener, I. Chatzakis, P. W. Wožniak, S. Ntoa, G. Margetis, Y. Rogers, and J. Niess

“Supporting Communication and Well-being with a Multi-Stakeholder Mobile App: Lessons Learned from a Field Study with ADHD Children and their Caregivers,” Proceedings of the ACM on Human-Computer Interaction, vol. 9, no. 2, 2025.

@article{Stefanidi_CSCW25,
TITLE = {Supporting Communication and Well-being with a Multi-Stakeholder Mobile App: {L}essons Learned from a Field Study with {ADHD} Children and their Caregivers},
AUTHOR = {Stefanidi, Evropi and Wagener, Nadine and Chatzakis, Ioannis and Wo{\v z}niak, Pawe{\l} W. and Ntoa, Stavroula and Margetis, George and Rogers, Yvonne and Niess, Jasmin},
LANGUAGE = {eng},
ISSN = {2573-0142},
DOI = {10.1145/3711075},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Proceedings of the ACM on Human-Computer Interaction},
VOLUME = {9},
NUMBER = {2},
PAGES = {1--37},
EID = {CSCW177},
}

Endnote

%0 Journal Article
%A Stefanidi, Evropi
%A Wagener, Nadine
%A Chatzakis, Ioannis
%A Wo&#382;niak, Pawe&#322; W.
%A Ntoa, Stavroula
%A Margetis, George
%A Rogers, Yvonne
%A Niess, Jasmin
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Supporting Communication and Well-being with a Multi-Stakeholder Mobile App: Lessons Learned from a Field Study with ADHD Children and their Caregivers : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-9015-2
%R 10.1145/3711075
%7 2025-05-02
%D 2025
%J Proceedings of the ACM on Human-Computer Interaction
%V 9
%N 2
%& 1
%P 1 - 37
%Z sequence number: CSCW177
%I ACM
%C New York, NY
%@ false

Article

D4D6

C. Stippel, F. Mujkanovic, T. Leimkühler, and P. Hermosilla

“Marching Neurons: Accurate Surface Extraction for Neural Implicit Shapes,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2025), vol. 44, no. 6, 2025.

Abstract

Accurate surface geometry representation is crucial in 3D visual computing. Explicit representations, such as polygonal meshes, and implicit representations, like signed distance functions, each have distinct advantages, making efficient conversions between them increasingly important. Conventional surface extraction methods for implicit representations, such as the widely used Marching Cubes algorithm, rely on spatial decomposition and sampling, leading to inaccuracies due to fixed and limited resolution. We introduce a novel approach for analytically extracting surfaces from neural implicit functions. Our method operates natively in parallel and can navigate large neural architectures. By leveraging the fact that each neuron partitions the domain, we develop a depth-first traversal strategy to efficiently track the encoded surface. The resulting meshes faithfully capture the full geometric information from the network without ad-hoc spatial discretization, achieving unprecedented accuracy across diverse shapes and network architectures while maintaining competitive speed.

BibTeX

@article{Stippel_SIGGRAPHAsia25,
TITLE = {Marching Neurons: {A}ccurate Surface Extraction for Neural Implicit Shapes},
AUTHOR = {Stippel, Christian and Mujkanovic, Felix and Leimk{\"u}hler, Thomas and Hermosilla, Pedro},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3763328},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
ABSTRACT = {Accurate surface geometry representation is crucial in 3D visual computing. Explicit representations, such as polygonal meshes, and implicit representations, like signed distance functions, each have distinct advantages, making efficient conversions between them increasingly important. Conventional surface extraction methods for implicit representations, such as the widely used Marching Cubes algorithm, rely on spatial decomposition and sampling, leading to inaccuracies due to fixed and limited resolution. We introduce a novel approach for analytically extracting surfaces from neural implicit functions. Our method operates natively in parallel and can navigate large neural architectures. By leveraging the fact that each neuron partitions the domain, we develop a depth-first traversal strategy to efficiently track the encoded surface. The resulting meshes faithfully capture the full geometric information from the network without ad-hoc spatial discretization, achieving unprecedented accuracy across diverse shapes and network architectures while maintaining competitive speed.},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {44},
NUMBER = {6},
PAGES = {1--12},
EID = {222},
BOOKTITLE = {SIGGRAPH Asia 2025 (ACM SIGGRAPH Asia 2025)},
}

Endnote

%0 Journal Article
%A Stippel, Christian
%A Mujkanovic, Felix
%A Leimk&#252;hler, Thomas
%A Hermosilla, Pedro
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
%T Marching Neurons: Accurate Surface Extraction for Neural Implicit Shapes : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-48C4-E
%R 10.1145/3763328
%7 2025-12-04
%D 2025
%X Accurate surface geometry representation is crucial in 3D visual computing. Explicit representations, such as polygonal meshes, and implicit representations, like signed distance functions, each have distinct advantages, making efficient conversions between them increasingly important. Conventional surface extraction methods for implicit representations, such as the widely used Marching Cubes algorithm, rely on spatial decomposition and sampling, leading to inaccuracies due to fixed and limited resolution. We introduce a novel approach for analytically extracting surfaces from neural implicit functions. Our method operates natively in parallel and can navigate large neural architectures. By leveraging the fact that each neuron partitions the domain, we develop a depth-first traversal strategy to efficiently track the encoded surface. The resulting meshes faithfully capture the full geometric information from the network without ad-hoc spatial discretization, achieving unprecedented accuracy across diverse shapes and network architectures while maintaining competitive speed.
%K Computer Science, Graphics, cs.GR,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Computer Vision and Pattern Recognition, cs.CV
%J ACM Transactions on Graphics
%V 44
%N 6
%& 1
%P 1 - 12
%Z sequence number: 222
%I ACM
%C New York, NY
%@ false
%B SIGGRAPH Asia 2025
%O ACM SIGGRAPH Asia 2025

Conference paper

P. Strohmeier, L. T. Vidal, G. Vega, C. N. Reed, A. Mazursky, E. AliAbbasi, A. Tajadura-Jiménez, and J. Steimle

“Sensorimotor Devices: Coupling Sensing and Actuation to Augment Bodily Experience,” in CHI ’25, CHI Conference on Human Factors in Computing Systems, Yokohama, Japan, 2025.

@inproceedings{strohmeier2025SensorimotorDevices,
TITLE = {Sensorimotor Devices: {C}oupling Sensing and Actuation to Augment Bodily Experience},
AUTHOR = {Strohmeier, Paul and Vidal, Laia Turmo and Vega, Gabriela and Reed, Courtney N. and Mazursky, Alex and AliAbbasi, Easa and Tajadura-Jim{\'e}nez, Ana and Steimle, J{\"u}rgen},
LANGUAGE = {eng},
ISBN = {979-8-4007-1395-8},
DOI = {10.1145/3706599.3706735},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {CHI '25, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Yamashita, Naomi and Evers, Vanessa and Yatani, Koji and Ding, Xianghua Sharon},
PAGES = {1--7},
EID = {799},
ADDRESS = {Yokohama, Japan},
}

Endnote

%0 Conference Proceedings
%A Strohmeier, Paul
%A Vidal, Laia Turmo
%A Vega, Gabriela
%A Reed, Courtney N.
%A Mazursky, Alex
%A AliAbbasi, Easa
%A Tajadura-Jim&#233;nez, Ana
%A Steimle, J&#252;rgen
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Sensorimotor Devices: Coupling Sensing and Actuation to Augment Bodily Experience : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D37D-4
%R 10.1145/3706599.3706735
%D 2025
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2025-04-26 - 2025-05-01
%C Yokohama, Japan
%B CHI '25
%E Yamashita, Naomi; Evers, Vanessa; Yatani, Koji; Ding, Xianghua Sharon
%P 1 - 7
%Z sequence number: 799
%I ACM
%@ 979-8-4007-1395-8

Thesis

D4IMPR-CS

D. Sumin

“Data-driven Methods for High-fidelity Material Appearance Reproduction in Additive 3D Printing,” Universität des Saarlandes, Saarbrücken, 2025.

@phdthesis{ThesisPhDSumin2025,
TITLE = {Data-driven Methods for High-fidelity Material Appearance Reproduction in Additive {3D} Printing},
AUTHOR = {Sumin, Denis},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-445122},
DOI = {10.22028/D291-44512},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025-02-18},
}

Endnote

%0 Thesis
%A Sumin, Denis
%Y Myszkowski, Karol
%A referee: Seidel, Hans-Peter
%A referee: Frisvad, Jeppe Rival
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Data-driven Methods for High-fidelity Material Appearance Reproduction in Additive 3D Printing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-3692-B
%R 10.22028/D291-44512
%U urn:nbn:de:bsz:291--ds-445122
%F OTHER: hdl:20.500.11880/39846
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2025
%8 18.02.2025
%P XI, 114 p.
%V phd
%9 phd
%U http://dx.doi.org/10.22028/D291-44512

Conference paper

G. Vega and P. Strohmeier

“VibroLink: A Wireless Vibro-Auditory Transmission System to Improve Situational Awareness During EVA,” in Advancing Human-Computer Interaction for Space Exploration (SpaceCHI 2025), Cologne, Germany, 2025.

@inproceedings{VegaSpaceCHI25,
TITLE = {{VibroLink}: {A} Wireless Vibro-Auditory Transmission System to Improve Situational Awareness During {EVA}},
AUTHOR = {Vega, Gabriela and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {978-3-95977-384-3},
URL = {urn:nbn:de:0030-drops-239955},
DOI = {10.4230/OASIcs.SpaceCHI.2025.5},
PUBLISHER = {Schloss Dagstuhl},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {Advancing Human-Computer Interaction for Space Exploration (SpaceCHI 2025)},
EDITOR = {Bensch, Leonie and Nilsson, Tommy and Nisser, Martin and Pataranutaporn, Pat and Schmidt, Albrecht and Sumini, Valentina},
PAGES = {1--9},
EID = {5},
SERIES = {Open Access Series in Informatics},
VOLUME = {130},
ADDRESS = {Cologne, Germany},
}

Endnote

%0 Conference Proceedings
%A Vega, Gabriela
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T VibroLink: A Wireless Vibro-Auditory Transmission System to Improve Situational Awareness During EVA : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-0A09-8
%R 10.4230/OASIcs.SpaceCHI.2025.5
%U urn:nbn:de:0030-drops-239955
%D 2025
%B 4th SpaceCHI Conference on Advancing Human-Computer Interaction in Space Exploration
%Z date of event: 2025-06-23 - 2025-06-24
%C Cologne, Germany
%B Advancing Human-Computer Interaction for Space Exploration
%E Bensch, Leonie; Nilsson, Tommy; Nisser, Martin; Pataranutaporn, Pat; Schmidt, Albrecht; Sumini, Valentina
%P 1 - 9
%Z sequence number: 5
%I Schloss Dagstuhl
%@ 978-3-95977-384-3
%B Open Access Series in Informatics
%N 130

Conference paper

N. Wagener, D. C. Albensoeder, L. Reicherts, P. W. Woźniak, Y. Rogers, and J. Niess

“TogetherReflect: Supporting Emotional Expression for Couples Through a Collaborative Virtual Reality Experience,” in CHI ’25, CHI Conference on Human Factors in Computing Sytems, Yokohama, Japan, 2025.

@inproceedings{wagener2025togetherreflect,
TITLE = {{TogetherReflect}: {S}upporting Emotional Expression for Couples Through a Collaborative Virtual Reality Experience},
AUTHOR = {Wagener, Nadine and Albensoeder, Daniel C. and Reicherts, Leon and Wo{\'Z}niak, Pawe{\l} W. and Rogers, Yvonne and Niess, Jasmin},
LANGUAGE = {eng},
ISBN = {979-8-4007-1394-1},
DOI = {10.1145/3706598.3713642},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {CHI '25, CHI Conference on Human Factors in Computing Sytems},
EDITOR = {Yamashita, Naomi and Evers, Vanessa and Yatani, Koji and Ding, Xianghua Sharon and Lee, Bongshin and Chetty, Marshini and Toups-Dugas, Phoebe},
PAGES = {1--16},
EID = {1130},
ADDRESS = {Yokohama, Japan},
}

Endnote

%0 Conference Proceedings
%A Wagener, Nadine
%A Albensoeder, Daniel C.
%A Reicherts, Leon
%A Wo&#378;niak, Pawe&#322; W.
%A Rogers, Yvonne
%A Niess, Jasmin
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T TogetherReflect: Supporting Emotional Expression for Couples Through a Collaborative Virtual Reality Experience : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D377-A
%R 10.1145/3706598.3713642
%D 2025
%B CHI Conference on Human Factors in Computing Sytems
%Z date of event: 2025-04-26 - 2025-05-01
%C Yokohama, Japan
%B CHI '25
%E Yamashita, Naomi; Evers, Vanessa; Yatani, Koji; Ding, Xianghua Sharon; Lee, Bongshin; Chetty, Marshini; Toups-Dugas, Phoebe
%P 1 - 16
%Z sequence number: 1130
%I ACM
%@ 979-8-4007-1394-1

Conference paper

N. Wagener, Y. Rogers, and J. Niess

“The Self-Care Technology Process Model (SCTpm): A Framework for Designing VR-Based Self-Care Technology,” in DIS ’25, ACM Designing Interactive Systems Conference, Madeira, Portugal, 2025.

@inproceedings{Wagener_DIS25,
TITLE = {The Self-Care Technology Process Model ({SCTpm}): {A} Framework for Designing {VR}-Based Self-Care Technology},
AUTHOR = {Wagener, Nadine and Rogers, Yvonne and Niess, Jasmin},
LANGUAGE = {eng},
ISBN = {979-8-4007-1485-6},
DOI = {10.1145/3715336.3735743},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {DIS '25, ACM Designing Interactive Systems Conference},
EDITOR = {Jardim Nunes, Nuno and Nisi, Valentina and Oakley, Ian and Yang, Qian and Zheng, Clement},
PAGES = {2237--2254},
ADDRESS = {Madeira, Portugal},
}

Endnote

%0 Conference Proceedings
%A Wagener, Nadine
%A Rogers, Yvonne
%A Niess, Jasmin
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T The Self-Care Technology Process Model (SCTpm): A Framework for Designing VR-Based Self-Care Technology : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-9021-4
%R 10.1145/3715336.3735743
%D 2025
%8 04.07.2025
%B ACM Designing Interactive Systems Conference
%Z date of event: 2025-07-05 - 2025-07-09
%C Madeira, Portugal
%B DIS '25
%E Jardim Nunes, Nuno; Nisi, Valentina; Oakley, Ian; Yang, Qian; Zheng, Clement
%P 2237 - 2254
%I ACM
%@ 979-8-4007-1485-6

Conference paper

C. Wang, Z. Xia, T. Leimkühler, K. Myszkowski, and X. Zhang

“LEDiff: Latent Exposure Diffusion for HDR Generation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025), Nashville, TN, USA, 2025.

@inproceedings{wang2024lediff,
TITLE = {{LEDiff}: {L}atent Exposure Diffusion for {HDR} Generation},
AUTHOR = {Wang, Chao and Xia, Zhihao and Leimk{\"u}hler, Thomas and Myszkowski, Karol and Zhang, Xuaner},
LANGUAGE = {eng},
ISBN = {979-8-3315-4364-8},
DOI = {10.1109/CVPR52734.2025.00051},
PUBLISHER = {IEEE},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)},
PAGES = {453--464},
ADDRESS = {Nashville, TN, USA},
}

Endnote

%0 Conference Proceedings
%A Wang, Chao
%A Xia, Zhihao
%A Leimk&#252;hler, Thomas
%A Myszkowski, Karol
%A Zhang, Xuaner
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T LEDiff: Latent Exposure Diffusion for HDR Generation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8B10-F
%R 10.1109/CVPR52734.2025.00051
%D 2025
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2025-06-11 - 2025-06-15
%C Nashville, TN, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 453 - 464
%I IEEE
%@ 979-8-3315-4364-8

Thesis

D4IMPR-CS

C. Wang

“Deep High Dynamic Range Imaging: Reconstruction, Generation and Display,” Universität des Saarlandes, Saarbrücken, 2025.

@phdthesis{ThesisPhDWang2025,
TITLE = {Deep High Dynamic Range Imaging: Reconstruction, Generation and Display},
AUTHOR = {Wang, Chao},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-458203},
DOI = {10.22028/D291-45820},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
}

Endnote

%0 Thesis
%A Wang, Chao
%Y Myszkowski, Karol
%Y Seidel, Hans-Peter
%A referee: Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Deep High Dynamic Range Imaging: Reconstruction, Generation and Display : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-A295-D
%R 10.22028/D291-45820
%U urn:nbn:de:bsz:291--ds-458203
%F OTHER: hdl:20.500.11880/40243
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2025
%P XI, 125 p.
%V phd
%9 phd
%U http://dx.doi.org/10.22028/D291-45820

Conference paper

C. N. Weerasinghe, R. Abeywardhane, U. Kodikara, S. Abewickrema, N. Sabnis, F. “Floyd” Mueller, and D. S. Elvitigala

“VibroBits: Visualising Body Centre of Pressure Using Vibrotactile Feedback for Sports Training,” in SportsHCI 2025, First Annual Conference on Human-Computer Interaction and Sports, Enschede The Netherlands, 2025.

@inproceedings{Weerasinghe_SportsHCI25,
TITLE = {{VibroBits}: {V}isualising Body Centre of Pressure Using Vibrotactile Feedback for Sports Training},
AUTHOR = {Weerasinghe, Chathura Nirmal and Abeywardhane, Ruchira and Kodikara, Uvindu and Abewickrema, Sasindu and Sabnis, Nihar and Mueller, Florian 'Floyd' and Elvitigala, Don Samitha},
LANGUAGE = {eng},
ISBN = {979-8-4007-1428-3},
DOI = {10.1145/3749385.3749418},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {SportsHCI 2025, First Annual Conference on Human-Computer Interaction and Sports},
EDITOR = {Jones, Michael D. and Lallemand, Carine},
PAGES = {1--4},
EID = {31},
ADDRESS = {Enschede The Netherlands},
}

Endnote

%0 Conference Proceedings
%A Weerasinghe, Chathura Nirmal
%A Abeywardhane, Ruchira
%A Kodikara, Uvindu
%A Abewickrema, Sasindu
%A Sabnis, Nihar
%A Mueller, Florian 'Floyd'
%A Elvitigala, Don Samitha
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T VibroBits: Visualising Body Centre of Pressure Using Vibrotactile Feedback for Sports Training : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-A6D6-F
%R 10.1145/3749385.3749418
%D 2025
%B First Annual Conference on Human-Computer Interaction and Sports
%Z date of event: 2025-11-17 - 2025-11-19
%C Enschede The Netherlands
%B SportsHCI 2025
%E Jones, Michael D.; Lallemand, Carine
%P 1 - 4
%Z sequence number: 31
%I ACM
%@ 979-8-4007-1428-3

Article

K. T. Wijaya, D.-H. Paek, and S.-H. Kong

“SwiftPCN: Fast, Implementation-efficient, and Accurate Point Cloud Completion Network with Flexible Output Resolution,” Neurocomputing, 2025.

@article{Tirta25,
TITLE = {{SwiftPCN}: {F}ast, implementation-efficient, and accurate point cloud completion network with flexible output resolution},
AUTHOR = {Wijaya, Kevin Tirta and Paek, Dong-Hee and Kong, Seung-Hyun},
LANGUAGE = {eng},
ISSN = {0925-2312},
DOI = {10.1016/j.neucom.2025.129387},
PUBLISHER = {Elsevier},
ADDRESS = {Amsterdam},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Neurocomputing},
EID = {129387},
}

Endnote

%0 Journal Article
%A Wijaya, Kevin Tirta
%A Paek, Dong-Hee
%A Kong, Seung-Hyun
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T SwiftPCN: Fast, Implementation-efficient, and Accurate Point Cloud Completion Network with Flexible Output Resolution : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-97BA-2
%R 10.1016/j.neucom.2025.129387
%7 2025-01-13
%D 2025
%J Neurocomputing
%Z sequence number: 129387
%I Elsevier
%C Amsterdam
%@ false

Article

K. T. Wijaya, N. Ansari, H.-P. Seidel, and V. Babaei

“Trustworthy Inverse Molecular Design via Alignment with Molecular Dynamics,” Advanced Science, vol. 12, no. 27, 2025.

@article{Wijaya2402.16930,
TITLE = {Trustworthy Inverse Molecular Design via Alignment with Molecular Dynamics},
AUTHOR = {Wijaya, Kevin Tirta and Ansari, Navid and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
ISSN = {2198-3844},
DOI = {10.1002/advs.202416356},
PUBLISHER = {Wiley-VCH},
ADDRESS = {Weinheim},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Advanced Science},
VOLUME = {12},
NUMBER = {27},
EID = {2416356},
}

Endnote

%0 Journal Article
%A Wijaya, Kevin Tirta
%A Ansari, Navid
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Trustworthy Inverse Molecular Design via Alignment with Molecular Dynamics : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-81E4-A
%R 10.1002/advs.202416356
%7 2025-05-08
%D 2025
%J Advanced Science
%O Adv. Sci.
%V 12
%N 27
%Z sequence number: 2416356
%I Wiley-VCH
%C Weinheim
%@ false

Article

K. T. Wijaya, C. R. Goenawan, and S.-H. Kong

“Enhancing Performance of 3D Point Completion Network using Consistency Loss,” Neurocomputing, vol. 618, 2025.

@article{Wijaya2025,
TITLE = {Enhancing Performance of {3D} Point Completion Network using Consistency Loss},
AUTHOR = {Wijaya, Kevin Tirta and Goenawan, Christofel Rio and Kong, Seung-Hyun},
LANGUAGE = {eng},
ISSN = {0925-2312},
DOI = {10.1016/j.neucom.2024.129037},
PUBLISHER = {Elsevier},
ADDRESS = {Amsterdam},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
JOURNAL = {Neurocomputing},
VOLUME = {618},
PAGES = {1--11},
EID = {129037},
}

Endnote

%0 Journal Article
%A Wijaya, Kevin Tirta
%A Goenawan, Christofel Rio
%A Kong, Seung-Hyun
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Enhancing Performance of 3D Point Completion Network using Consistency Loss : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-6BDB-0
%R 10.1016/j.neucom.2024.129037
%7 2024-11-30
%D 2025
%J Neurocomputing
%V 618
%& 1
%P 1 - 11
%Z sequence number: 129037
%I Elsevier
%C Amsterdam
%@ false

Conference paper

K. T. Wijaya, M. Sun, M. Guo, H.-P. Seidel, W. Matusik, and V. Babaei

“Post Hoc Regression Refinement via Pairwise Rankings,” in Advances in Neural Information Processing Systems 38, San Diego, CA, USA.

Abstract

Accurate prediction of continuous properties is essential to many scientific and engineering tasks. Although deep-learning regressors excel with abundant labels, their accuracy deteriorates in data-scarce regimes. We introduce RankRefine, a model-agnostic, plug-and-play post hoc method that refines regression with expert knowledge coming from pairwise rankings. Given a query item and a small reference set with known properties, RankRefine combines the base regressor's output with a rank-based estimate via inverse variance weighting, requiring no retraining. In molecular property prediction task, RankRefine achieves up to 10% relative reduction in mean absolute error using only 20 pairwise comparisons obtained through a general-purpose large language model (LLM) with no finetuning. As rankings provided by human experts or general-purpose LLMs are sufficient for improving regression across diverse domains, RankRefine offers practicality and broad applicability, especially in low-data settings.

BibTeX

@inproceedings{Wijaya_NeurIPS25a,
TITLE = {Post Hoc Regression Refinement via Pairwise Rankings},
AUTHOR = {Wijaya, Kevin Tirta and Sun, Michael and Guo, Minghao and Seidel, Hans-Peter and Matusik, Wojciech and Babaei, Vahid},
LANGUAGE = {eng},
PUBLISHER = {Curran Associates, Inc.},
YEAR = {2025},
PUBLREMARK = {Accepted},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Accurate prediction of continuous properties is essential to many scientific and engineering tasks. Although deep-learning regressors excel with abundant labels, their accuracy deteriorates in data-scarce regimes. We introduce RankRefine, a model-agnostic, plug-and-play post hoc method that refines regression with expert knowledge coming from pairwise rankings. Given a query item and a small reference set with known properties, RankRefine combines the base regressor's output with a rank-based estimate via inverse variance weighting, requiring no retraining. In molecular property prediction task, RankRefine achieves up to 10% relative reduction in mean absolute error using only 20 pairwise comparisons obtained through a general-purpose large language model (LLM) with no finetuning. As rankings provided by human experts or general-purpose LLMs are sufficient for improving regression across diverse domains, RankRefine offers practicality and broad applicability, especially in low-data settings.},
BOOKTITLE = {Advances in Neural Information Processing Systems 38},
ADDRESS = {San Diego, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Wijaya, Kevin Tirta
%A Sun, Michael
%A Guo, Minghao
%A Seidel, Hans-Peter
%A Matusik, Wojciech
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Post Hoc Regression Refinement via Pairwise Rankings : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-FAD1-7
%D 2025
%B 39th Annual Conference on Neural Information Processing Systems
%Z date of event: 2025-12-02 - 2025-12-07
%C San Diego, CA, USA 
%X Accurate prediction of continuous properties is essential to many scientific and engineering tasks. Although deep-learning regressors excel with abundant labels, their accuracy deteriorates in data-scarce regimes. We introduce RankRefine, a model-agnostic, plug-and-play post hoc method that refines regression with expert knowledge coming from pairwise rankings. Given a query item and a small reference set with known properties, RankRefine combines the base regressor's output with a rank-based estimate via inverse variance weighting, requiring no retraining. In molecular property prediction task, RankRefine achieves up to 10% relative reduction in mean absolute error using only 20 pairwise comparisons obtained through a general-purpose large language model (LLM) with no finetuning. As rankings provided by human experts or general-purpose LLMs are sufficient for improving regression across diverse domains, RankRefine offers practicality and broad applicability, especially in low-data settings.
%K Computer Science, Learning, cs.LG,Computer Science, Artificial Intelligence, cs.AI
%B Advances in Neural Information Processing Systems 38
%I Curran Associates, Inc.

Conference paper

D. Wittchen, A. Ramian, N. Sabnis, C. Chlebowski, R. Böhme, G. Freitag, B. Fruchard, and D. Degraen

“CollabJam: Studying Collaborative Haptic Experience Design for On-Body Vibrotactile Patterns,” in CHI ’25, CHI Conference on Human Factors in Computing Sytems, Yokohama, Japan, 2025.

@inproceedings{Wittchen2025CollabJam,
TITLE = {CollabJam: {S}tudying Collaborative Haptic Experience Design for On-Body Vibrotactile Patterns},
AUTHOR = {Wittchen, Dennis and Ramian, Alexander and Sabnis, Nihar and Chlebowski, Christopher and B{\"o}hme, Richard and Freitag, Georg and Fruchard, Bruno and Degraen, Donald},
LANGUAGE = {eng},
ISBN = {979-8-4007-1394-1},
DOI = {10.1145/3706598.3713469},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {CHI '25, CHI Conference on Human Factors in Computing Sytems},
EDITOR = {Yamashita, Naomi and Evers, Vanessa and Yatani, Koji and Ding, Xianghua Sharon and Lee, Bonshin and Chetty, Marshini and Toups-Duas, Phoebe},
PAGES = {1--20},
EID = {1132},
ADDRESS = {Yokohama, Japan},
}

Endnote

%0 Conference Proceedings
%A Wittchen, Dennis
%A Ramian, Alexander
%A Sabnis, Nihar
%A Chlebowski, Christopher
%A B&#246;hme, Richard
%A Freitag, Georg
%A Fruchard, Bruno
%A Degraen, Donald
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T CollabJam: Studying Collaborative Haptic Experience Design for On-Body Vibrotactile Patterns : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D39B-1
%R 10.1145/3706598.3713469
%D 2025
%B CHI Conference on Human Factors in Computing Sytems
%Z date of event: 2025-04-26 - 2025-05-01
%C Yokohama, Japan
%B CHI '25
%E Yamashita, Naomi; Evers, Vanessa; Yatani, Koji; Ding, Xianghua Sharon; Lee, Bonshin; Chetty, Marshini; Toups-Duas, Phoebe
%P 1 - 20
%Z sequence number: 1132
%I ACM
%@ 979-8-4007-1394-1

Conference paper

L. Wu, S. Bi, Z. Xu, H. Tan, K. Zhang, F. Luan, H. Lu, and R. Ramamoorthi

“Neural BRDF Importance Sampling by Reparameterization,” in SIGGRAPH 2025 Conference Papers, Vancouver, Canada, 2025.

@inproceedings{Wu_SIGGRAPH25,
TITLE = {Neural {BRDF} Importance Sampling by Reparameterization},
AUTHOR = {Wu, Liwen and Bi, Sai and Xu, Zexiang and Tan, Hao and Zhang, Kai and Luan, Fujun and Lu, Haolin and Ramamoorthi, Ravi},
LANGUAGE = {eng},
ISBN = {979-8-4007-1540-2},
DOI = {10.1145/3721238.3730679},
PUBLISHER = {ACM},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
DATE = {2025},
BOOKTITLE = {SIGGRAPH 2025 Conference Papers},
EDITOR = {Alford, Ginger and Zhang, Hao Richard and Schulz, Adriana},
PAGES = {1--11},
EID = {163},
ADDRESS = {Vancouver, Canada},
}

Endnote

%0 Conference Proceedings
%A Wu, Liwen
%A Bi, Sai
%A Xu, Zexiang
%A Tan, Hao
%A Zhang, Kai
%A Luan, Fujun
%A Lu, Haolin
%A Ramamoorthi, Ravi
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Neural BRDF Importance Sampling by Reparameterization : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0012-6EE4-0
%R 10.1145/3721238.3730679
%D 2025
%B ACM SIGGRAPH Conference
%Z date of event: 2025-08-10 - 2025-08-14
%C Vancouver, Canada
%B SIGGRAPH 2025 Conference Papers
%E Alford, Ginger; Zhang, Hao Richard; Schulz, Adriana
%P 1 - 11
%Z sequence number: 163
%I ACM
%@ 979-8-4007-1540-2

2024

Article

N. Ansari, V. Babaei, and M. M. Najafpour

“Enhancing Catalysis Studies with Chat Generative Pre-trained Transformer (ChatGPT): Conversation with ChatGPT,” Dalton Transactions, vol. 53, no. 8, 2024.

@article{Ansari24,
TITLE = {Enhancing catalysis studies with chat generative pre-trained transformer ({ChatGPT}): {C}onversation with {ChatGPT}},
AUTHOR = {Ansari, Navid and Babaei, Vahid and Najafpour, Mohammad Mahdi},
LANGUAGE = {eng},
ISSN = {1477-9226},
DOI = {10.1039/d3dt04178f},
PUBLISHER = {Royal Society of Chemistry},
ADDRESS = {Cambridge, UK},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
JOURNAL = {Dalton Transactions},
VOLUME = {53},
NUMBER = {8},
PAGES = {3534--3547},
}

Endnote

%0 Journal Article
%A Ansari, Navid
%A Babaei, Vahid
%A Najafpour, Mohammad Mahdi
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Enhancing Catalysis Studies with Chat Generative Pre-trained Transformer (ChatGPT): Conversation with ChatGPT : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-690B-1
%R 10.1039/d3dt04178f
%7 2024
%D 2024
%J Dalton Transactions
%O Dalton Trans.
%V 53
%N 8
%& 3534
%P 3534 - 3547
%I Royal Society of Chemistry
%C Cambridge, UK
%@ false

Article

U. Çoğalan, M. Bemana, H.-P. Seidel, and K. Myszkowski

“Enhancing Image Quality Prediction with Self-supervised Visual Masking,” Computer Graphics Forum (Proc. EUROGRAPHICS 2024), vol. 43, no. 2, 2024.

@article{cogalan_Eurographics24,
TITLE = {Enhancing Image Quality Prediction with Self-supervised Visual Masking},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur and Bemana, Mojtaba and Seidel, Hans-Peter and Myszkowski, Karol},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.15051},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {43},
NUMBER = {2},
PAGES = {1--12},
EID = {e15051},
BOOKTITLE = {45th Annual Conference of the European Association for Computer Graphics (EUROGRAPHICS 2024)},
EDITOR = {Bermano, A. and Kaolgerakis, E.},
}

Endnote

%0 Journal Article
%A &#199;o&#287;alan, U&#287;ur
%A Bemana, Mojtaba
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Enhancing Image Quality Prediction with Self-supervised Visual Masking
 : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-4DAC-A
%R 10.1111/cgf.15051
%7 2024-04-27
%D 2024
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 43
%N 2
%& 1
%P 1 - 12
%Z sequence number: e15051
%I Blackwell-Wiley
%C Oxford
%@ false
%B 45th Annual Conference of the European Association for Computer Graphics
%O EUROGRAPHICS 2024 EG 2024 Limassol, Cyprus, April 22-26
%U https://onlinelibrary.wiley.com/doi/epdf/10.1111/cgf.15051

Thesis

D4IMPR-CS

U. Çoğalan

“Machine Learning Solutions for High Dynamic Range Video Processing and Image Quality Evaluation,” Universität des Saarlandes, Saarbrücken, 2024.

@phdthesis{ThesisPhDCogalan2024,
TITLE = {Machine Learning Solutions for High Dynamic Range Video Processing and Image Quality Evaluation},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-433804},
DOI = {10.22028/D291-43380},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
}

Endnote

%0 Thesis
%A &#199;o&#287;alan, U&#287;ur
%Y Myszkowski, Karol
%A referee: Seidel, Hans-Peter
%A referee: Serrano, Ana
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Machine Learning Solutions for High Dynamic Range Video Processing and Image Quality Evaluation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-7AF9-D
%R 10.22028/D291-43380
%U urn:nbn:de:bsz:291--ds-433804
%F OTHER: hdl:20.500.11880/39031
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2024
%P X, 101 p.
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/39031

Conference paper

D4D6

J. K. Didion, K. Wolski, D. Wittchen, D. Coyle, T. Leimkühler, and P. Strohmeier

“Who did it? How User Agency is influenced by Visual Properties of Generated Images,” in UIST ’24, 37th Annual ACM Symposium on User Interface Software and Technology, Pittsburgh, PA, USA, 2024.

@inproceedings{Didion_UIST24,
TITLE = {Who did it? {H}ow User Agency is influenced by Visual Properties of Generated Images},
AUTHOR = {Didion, Johanna Katharina and Wolski, Krzysztof and Wittchen, Dennis and Coyle, David and Leimk{\"u}hler, Thomas and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-0628-8},
DOI = {10.1145/3654777.3676335},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {UIST '24, 37th Annual ACM Symposium on User Interface Software and Technology},
EDITOR = {Yao, Lining and Goel, Mayank and Ion, Alexandra and Lopes, Pedro},
PAGES = {1--17},
EID = {94},
ADDRESS = {Pittsburgh, PA, USA},
}

Endnote

%0 Conference Proceedings
%A Didion, Johanna Katharina
%A Wolski, Krzysztof
%A Wittchen, Dennis
%A Coyle, David
%A Leimk&#252;hler, Thomas
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Who did it? How User Agency is influenced by Visual Properties of Generated Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-106C-3
%R 10.1145/3654777.3676335
%D 2024
%B 37th Annual ACM Symposium on User Interface Software and
Technology
%Z date of event: 2024-10-13 - 2024-10-16
%C Pittsburgh, PA, USA
%B UIST '24
%E Yao, Lining; Goel, Mayank; Ion, Alexandra; Lopes, Pedro
%P 1 - 17
%Z sequence number: 94
%I ACM
%@ 979-8-4007-0628-8

Conference paper

Y. Ding, N. Sabnis, and P. Strohmeier

“Motionless Movement: Towards Vibrotactile Kinesthetic Displays,” in CHI ’24, CHI Conference on Human Factors in Computing Sytems, Honolulu, HI, USA, 2024.

@inproceedings{Ding_CHI24,
TITLE = {Motionless Movement: {T}owards Vibrotactile Kinesthetic Displays},
AUTHOR = {Ding, Yuran and Sabnis, Nihar and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-0330-0},
DOI = {10.1145/3613904.3642499},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {CHI '24, CHI Conference on Human Factors in Computing Sytems},
EDITOR = {Mueller, Florian Floyd and Kyburz, Penny and Williamson, Julie R. and Sas, Corina and Wilson, Max L. and Dugas, Phoebe Toups and Shklovski, Irina},
PAGES = {1--16},
EID = {418},
ADDRESS = {Honolulu, HI, USA},
}

Endnote

%0 Conference Proceedings
%A Ding, Yuran
%A Sabnis, Nihar
%A Strohmeier, Paul
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Motionless Movement: Towards Vibrotactile Kinesthetic Displays : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-F1D0-5
%R 10.1145/3613904.3642499
%D 2024
%B CHI Conference on Human Factors in Computing Sytems
%Z date of event: 2024-05-11 - 2024-05-16
%C Honolulu, HI, USA
%B CHI '24
%E Mueller, Florian Floyd; Kyburz, Penny; Williamson, Julie R.; Sas, Corina; Wilson, Max L.; Dugas, Phoebe Toups; Shklovski, Irina
%P 1 - 16
%Z sequence number: 418
%I ACM
%@ 979-8-4007-0330-0

Conference paper

Y. Ding, N. Sabnis, and P. Strohmeier

“Motionless Movement: Towards Vibrotactile Kinesthetic Displays,” in Demo EUROHAPTICS 2024, Lille, France, 2024.

@inproceedings{motionlessMovement_demo,
TITLE = {Motionless Movement: {T}owards Vibrotactile Kinesthetic Displays},
AUTHOR = {Ding, Yuran and Sabnis, Nihar and Strohmeier, Paul},
LANGUAGE = {eng},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {Demo EUROHAPTICS 2024},
ADDRESS = {Lille, France},
}

Endnote

%0 Conference Proceedings
%A Ding, Yuran
%A Sabnis, Nihar
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Motionless Movement: Towards Vibrotactile Kinesthetic Displays : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-E948-7
%D 2024
%B Eurohaptics Conference
%Z date of event: 2024-06-30 - 2024-07-03
%C Lille, France
%B Demo EUROHAPTICS 2024
%U https://www.researchgate.net/publication/387485077_Demonstrating_Motionless_Movement_Towards_Vibrotactile_Kinesthetic_Displays

Conference paper

K. Fabian, D. Wittchen, and P. Strohmeier

“3D-Printed Cells for Creating Variable Softness,” in TEI ’24, 18th International Conference on Tangible, Embedded and Embodied Interaction, Cork, Ireland, 2024.

@inproceedings{FabianTEI24,
TITLE = {3{D}-Printed Cells for Creating Variable Softness},
AUTHOR = {Fabian, Konrad and Wittchen, Dennis and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-0402-4},
DOI = {10.1145/3623509.3635249},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {TEI '24, 18th International Conference on Tangible, Embedded and Embodied Interaction},
PAGES = {1--7},
EID = {69},
ADDRESS = {Cork, Ireland},
}

Endnote

%0 Conference Proceedings
%A Fabian, Konrad
%A Wittchen, Dennis
%A Strohmeier, Paul
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T 3D-Printed Cells for Creating Variable Softness : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8182-8
%R 10.1145/3623509.3635249
%D 2024
%B 18th International Conference on Tangible, Embedded and Embodied Interaction
%Z date of event: 2024-02-11 - 2024-02-14
%C Cork, Ireland
%B TEI '24
%P 1 - 7
%Z sequence number: 69
%I ACM
%@ 979-8-4007-0402-4

Conference paper

R. Freire, V. Martinez-Missir, C. N. Reed, and P. Strohmeier

“Liminal Space: A Performance with RaveNET,” in TEI ’24, 18th International Conference on Tangible, Embedded and Embodied Interaction, Cork, Ireland, 2024.

@inproceedings{FreireTEI24a,
TITLE = {Liminal Space: {A} Performance with {RaveNET}},
AUTHOR = {Freire, Rachel and Martinez-Missir, Valentin and Reed, Courtney N. and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-0402-4},
DOI = {10.1145/3623509.3635337},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {TEI '24, 18th International Conference on Tangible, Embedded and Embodied Interaction},
PAGES = {1--4},
EID = {105},
ADDRESS = {Cork, Ireland},
}

Endnote

%0 Conference Proceedings
%A Freire, Rachel
%A Martinez-Missir, Valentin
%A Reed, Courtney N.
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Liminal Space: A Performance with RaveNET : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-818A-0
%R 10.1145/3623509.3635337
%D 2024
%B 18th International Conference on Tangible, Embedded and Embodied Interaction
%Z date of event: 2024-02-11 - 2024-02-14
%C Cork, Ireland
%B TEI '24
%P 1 - 4
%Z sequence number: 105
%I ACM
%@ 979-8-4007-0402-4

Conference paper

R. Freire, V. Martinez-Missir, C. N. Reed, and P. Strohmeier

“RaveNET: Connecting People and Exploring Liminal Space through Wearable Networks in Music Performance,” in TEI ’24, 18th International Conference on Tangible, Embedded and Embodied Interaction, Cork, Ireland, 2024.

@inproceedings{FreireTEI24,
TITLE = {{RaveNET}: {C}onnecting People and Exploring Liminal Space through Wearable Networks in Music Performance},
AUTHOR = {Freire, Rachel and Martinez-Missir, Valentin and Reed, Courtney N. and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-0402-4},
DOI = {10.1145/3623509.3635270},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {TEI '24, 18th International Conference on Tangible, Embedded and Embodied Interaction},
PAGES = {1--8},
EID = {89},
ADDRESS = {Cork, Ireland},
}

Endnote

%0 Conference Proceedings
%A Freire, Rachel
%A Martinez-Missir, Valentin
%A Reed, Courtney N.
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T RaveNET: Connecting People and Exploring Liminal Space through Wearable Networks in Music Performance : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8187-3
%R 10.1145/3623509.3635270
%D 2024
%B 18th International Conference on Tangible, Embedded and Embodied Interaction
%Z date of event: 2024-02-11 - 2024-02-14
%C Cork, Ireland
%B TEI '24
%P 1 - 8
%Z sequence number: 89
%I ACM
%@ 979-8-4007-0402-4

Conference paper

D4D6

X. Huang, C. Salaün, C. Vasconcelos, C. Theobalt, C. Oztireli, and G. Singh

“Blue Noise for Diffusion Models,” in Proceedings SIGGRAPH 2024 Conference Papers, Denver, CO, USA, 2024.

@inproceedings{Huang_SIGGRAPH24,
TITLE = {Blue Noise for Diffusion Models},
AUTHOR = {Huang, Xingchang and Sala{\"u}n, Corentin and Vasconcelos, Cristina and Theobalt, Christian and Oztireli, Cengiz and Singh, Gurprit},
LANGUAGE = {eng},
ISBN = {979-8-4007-0525-0},
DOI = {10.1145/3641519.3657435},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {Proceedings SIGGRAPH 2024 Conference Papers},
EDITOR = {Burgano, Andres and Zorin, Denis and Jarosz, Wojciech},
PAGES = {1--11},
EID = {28},
ADDRESS = {Denver, CO, USA},
}

Endnote

%0 Conference Proceedings
%A Huang, Xingchang
%A Sala&#252;n, Corentin
%A Vasconcelos, Cristina
%A Theobalt, Christian
%A Oztireli, Cengiz
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Blue Noise for Diffusion Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-EEB4-A
%R 10.1145/3641519.3657435
%D 2024
%B ACM SIGGRAPH Conference
%Z date of event: 2024-07-28 - 2024-08-01
%C Denver, CO, USA
%B Proceedings SIGGRAPH 2024 Conference Papers
%E Burgano, Andres; Zorin, Denis; Jarosz, Wojciech
%P 1 - 11
%Z sequence number: 28
%I ACM
%@ 979-8-4007-0525-0

Conference paper

D. Jiménez Navarro, X. Peng, Y. Zhang, K. Myszkowski, H.-P. Seidel, Q. Sun, and A. Serrano

“Accelerating Saccadic Response through Spatial and Temporal Cross-Modal Misalignments,” in Proceedings SIGGRAPH 2024 Conference Papers, Denver, CO, USA, 2024.

@inproceedings{Navarro_SIGGRAPH24,
TITLE = {Accelerating Saccadic Response through Spatial and Temporal Cross-Modal Misalignments},
AUTHOR = {Jim{\'e}nez Navarro, Daniel and Peng, Xi and Zhang, Yunxiang and Myszkowski, Karol and Seidel, Hans-Peter and Sun, Qi and Serrano, Ana},
LANGUAGE = {eng},
ISBN = {979-8-4007-0525-0},
DOI = {10.1145/3641519.3657432},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {Proceedings SIGGRAPH 2024 Conference Papers},
EDITOR = {Burgano, Andres and Zorin, Denis and Jarosz, Wojciech},
PAGES = {1--12},
EID = {129},
ADDRESS = {Denver, CO, USA},
}

Endnote

%0 Conference Proceedings
%A Jim&#233;nez Navarro, Daniel
%A Peng, Xi
%A Zhang, Yunxiang
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Sun, Qi
%A Serrano, Ana
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Accelerating Saccadic Response through Spatial and Temporal Cross-Modal Misalignments : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-FC10-3
%R 10.1145/3641519.3657432
%D 2024
%B ACM SIGGRAPH Conference
%Z date of event: 2024-07-28 - 2024-08-01
%C Denver, CO, USA
%B Proceedings SIGGRAPH 2024 Conference Papers
%E Burgano, Andres; Zorin, Denis; Jarosz, Wojciech
%P 1 - 12
%Z sequence number: 129
%I ACM
%@ 979-8-4007-0525-0

Conference paper

A. Jingu, N. Sabnis, P. Strohmeier, and J. Steimle

“Shaping Compliance: Inducing Haptic Illusion of Compliance in Different Shapes with Electrotactile Grains,” in Demo EUROHAPTICS 2024, Lille, France, 2024.

@inproceedings{touchShape_demo,
TITLE = {Shaping Compliance: {I}nducing Haptic Illusion of Compliance in Different Shapes with Electrotactile Grains},
AUTHOR = {Jingu, Arata and Sabnis, Nihar and Strohmeier, Paul and Steimle, J{\"u}rgen},
LANGUAGE = {eng},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {Demo EUROHAPTICS 2024},
ADDRESS = {Lille, France},
}

Endnote

%0 Conference Proceedings
%A Jingu, Arata
%A Sabnis, Nihar
%A Strohmeier, Paul
%A Steimle, J&#252;rgen
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Shaping Compliance: Inducing Haptic Illusion of Compliance in Different Shapes with Electrotactile Grains : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D3F1-F
%D 2024
%B Eurohaptics Conference
%Z date of event: 2024-06-30 - 2024-07-03
%C Lille, France
%B Demo EUROHAPTICS 2024
%U https://www.researchgate.net/publication/387485195_Damonstration_of_Shaping_Compliance_Haptic_Illusion_of_Compliance_using_Electrotactile_Grains

Conference paper

A. Jingu, N. Sabnis, P. Strohmeier, and J. Steimle

“Shaping Compliance: Inducing Haptic Illusion of Compliance in Different Shapes with Electrotactile Grains,” in CHI ’24, CHI Conference on Human Factors in Computing Sytems, Honolulu, HI, USA, 2024.

@inproceedings{Jingu_CHI24,
TITLE = {Shaping Compliance: {I}nducing Haptic Illusion of Compliance in Different Shapes with Electrotactile Grains},
AUTHOR = {Jingu, Arata and Sabnis, Nihar and Strohmeier, Paul and Steimle, J{\"u}rgen},
LANGUAGE = {eng},
ISBN = {979-8-4007-0330-0},
DOI = {10.1145/3613904.36419},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {CHI '24, CHI Conference on Human Factors in Computing Sytems},
EDITOR = {Mueller, Florian Floyd and Kyburz, Penny and Williamson, Julie R. and Sas, Corina and Wilson, Max L. and Dugas, Phoebe Toups and Shklovski, Irina},
PAGES = {1--13},
EID = {432},
ADDRESS = {Honolulu, HI, USA},
}

Endnote

%0 Conference Proceedings
%A Jingu, Arata
%A Sabnis, Nihar
%A Strohmeier, Paul
%A Steimle, J&#252;rgen
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Shaping Compliance: Inducing Haptic Illusion of Compliance in Different Shapes with Electrotactile Grains : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-F1DB-A
%R 10.1145/3613904.36419
%D 2024
%B CHI Conference on Human Factors in Computing Sytems
%Z date of event: 2024-05-11 - 2024-05-16
%C Honolulu, HI, USA
%B CHI '24
%E Mueller, Florian Floyd; Kyburz, Penny; Williamson, Julie R.; Sas, Corina; Wilson, Max L.; Dugas, Phoebe Toups; Shklovski, Irina
%P 1 - 13
%Z sequence number: 432
%I ACM
%@ 979-8-4007-0330-0

Conference paper

E. Luci, F. Pellacini, and V. Babaei

“Differentiable Modeling of Material Spreading in Inkjet Printing for Appearance Prediction,” in Proceedings SIGGRAPH Asia 2024 Conference Papers, Tokyo, Japan, 2024.

@inproceedings{LuciSA24,
TITLE = {Differentiable Modeling of Material Spreading in Inkjet Printing for Appearance Prediction},
AUTHOR = {Luci, Emiliano and Pellacini, Fabio and Babaei, Vahid},
LANGUAGE = {eng},
ISBN = {979-8-4007-1131-2},
DOI = {10.1145/3680528.3687598},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {Proceedings SIGGRAPH Asia 2024 Conference Papers},
PAGES = {1--10},
EID = {72},
ADDRESS = {Tokyo, Japan},
}

Endnote

%0 Conference Proceedings
%A Luci, Emiliano
%A Pellacini, Fabio
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Differentiable Modeling of Material Spreading in Inkjet Printing for Appearance Prediction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-D788-5
%R 10.1145/3680528.3687598
%D 2024
%B 17th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia
%Z date of event: 2024-12-03 - 2024-12-06
%C Tokyo, Japan
%B Proceedings SIGGRAPH Asia 2024 Conference Papers
%P 1 - 10
%Z sequence number: 72
%I ACM
%@ 979-8-4007-1131-2

Article

D4D6

F. Mujkanovic, N. E. Nsampi, C. Theobalt, H.-P. Seidel, and T. Leimkühler

“Neural Gaussian Scale-Space Fields,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2024), vol. 43, no. 4, 2024.

@article{Mujkanovic24,
TITLE = {Neural {G}aussian Scale-Space Fields},
AUTHOR = {Mujkanovic, Felix and Nsampi, Ntumba Elie and Theobalt, Christian and Seidel, Hans-Peter and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3658163},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {43},
NUMBER = {4},
PAGES = {1--15},
EID = {134},
BOOKTITLE = {Proceedings of the SIGGRAPH Conference (ACM SIGGRAPH 2024)},
}

Endnote

%0 Journal Article
%A Mujkanovic, Felix
%A Nsampi, Ntumba Elie
%A Theobalt, Christian
%A Seidel, Hans-Peter
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Neural Gaussian Scale-Space Fields : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-EE1D-6
%R 10.1145/3658163
%7 2024-07-19
%D 2024
%J ACM Transactions on Graphics
%V 43
%N 4
%& 1
%P 1 - 15
%Z sequence number: 134
%I ACM
%C New York, NY
%@ false
%B Proceedings of the SIGGRAPH Conference
%O ACM SIGGRAPH 2024 Denver, CO, USA, July 28 - Aug 1

Article

X. Peng, Y. Zhang, D. Jiménez Navarro, A. Serrano, K. Myszkowski, and Q. Sun

“Measuring and Predicting Multisensory Reaction Latency: A Probabilistic Model for Visual-Auditory Integration,” IEEE Transactions on Visualization and Computer Graphics, vol. 30, no. 11, 2024.

@article{Peng_TVCG24,
TITLE = {Measuring and Predicting Multisensory Reaction Latency: {A} Probabilistic Model for Visual-Auditory Integration},
AUTHOR = {Peng, Xi and Zhang, Yunxiang and Jim{\'e}nez Navarro, Daniel and Serrano, Ana and Myszkowski, Karol and Sun, Qi},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2024.3456185},
PUBLISHER = {IEEE Computer Society},
ADDRESS = {New York, NY},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics},
VOLUME = {30},
NUMBER = {11},
PAGES = {7364--7374},
}

Endnote

%0 Journal Article
%A Peng, Xi
%A Zhang, Yunxiang
%A Jim&#233;nez Navarro, Daniel
%A Serrano, Ana
%A Myszkowski, Karol
%A Sun, Qi
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Measuring and Predicting Multisensory Reaction Latency: A Probabilistic Model for Visual-Auditory Integration : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-FC2E-3
%R 10.1109/TVCG.2024.3456185
%7 2024-09-09
%D 2024
%J IEEE Transactions on Visualization and Computer Graphics
%V 30
%N 11
%& 7364
%P 7364 - 7374
%I IEEE Computer Society
%C New York, NY
%@ false

Conference paper

V. Rallabandi, A. C. Haynes, C. N. Reed, and P. Strohmeier

“Base and Stitch: Evaluating eTextile Interfaces from a Material-Centric View,” in TEI ’24, 18th International Conference on Tangible, Embedded and Embodied Interaction, Cork, Ireland, 2024.

@inproceedings{RallabandiTEI24,
TITLE = {Base and Stitch: {E}valuating eTextile Interfaces from a Material-Centric View},
AUTHOR = {Rallabandi, Vineetha and Haynes, Alice C and Reed, Courtney N. and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-0402-4},
DOI = {10.1145/3623509.3633363},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {TEI '24, 18th International Conference on Tangible, Embedded and Embodied Interaction},
PAGES = {1--13},
EID = {15},
ADDRESS = {Cork, Ireland},
}

Endnote

%0 Conference Proceedings
%A Rallabandi, Vineetha
%A Haynes, Alice C
%A Reed, Courtney N.
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Base and Stitch: Evaluating eTextile Interfaces from a Material-Centric View : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-817B-2
%R 10.1145/3623509.3633363
%D 2024
%B 18th International Conference on Tangible, Embedded and Embodied Interaction
%Z date of event: 2024-02-11 - 2024-02-14
%C Cork, Ireland
%B TEI '24
%P 1 - 13
%Z sequence number: 15
%I ACM
%@ 979-8-4007-0402-4

Conference paper

L. Ruan, M. Bálint, M. Bemana, K. Wolski, H.-P. Seidel, K. Myszkowski, and B. Chen

“Self-Supervised Video Defocus Deblurring with Atlas Learning,” in Proceedings SIGGRAPH 2024 Conference Papers, Denver, CO, USA, 2024.

@inproceedings{Ruan_SIGGRAPH24,
TITLE = {Self-Supervised Video Defocus Deblurring with Atlas Learning},
AUTHOR = {Ruan, Lingyan and B{\'a}lint, Martin and Bemana, Mojtaba and Wolski, Krzysztof and Seidel, Hans-Peter and Myszkowski, Karol and Chen, Bin},
LANGUAGE = {eng},
ISBN = {979-8-4007-0525-0},
DOI = {10.1145/3641519.3657524},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {Proceedings SIGGRAPH 2024 Conference Papers},
EDITOR = {Burgano, Andres and Zorin, Denis and Jarosz, Wojciech},
PAGES = {1--11},
EID = {120},
ADDRESS = {Denver, CO, USA},
}

Endnote

%0 Conference Proceedings
%A Ruan, Lingyan
%A B&#225;lint, Martin
%A Bemana, Mojtaba
%A Wolski, Krzysztof
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%A Chen, Bin
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Self-Supervised Video Defocus Deblurring with Atlas Learning : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-FC3F-0
%R 10.1145/3641519.3657524
%D 2024
%B ACM SIGGRAPH Conference
%Z date of event: 2024-07-28 - 2024-08-01
%C Denver, CO, USA
%B Proceedings SIGGRAPH 2024 Conference Papers
%E Burgano, Andres; Zorin, Denis; Jarosz, Wojciech
%P 1 - 11
%Z sequence number: 120
%I ACM
%@ 979-8-4007-0525-0

Conference paper

N. Sabnis and T. Nagashima

“Empowering Learners: Chatbot-Mediated ‘Learning-by-Teaching,’” in CHI ’24, CHI Conference on Human Factors in Computing Sytems, Honolulu, HI, USA, 2024.

@inproceedings{Sabnis_CHI24,
TITLE = {Empowering Learners: {C}hatbot-Mediated 'Learning-by-Teaching'},
AUTHOR = {Sabnis, Nihar and Nagashima, Tomohiro},
LANGUAGE = {eng},
ISBN = {979-8-4007-0331-7},
DOI = {10.1145/3613905.3650754},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {CHI '24, CHI Conference on Human Factors in Computing Sytems},
EDITOR = {Mueller, Florian Floyd and Kyburz, Penny and Williamson, Julie R. and Sas, Corina},
PAGES = {1--9},
EID = {122},
ADDRESS = {Honolulu, HI, USA},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Nagashima, Tomohiro
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Empowering Learners: Chatbot-Mediated 'Learning-by-Teaching'  : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-FCD5-5
%R 10.1145/3613905.3650754
%D 2024
%B CHI Conference on Human Factors in Computing Sytems
%Z date of event: 2024-05-11 - 2024-05-16
%C Honolulu, HI, USA
%B CHI '24
%E Mueller, Florian Floyd; Kyburz, Penny; Williamson, Julie R.; Sas, Corina
%P 1 - 9
%Z sequence number: 122
%I ACM
%@ 979-8-4007-0331-7

Conference paper

N. Sabnis, D. Wittchen, G. Vega, C. N. Reed, and P. Strohmeier

“Design and Experience Tactile Symbols using Continuous and Motion-Coupled Vibration,” in Demo EUROHAPTICS 2024, Lille, France, 2024.

@inproceedings{TactileSymbols_demo,
TITLE = {Design and Experience Tactile Symbols using Continuous and Motion-Coupled Vibration},
AUTHOR = {Sabnis, Nihar and Wittchen, Dennis and Vega, Gabriela and Reed, Courtney N. and Strohmeier, Paul},
LANGUAGE = {eng},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {Demo EUROHAPTICS 2024},
ADDRESS = {Lille, France},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Wittchen, Dennis
%A Vega, Gabriela
%A Reed, Courtney N.
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Design and Experience Tactile Symbols using Continuous and Motion-Coupled Vibration : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D3E8-A
%D 2024
%B Eurohaptics Conference
%Z date of event: 2024-06-30 - 2024-07-03
%C Lille, France
%B Demo EUROHAPTICS 2024
%U https://www.researchgate.net/publication/387485081_Design_and_Experience_Tactile_Symbols_using_Continuous_and_Motion-Coupled_Vibration

Conference paper

G. Singh and J. Wenzel

“MCMC: Bridging Rendering, Optimization and Generative AI,” in Proceedings SIGGRAPH Asia 2024 Courses, Tokyo, Japan, 2024.

@inproceedings{Gurprit_SIGGRAPHAsia24,
TITLE = {{MCMC}: {B}ridging Rendering, Optimization and Generative {AI}},
AUTHOR = {Singh, Gurprit and Wenzel, Jakob},
LANGUAGE = {eng},
ISBN = {979-8-4007-1135-0},
DOI = {10.1145/3680532.3689592},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {Proceedings SIGGRAPH Asia 2024 Courses},
EDITOR = {Igarashi, Takeo and Huang, Hui},
PAGES = {1--27},
EID = {8},
ADDRESS = {Tokyo, Japan},
}

Endnote

%0 Conference Proceedings
%A Singh, Gurprit
%A Wenzel, Jakob
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T MCMC: Bridging Rendering, Optimization and Generative AI : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-EC4A-2
%R 10.1145/3680532.3689592
%D 2024
%B 17th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia
%Z date of event: 2024-12-03 - 2024-12-06
%C Tokyo, Japan
%B Proceedings SIGGRAPH Asia 2024 Courses
%E Igarashi, Takeo; Huang, Hui
%P 1 - 27
%Z sequence number: 8
%I ACM
%@ 979-8-4007-1135-0

Thesis

D4IMPR-CSD6

E. Tretschk

“Representing and Reconstructing General Non-rigid Objects with Neural Models,” Universität des Saarlandes, Saarbrücken, 2024.

@phdthesis{Tretschk_PhD2024,
TITLE = {Representing and Reconstructing General Non-rigid Objects with Neural Models},
AUTHOR = {Tretschk, Edith},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-416509},
DOI = {doi:10.22028/D291-41650},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
}

Endnote

%0 Thesis
%A Tretschk, Edith
%Y Theobalt, Christian
%A referee: Seidel, Hans-Peter
%A referee: Agapito, Lourdes
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Representing and Reconstructing General Non-rigid Objects with Neural Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-2406-2
%R doi:10.22028/D291-41650
%U urn:nbn:de:bsz:291--ds-416509
%F OTHER: hdl:20.500.11880/37392
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2024
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/37392

Paper

J. Vandersanden, S. Holl, X. Huang, and G. Singh

“Edge-Preserving Noise for Diffusion Models,” 2024. [Online]. Available: https://arxiv.org/abs/2410.01540.

Abstract

Classical generative diffusion models learn an isotropic Gaussian denoising
process, treating all spatial regions uniformly, thus neglecting potentially
valuable structural information in the data. Inspired by the long-established
work on anisotropic diffusion in image processing, we present a novel
edge-preserving diffusion model that is a generalization of denoising diffusion
probablistic models (DDPM). In particular, we introduce an edge-aware noise
scheduler that varies between edge-preserving and isotropic Gaussian noise. We
show that our model's generative process converges faster to results that more
closely match the target distribution. We demonstrate its capability to better
learn the low-to-mid frequencies within the dataset, which plays a crucial role
in representing shapes and structural information. Our edge-preserving
diffusion process consistently outperforms state-of-the-art baselines in
unconditional image generation. It is also more robust for generative tasks
guided by a shape-based prior, such as stroke-to-image generation. We present
qualitative and quantitative results showing consistent improvements (FID
score) of up to 30% for both tasks. We provide source code and supplementary
content via the public domain edge-preserving-diffusion.mpi-inf.mpg.de .

BibTeX

@online{Vandersanden2410.01540,
TITLE = {Edge-Preserving Noise for Diffusion Models},
AUTHOR = {Vandersanden, Jente and Holl, Sascha and Huang, Xingchang and Singh, Gurprit},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2410.01540},
EPRINT = {2410.01540},
EPRINTTYPE = {arXiv},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Classical generative diffusion models learn an isotropic Gaussian denoising<br>process, treating all spatial regions uniformly, thus neglecting potentially<br>valuable structural information in the data. Inspired by the long-established<br>work on anisotropic diffusion in image processing, we present a novel<br>edge-preserving diffusion model that is a generalization of denoising diffusion<br>probablistic models (DDPM). In particular, we introduce an edge-aware noise<br>scheduler that varies between edge-preserving and isotropic Gaussian noise. We<br>show that our model's generative process converges faster to results that more<br>closely match the target distribution. We demonstrate its capability to better<br>learn the low-to-mid frequencies within the dataset, which plays a crucial role<br>in representing shapes and structural information. Our edge-preserving<br>diffusion process consistently outperforms state-of-the-art baselines in<br>unconditional image generation. It is also more robust for generative tasks<br>guided by a shape-based prior, such as stroke-to-image generation. We present<br>qualitative and quantitative results showing consistent improvements (FID<br>score) of up to 30% for both tasks. We provide source code and supplementary<br>content via the public domain edge-preserving-diffusion.mpi-inf.mpg.de .<br>},
}

Endnote

%0 Report
%A Vandersanden, Jente
%A Holl, Sascha
%A Huang, Xingchang
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Edge-Preserving Noise for Diffusion Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8AF7-C
%U https://arxiv.org/abs/2410.01540
%D 2024
%X   Classical generative diffusion models learn an isotropic Gaussian denoising<br>process, treating all spatial regions uniformly, thus neglecting potentially<br>valuable structural information in the data. Inspired by the long-established<br>work on anisotropic diffusion in image processing, we present a novel<br>edge-preserving diffusion model that is a generalization of denoising diffusion<br>probablistic models (DDPM). In particular, we introduce an edge-aware noise<br>scheduler that varies between edge-preserving and isotropic Gaussian noise. We<br>show that our model's generative process converges faster to results that more<br>closely match the target distribution. We demonstrate its capability to better<br>learn the low-to-mid frequencies within the dataset, which plays a crucial role<br>in representing shapes and structural information. Our edge-preserving<br>diffusion process consistently outperforms state-of-the-art baselines in<br>unconditional image generation. It is also more robust for generative tasks<br>guided by a shape-based prior, such as stroke-to-image generation. We present<br>qualitative and quantitative results showing consistent improvements (FID<br>score) of up to 30% for both tasks. We provide source code and supplementary<br>content via the public domain edge-preserving-diffusion.mpi-inf.mpg.de .<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Conference paper

G. Vega, V. Martinez-Missir, D. Wittchen, N. Sabnis, K. Cochrane, A. Girouard, and P. Strohmeier

“vARitouch: Back of the Finger Device for Adding Variable Compliance to Rigid Objects,” in Demo EUROHAPTICS 2024, Lille, France, 2024.

@inproceedings{variTouch_demo,
TITLE = {{vARitouch}: {B}ack of the Finger Device for Adding Variable Compliance to Rigid Objects},
AUTHOR = {Vega, Gabriela and Martinez-Missir, Valentin and Wittchen, Dennis and Sabnis, Nihar and Cochrane, Karen and Girouard, Audrey and Strohmeier, Paul},
LANGUAGE = {eng},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {Demo EUROHAPTICS 2024},
ADDRESS = {Lille, France},
}

Endnote

%0 Conference Proceedings
%A Vega, Gabriela
%A Martinez-Missir, Valentin
%A Wittchen, Dennis
%A Sabnis, Nihar
%A Cochrane, Karen
%A Girouard, Audrey
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T vARitouch: Back of the Finger Device for Adding Variable Compliance to Rigid Objects : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-D3EC-6
%D 2024
%B Eurohaptics Conference
%Z date of event: 2024-06-30 - 2024-07-03
%C Lille, France
%B Demo EUROHAPTICS 2024
%U https://www.researchgate.net/publication/387485191_Demo_of_vARitouch_Back_of_the_Finger_Device_for_Adding_Variable_Compliance_to_Rigid_Objects

Conference paper

G. Vega, V. Martinez-Missir, D. Wittchen, N. Sabnis, A. Girouard, K. A. Cochrane, and P. Strohmeier

“vARitouch: Back of the Finger Device for Adding Variable Compliance to Rigid Objects,” in CHI ’24, CHI Conference on Human Factors in Computing Sytems, Honolulu, HI, USA, 2024.

@inproceedings{Vega_CHI24,
TITLE = {{vARitouch}: {B}ack of the Finger Device for Adding Variable Compliance to Rigid Objects},
AUTHOR = {Vega, Gabriela and Martinez-Missir, Valentin and Wittchen, Dennis and Sabnis, Nihar and Girouard, Audrey and Cochrane, Karen Anne and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {979-8-4007-0330-0},
DOI = {10.1145/3613904.3642828},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {CHI '24, CHI Conference on Human Factors in Computing Sytems},
EDITOR = {Mueller, Florian Floyd and Kyburz, Penny and Williamson, Julie R. and Sas, Corina and Wilson, Max L. and Dugas, Phoebe Toups and Shklovski, Irina},
PAGES = {1--20},
EID = {414},
ADDRESS = {Honolulu, HI, USA},
}

Endnote

%0 Conference Proceedings
%A Vega, Gabriela
%A Martinez-Missir, Valentin
%A Wittchen, Dennis
%A Sabnis, Nihar
%A Girouard, Audrey
%A Cochrane, Karen Anne
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T vARitouch: Back of the Finger Device for Adding Variable Compliance to Rigid Objects : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-F1E2-1
%R 10.1145/3613904.3642828
%D 2024
%B CHI Conference on Human Factors in Computing Sytems
%Z date of event: 2024-05-11 - 2024-05-16
%C Honolulu, HI, USA
%B CHI '24
%E Mueller, Florian Floyd; Kyburz, Penny; Williamson, Julie R.; Sas, Corina; Wilson, Max L.; Dugas, Phoebe Toups; Shklovski, Irina
%P 1 - 20
%Z sequence number: 414
%I ACM
%@ 979-8-4007-0330-0

Paper

S. R. Vutukur, M. Ba, B. Busam, M. Kayser, and G. Singh

“SABER-6D: Shape Representation Based Implicit Object Pose Estimation,” 2024. [Online]. Available: https://arxiv.org/abs/2408.05867.

Abstract

In this paper, we propose a novel encoder-decoder architecture, named SABER,
to learn the 6D pose of the object in the embedding space by learning shape
representation at a given pose. This model enables us to learn pose by
performing shape representation at a target pose from RGB image input. We
perform shape representation as an auxiliary task which helps us in learning
rotations space for an object based on 2D images. An image encoder predicts the
rotation in the embedding space and the DeepSDF based decoder learns to
represent the object's shape at the given pose. As our approach is shape based,
the pipeline is suitable for any type of object irrespective of the symmetry.
Moreover, we need only a CAD model of the objects to train SABER. Our pipeline
is synthetic data based and can also handle symmetric objects without symmetry
labels and, thus, no additional labeled training data is needed. The
experimental evaluation shows that our method achieves close to benchmark
results for both symmetric objects and asymmetric objects on Occlusion-LineMOD,
and T-LESS datasets.

BibTeX

@online{Vutukur2408.05867,
TITLE = {{SABER}-{6D}: Shape Representation Based Implicit Object Pose Estimation},
AUTHOR = {Vutukur, Shishir Reddy and Ba, Mengkejiergeli and Busam, Benjamin and Kayser, Matthias and Singh, Gurprit},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2408.05867},
EPRINT = {2408.05867},
EPRINTTYPE = {arXiv},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
ABSTRACT = {In this paper, we propose a novel encoder-decoder architecture, named SABER,<br>to learn the 6D pose of the object in the embedding space by learning shape<br>representation at a given pose. This model enables us to learn pose by<br>performing shape representation at a target pose from RGB image input. We<br>perform shape representation as an auxiliary task which helps us in learning<br>rotations space for an object based on 2D images. An image encoder predicts the<br>rotation in the embedding space and the DeepSDF based decoder learns to<br>represent the object's shape at the given pose. As our approach is shape based,<br>the pipeline is suitable for any type of object irrespective of the symmetry.<br>Moreover, we need only a CAD model of the objects to train SABER. Our pipeline<br>is synthetic data based and can also handle symmetric objects without symmetry<br>labels and, thus, no additional labeled training data is needed. The<br>experimental evaluation shows that our method achieves close to benchmark<br>results for both symmetric objects and asymmetric objects on Occlusion-LineMOD,<br>and T-LESS datasets.<br>},
}

Endnote

%0 Report
%A Vutukur, Shishir Reddy
%A Ba, Mengkejiergeli
%A Busam, Benjamin
%A Kayser, Matthias
%A Singh, Gurprit
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T SABER-6D: Shape Representation Based Implicit Object Pose Estimation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8AF4-F
%U https://arxiv.org/abs/2408.05867
%D 2024
%X   In this paper, we propose a novel encoder-decoder architecture, named SABER,<br>to learn the 6D pose of the object in the embedding space by learning shape<br>representation at a given pose. This model enables us to learn pose by<br>performing shape representation at a target pose from RGB image input. We<br>perform shape representation as an auxiliary task which helps us in learning<br>rotations space for an object based on 2D images. An image encoder predicts the<br>rotation in the embedding space and the DeepSDF based decoder learns to<br>represent the object's shape at the given pose. As our approach is shape based,<br>the pipeline is suitable for any type of object irrespective of the symmetry.<br>Moreover, we need only a CAD model of the objects to train SABER. Our pipeline<br>is synthetic data based and can also handle symmetric objects without symmetry<br>labels and, thus, no additional labeled training data is needed. The<br>experimental evaluation shows that our method achieves close to benchmark<br>results for both symmetric objects and asymmetric objects on Occlusion-LineMOD,<br>and T-LESS datasets.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

D4D6

C. Wang, K. Wolski, B. Kerbl, A. Serrano, M. Bemana, H.-P. Seidel, K. Myszkowski, and T. Leimkühler

“Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field,” Computer Graphics Forum (Proc. Pacific Graphics 2024), vol. 43, no. 7, 2024.

@article{WangPG24,
TITLE = {Cinematic {G}aussians: {R}eal-Time {HDR} Radiance Fields with Depth of Field},
AUTHOR = {Wang, Chao and Wolski, Krzysztof and Kerbl, Bernhard and Serrano, Ana and Bemana, Mojtaba and Seidel, Hans-Peter and Myszkowski, Karol and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.15214},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
JOURNAL = {Computer Graphics Forum (Proc. Pacific Graphics)},
VOLUME = {43},
NUMBER = {7},
PAGES = {1--13},
BOOKTITLE = {The 32nd Pacific Conference on Computer Graphics and Applications (Pacific Graphics 2024)},
EDITOR = {Chen, Renjie and Ritschel, Tobias and Whiting, Emily},
}

Endnote

%0 Journal Article
%A Wang, Chao
%A Wolski, Krzysztof
%A Kerbl, Bernhard
%A Serrano, Ana
%A Bemana, Mojtaba
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Cinematic Gaussians: Real-Time HDR Radiance Fields with Depth of Field : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-FC3A-5
%R 10.1111/cgf.15214
%7 2024
%D 2024
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 43
%N 7
%& 1
%P 1 - 13
%I Blackwell-Wiley
%C Oxford
%@ false
%B The 32nd Pacific Conference on Computer Graphics and Applications
%O Pacific Graphics 2024 PG 2024 Huangshan (Yellow Mountain), China ; 
October 13 &#8211; 16, 2024

Paper

K. T. Wijaya, M. Guo, M. Sun, H.-P. Seidel, W. Matusik, and V. Babaei

“Two-Stage Pretraining for Molecular Property Prediction in the Wild,” 2024. [Online]. Available: https://arxiv.org/abs/2411.03537.

Abstract

Accurate property prediction is crucial for accelerating the discovery of new
molecules. Although deep learning models have achieved remarkable success,
their performance often relies on large amounts of labeled data that are
expensive and time-consuming to obtain. Thus, there is a growing need for
models that can perform well with limited experimentally-validated data. In
this work, we introduce MoleVers, a versatile pretrained model designed for
various types of molecular property prediction in the wild, i.e., where
experimentally-validated molecular property labels are scarce. MoleVers adopts
a two-stage pretraining strategy. In the first stage, the model learns
molecular representations from large unlabeled datasets via masked atom
prediction and dynamic denoising, a novel task enabled by a new branching
encoder architecture. In the second stage, MoleVers is further pretrained using
auxiliary labels obtained with inexpensive computational methods, enabling
supervised learning without the need for costly experimental data. This
two-stage framework allows MoleVers to learn representations that generalize
effectively across various downstream datasets. We evaluate MoleVers on a new
benchmark comprising 22 molecular datasets with diverse types of properties,
the majority of which contain 50 or fewer training labels reflecting real-world
conditions. MoleVers achieves state-of-the-art results on 20 out of the 22
datasets, and ranks second among the remaining two, highlighting its ability to
bridge the gap between data-hungry models and real-world conditions where
practically-useful labels are scarce.

BibTeX

@online{Wijaya2411.03537,
TITLE = {Two-Stage Pretraining for Molecular Property Prediction in the Wild},
AUTHOR = {Wijaya, Kevin Tirta and Guo, Minghao and Sun, Michael and Seidel, Hans-Peter and Matusik, Wojciech and Babaei, Vahid},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2411.03537},
EPRINT = {2411.03537},
EPRINTTYPE = {arXiv},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Accurate property prediction is crucial for accelerating the discovery of new<br>molecules. Although deep learning models have achieved remarkable success,<br>their performance often relies on large amounts of labeled data that are<br>expensive and time-consuming to obtain. Thus, there is a growing need for<br>models that can perform well with limited experimentally-validated data. In<br>this work, we introduce MoleVers, a versatile pretrained model designed for<br>various types of molecular property prediction in the wild, i.e., where<br>experimentally-validated molecular property labels are scarce. MoleVers adopts<br>a two-stage pretraining strategy. In the first stage, the model learns<br>molecular representations from large unlabeled datasets via masked atom<br>prediction and dynamic denoising, a novel task enabled by a new branching<br>encoder architecture. In the second stage, MoleVers is further pretrained using<br>auxiliary labels obtained with inexpensive computational methods, enabling<br>supervised learning without the need for costly experimental data. This<br>two-stage framework allows MoleVers to learn representations that generalize<br>effectively across various downstream datasets. We evaluate MoleVers on a new<br>benchmark comprising 22 molecular datasets with diverse types of properties,<br>the majority of which contain 50 or fewer training labels reflecting real-world<br>conditions. MoleVers achieves state-of-the-art results on 20 out of the 22<br>datasets, and ranks second among the remaining two, highlighting its ability to<br>bridge the gap between data-hungry models and real-world conditions where<br>practically-useful labels are scarce.<br>},
}

Endnote

%0 Report
%A Wijaya, Kevin Tirta
%A Guo, Minghao
%A Sun, Michael
%A Seidel, Hans-Peter
%A Matusik, Wojciech
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Two-Stage Pretraining for Molecular Property Prediction in the Wild : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-81DB-5
%U https://arxiv.org/abs/2411.03537
%D 2024
%X   Accurate property prediction is crucial for accelerating the discovery of new<br>molecules. Although deep learning models have achieved remarkable success,<br>their performance often relies on large amounts of labeled data that are<br>expensive and time-consuming to obtain. Thus, there is a growing need for<br>models that can perform well with limited experimentally-validated data. In<br>this work, we introduce MoleVers, a versatile pretrained model designed for<br>various types of molecular property prediction in the wild, i.e., where<br>experimentally-validated molecular property labels are scarce. MoleVers adopts<br>a two-stage pretraining strategy. In the first stage, the model learns<br>molecular representations from large unlabeled datasets via masked atom<br>prediction and dynamic denoising, a novel task enabled by a new branching<br>encoder architecture. In the second stage, MoleVers is further pretrained using<br>auxiliary labels obtained with inexpensive computational methods, enabling<br>supervised learning without the need for costly experimental data. This<br>two-stage framework allows MoleVers to learn representations that generalize<br>effectively across various downstream datasets. We evaluate MoleVers on a new<br>benchmark comprising 22 molecular datasets with diverse types of properties,<br>the majority of which contain 50 or fewer training labels reflecting real-world<br>conditions. MoleVers achieves state-of-the-art results on 20 out of the 22<br>datasets, and ranks second among the remaining two, highlighting its ability to<br>bridge the gap between data-hungry models and real-world conditions where<br>practically-useful labels are scarce.<br>
%K Computer Science, Learning, cs.LG,Computer Science, Artificial Intelligence, cs.AI, Physics, Chemical Physics, physics.chem-ph,Quantitative Biology, Biomolecules, q-bio.BM

Article

K. T. Wijaya, D.-H. Paek, and S.-H. Kong

“Advanced Feature Learning on Point Clouds Using Multi-Resolution Features and Learnable Pooling,” Remote Sensing, vol. 16, no. 11, 2024.

@article{Wijaya24,
TITLE = {Advanced Feature Learning on Point Clouds Using Multi-Resolution Features and Learnable Pooling},
AUTHOR = {Wijaya, Kevin Tirta and Paek, Dong-Hee and Kong, Seung-Hyun},
LANGUAGE = {eng},
ISSN = {2072-4292},
DOI = {10.3390/rs16111835},
PUBLISHER = {Molecular Diversity Preservation International (MDPI)},
ADDRESS = {Basel},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
JOURNAL = {Remote Sensing},
VOLUME = {16},
NUMBER = {11},
EID = {1835},
}

Endnote

%0 Journal Article
%A Wijaya, Kevin Tirta
%A Paek, Dong-Hee
%A Kong, Seung-Hyun
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Advanced Feature Learning on Point Clouds Using Multi-Resolution Features and Learnable Pooling : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-7697-2
%R 10.3390/rs16111835
%7 2024
%D 2024
%J Remote Sensing
%V 16
%N 11
%Z sequence number: 1835
%I Molecular Diversity Preservation International (MDPI) 
%C Basel
%@ false

Conference paper

D. Wittchen, N. Sabnis, T. R. Nachtigall, F. F. Mueller, P. Strohmeier, and D. S. Elvitigala

“Foot Augmentation 101: Design your own Augmented Experiences,” in TEI ’24, 18th International Conference on Tangible, Embedded and Embodied Interaction, Cork, Ireland, 2024.

@inproceedings{WittchenTEI24,
TITLE = {Foot Augmentation 101: {D}esign your own Augmented Experiences},
AUTHOR = {Wittchen, Dennis and Sabnis, Nihar and Nachtigall, Troy Robert and Mueller, Florian Floyd and Strohmeier, Paul and Elvitigala, Don Samitha},
LANGUAGE = {eng},
ISBN = {979-8-4007-0402-4},
DOI = {10.1145/3623509.3634744},
PUBLISHER = {ACM},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {TEI '24, 18th International Conference on Tangible, Embedded and Embodied Interaction},
PAGES = {1--4},
EID = {59},
ADDRESS = {Cork, Ireland},
}

Endnote

%0 Conference Proceedings
%A Wittchen, Dennis
%A Sabnis, Nihar
%A Nachtigall, Troy Robert
%A Mueller, Florian Floyd
%A Strohmeier, Paul
%A Elvitigala, Don Samitha
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Foot Augmentation 101: Design your own Augmented Experiences : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8180-A
%R 10.1145/3623509.3634744
%D 2024
%B 18th International Conference on Tangible, Embedded and Embodied Interaction
%Z date of event: 2024-02-11 - 2024-02-14
%C Cork, Ireland
%B TEI '24
%P 1 - 4
%Z sequence number: 59
%I ACM
%@ 979-8-4007-0402-4

Article

D4D6

K. Wolski, A. Djeacoumar, A. Javanmardi, H.-P. Seidel, C. Theobalt, G. Cordonnier, K. Myszkowski, G. Drettakis, X. Pan, and T. Leimkühler

“Learning Images Across Scales Using Adversarial Training,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2024), vol. 43, no. 4, 2024.

@article{Wolski24,
TITLE = {Learning Images Across Scales Using Adversarial Training},
AUTHOR = {Wolski, Krzysztof and Djeacoumar, Adarsh and Javanmardi, Alireza and Seidel, Hans-Peter and Theobalt, Christian and Cordonnier, Guillaume and Myszkowski, Karol and Drettakis, George and Pan, Xingang and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3658190},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {43},
NUMBER = {4},
PAGES = {1--13},
EID = {131},
BOOKTITLE = {Proceedings of the SIGGRAPH Conference (ACM SIGGRAPH 2024)},
}

Endnote

%0 Journal Article
%A Wolski, Krzysztof
%A Djeacoumar, Adarsh
%A Javanmardi, Alireza
%A Seidel, Hans-Peter
%A Theobalt, Christian
%A Cordonnier, Guillaume
%A Myszkowski, Karol
%A Drettakis, George
%A Pan, Xingang
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Learning Images Across Scales Using Adversarial Training : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-EED1-9
%R 10.1145/3658190
%7 2024-07-19
%D 2024
%J ACM Transactions on Graphics
%V 43
%N 4
%& 1
%P 1 - 13
%Z sequence number: 131
%I ACM
%C New York, NY
%@ false
%B Proceedings of the SIGGRAPH Conference
%O ACM SIGGRAPH 2024 Denver, CO, USA, July 28 - Aug 1

2023

Paper

N. Ansari, H.-P. Seidel, and V. Babaei

“Large-batch, Iteration-efficient Neural Bayesian Design Optimization,” 2023. [Online]. Available: https://arxiv.org/abs/2306.01095.

Abstract

Bayesian optimization (BO) provides a powerful framework for optimizing
black-box, expensive-to-evaluate functions. It is therefore an attractive tool
for engineering design problems, typically involving multiple objectives.
Thanks to the rapid advances in fabrication and measurement methods as well as
parallel computing infrastructure, querying many design problems can be heavily
parallelized. This class of problems challenges BO with an unprecedented setup
where it has to deal with very large batches, shifting its focus from sample
efficiency to iteration efficiency. We present a novel Bayesian optimization
framework specifically tailored to address these limitations. Our key
contribution is a highly scalable, sample-based acquisition function that
performs a non-dominated sorting of not only the objectives but also their
associated uncertainty. We show that our acquisition function in combination
with different Bayesian neural network surrogates is effective in
data-intensive environments with a minimal number of iterations. We demonstrate
the superiority of our method by comparing it with state-of-the-art
multi-objective optimizations. We perform our evaluation on two real-world
problems -- airfoil design and 3D printing -- showcasing the applicability and
efficiency of our approach. Our code is available at:
github.com/an-on-ym-ous/lbn_mobo

BibTeX

@online{Ansari-et-al_2023,
TITLE = {Large-batch, Iteration-efficient Neural Bayesian Design Optimization},
AUTHOR = {Ansari, Navid and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2306.01095},
DOI = {10.48550/arXiv.2306.01095},
EPRINT = {2306.01095},
EPRINTTYPE = {arXiv},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
ABSTRACT = {Bayesian optimization (BO) provides a powerful framework for optimizing<br>black-box, expensive-to-evaluate functions. It is therefore an attractive tool<br>for engineering design problems, typically involving multiple objectives.<br>Thanks to the rapid advances in fabrication and measurement methods as well as<br>parallel computing infrastructure, querying many design problems can be heavily<br>parallelized. This class of problems challenges BO with an unprecedented setup<br>where it has to deal with very large batches, shifting its focus from sample<br>efficiency to iteration efficiency. We present a novel Bayesian optimization<br>framework specifically tailored to address these limitations. Our key<br>contribution is a highly scalable, sample-based acquisition function that<br>performs a non-dominated sorting of not only the objectives but also their<br>associated uncertainty. We show that our acquisition function in combination<br>with different Bayesian neural network surrogates is effective in<br>data-intensive environments with a minimal number of iterations. We demonstrate<br>the superiority of our method by comparing it with state-of-the-art<br>multi-objective optimizations. We perform our evaluation on two real-world<br>problems -- airfoil design and 3D printing -- showcasing the applicability and<br>efficiency of our approach. Our code is available at:<br>https://github.com/an-on-ym-ous/lbn_mobo<br>},
}

Endnote

%0 Report
%A Ansari, Navid
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Large-batch, Iteration-efficient Neural Bayesian Design Optimization : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-7BB3-D
%R 10.48550/arXiv.2306.01095
%U https://arxiv.org/abs/2306.01095
%D 2023
%X   Bayesian optimization (BO) provides a powerful framework for optimizing<br>black-box, expensive-to-evaluate functions. It is therefore an attractive tool<br>for engineering design problems, typically involving multiple objectives.<br>Thanks to the rapid advances in fabrication and measurement methods as well as<br>parallel computing infrastructure, querying many design problems can be heavily<br>parallelized. This class of problems challenges BO with an unprecedented setup<br>where it has to deal with very large batches, shifting its focus from sample<br>efficiency to iteration efficiency. We present a novel Bayesian optimization<br>framework specifically tailored to address these limitations. Our key<br>contribution is a highly scalable, sample-based acquisition function that<br>performs a non-dominated sorting of not only the objectives but also their<br>associated uncertainty. We show that our acquisition function in combination<br>with different Bayesian neural network surrogates is effective in<br>data-intensive environments with a minimal number of iterations. We demonstrate<br>the superiority of our method by comparing it with state-of-the-art<br>multi-objective optimizations. We perform our evaluation on two real-world<br>problems -- airfoil design and 3D printing -- showcasing the applicability and<br>efficiency of our approach. Our code is available at:<br>https://github.com/an-on-ym-ous/lbn_mobo<br>
%K Computer Science, Learning, cs.LG,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Computational Engineering, Finance, and Science, cs.CE

Thesis

D4IMPR-CS

E. Arabadzhiyska

“Perceptually Driven Methods for Improved Gaze-contingent Rendering,” Universität des Saarlandes, Saarbrücken, 2023.

@phdthesis{Arabadzhiyska_PhD2023,
TITLE = {Perceptually Driven Methods for Improved Gaze-contingent Rendering},
AUTHOR = {Arabadzhiyska, Elena},
URL = {urn:nbn:de:bsz:291--ds-402489},
DOI = {10.22028/D291-40248},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
}

Endnote

%0 Thesis
%A Arabadzhiyska, Elena
%Y Didyk, Piotr
%A referee: Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perceptually Driven Methods for Improved Gaze-contingent Rendering : 
%U http://hdl.handle.net/21.11116/0000-000D-87B0-3
%U urn:nbn:de:bsz:291--ds-402489
%F OTHER: hdl:20.500.11880/36181
%R 10.22028/D291-40248
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2023
%P xxiv, 119 p.
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/36181

Article

E. Arabadzhiyska, C. Tursun, H.-P. Seidel, and P. Didyk

“Practical Saccade Prediction for Head-mounted Displays: Towards a Comprehensive Model,” ACM Transactions on Applied Perception, vol. 20, no. 1, 2023.

@article{Arabadzhiyska23,
TITLE = {Practical Saccade Prediction for Head-Mounted Displays: {T}owards a Comprehensive Model},
AUTHOR = {Arabadzhiyska, Elena and Tursun, Cara and Seidel, Hans-Peter and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {1544-3558},
DOI = {10.1145/3568311},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
JOURNAL = {ACM Transactions on Applied Perception},
VOLUME = {20},
NUMBER = {1},
PAGES = {1--23},
EID = {2},
}

Endnote

%0 Journal Article
%A Arabadzhiyska, Elena
%A Tursun, Cara
%A Seidel, Hans-Peter
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Practical Saccade Prediction for Head-mounted Displays: Towards a Comprehensive Model : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-B76B-E
%R 10.1145/3568311
%7 2023
%D 2023
%J ACM Transactions on Applied Perception
%V 20
%N 1
%& 1
%P 1 - 23
%Z sequence number: 2
%I ACM
%C New York, NY
%@ false
%U https://dl.acm.org/doi/pdf/10.1145/3568311

Proceedings

V. Babaei and M. Skouras

Eds., EUROGRAPHICS 2023 : Short Papers. Eurographics Association, 2023.

@proceedings{BabaeiEGEditor23,
TITLE = {EUROGRAPHICS 2023 : Short Papers},
EDITOR = {Babaei, Vahid and Skouras, Melina},
LANGUAGE = {eng},
ISBN = {978-3-03868-209-7},
PUBLISHER = {Eurographics Association},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
PAGES = {VII, 60 p.},
ADDRESS = {Saarbr{\"u}cken, Germany},
}

Endnote

%0 Conference Proceedings
%E Babaei, Vahid
%E Skouras, Melina
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T EUROGRAPHICS 2023 : Short Papers : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-81D9-7
%@ 978-3-03868-209-7
%I Eurographics Association
%D 2023
%B The European Association for Computer Graphics 44th Annual Conference
%Z date of event: 2023-05-08 - 2023-05-12
%D 2023
%C Saarbr&#252;cken, Germany
%P VII, 60 p.

Conference paper

M. Bálint, K. Myszkowski, H.-P. Seidel, and G. Singh

“Joint Sampling and Optimisation for Inverse Rendering,” in Proceedings SIGGRAPH Asia 2023 Conference Papers, Sydney, Australia, 2023.

@inproceedings{DBLP:journals/corr/abs-2309-15676,
TITLE = {Joint Sampling and Optimisation for Inverse Rendering},
AUTHOR = {B{\'a}lint, Martin and Myszkowski, Karol and Seidel, Hans-Peter and Singh, Gurprit},
LANGUAGE = {eng},
ISBN = {979-8-4007-0315-7},
DOI = {10.1145/3610548.3618244},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {Proceedings SIGGRAPH Asia 2023 Conference Papers},
EDITOR = {Kim, June and Lin, Ming C. and Bickel, Bernd},
PAGES = {1--10},
EID = {29},
ADDRESS = {Sydney, Australia},
}

Endnote

%0 Conference Proceedings
%A B&#225;lint, Martin
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Joint Sampling and Optimisation for Inverse Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-7B9B-9
%R 10.1145/3610548.3618244
%D 2023
%B SIGGRAPH Asia 2023 Conference
%Z date of event: 2023-12-12 - 2023-12-15
%C Sydney, Australia
%B Proceedings SIGGRAPH Asia 2023 Conference Papers
%E Kim, June; Lin, Ming C.; Bickel, Bernd
%P 1 - 10
%Z sequence number: 29
%I ACM
%@ 979-8-4007-0315-7
%U https://dl.acm.org/doi/pdf/10.1145/3610548.3618244

Conference paper

M. Bálint, K. Wolski, K. Myszkowski, H.-P. Seidel, and R. Mantiuk

“Neural Partitioning Pyramids for Denoising Monte Carlo Renderings,” in Proceedings SIGGRAPH 2023 Conference Papers, Los Angeles, CA, USA, 2023.

@inproceedings{Balint_SIGGRAPH23,
TITLE = {Neural Partitioning Pyramids for Denoising {Monte Carlo} Renderings},
AUTHOR = {B{\'a}lint, Martin and Wolski, Krzysztof and Myszkowski, Karol and Seidel, Hans-Peter and Mantiuk, Rafa{\l}},
LANGUAGE = {eng},
ISBN = {979-8-4007-0159-7},
DOI = {10.1145/3588432.3591562},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {Proceedings SIGGRAPH 2023 Conference Papers},
EDITOR = {Brunvand, Erik and Sheffer, Alla and Wimmer, Michael},
PAGES = {1--11},
EID = {60},
ADDRESS = {Los Angeles, CA, USA},
}

Endnote

%0 Conference Proceedings
%A B&#225;lint, Martin
%A Wolski, Krzysztof
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Mantiuk, Rafa&#322;
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Neural Partitioning Pyramids for Denoising Monte Carlo Renderings : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-3740-C
%R 10.1145/3588432.3591562
%D 2023
%B ACM SIGGRAPH Conference
%Z date of event: 2023-08-06 - 2023-08-10
%C Los Angeles, CA, USA
%B Proceedings SIGGRAPH 2023 Conference Papers
%E Brunvand, Erik; Sheffer, Alla; Wimmer, Michael
%P 1 - 11
%Z sequence number: 60
%I ACM
%@ 979-8-4007-0159-7

Thesis

D4IMPR-CS

M. Bemana

“Efficient Image-Based Rendering,” Universität des Saarlandes, Saarbrücken, 2023.

@phdthesis{Bemana_PhD2023,
TITLE = {Efficient Image-Based Rendering},
AUTHOR = {Bemana, Mojtaba},
URL = {urn:nbn:de:bsz:291-scidok-2279},
DOI = {10.22028/D291-25735},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
}

Endnote

%0 Thesis
%A Bemana, Mojtaba
%Y Myszkowski, K.
%A referee: Seidel, Hans-Peter
%A referee: Kellnhofer, Petr
%A referee: Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Efficient Image-Based Rendering : 
%U http://hdl.handle.net/21.11116/0000-000D-D171-7
%R 10.22028/D291-25735
%U urn:nbn:de:bsz:291-scidok-2279
%F OTHER: hdl:20.500.11880/25791
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2023
%P xi, 112 p.
%V phd
%9 phd
%U https://scidok.sulb.uni-saarland.de/handle/20.500.11880/36287

Conference paper

B. Chen, A. Jindal, M. Piovarci, C. Wang, H.-P. Seidel, P. Didyk, K. Myszkowski, A. Serrano, and R. K. Mantiuk

“The Effect of Display Capabilities on the Gloss Consistency between Real and Virtual Objects,” in Proceedings SIGGRAPH Asia 2023 Conference Papers, Sydney, Australia, 2023.

@inproceedings{DBLP:conf/siggrapha/ChenJPWSDMSM23,
TITLE = {The Effect of Display Capabilities on the Gloss Consistency between Real and Virtual Objects},
AUTHOR = {Chen, Bin and Jindal, Akshay and Piovarci, Michal and Wang, Chao and Seidel, Hans-Peter and Didyk, Piotr and Myszkowski, Karol and Serrano, Ana and Mantiuk, Rafal K.},
LANGUAGE = {eng},
ISBN = {979-8-4007-0315-7},
DOI = {10.1145/3610548.3618226},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {Proceedings SIGGRAPH Asia 2023 Conference Papers},
EDITOR = {Kim, June and Lin, Ming C. and Bickel, Bernd},
PAGES = {1--11},
EID = {90},
ADDRESS = {Sydney, Australia},
}

Endnote

%0 Conference Proceedings
%A Chen, Bin
%A Jindal, Akshay
%A Piovarci, Michal
%A Wang, Chao
%A Seidel, Hans-Peter
%A Didyk, Piotr
%A Myszkowski, Karol
%A Serrano, Ana
%A Mantiuk, Rafal K.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T The Effect of Display Capabilities on the Gloss Consistency between
Real and Virtual Objects : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-7BC2-C
%R 10.1145/3610548.3618226
%D 2023
%B SIGGRAPH Asia 2023 Conference
%Z date of event: 2023-12-12 - 2023-12-15
%C Sydney, Australia
%B Proceedings SIGGRAPH Asia 2023 Conference Papers
%E Kim, June; Lin, Ming C.; Bickel, Bernd
%P 1 - 11
%Z sequence number: 90
%I ACM
%@ 979-8-4007-0315-7
%U https://dl.acm.org/doi/pdf/10.1145/3610548.3618226

Article

U. Çoğalan, M. Bemana, H.-P. Seidel, and K. Myszkowski

“Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors,” Computer Graphics Forum (Proc. EUROGRAPHICS 2023), vol. 42, no. 2, 2023.

@article{Cogalan_Eurographics23,
TITLE = {Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur and Bemana, Mojtaba and Seidel, Hans-Peter and Myszkowski, Karol},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.14748},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {42},
NUMBER = {2},
PAGES = {119--131},
BOOKTITLE = {The European Association for Computer Graphics 43rdAnnual Conference (EUROGRAPHICS 2023)},
}

Endnote

%0 Journal Article
%A &#199;o&#287;alan, U&#287;ur
%A Bemana, Mojtaba
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Video Frame Interpolation for High Dynamic Range Sequences 
Captured with Dual-exposure Sensors : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-F953-E
%R 10.1111/cgf.14748
%7 2023
%D 2023
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 42
%N 2
%& 119
%P 119 - 131
%I Blackwell-Wiley
%C Oxford
%@ false
%B The European Association for Computer Graphics 43rdAnnual Conference
%O EUROGRAPHICS 2023 EG 2023 Saarbr&#252;cken, Germany, May 8-12, 2023

Article

D6D4D2

M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll, and C. Theobalt

“A Deeper Look into DeepCap,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, 2023.

Abstract

Human performance capture is a highly important computer vision problem with
many applications in movie production and virtual/augmented reality. Many
previous performance capture approaches either required expensive multi-view
setups or did not recover dense space-time coherent geometry with
frame-to-frame correspondences. We propose a novel deep learning approach for
monocular dense human performance capture. Our method is trained in a weakly
supervised manner based on multi-view supervision completely removing the need
for training data with 3D ground truth annotations. The network architecture is
based on two separate networks that disentangle the task into a pose estimation
and a non-rigid surface deformation step. Extensive qualitative and
quantitative evaluations show that our approach outperforms the state of the
art in terms of quality and robustness. This work is an extended version of
DeepCap where we provide more detailed explanations, comparisons and results as
well as applications.

BibTeX

@article{Habermann2111.10563,
TITLE = {A Deeper Look into {DeepCap}},
AUTHOR = {Habermann, Marc and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0162-8828},
DOI = {10.1109/TPAMI.2021.3093553},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
ABSTRACT = {Human performance capture is a highly important computer vision problem with<br>many applications in movie production and virtual/augmented reality. Many<br>previous performance capture approaches either required expensive multi-view<br>setups or did not recover dense space-time coherent geometry with<br>frame-to-frame correspondences. We propose a novel deep learning approach for<br>monocular dense human performance capture. Our method is trained in a weakly<br>supervised manner based on multi-view supervision completely removing the need<br>for training data with 3D ground truth annotations. The network architecture is<br>based on two separate networks that disentangle the task into a pose estimation<br>and a non-rigid surface deformation step. Extensive qualitative and<br>quantitative evaluations show that our approach outperforms the state of the<br>art in terms of quality and robustness. This work is an extended version of<br>DeepCap where we provide more detailed explanations, comparisons and results as<br>well as applications.<br>},
JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
VOLUME = {45},
NUMBER = {4},
PAGES = {4009--4002},
}

Endnote

%0 Journal Article
%A Habermann, Marc
%A Xu, Weipeng
%A Zollh&#246;fer, Michael
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T A Deeper Look into DeepCap : (Invited Paper)
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-8C33-0
%R 10.1109/TPAMI.2021.3093553
%7 2021
%D 2023
%X   Human performance capture is a highly important computer vision problem with<br>many applications in movie production and virtual/augmented reality. Many<br>previous performance capture approaches either required expensive multi-view<br>setups or did not recover dense space-time coherent geometry with<br>frame-to-frame correspondences. We propose a novel deep learning approach for<br>monocular dense human performance capture. Our method is trained in a weakly<br>supervised manner based on multi-view supervision completely removing the need<br>for training data with 3D ground truth annotations. The network architecture is<br>based on two separate networks that disentangle the task into a pose estimation<br>and a non-rigid surface deformation step. Extensive qualitative and<br>quantitative evaluations show that our approach outperforms the state of the<br>art in terms of quality and robustness. This work is an extended version of<br>DeepCap where we provide more detailed explanations, comparisons and results as<br>well as applications.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%J IEEE Transactions on Pattern Analysis and Machine Intelligence
%O IEEE Trans. Pattern Anal. Mach. Intell.
%V 45
%N 4
%& 4009
%P 4009 - 4002
%I IEEE
%C Piscataway, NJ
%@ false

Thesis

D4IMPR-CSD2

I. Habibie

“Learning-based 3D Human Motion Capture and Animation Synthesis,” Universität des Saarlandes, Saarbrücken, 2023.

@phdthesis{Habibie_PhD2023,
TITLE = {Learning-based {3D} Human Motion Capture and Animation Synthesis},
AUTHOR = {Habibie, Ikhsanul},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-400122},
DOI = {10.22028/D291-40012},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
}

Endnote

%0 Thesis
%A Habibie, Ikhsanul
%Y Theobalt, Christian
%A referee: Neff, Michael Paul
%A referee: Kr&#252;ger, Antonio
%A referee: Pons-Moll, Gerard
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Learning-based 3D Human Motion Capture and Animation Synthesis : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000D-7336-5
%R 10.22028/D291-40012
%U urn:nbn:de:bsz:291--ds-400122
%F OTHER: hdl:20.500.11880/36046
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2023
%P xviii, 118 p.
%V phd
%9 phd
%U https://scidok.sulb.uni-saarland.de/handle/20.500.11880/36046

Conference paper

A. Haynes, C. N. Reed, C. Nordmoen, and S. Skach

“Being Meaningful: Weaving Soma-Reflective Technological Mediations into the Fabric of Daily Life,” in TEI ’23, Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction, Warsaw, Poland, 2023.

@inproceedings{Haynes_TEI23,
TITLE = {Being Meaningful: {W}eaving Soma-Reflective Technological Mediations into the Fabric of Daily Life},
AUTHOR = {Haynes, Alice and Reed, Courtney N. and Nordmoen, Charlotte and Skach, Sophie},
LANGUAGE = {eng},
ISBN = {978-1-4503-9977-7},
DOI = {10.1145/3569009.3571844},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {TEI '23, Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction},
PAGES = {1--5},
EID = {68},
ADDRESS = {Warsaw, Poland},
}

Endnote

%0 Conference Proceedings
%A Haynes, Alice
%A Reed, Courtney N.
%A Nordmoen, Charlotte
%A Skach, Sophie
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Being Meaningful: Weaving Soma-Reflective Technological Mediations into the Fabric of Daily Life : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-BDEB-7
%R 10.1145/3569009.3571844
%D 2023
%B Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction
%Z date of event: 2023-02-26 - 2023-03-01
%C Warsaw, Poland
%B TEI '23
%P 1 - 5
%Z sequence number: 68
%I ACM
%@ 978-1-4503-9977-7

Article

X. Huang, T. Ritschel, H.-P. Seidel, P. Memari, and G. Singh

“Patternshop: Editing Point Patterns by Image Manipulation,” ACM Transactions on Graphics, vol. 42, no. 4, 2023.

@article{Huang2023,
TITLE = {Patternshop: {E}diting Point Patterns by Image Manipulation},
AUTHOR = {Huang, Xingchang and Ritschel, Tobias and Seidel, Hans-Peter and Memari, Pooran and Singh, Gurprit},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3592418},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {42},
NUMBER = {4},
PAGES = {1--14},
EID = {53},
}

Endnote

%0 Journal Article
%A Huang, Xingchang
%A Ritschel, Tobias
%A Seidel, Hans-Peter
%A Memari, Pooran
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Patternshop: Editing Point Patterns by Image Manipulation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000D-FE1D-6
%R 10.1145/3592418
%7 2023
%D 2023
%J ACM Transactions on Graphics
%V 42
%N 4
%& 1
%P 1 - 14
%Z sequence number: 53
%I ACM
%C New York, NY
%@ false

Article

C. Jambon, B. Kerbl, G. Kopanas, S. Diolatzis, T. Leimkühler, and G. Drettakis

“NeRFshop: Interactive Editing of Neural Radiance Fields,” Proceedings of the ACM on Computer Graphics and Interactive Techniques, vol. 6, no. 1, 2023.

@article{JKKDLD23,
TITLE = {{NeRFshop}: {I}nteractive Editing of Neural Radiance Fields},
AUTHOR = {Jambon, Cl{\'e}ment and Kerbl, Bernhard and Kopanas, Georgios and Diolatzis, Stavros and Leimk{\"u}hler, Thomas and Drettakis, George},
LANGUAGE = {eng},
ISSN = {2577-6193},
DOI = {10.1145/3585499},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
JOURNAL = {Proceedings of the ACM on Computer Graphics and Interactive Techniques},
VOLUME = {6},
NUMBER = {1},
PAGES = {1--21},
EID = {1},
}

Endnote

%0 Journal Article
%A Jambon, Cl&#233;ment
%A Kerbl, Bernhard
%A Kopanas, Georgios
%A Diolatzis, Stavros
%A Leimk&#252;hler, Thomas
%A Drettakis, George
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T NeRFshop: Interactive Editing of Neural Radiance Fields : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-BEE0-1
%R 10.1145/3585499
%7 2023-05-16
%D 2023
%J Proceedings of the ACM on Computer Graphics and Interactive Techniques
%V 6
%N 1
%& 1
%P 1 - 21
%Z sequence number: 1
%I ACM
%C New York, NY
%@ false
%U http://www-sop.inria.fr/reves/Basilic/2023/JKKDLD23

Conference paper

M. Korać, C. Salaün, I. Georgiev, P. Grittmann, P. Slusallek, K. Myszkowski, and G. Singh

“Perceptual Error Optimization for Monte Carlo Animation Rendering,” in Proceedings SIGGRAPH Asia 2023 Conference Papers, Sidney, Australia, 2023.

@inproceedings{KoracSIGGRAPHASIA23,
TITLE = {Perceptual Error Optimization for {Monte Carlo} Animation Rendering},
AUTHOR = {Kora{\'c}, Mi{\v s}a and Sala{\"u}n, Corentin and Georgiev, Iliyan and Grittmann, Pascal and Slusallek, Philipp and Myszkowski, Karol and Singh, Gurprit},
LANGUAGE = {eng},
ISBN = {979-8-4007-0315-7},
DOI = {10.1145/3610548.3618146},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {Proceedings SIGGRAPH Asia 2023 Conference Papers},
EDITOR = {Kim, June and Lin, Ming C. and Bickel, Bernd},
PAGES = {1--10},
EID = {89},
ADDRESS = {Sidney, Australia},
}

Endnote

%0 Conference Proceedings
%A Kora&#263;, Mi&#353;a
%A Sala&#252;n, Corentin
%A Georgiev, Iliyan
%A Grittmann, Pascal
%A Slusallek, Philipp
%A Myszkowski, Karol
%A Singh, Gurprit
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perceptual Error Optimization for Monte Carlo Animation Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8174-9
%R 10.1145/3610548.3618146
%D 2023
%B SIGGRAPH Asia 2023 Conference
%Z date of event: 2023-12-12 - 2023-12-15
%C Sidney, Australia
%B Proceedings SIGGRAPH Asia 2023 Conference Papers
%E Kim, June; Lin, Ming C.; Bickel, Bernd
%P 1 - 10
%Z sequence number: 89
%I ACM
%@ 979-8-4007-0315-7

Conference paper

K. Liao, T. Tricard, M. Piovarči, H.-P. Seidel, and V. Babaei

“Learning Deposition Policies for Fused Multi-Material 3D Printing,” in IEEE International Conference on Robotics and Automation (ICRA 2023), London, UK, 2023.

@inproceedings{Liao_ICRA2023,
TITLE = {Learning Deposition Policies for Fused Multi-Material {3D} Printing},
AUTHOR = {Liao, Kang and Tricard, Thibault and Piovar{\v c}i, Michal and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
ISBN = {979-8-3503-2365-8},
DOI = {10.1109/ICRA48891.2023.10160465},
PUBLISHER = {IEEE},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {IEEE International Conference on Robotics and Automation (ICRA 2023)},
PAGES = {12345--12352},
ADDRESS = {London, UK},
}

Endnote

%0 Conference Proceedings
%A Liao, Kang
%A Tricard, Thibault
%A Piovar&#269;i, Michal
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Learning Deposition Policies for Fused Multi-Material 3D Printing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-44C2-C
%R 10.1109/ICRA48891.2023.10160465
%D 2023
%B IEEE International Conference on Robotics and Automation
%Z date of event: 2023-05-29 - 2023-06-02
%C London, UK
%B IEEE International Conference on Robotics and Automation
%P 12345 - 12352
%I IEEE
%@ 979-8-3503-2365-8

Article

D4D6

N. E. Nsampi, A. Djeacoumar, H.-P. Seidel, T. Ritschel, and T. Leimkühler

“Neural Field Convolutions by Repeated Differentiation,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2023), vol. 42, no. 6, 2023.

@article{Nsampi_SIGGRAPHASIA23,
TITLE = {Neural Field Convolutions by Repeated Differentiation},
AUTHOR = {Nsampi, Ntumba Elie and Djeacoumar, Adarsh and Seidel, Hans-Peter and Ritschel, Tobias and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3618340},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {42},
NUMBER = {6},
PAGES = {1--11},
EID = {206},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2023},
}

Endnote

%0 Journal Article
%A Nsampi, Ntumba Elie
%A Djeacoumar, Adarsh
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Neural Field Convolutions by Repeated Differentiation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-691E-C
%R 10.1145/3618340
%7 2023
%D 2023
%J ACM Transactions on Graphics
%V 42
%N 6
%& 1
%P 1 - 11
%Z sequence number: 206
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2023

%O ACM SIGGRAPH Asia 2023 Sydney, Australia, 12-15 December 2023 SA '23 SA 2023

Conference paper

C. N. Reed, P. Strohmeier, and A. McPherson

“Negotiating Experience and Communicating Information Through Abstract Metaphor,” in CHI ’23, CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 2023.

@inproceedings{Reed_CHI2023,
TITLE = {Negotiating Experience and Communicating Information Through Abstract Metaphor},
AUTHOR = {Reed, Courtney N. and Strohmeier, Paul and McPherson, Andrew},
LANGUAGE = {eng},
ISBN = {978-1-4503-9421-5},
DOI = {10.1145/3544548.3580700},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {CHI '23, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Schmidt, Albrecht and V{\"a}{\"a}n{\"a}nen, Kaisa and Goyal, Tesh and Kristensson, Per Ola and Peters, Anica and Mueller, Stefanie and Williamson, Julie R. and Wilson, Max L.},
PAGES = {1--16},
EID = {185},
ADDRESS = {Hamburg, Germany},
}

Endnote

%0 Conference Proceedings
%A Reed, Courtney N.
%A Strohmeier, Paul
%A McPherson, Andrew
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Negotiating Experience and Communicating Information Through Abstract Metaphor : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A035-3
%R 10.1145/3544548.3580700
%D 2023
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2023-04-23 - 2023-04-28
%C Hamburg, Germany
%B CHI '23
%E Schmidt, Albrecht; V&#228;&#228;n&#228;nen, Kaisa; Goyal, Tesh; Kristensson, Per Ola; Peters, Anica; Mueller, Stefanie; Williamson, Julie R.; Wilson, Max L.
%P 1 - 16
%Z sequence number: 185
%I ACM
%@ 978-1-4503-9421-5

Conference paper

C. N. Reed and A. P. McPherson

“The Body as Sound: Unpacking Vocal Embodiment through Auditory Biofeedback,” in TEI ’23, Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction, Warsaw, Poland, 2023.

@inproceedings{Reed_TEI23,
TITLE = {The Body as Sound: {U}npacking Vocal Embodiment through Auditory Biofeedback},
AUTHOR = {Reed, Courtney N. and McPherson, Andrew P.},
LANGUAGE = {eng},
ISBN = {978-1-4503-9977-7},
DOI = {10.1145/3569009.3572738},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {TEI '23, Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction},
PAGES = {1--15},
EID = {7},
ADDRESS = {Warsaw, Poland},
}

Endnote

%0 Conference Proceedings
%A Reed, Courtney N.
%A McPherson, Andrew P.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T The Body as Sound: Unpacking Vocal Embodiment through Auditory Biofeedback : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A02B-F
%R 10.1145/3569009.3572738
%D 2023
%B Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction
%Z date of event: 2023-02-26 - 2023-03-01
%C Warsaw, Poland
%B TEI '23
%P 1 - 15
%Z sequence number: 7
%I ACM
%@ 978-1-4503-9977-7

Conference paper

C. N. Reed

“As the Luthiers Do: Designing with a Living, Growing, Changing Body-Material,” in CHI’ 23 Workshop - Body x Materials, Hamburg, Germany, 2023.

@inproceedings{reed2023bodyx,
TITLE = {As the Luthiers Do: {D}esigning with a Living, Growing, Changing Body-Material},
AUTHOR = {Reed, Courtney N.},
LANGUAGE = {eng},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {CHI{\textquoteright} 23 Workshop -- Body x Materials},
ADDRESS = {Hamburg, Germany},
}

Endnote

%0 Conference Proceedings
%A Reed, Courtney N.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
%T As the Luthiers Do: Designing with a Living, Growing, Changing Body-Material : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-BDD8-C
%D 2023
%B ACM CHI Workshop on Body X Materials
%Z date of event: 2023-04-23 - 2023-04-23
%C Hamburg, Germany
%B CHI&#8217; 23 Workshop - Body x Materials
%U https://rca-media2.rca.ac.uk/documents/33_-_CHI23_bodyluthiers-reed-bodyxmaterials_-_Courtney_Reed.pdf

Paper

L. Ruan, M. Bemana, H.-P. Seidel, K. Myszkowski, and B. Chen

“Revisiting Image Deblurring with an Efficient ConvNet,” 2023. [Online]. Available: https://arxiv.org/abs/2302.02234.

Abstract

Image deblurring aims to recover the latent sharp image from its blurry
counterpart and has a wide range of applications in computer vision. The
Convolution Neural Networks (CNNs) have performed well in this domain for many
years, and until recently an alternative network architecture, namely
Transformer, has demonstrated even stronger performance. One can attribute its
superiority to the multi-head self-attention (MHSA) mechanism, which offers a
larger receptive field and better input content adaptability than CNNs.
However, as MHSA demands high computational costs that grow quadratically with
respect to the input resolution, it becomes impractical for high-resolution
image deblurring tasks. In this work, we propose a unified lightweight CNN
network that features a large effective receptive field (ERF) and demonstrates
comparable or even better performance than Transformers while bearing less
computational costs. Our key design is an efficient CNN block dubbed LaKD,
equipped with a large kernel depth-wise convolution and spatial-channel mixing
structure, attaining comparable or larger ERF than Transformers but with a
smaller parameter scale. Specifically, we achieve +0.17dB / +0.43dB PSNR over
the state-of-the-art Restormer on defocus / motion deblurring benchmark
datasets with 32% fewer parameters and 39% fewer MACs. Extensive experiments
demonstrate the superior performance of our network and the effectiveness of
each module. Furthermore, we propose a compact and intuitive ERFMeter metric
that quantitatively characterizes ERF, and shows a high correlation to the
network performance. We hope this work can inspire the research community to
further explore the pros and cons of CNN and Transformer architectures beyond
image deblurring tasks.

BibTeX

@online{ruan2023revisiting,
TITLE = {Revisiting Image Deblurring with an Efficient {ConvNet}},
AUTHOR = {Ruan, Lingyan and Bemana, Mojtaba and Seidel, Hans-Peter and Myszkowski, Karol and Chen, Bin},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2302.02234},
EPRINT = {2302.02234},
EPRINTTYPE = {arXiv},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Image deblurring aims to recover the latent sharp image from its blurry<br>counterpart and has a wide range of applications in computer vision. The<br>Convolution Neural Networks (CNNs) have performed well in this domain for many<br>years, and until recently an alternative network architecture, namely<br>Transformer, has demonstrated even stronger performance. One can attribute its<br>superiority to the multi-head self-attention (MHSA) mechanism, which offers a<br>larger receptive field and better input content adaptability than CNNs.<br>However, as MHSA demands high computational costs that grow quadratically with<br>respect to the input resolution, it becomes impractical for high-resolution<br>image deblurring tasks. In this work, we propose a unified lightweight CNN<br>network that features a large effective receptive field (ERF) and demonstrates<br>comparable or even better performance than Transformers while bearing less<br>computational costs. Our key design is an efficient CNN block dubbed LaKD,<br>equipped with a large kernel depth-wise convolution and spatial-channel mixing<br>structure, attaining comparable or larger ERF than Transformers but with a<br>smaller parameter scale. Specifically, we achieve +0.17dB / +0.43dB PSNR over<br>the state-of-the-art Restormer on defocus / motion deblurring benchmark<br>datasets with 32% fewer parameters and 39% fewer MACs. Extensive experiments<br>demonstrate the superior performance of our network and the effectiveness of<br>each module. Furthermore, we propose a compact and intuitive ERFMeter metric<br>that quantitatively characterizes ERF, and shows a high correlation to the<br>network performance. We hope this work can inspire the research community to<br>further explore the pros and cons of CNN and Transformer architectures beyond<br>image deblurring tasks.<br>},
}

Endnote

%0 Report
%A Ruan, Lingyan
%A Bemana, Mojtaba
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%A Chen, Bin
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Revisiting Image Deblurring with an Efficient ConvNet : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-C7B9-3
%U https://arxiv.org/abs/2302.02234
%D 2023
%X   Image deblurring aims to recover the latent sharp image from its blurry<br>counterpart and has a wide range of applications in computer vision. The<br>Convolution Neural Networks (CNNs) have performed well in this domain for many<br>years, and until recently an alternative network architecture, namely<br>Transformer, has demonstrated even stronger performance. One can attribute its<br>superiority to the multi-head self-attention (MHSA) mechanism, which offers a<br>larger receptive field and better input content adaptability than CNNs.<br>However, as MHSA demands high computational costs that grow quadratically with<br>respect to the input resolution, it becomes impractical for high-resolution<br>image deblurring tasks. In this work, we propose a unified lightweight CNN<br>network that features a large effective receptive field (ERF) and demonstrates<br>comparable or even better performance than Transformers while bearing less<br>computational costs. Our key design is an efficient CNN block dubbed LaKD,<br>equipped with a large kernel depth-wise convolution and spatial-channel mixing<br>structure, attaining comparable or larger ERF than Transformers but with a<br>smaller parameter scale. Specifically, we achieve +0.17dB / +0.43dB PSNR over<br>the state-of-the-art Restormer on defocus / motion deblurring benchmark<br>datasets with 32% fewer parameters and 39% fewer MACs. Extensive experiments<br>demonstrate the superior performance of our network and the effectiveness of<br>each module. Furthermore, we propose a compact and intuitive ERFMeter metric<br>that quantitatively characterizes ERF, and shows a high correlation to the<br>network performance. We hope this work can inspire the research community to<br>further explore the pros and cons of CNN and Transformer architectures beyond<br>image deblurring tasks.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

N. Sabnis, D. Wittchen, C. N. Reed, N. Pourjafarian, J. Steimle, and P. Strohmeier

“Haptic Servos: Self-Contained Vibrotactile Rendering System for Creating or Augmenting Material Experiences,” in CHI ’23, CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 2023.

@inproceedings{Sabnis_CHI2023,
TITLE = {Haptic Servos: {S}elf-Contained Vibrotactile Rendering System for Creating or Augmenting Material Experiences},
AUTHOR = {Sabnis, Nihar and Wittchen, Dennis and Reed, Courtney N. and Pourjafarian, Narjes and Steimle, J{\"u}rgen and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {978-1-4503-9421-5},
DOI = {10.1145/3544548.3580716},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {CHI '23, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Schmidt, Albrecht and V{\"a}{\"a}n{\"a}nen, Kaisa and Goyal, Tesh and Kristensson, Per Ola and Peters, Anica and Mueller, Stefanie and Williamson, Julie R. and Wilson, Max L.},
PAGES = {1--17},
EID = {522},
ADDRESS = {Hamburg, Germany},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Wittchen, Dennis
%A Reed, Courtney N.
%A Pourjafarian, Narjes
%A Steimle, J&#252;rgen
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Haptic Servos: Self-Contained Vibrotactile Rendering System for Creating or Augmenting Material Experiences : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A03C-C
%R 10.1145/3544548.3580716
%D 2023
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2023-04-23 - 2023-04-28
%C Hamburg, Germany
%B CHI '23
%E Schmidt, Albrecht; V&#228;&#228;n&#228;nen, Kaisa; Goyal, Tesh; Kristensson, Per Ola; Peters, Anica; Mueller, Stefanie; Williamson, Julie R.; Wilson, Max L.
%P 1 - 17
%Z sequence number: 522
%I ACM
%@ 978-1-4503-9421-5

Conference paper

N. Sabnis, D. Wittchen, G. Vega, C. N. Reed, and P. Strohmeier

“Tactile Symbols with Continuous and Motion-Coupled Vibration: An Exploration of using Embodied Experiences for Hermeneutic Design,” in CHI ’23, CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 2023.

@inproceedings{Sabnis_CHI2023B,
TITLE = {Tactile Symbols with Continuous and Motion-Coupled Vibration: {A}n Exploration of using Embodied Experiences for Hermeneutic Design},
AUTHOR = {Sabnis, Nihar and Wittchen, Dennis and Vega, Gabriela and Reed, Courtney N. and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {978-1-4503-9421-5},
DOI = {10.1145/3544548.3581356},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {CHI '23, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Schmidt, Albrecht and V{\"a}{\"a}n{\"a}nen, Kaisa and Goyal, Tesh and Kristensson, Per Ola and Peters, Anica and Mueller, Stefanie and Williamson, Julie R. and Wilson, Max L.},
PAGES = {1--19},
EID = {688},
ADDRESS = {Hamburg, Germany},
}

Endnote

%0 Conference Proceedings
%A Sabnis, Nihar
%A Wittchen, Dennis
%A Vega, Gabriela
%A Reed, Courtney N.
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Tactile Symbols with Continuous and Motion-Coupled Vibration: An Exploration of using Embodied Experiences for Hermeneutic Design : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A042-4
%R 10.1145/3544548.3581356
%D 2023
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2023-04-23 - 2023-04-28
%C Hamburg, Germany
%B CHI '23
%E Schmidt, Albrecht; V&#228;&#228;n&#228;nen, Kaisa; Goyal, Tesh; Kristensson, Per Ola; Peters, Anica; Mueller, Stefanie; Williamson, Julie R.; Wilson, Max L.
%P 1 - 19
%Z sequence number: 688
%I ACM
%@ 978-1-4503-9421-5

Proceedings

G. Singh and M. R. Chu

Eds., EUROGRAPHICS 2023 : Posters. Eurographics Association, 2023.

@proceedings{SinghEGEditor23,
TITLE = {EUROGRAPHICS 2023 : Posters},
EDITOR = {Singh, Gurprit and Chu, Mengyu Rachel},
LANGUAGE = {eng},
ISBN = {978-3-03868-211-0},
DOI = {10.2312/egp.20232002},
PUBLISHER = {Eurographics Association},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
PAGES = {18 p.},
ADDRESS = {Saarbr{\"u}cken, Germany},
}

Endnote

%0 Conference Proceedings
%E Singh, Gurprit
%E Chu, Mengyu Rachel
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T EUROGRAPHICS 2023 : Posters : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8B02-F
%@ 978-3-03868-211-0
%R 10.2312/egp.20232002
%I Eurographics Association
%D 2023
%B The European Association for Computer Graphics 44th Annual Conference
%Z date of event: 2023-05-08 - 2023-05-12
%D 2023
%C Saarbr&#252;cken, Germany
%P 18 p.

Conference paper

J. Steimle, M. Muehlhaus, M. L. Nicolae, A. S. Nittala, N. Pourjafarian, A. Sharma, M. Teyssier, M. Koelle, B. Fruchard, and P. Strohmeier

“Design and Fabrication of Body-Based Interfaces (Demo of Saarland HCI Lab),” in CHI ’23, CHI Conference on Human Factors in Computing Systems, Hamburg, Germany, 2023.

@inproceedings{Steimle_CHIEA2023,
TITLE = {Design and Fabrication of Body-Based Interfaces (Demo of {Saarland HCI Lab})},
AUTHOR = {Steimle, J{\"u}rgen and Muehlhaus, Marie and Nicolae, Madalina Luciana and Nittala, Aditya Shekhar and Pourjafarian, Narjes and Sharma, Adwait and Teyssier, Marc and Koelle, Marion and Fruchard, Bruno and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {978-1-4503-9422-2},
DOI = {10.1145/3544549.3583916},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {CHI '23, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Schmidt, Albrecht and V{\"a}{\"a}n{\"a}en, Kaisa and Goyal, Tesh and Peters, Anicia},
PAGES = {1--4},
EID = {446},
ADDRESS = {Hamburg, Germany},
}

Endnote

%0 Conference Proceedings
%A Steimle, J&#252;rgen
%A Muehlhaus, Marie
%A Nicolae, Madalina Luciana
%A Nittala, Aditya Shekhar
%A Pourjafarian, Narjes
%A Sharma, Adwait
%A Teyssier, Marc
%A Koelle, Marion
%A Fruchard, Bruno
%A Strohmeier, Paul
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Design and Fabrication of Body-Based Interfaces (Demo of Saarland HCI Lab) : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-8191-7
%R 10.1145/3544549.3583916
%D 2023
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2023-04-23 - 2023-04-28
%C Hamburg, Germany
%B CHI '23
%E Schmidt, Albrecht; V&#228;&#228;n&#228;en, Kaisa; Goyal, Tesh; Peters, Anicia
%P 1 - 4
%Z sequence number: 446
%I ACM
%@ 978-1-4503-9422-2

Article

L. Surace, M. Wernikowski, C. Tursun, K. Myszkowski, R. Mantiuk, and P. Didyk

“Learning GAN-based Foveated Reconstruction to Recover Perceptually Important Image Features,” ACM Transactions on Applied Perception, vol. 20, no. 2, 2023.

@article{Surace23,
TITLE = {Learning {GAN}-based Foveated Reconstruction to Recover Perceptually Important Image Features},
AUTHOR = {Surace, Luca and Wernikowski, Marek and Tursun, Cara and Myszkowski, Karol and Mantiuk, Rados{\l}aw and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {1544-3558},
DOI = {10.1145/3583072},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
JOURNAL = {ACM Transactions on Applied Perception},
VOLUME = {20},
NUMBER = {2},
PAGES = {1--23},
EID = {7},
}

Endnote

%0 Journal Article
%A Surace, Luca
%A Wernikowski, Marek
%A Tursun, Cara
%A Myszkowski, Karol
%A Mantiuk, Rados&#322;aw
%A Didyk, Piotr
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Learning GAN-based Foveated Reconstruction to Recover Perceptually Important Image Features : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A00D-1
%R 10.1145/3583072
%7 2023-04-21
%D 2023
%J ACM Transactions on Applied Perception
%V 20
%N 2
%& 1
%P 1 - 23
%Z sequence number: 7
%I ACM
%C New York, NY
%@ false
%U https://doi.org/10.1145/3583072

Article

D4D6

C. Wang, A. Serrano, X. Pan, K. Wolski, B. Chen, K. Myszkowski, H.-P. Seidel, C. Theobalt, and T. Leimkühler

“An Implicit Neural Representation for the Image Stack: Depth, All in Focus, and High Dynamic Range,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2023), vol. 42, no. 6, 2023.

@article{Wang_SIGGRAPHASIA23,
TITLE = {An Implicit Neural Representation for the Image Stack: {D}epth, All in Focus, and High Dynamic Range},
AUTHOR = {Wang, Chao and Serrano, Ana and Pan, Xingang and Wolski, Krzysztof and Chen, Bin and Myszkowski, Karol and Seidel, Hans-Peter and Theobalt, Christian and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3618367},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {42},
NUMBER = {6},
PAGES = {1--11},
EID = {221},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2023},
}

Endnote

%0 Journal Article
%A Wang, Chao
%A Serrano, Ana
%A Pan, Xingang
%A Wolski, Krzysztof
%A Chen, Bin
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Theobalt, Christian
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T An Implicit Neural Representation for the Image Stack: Depth, All in Focus, and High Dynamic Range : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000D-B80B-8
%R 10.1145/3618367
%7 2023
%D 2023
%J ACM Transactions on Graphics
%V 42
%N 6
%& 1
%P 1 - 11
%Z sequence number: 221
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2023

%O ACM SIGGRAPH Asia 2023 Sydney, Australia, 12-15 December 2023 SA '23 SA 2023

Conference paper

D4D6

C. Wang, A. Serrano, X. Pan, B. Chen, H.-P. Seidel, C. Theobalt, K. Myszkowski, and T. Leimkühler

“GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild,” in IEEE/CVF International Conference on Computer Vision (ICCV 2023), Paris, France, 2023.

@inproceedings{wang2023glowgan,
TITLE = {{GlowGAN}: {U}nsupervised Learning of {HDR} Images from {LDR} Images in the Wild},
AUTHOR = {Wang, Chao and Serrano, Ana and Pan, Xingang and Chen, Bin and Seidel, Hans-Peter and Theobalt, Christian and Myszkowski, Karol and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
ISBN = {979-8-3503-0718-4},
DOI = {10.1109/ICCV51070.2023.00964},
PUBLISHER = {IEEE},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {IEEE/CVF International Conference on Computer Vision (ICCV 2023)},
PAGES = {10475--10485},
ADDRESS = {Paris, France},
}

Endnote

%0 Conference Proceedings
%A Wang, Chao
%A Serrano, Ana
%A Pan, Xingang
%A Chen, Bin
%A Seidel, Hans-Peter
%A Theobalt, Christian
%A Myszkowski, Karol
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T GlowGAN: Unsupervised Learning of HDR Images from LDR 
Images in the Wild : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000D-B7FC-9
%R 10.1109/ICCV51070.2023.00964
%D 2023
%B IEEE/CVF International Conference on Computer Vision
%Z date of event: 2023-10-01 - 2023-10-06
%C Paris, France
%B IEEE/CVF International Conference on Computer Vision
%P 10475 - 10485
%I IEEE
%@ 979-8-3503-0718-4

Article

A. Weinrauch, H.-P. Seidel, D. Mlakar, M. Steinberger, and R. Zayer

“A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and Reeb Graph Construction on Surfaces,” Computer Graphics Forum, vol. 42, no. 2, 2023.

@article{Weinrauch_CGF23,
TITLE = {A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and {Reeb} Graph Construction on Surfaces},
AUTHOR = {Weinrauch, Alexander and Seidel, Hans-Peter and Mlakar, Daniel and Steinberger, Markus and Zayer, Rhaleb},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.14763},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
JOURNAL = {Computer Graphics Forum},
VOLUME = {42},
NUMBER = {2},
PAGES = {309--320},
}

Endnote

%0 Journal Article
%A Weinrauch, Alexander
%A Seidel, Hans-Peter
%A Mlakar, Daniel
%A Steinberger, Markus
%A Zayer, Rhaleb
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and Reeb Graph Construction on Surfaces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-B851-9
%R 10.1111/cgf.14763
%7 2023-05-23
%D 2023
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 42
%N 2
%& 309
%P 309 - 320
%I Blackwell-Wiley
%C Oxford
%@ false
%U https://doi.org/10.1111/cgf.14763

Conference paper

D. Wittchen, V. Marinez-Missir, S. Mavali, N. Sabnis, C. N. Reed, and P. Strohmeier

“Designing Interactive Shoes for Tactile Augmented Reality,” in AHs ’23, Augmented Humans International Conference, Glasgow, UK, 2023.

@inproceedings{Wittchen_AHs2023,
TITLE = {Designing Interactive Shoes for Tactile Augmented Reality},
AUTHOR = {Wittchen, Dennis and Marinez-Missir, Valenin and Mavali, Sina and Sabnis, Nihar and Reed, Courtney N. and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {978-1-4503-9984-5},
DOI = {10.1145/3582700.3582728},
PUBLISHER = {ACM},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {AHs '23, Augmented Humans International Conference},
PAGES = {1--14},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Wittchen, Dennis
%A Marinez-Missir, Valenin
%A Mavali, Sina
%A Sabnis, Nihar
%A Reed, Courtney N.
%A Strohmeier, Paul
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Designing Interactive Shoes for Tactile Augmented Reality : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A04D-9
%R 10.1145/3582700.3582728
%D 2023
%B Augmented Humans International Conference
%Z date of event: 2023-03-12 - 2023-03-14
%C Glasgow, UK
%B AHs '23
%P 1 - 14
%I ACM
%@ 978-1-4503-9984-5

Thesis

D4IMPR-CS

K. Wolski

“Design and Applications of Perception-based Mesh, Image, and Display-related Quality Metrics,” Universität des Saarlandes, Saarbrücken, 2023.

@phdthesis{ThesisPhDWolski2023,
TITLE = {Design and Applications of Perception-based Mesh, Image, and Display-related Quality Metrics},
AUTHOR = {Wolski, Krzysztof},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-417567},
DOI = {10.22028/D291-41756},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
}

Endnote

%0 Thesis
%A Wolski, Krzysztof
%Y Myszkowski, Karol
%A referee: Seidel, Hans-Peter
%A referee: O'Sullivan, Carol
%A referee: Mantiuk, Rafa&#322;
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Design and Applications of Perception-based Mesh, Image, and Display-related Quality Metrics : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-54B2-9
%R 10.22028/D291-41756 
%U urn:nbn:de:bsz:291--ds-417567
%F OTHER: hdl:20.500.11880/37411
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2023
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/37411

2022

Article

N. Ansari, H.-P. Seidel, and V. Babaei

“Mixed Integer Neural Inverse Design,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2022), vol. 41, no. 4, 2022.

@article{Ansari22,
TITLE = {Mixed Integer Neural Inverse Design},
AUTHOR = {Ansari, Navid and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3528223.3530083},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {41},
NUMBER = {4},
PAGES = {1--14},
EID = {151},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2022},
}

Endnote

%0 Journal Article
%A Ansari, Navid
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Mixed Integer Neural Inverse Design : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-1678-5
%R 10.1145/3528223.3530083
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 4
%& 1
%P 1 - 14
%Z sequence number: 151
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2022
%O ACM SIGGRAPH 2022

Conference paper

N. Ansari, H.-P. Seidel, N. Vahidi Ferdowsi, and V. Babaei

“Autoinverse: Uncertainty Aware Inversion of Neural Networks,” in Advances in Neural Information Processing Systems 35 (NeurIPS 2022), New Orleans, LA, USA, 2022.

@inproceedings{Ansari_Neurips22,
TITLE = {Autoinverse: {U}ncertainty Aware Inversion of Neural Networks},
AUTHOR = {Ansari, Navid and Seidel, Hans-Peter and Vahidi Ferdowsi, Nima and Babaei, Vahid},
LANGUAGE = {eng},
PUBLISHER = {Curran Associates, Inc},
YEAR = {2022},
BOOKTITLE = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
EDITOR = {Koyejo, S. and Mohamed, S. and Agarwal, A. and Belgrave, D. and Cho, K. and Oh, A.},
PAGES = {8675--8686},
ADDRESS = {New Orleans, LA, USA},
}

Endnote

%0 Conference Proceedings
%A Ansari, Navid
%A Seidel, Hans-Peter
%A Vahidi Ferdowsi, Nima
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Autoinverse: Uncertainty Aware Inversion of Neural Networks : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-16F6-6
%D 2022
%B 36th Conference on Neural Information Processing Systems
%Z date of event: 2022-11-28 - 2022-12-09
%C New Orleans, LA, USA
%B Advances in Neural Information Processing Systems 35
%E Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; Oh, A.
%P 8675 - 8686
%I Curran Associates, Inc
%U https://openreview.net/pdf?id=dNyCj1AbOb

Paper

E. Arabadzhiyska, C. Tursun, H.-P. Seidel, and P. Didyk

“Practical Saccade Prediction for Head-Mounted Displays: Towards a Comprehensive Model,” 2022. [Online]. Available: https://arxiv.org/abs/2205.01624.

Abstract

Eye-tracking technology is an integral component of new display devices such
as virtual and augmented reality headsets. Applications of gaze information
range from new interaction techniques exploiting eye patterns to
gaze-contingent digital content creation. However, system latency is still a
significant issue in many of these applications because it breaks the
synchronization between the current and measured gaze positions. Consequently,
it may lead to unwanted visual artifacts and degradation of user experience. In
this work, we focus on foveated rendering applications where the quality of an
image is reduced towards the periphery for computational savings. In foveated
rendering, the presence of latency leads to delayed updates to the rendered
frame, making the quality degradation visible to the user. To address this
issue and to combat system latency, recent work proposes to use saccade landing
position prediction to extrapolate the gaze information from delayed
eye-tracking samples. While the benefits of such a strategy have already been
demonstrated, the solutions range from simple and efficient ones, which make
several assumptions about the saccadic eye movements, to more complex and
costly ones, which use machine learning techniques. Yet, it is unclear to what
extent the prediction can benefit from accounting for additional factors. This
paper presents a series of experiments investigating the importance of
different factors for saccades prediction in common virtual and augmented
reality applications. In particular, we investigate the effects of saccade
orientation in 3D space and smooth pursuit eye-motion (SPEM) and how their
influence compares to the variability across users. We also present a simple
yet efficient correction method that adapts the existing saccade prediction
methods to handle these factors without performing extensive data collection.

BibTeX

@online{Arabadzhiyska2205.01624,
TITLE = {Practical Saccade Prediction for Head-Mounted Displays: Towards a Comprehensive Model},
AUTHOR = {Arabadzhiyska, Elena and Tursun, Cara and Seidel, Hans-Peter and Didyk, Piotr},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2205.01624},
EPRINT = {2205.01624},
EPRINTTYPE = {arXiv},
YEAR = {2022},
ABSTRACT = {Eye-tracking technology is an integral component of new display devices such<br>as virtual and augmented reality headsets. Applications of gaze information<br>range from new interaction techniques exploiting eye patterns to<br>gaze-contingent digital content creation. However, system latency is still a<br>significant issue in many of these applications because it breaks the<br>synchronization between the current and measured gaze positions. Consequently,<br>it may lead to unwanted visual artifacts and degradation of user experience. In<br>this work, we focus on foveated rendering applications where the quality of an<br>image is reduced towards the periphery for computational savings. In foveated<br>rendering, the presence of latency leads to delayed updates to the rendered<br>frame, making the quality degradation visible to the user. To address this<br>issue and to combat system latency, recent work proposes to use saccade landing<br>position prediction to extrapolate the gaze information from delayed<br>eye-tracking samples. While the benefits of such a strategy have already been<br>demonstrated, the solutions range from simple and efficient ones, which make<br>several assumptions about the saccadic eye movements, to more complex and<br>costly ones, which use machine learning techniques. Yet, it is unclear to what<br>extent the prediction can benefit from accounting for additional factors. This<br>paper presents a series of experiments investigating the importance of<br>different factors for saccades prediction in common virtual and augmented<br>reality applications. In particular, we investigate the effects of saccade<br>orientation in 3D space and smooth pursuit eye-motion (SPEM) and how their<br>influence compares to the variability across users. We also present a simple<br>yet efficient correction method that adapts the existing saccade prediction<br>methods to handle these factors without performing extensive data collection.<br>},
}

Endnote

%0 Report
%A Arabadzhiyska, Elena
%A Tursun, Cara
%A Seidel, Hans-Peter
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Practical Saccade Prediction for Head-Mounted Displays: Towards a
  Comprehensive Model : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-16E3-B
%U https://arxiv.org/abs/2205.01624
%D 2022
%X   Eye-tracking technology is an integral component of new display devices such<br>as virtual and augmented reality headsets. Applications of gaze information<br>range from new interaction techniques exploiting eye patterns to<br>gaze-contingent digital content creation. However, system latency is still a<br>significant issue in many of these applications because it breaks the<br>synchronization between the current and measured gaze positions. Consequently,<br>it may lead to unwanted visual artifacts and degradation of user experience. In<br>this work, we focus on foveated rendering applications where the quality of an<br>image is reduced towards the periphery for computational savings. In foveated<br>rendering, the presence of latency leads to delayed updates to the rendered<br>frame, making the quality degradation visible to the user. To address this<br>issue and to combat system latency, recent work proposes to use saccade landing<br>position prediction to extrapolate the gaze information from delayed<br>eye-tracking samples. While the benefits of such a strategy have already been<br>demonstrated, the solutions range from simple and efficient ones, which make<br>several assumptions about the saccadic eye movements, to more complex and<br>costly ones, which use machine learning techniques. Yet, it is unclear to what<br>extent the prediction can benefit from accounting for additional factors. This<br>paper presents a series of experiments investigating the importance of<br>different factors for saccades prediction in common virtual and augmented<br>reality applications. In particular, we investigate the effects of saccade<br>orientation in 3D space and smooth pursuit eye-motion (SPEM) and how their<br>influence compares to the variability across users. We also present a simple<br>yet efficient correction method that adapts the existing saccade prediction<br>methods to handle these factors without performing extensive data collection.<br>
%K Computer Science, Human-Computer Interaction, cs.HC,Computer Science, Graphics, cs.GR

Conference paper

M. Bemana, K. Myszkowski, J. R. Frisvad, H.-P. Seidel, and T. Ritschel

“Eikonal Fields for Refractive Novel-View Synthesis,” in Proceedings SIGGRAPH 2022 Conference Papers Proceedings (ACM SIGGRAPH 2022), Vancouver, Canada, 2022.

@inproceedings{Bemana_SIGGRAPH22,
TITLE = {Eikonal Fields for Refractive Novel-View Synthesis},
AUTHOR = {Bemana, Mojtaba and Myszkowski, Karol and Frisvad, Jeppe Revall and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISBN = {978-1-4503-9337-9},
DOI = {10.1145/3528233.3530706},
PUBLISHER = {ACM},
YEAR = {2022},
BOOKTITLE = {Proceedings SIGGRAPH 2022 Conference Papers Proceedings (ACM SIGGRAPH 2022)},
EDITOR = {Nandigjav, Munkhtsetseg and Mitra, Niloy J. and Hertzmann, Aaron},
PAGES = {1--9},
EID = {39},
ADDRESS = {Vancouver, Canada},
}

Endnote

%0 Conference Proceedings
%A Bemana, Mojtaba
%A Myszkowski, Karol
%A Frisvad, Jeppe Revall
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Eikonal Fields for Refractive Novel-View Synthesis : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-BA61-7
%R 10.1145/3528233.3530706
%D 2022
%B ACM SIGGRAPH
%Z date of event: 2022-08-07 - 2022-08-11
%C Vancouver, Canada
%B Proceedings SIGGRAPH 2022 Conference Papers Proceedings
%E Nandigjav, Munkhtsetseg; Mitra, Niloy J.; Hertzmann, Aaron
%P 1 - 9
%Z sequence number: 39
%I ACM
%@ 978-1-4503-9337-9

Conference paper

B. Chen, M. Piovarči, C. Wang, H.-P. Seidel, P. Didyk, K. Myszkowski, and A. Serrano

“Gloss Management for Consistent Reproduction of Real and Virtual Objects,” in Proceedings SIGGRAPH Asia 2022 (ACM SIGGRAPH Asia 2022), Daegu, Republic of Korea, 2022.

@inproceedings{ChenSA22,
TITLE = {Gloss Management for Consistent Reproduction of Real and Virtual Objects},
AUTHOR = {Chen, Bin and Piovar{\v c}i, Michal and Wang, Chao and Seidel, Hans-Peter and Didyk, Piotr and Myszkowski, Karol and Serrano, Ana},
LANGUAGE = {eng},
ISBN = {978-1-4503-9470-3},
DOI = {10.1145/3550469.3555406},
PUBLISHER = {ACM},
YEAR = {2022},
BOOKTITLE = {Proceedings SIGGRAPH Asia 2022 (ACM SIGGRAPH Asia 2022)},
EDITOR = {Jung, Soon Ki and Lee, Jehee and Bargteil, Adam},
PAGES = {1--9},
EID = {35},
ADDRESS = {Daegu, Republic of Korea},
}

Endnote

%0 Conference Proceedings
%A Chen, Bin
%A Piovar&#269;i, Michal
%A Wang, Chao
%A Seidel, Hans-Peter
%A Didyk, Piotr
%A Myszkowski, Karol
%A Serrano, Ana
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Gloss Management for Consistent Reproduction of Real and Virtual Objects : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-167F-E
%R 10.1145/3550469.3555406
%D 2022
%B SIGGRAPH Asia 2022
%Z date of event: 2022-12-06 - 2022-12-09
%C Daegu, Republic of Korea
%B Proceedings SIGGRAPH Asia 2022
%E Jung, Soon Ki; Lee, Jehee; Bargteil, Adam
%P 1 - 9
%Z sequence number: 35
%I ACM
%@ 978-1-4503-9470-3

Article

V. Chizhov, I. Georgiev, K. Myszkowski, and G. Singh

“Perceptual Error Optimization for Monte Carlo Rendering,” ACM Transactions on Graphics, vol. 41, no. 3, 2022.

@article{ChizhovTOG22,
TITLE = {Perceptual Error Optimization for {Monte Carlo} Rendering},
AUTHOR = {Chizhov, Vassillen and Georgiev, Iliyan and Myszkowski, Karol and Singh, Gurprit},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3504002},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {41},
NUMBER = {3},
PAGES = {1--17},
EID = {26},
}

Endnote

%0 Journal Article
%A Chizhov, Vassillen
%A Georgiev, Iliyan
%A Myszkowski, Karol
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perceptual Error Optimization for Monte Carlo Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-BA49-3
%R 10.1145/3504002
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 3
%& 1
%P 1 - 17
%Z sequence number: 26
%I ACM
%C New York, NY
%@ false

Article

D6D4

M. Chu, L. Liu, Q. Zheng, E. Franz, H.-P. Seidel, C. Theobalt, and R. Zayer

“Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data,” ACM Transactions on Graphics, vol. 41, no. 4, 2022.

@article{Chu2022,
TITLE = {Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data},
AUTHOR = {Chu, Mengyu and Liu, Lingjie and Zheng, Quan and Franz, Erik and Seidel, Hans-Peter and Theobalt, Christian and Zayer, Rhaleb},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3528223.3530169},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {41},
NUMBER = {4},
PAGES = {1--14},
EID = {119},
}

Endnote

%0 Journal Article
%A Chu, Mengyu
%A Liu, Lingjie
%A Zheng, Quan
%A Franz, Erik
%A Seidel, Hans-Peter
%A Theobalt, Christian
%A Zayer, Rhaleb
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Physics Informed Neural Fields for Smoke Reconstruction with Sparse Data : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000B-6561-6
%R 10.1145/3528223.3530169
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 4
%& 1
%P 1 - 14
%Z sequence number: 119
%I ACM
%C New York, NY
%@ false
%U https://people.mpi-inf.mpg.de/~mchu/projects/PI-NeRF/

Article

U. Çoğalan, M. Bemana, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“Learning HDR Video Reconstruction for Dual-Exposure Sensors with Temporally-alternating Exposures,” Computers and Graphics, vol. 105, 2022.

@article{Cogalan2022,
TITLE = {Learning {HDR} Video Reconstruction for Dual-Exposure Sensors with Temporally-alternating Exposures},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur and Bemana, Mojtaba and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {0097-8493},
DOI = {10.1016/j.cag.2022.04.008},
PUBLISHER = {Elsevier},
ADDRESS = {Amsterdam},
YEAR = {2022},
JOURNAL = {Computers and Graphics},
VOLUME = {105},
PAGES = {57--72},
}

Endnote

%0 Journal Article
%A &#199;o&#287;alan, U&#287;ur
%A Bemana, Mojtaba
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Learning HDR Video Reconstruction for Dual-Exposure Sensors with Temporally-alternating Exposures : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-9D95-D
%R 10.1016/j.cag.2022.04.008
%7 2022
%D 2022
%J Computers and Graphics
%V 105
%& 57
%P 57 - 72
%I Elsevier
%C Amsterdam
%@ false

Paper

U. Çoğalan, M. Bemana, H.-P. Seidel, and K. Myszkowski

“Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors,” 2022. [Online]. Available: https://arxiv.org/abs/2206.09485.

Abstract

Video frame interpolation (VFI) enables many important applications that
might involve the temporal domain, such as slow motion playback, or the spatial
domain, such as stop motion sequences. We are focusing on the former task,
where one of the key challenges is handling high dynamic range (HDR) scenes in
the presence of complex motion. To this end, we explore possible advantages of
dual-exposure sensors that readily provide sharp short and blurry long
exposures that are spatially registered and whose ends are temporally aligned.
This way, motion blur registers temporally continuous information on the scene
motion that, combined with the sharp reference, enables more precise motion
sampling within a single camera shot. We demonstrate that this facilitates a
more complex motion reconstruction in the VFI task, as well as HDR frame
reconstruction that so far has been considered only for the originally captured
frames, not in-between interpolated frames. We design a neural network trained
in these tasks that clearly outperforms existing solutions. We also propose a
metric for scene motion complexity that provides important insights into the
performance of VFI methods at the test time.

BibTeX

@online{Cogalan2206.09485,
TITLE = {Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur and Bemana, Mojtaba and Seidel, Hans-Peter and Myszkowski, Karol},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2206.09485},
EPRINT = {2206.09485},
EPRINTTYPE = {arXiv},
YEAR = {2022},
ABSTRACT = {Video frame interpolation (VFI) enables many important applications that<br>might involve the temporal domain, such as slow motion playback, or the spatial<br>domain, such as stop motion sequences. We are focusing on the former task,<br>where one of the key challenges is handling high dynamic range (HDR) scenes in<br>the presence of complex motion. To this end, we explore possible advantages of<br>dual-exposure sensors that readily provide sharp short and blurry long<br>exposures that are spatially registered and whose ends are temporally aligned.<br>This way, motion blur registers temporally continuous information on the scene<br>motion that, combined with the sharp reference, enables more precise motion<br>sampling within a single camera shot. We demonstrate that this facilitates a<br>more complex motion reconstruction in the VFI task, as well as HDR frame<br>reconstruction that so far has been considered only for the originally captured<br>frames, not in-between interpolated frames. We design a neural network trained<br>in these tasks that clearly outperforms existing solutions. We also propose a<br>metric for scene motion complexity that provides important insights into the<br>performance of VFI methods at the test time.<br>},
}

Endnote

%0 Report
%A &#199;o&#287;alan, U&#287;ur
%A Bemana, Mojtaba
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Video Frame Interpolation for High Dynamic Range Sequences Captured with Dual-exposure Sensors : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-16E8-6
%U https://arxiv.org/abs/2206.09485
%D 2022
%X   Video frame interpolation (VFI) enables many important applications that<br>might involve the temporal domain, such as slow motion playback, or the spatial<br>domain, such as stop motion sequences. We are focusing on the former task,<br>where one of the key challenges is handling high dynamic range (HDR) scenes in<br>the presence of complex motion. To this end, we explore possible advantages of<br>dual-exposure sensors that readily provide sharp short and blurry long<br>exposures that are spatially registered and whose ends are temporally aligned.<br>This way, motion blur registers temporally continuous information on the scene<br>motion that, combined with the sharp reference, enables more precise motion<br>sampling within a single camera shot. We demonstrate that this facilitates a<br>more complex motion reconstruction in the VFI task, as well as HDR frame<br>reconstruction that so far has been considered only for the originally captured<br>frames, not in-between interpolated frames. We design a neural network trained<br>in these tasks that clearly outperforms existing solutions. We also propose a<br>metric for scene motion complexity that provides important insights into the<br>performance of VFI methods at the test time.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

J. Hladký, M. Stengel, N. Vining, B. Kerbl, H.-P. Seidel, and M. Steinberger

“QuadStream: A Quad-Based Scene Streaming Architecture for Novel Viewpoint Reconstruction,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2022), vol. 41, no. 6, 2022.

@article{HladkySIGGRAPHAsia22,
TITLE = {QuadStream: {A} Quad-Based Scene Streaming Architecture for Novel Viewpoint Reconstruction},
AUTHOR = {Hladk{\'y}, Jozef and Stengel, Michael and Vining, Nicholas and Kerbl, Bernhard and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3550454.3555524},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {41},
NUMBER = {6},
PAGES = {1--13},
EID = {233},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2022},
}

Endnote

%0 Journal Article
%A Hladk&#253;, Jozef
%A Stengel, Michael
%A Vining, Nicholas
%A Kerbl, Bernhard
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T QuadStream: A Quad-Based Scene Streaming Architecture for Novel Viewpoint Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-208B-3
%R 10.1145/3550454.3555524
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 6
%& 1
%P 1 - 13
%Z sequence number: 233
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2022

%O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

Thesis

D4IMPR-CS

J. Hladký

“Latency Hiding and High Fidelity Novel View Synthesis on Thin Clients Using Decoupled Streaming Rendering from Powerful Servers,” Universität des Saarlandes, Saarbrücken, 2022.

@phdthesis{Hladky_PhD22,
TITLE = {Latency Hiding and High Fidelity Novel View Synthesis on Thin Clients Using Decoupled Streaming Rendering from Powerful Servers},
AUTHOR = {Hladk{\'y}, Jozef},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-376882},
DOI = {10.22028/D291-37688},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2022},
DATE = {2022},
}

Endnote

%0 Thesis
%A Hladk&#253;, Jozef
%Y Seidel, Hans-Peter
%A referee: Steinberger, Markus
%A referee: Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Latency Hiding and High Fidelity Novel View Synthesis on Thin Clients Using Decoupled Streaming Rendering from Powerful Servers : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-24A1-2
%R 10.22028/D291-37688
%U urn:nbn:de:bsz:291--ds-376882
%F OTHER: hdl:20.500.11880/34640
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2022
%V phd
%9 phd
%U https://scidok.sulb.uni-saarland.de/handle/20.500.11880/34640

Article

X. Huang, P. Memari, H.-P. Seidel, and G. Singh

“Point-Pattern Synthesis using Gabor and Random Filters,” Computer Graphics Forum (Proc. Eurographics Symposium on Rendering 2022), vol. 41, no. 4, 2022.

@article{Huang_EGSR2022,
TITLE = {Point-Pattern Synthesis using {Gabor} and Random Filters},
AUTHOR = {Huang, Xingchang and Memari, Pooran and Seidel, Hans-Peter and Singh, Gurprit},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.14596},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford},
YEAR = {2022},
DATE = {2022},
JOURNAL = {Computer Graphics Forum (Proc. Eurographics Symposium on Rendering)},
VOLUME = {41},
NUMBER = {4},
PAGES = {169--179},
BOOKTITLE = {Eurographics Symposium on Rendering 2022},
EDITOR = {Ghosh, Abhijeet and Wei, Li-Yi and Wilkie, Alexander},
}

Endnote

%0 Journal Article
%A Huang, Xingchang
%A Memari, Pooran
%A Seidel, Hans-Peter
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Point-Pattern Synthesis using Gabor and Random Filters : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-1675-8
%R 10.1111/cgf.14596
%7 2022
%D 2022
%J Computer Graphics Forum
%V 41
%N 4
%& 169
%P 169 - 179
%I Wiley-Blackwell
%C Oxford
%@ false
%B Eurographics Symposium on Rendering 2022
%O Eurographics Symposium on Rendering 2022 EGSR 2022 Prague, Czech Republic & Virtual ; 4 - 6 July 2022
%U https://onlinelibrary.wiley.com/share/X44DPUPXHCYNCUKSEBEE?target=10.1111/cgf.14596

Article

G. Kopanas, T. Leimkühler, G. Rainer, C. Jambon, and G. Drettakis

“Neural Point Catacaustics for Novel-View Synthesis of Reflections,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2022), vol. 41, no. 6, 2022.

@article{KopanasSIGGRAPHAsia22,
TITLE = {Neural Point Catacaustics for Novel-View Synthesis of Reflections},
AUTHOR = {Kopanas, Georgios and Leimk{\"u}hler, Thomas and Rainer, Gilles and Jambon, Cl{\'e}ment and Drettakis, George},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3550454.3555497},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {41},
NUMBER = {6},
PAGES = {1--15},
EID = {201},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2022},
}

Endnote

%0 Journal Article
%A Kopanas, Georgios
%A Leimk&#252;hler, Thomas
%A Rainer, Gilles
%A Jambon, Cl&#233;ment
%A Drettakis, George
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T Neural Point Catacaustics for Novel-View Synthesis of Reflections : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-209B-1
%R 10.1145/3550454.3555497
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 6
%& 1
%P 1 - 15
%Z sequence number: 201
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2022

%O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

Paper

J. Li, B. Chen, G. Zan, G. Qian, P. Pianetta, and Y. Liu

“Subspace Modeling for Fast and High-sensitivity X-ray Chemical Imaging,” 2022. [Online]. Available: https://arxiv.org/abs/2201.00259.

Abstract

Resolving morphological chemical phase transformations at the nanoscale is of
vital importance to many scientific and industrial applications across various
disciplines. The TXM-XANES imaging technique, by combining full field
transmission X-ray microscopy (TXM) and X-ray absorption near edge structure
(XANES), has been an emerging tool which operates by acquiring a series of
microscopy images with multi-energy X-rays and fitting to obtain the chemical
map. Its capability, however, is limited by the poor signal-to-noise ratios due
to the system errors and low exposure illuminations for fast acquisition. In
this work, by exploiting the intrinsic properties and subspace modeling of the
TXM-XANES imaging data, we introduce a simple and robust denoising approach to
improve the image quality, which enables fast and high-sensitivity chemical
imaging. Extensive experiments on both synthetic and real datasets demonstrate
the superior performance of the proposed method.

BibTeX

@online{li2022subspace,
TITLE = {Subspace Modeling for Fast and High-sensitivity X-ray Chemical Imaging},
AUTHOR = {Li, Jizhou and Chen, Bin and Zan, Guibin and Qian, Guannan and Pianetta, Piero and Liu, Yijin},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2201.00259},
EPRINT = {2201.00259},
EPRINTTYPE = {arXiv},
YEAR = {2022},
ABSTRACT = {Resolving morphological chemical phase transformations at the nanoscale is of<br>vital importance to many scientific and industrial applications across various<br>disciplines. The TXM-XANES imaging technique, by combining full field<br>transmission X-ray microscopy (TXM) and X-ray absorption near edge structure<br>(XANES), has been an emerging tool which operates by acquiring a series of<br>microscopy images with multi-energy X-rays and fitting to obtain the chemical<br>map. Its capability, however, is limited by the poor signal-to-noise ratios due<br>to the system errors and low exposure illuminations for fast acquisition. In<br>this work, by exploiting the intrinsic properties and subspace modeling of the<br>TXM-XANES imaging data, we introduce a simple and robust denoising approach to<br>improve the image quality, which enables fast and high-sensitivity chemical<br>imaging. Extensive experiments on both synthetic and real datasets demonstrate<br>the superior performance of the proposed method.<br>},
}

Endnote

%0 Report
%A Li, Jizhou
%A Chen, Bin
%A Zan, Guibin
%A Qian, Guannan
%A Pianetta, Piero
%A Liu, Yijin
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
%T Subspace Modeling for Fast and High-sensitivity X-ray Chemical Imaging : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-C7BE-E
%U https://arxiv.org/abs/2201.00259
%D 2022
%X   Resolving morphological chemical phase transformations at the nanoscale is of<br>vital importance to many scientific and industrial applications across various<br>disciplines. The TXM-XANES imaging technique, by combining full field<br>transmission X-ray microscopy (TXM) and X-ray absorption near edge structure<br>(XANES), has been an emerging tool which operates by acquiring a series of<br>microscopy images with multi-energy X-rays and fitting to obtain the chemical<br>map. Its capability, however, is limited by the poor signal-to-noise ratios due<br>to the system errors and low exposure illuminations for fast acquisition. In<br>this work, by exploiting the intrinsic properties and subspace modeling of the<br>TXM-XANES imaging data, we introduce a simple and robust denoising approach to<br>improve the image quality, which enables fast and high-sensitivity chemical<br>imaging. Extensive experiments on both synthetic and real datasets demonstrate<br>the superior performance of the proposed method.<br>
%K eess.IV,Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Multimedia, cs.MM

Article

F. Mueller, N. Semertzidis, J. Andres, M. Weigel, S. Nanayakkara, R. Patibanda, Z. Li, P. Strohmeier, J. Knibbe, S. Greuter, M. Obrist, P. Maes, D. Wang, K. Wolf, L. Gerber, J. Marshall, K. Kunze, J. Grudin, H. Reiterer, and R. Byrne

“Human–Computer Integration: Towards Integrating the Human Body with the Computational Machine,” Foundations and Trends in Human-Computer Interaction, vol. 16, no. 1, 2022.

@article{Mueller22,
TITLE = {Human--Computer Integration: {T}owards Integrating the Human Body with the Computational Machine},
AUTHOR = {Mueller, Florian and Semertzidis, Nathan and Andres, Josh and Weigel, Martin and Nanayakkara, Suranga and Patibanda, Rakesh and Li, Zhuying and Strohmeier, Paul and Knibbe, Jarrod and Greuter, Stefan and Obrist, Marianna and Maes, Pattie and Wang, Dakuo and Wolf, Katrin and Gerber, Liz and Marshall, Joe and Kunze, Kai and Grudin, Jonathan and Reiterer, Harald and Byrne, Richard},
LANGUAGE = {eng},
ISSN = {1551-3955},
ISBN = {978-1-63828-068-2},
DOI = {10.1561/1100000086},
YEAR = {2022},
JOURNAL = {Foundations and Trends in Human-Computer Interaction},
VOLUME = {16},
NUMBER = {1},
PAGES = {1--64},
}

Endnote

%0 Journal Article
%A Mueller, Florian
%A Semertzidis, Nathan
%A Andres, Josh
%A Weigel, Martin
%A Nanayakkara, Suranga
%A Patibanda, Rakesh
%A Li, Zhuying
%A Strohmeier, Paul
%A Knibbe, Jarrod
%A Greuter, Stefan
%A Obrist, Marianna
%A Maes, Pattie
%A Wang, Dakuo
%A Wolf, Katrin
%A Gerber, Liz
%A Marshall, Joe
%A Kunze, Kai
%A Grudin, Jonathan
%A Reiterer, Harald
%A Byrne, Richard
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Human&#8211;Computer Integration: Towards Integrating the Human Body with the Computational Machine : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-1734-0
%R 10.1561/1100000086
%@ 978-1-63828-068-2
%7 2022
%D 2022
%J Foundations and Trends in Human-Computer Interaction
%O Foundations and Trends&#174; in Human-Computer Interaction
%V 16
%N 1
%& 1
%P 1 - 64
%@ false

Article

J. Panetta, H. Mohammadian, E. Luci, and V. Babaei

“Shape from Release: Inverse Design and Fabrication of Controlled Release Structures,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2022), vol. 41, no. 6, 2022.

@article{PanettalSIGGRAPHAsia22,
TITLE = {Shape from Release: Inverse Design and Fabrication of Controlled Release Structures},
AUTHOR = {Panetta, Julian and Mohammadian, Haleh and Luci, Emiliano and Babaei, Vahid},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3550454.3555518},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {41},
NUMBER = {6},
PAGES = {1--14},
EID = {274},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2022},
}

Endnote

%0 Journal Article
%A Panetta, Julian
%A Mohammadian, Haleh
%A Luci, Emiliano
%A Babaei, Vahid
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Shape from Release: Inverse Design and Fabrication of Controlled Release Structures : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000B-5E7D-1
%R 10.1145/3550454.3555518
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 6
%& 1
%P 1 - 14
%Z sequence number: 274
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2022

%O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

Article

M. Piovarči, M. Foshey, J. Xu, T. Erps, V. Babaei, P. Didyk, S. Rusinkiewicz, W. Matusik, and B. Bickel

“Closed-Loop Control of Direct Ink Writing via Reinforcement Learning,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2022), vol. 41, no. 4, 2022.

@article{PiovarciSIGGRAPH22,
TITLE = {Closed-Loop Control of Direct Ink Writing via Reinforcement Learning},
AUTHOR = {Piovar{\v c}i, Michal and Foshey, Michael and Xu, Jie and Erps, Timmothy and Babaei, Vahid and Didyk, Piotr and Rusinkiewicz, Szymon and Matusik, Wojciech and Bickel, Bernd},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3528223.3530144},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {41},
NUMBER = {4},
PAGES = {1--10},
EID = {112},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2022},
}

Endnote

%0 Journal Article
%A Piovar&#269;i, Michal
%A Foshey, Michael
%A Xu, Jie
%A Erps, Timmothy
%A Babaei, Vahid
%A Didyk, Piotr
%A Rusinkiewicz, Szymon
%A Matusik, Wojciech
%A Bickel, Bernd
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
%T Closed-Loop Control of Direct Ink Writing via Reinforcement Learning : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-178D-C
%R 10.1145/3528223.3530144
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 4
%& 1
%P 1 - 10
%Z sequence number: 112
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2022
%O ACM SIGGRAPH 2022

Conference paper

N. Pourjafarian, M. Koelle, F. Mjaku, P. Strohmeier, and J. Steimle

“Print-A-Sketch: A Handheld Printer for Physical Sketching of Circuits and Sensors on Everyday Surfaces,” in CHI ’22, CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 2022.

@inproceedings{Pourjafarian_CHI2022,
TITLE = {{Print-A-Sketch}: {A} Handheld Printer for Physical Sketching of Circuits and Sensors on Everyday Surfaces},
AUTHOR = {Pourjafarian, Narjes and Koelle, Marion and Mjaku, Fjolla and Strohmeier, Paul and Steimle, J{\"u}rgen},
LANGUAGE = {eng},
ISBN = {9781450391573},
DOI = {10.1145/3491102.3502074},
PUBLISHER = {ACM},
YEAR = {2022},
BOOKTITLE = {CHI '22, CHI Conference on Human Factors in Computing Systems},
PAGES = {1--17},
EID = {270},
ADDRESS = {New Orleans, LA, USA},
}

Endnote

%0 Conference Proceedings
%A Pourjafarian, Narjes
%A Koelle, Marion
%A Mjaku, Fjolla
%A Strohmeier, Paul
%A Steimle, J&#252;rgen
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Print-A-Sketch: A Handheld Printer for Physical Sketching of
Circuits and Sensors on Everyday Surfaces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-215D-9
%R 10.1145/3491102.3502074
%D 2022
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2022-04-29 - 2022-05-05
%C New Orleans, LA, USA
%B CHI '22
%P 1 - 17
%Z sequence number: 270
%I ACM
%@ 9781450391573

Conference paper

D4D2

S. Rao, M. Böhle, and B. Schiele

“Towards Better Understanding Attribution Methods,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), New Orleans, LA, USA, 2022.

@inproceedings{Rao_CVPR2022,
TITLE = {Towards Better Understanding Attribution Methods},
AUTHOR = {Rao, Sukrut and B{\"o}hle, Moritz and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {978-1-6654-6946-3},
DOI = {10.1109/CVPR52688.2022.00998},
PUBLISHER = {IEEE},
YEAR = {2022},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)},
PAGES = {10213--10222},
ADDRESS = {New Orleans, LA, USA},
}

Endnote

%0 Conference Proceedings
%A Rao, Sukrut
%A B&#246;hle, Moritz
%A Schiele, Bernt
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Towards Better Understanding Attribution Methods : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-6F91-6
%R 10.1109/CVPR52688.2022.00998
%D 2022
%B 35th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2022-06-19 - 2022-06-24
%C New Orleans, LA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 10213 - 10222
%I IEEE
%@ 978-1-6654-6946-3

Conference paper

C. N. Reed, C. Nordmoen, A. Martelloni, G. Lepri, N. Robson, E. Zayas-Garin, K. Cotton, L. Mice, and A. McPherson

“Exploring Experiences with New Musical Instruments through Micro-phenomenology,” in NIME 2022, International Conference on New Interfaces for Musical Expression, Auckland, New Zealand, 2022.

@inproceedings{Reed2022Exploring,
TITLE = {Exploring Experiences with New Musical Instruments through Micro-phenomenology},
AUTHOR = {Reed, Courtney N. and Nordmoen, Charlotte and Martelloni, Andrea and Lepri, Giacomo and Robson, Nicole and Zayas-Garin, Eevee and Cotton, Kelsey and Mice, Lia and McPherson, Andrew},
LANGUAGE = {eng},
DOI = {10.21428/92fbeb44.b304e4b1},
PUBLISHER = {PubPub},
YEAR = {2022},
BOOKTITLE = {NIME 2022, International Conference on New Interfaces for Musical Expression},
ADDRESS = {Auckland, New Zealand},
}

Endnote

%0 Conference Proceedings
%A Reed, Courtney N.
%A Nordmoen, Charlotte
%A Martelloni, Andrea
%A Lepri, Giacomo
%A Robson, Nicole
%A Zayas-Garin, Eevee
%A Cotton, Kelsey
%A Mice, Lia
%A McPherson, Andrew
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Exploring Experiences with New Musical Instruments through Micro-phenomenology : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A023-7
%R 10.21428/92fbeb44.b304e4b1
%D 2022
%B International Conference on New Interfaces for Musical Expression
%Z date of event: 2022-06-28 - 2022-07-01
%C Auckland, New Zealand
%B NIME 2022
%I PubPub

Conference paper

C. N. Reed, S. Skach, P. Strohmeier, and A. P. McPherson

“Singing Knit: Soft Knit Biosensing for Augmenting Vocal Performances,” in AHs ’22, Augmented Humans International Conference, Munich, Germany (Hybrid), 2022.

@inproceedings{Reed_AHs2022,
TITLE = {Singing Knit: {S}oft Knit Biosensing for Augmenting Vocal Performances},
AUTHOR = {Reed, Courtney N. and Skach, Sophie and Strohmeier, Paul and McPherson, Andrew P.},
LANGUAGE = {eng},
DOI = {10.1145/3519391.3519412},
PUBLISHER = {ACM},
YEAR = {2022},
BOOKTITLE = {AHs '22, Augmented Humans International Conference},
PAGES = {170--183},
ADDRESS = {Munich, Germany (Hybrid)},
}

Endnote

%0 Conference Proceedings
%A Reed, Courtney N.
%A Skach, Sophie
%A Strohmeier, Paul
%A McPherson, Andrew P.
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Singing Knit: Soft Knit Biosensing for Augmenting Vocal
Performances : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-2178-A
%R 10.1145/3519391.3519412
%D 2022
%B Augmented Humans International Conference
%Z date of event: 2022-03-13 - 2022-03-15
%C Munich, Germany (Hybrid)
%B AHs '22
%P 170 - 183
%I ACM

Conference paper

L. Ruan, B. Chen, J. Li, and M. Lam

“Learning to Deblur using Light Field Generated and Real Defocus Images,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), New Orleans, LA, USA, 2022.

Abstract

Although considerable progress has been made in semantic scene understanding
under clear weather, it is still a tough problem under adverse weather
conditions, such as dense fog, due to the uncertainty caused by imperfect
observations. Besides, difficulties in collecting and labeling foggy images
hinder the progress of this field. Considering the success in semantic scene
understanding under clear weather, we think it is reasonable to transfer
knowledge learned from clear images to the foggy domain. As such, the problem
becomes to bridge the domain gap between clear images and foggy images. Unlike
previous methods that mainly focus on closing the domain gap caused by fog --
defogging the foggy images or fogging the clear images, we propose to alleviate
the domain gap by considering fog influence and style variation simultaneously.
The motivation is based on our finding that the style-related gap and the
fog-related gap can be divided and closed respectively, by adding an
intermediate domain. Thus, we propose a new pipeline to cumulatively adapt
style, fog and the dual-factor (style and fog). Specifically, we devise a
unified framework to disentangle the style factor and the fog factor
separately, and then the dual-factor from images in different domains.
Furthermore, we collaborate the disentanglement of three factors with a novel
cumulative loss to thoroughly disentangle these three factors. Our method
achieves the state-of-the-art performance on three benchmarks and shows
generalization ability in rainy and snowy scenes.

BibTeX

@inproceedings{Ruan_CVPR2022,
TITLE = {Learning to Deblur using Light Field Generated and Real Defocus Images},
AUTHOR = {Ruan, Lingyan and Chen, Bin and Li, Jizhou and Lam, Miuling},
LANGUAGE = {eng},
ISBN = {978-1-6654-6946-3},
DOI = {10.1109/CVPR52688.2022.01582},
PUBLISHER = {IEEE},
YEAR = {2022},
ABSTRACT = {Although considerable progress has been made in semantic scene understanding<br>under clear weather, it is still a tough problem under adverse weather<br>conditions, such as dense fog, due to the uncertainty caused by imperfect<br>observations. Besides, difficulties in collecting and labeling foggy images<br>hinder the progress of this field. Considering the success in semantic scene<br>understanding under clear weather, we think it is reasonable to transfer<br>knowledge learned from clear images to the foggy domain. As such, the problem<br>becomes to bridge the domain gap between clear images and foggy images. Unlike<br>previous methods that mainly focus on closing the domain gap caused by fog --<br>defogging the foggy images or fogging the clear images, we propose to alleviate<br>the domain gap by considering fog influence and style variation simultaneously.<br>The motivation is based on our finding that the style-related gap and the<br>fog-related gap can be divided and closed respectively, by adding an<br>intermediate domain. Thus, we propose a new pipeline to cumulatively adapt<br>style, fog and the dual-factor (style and fog). Specifically, we devise a<br>unified framework to disentangle the style factor and the fog factor<br>separately, and then the dual-factor from images in different domains.<br>Furthermore, we collaborate the disentanglement of three factors with a novel<br>cumulative loss to thoroughly disentangle these three factors. Our method<br>achieves the state-of-the-art performance on three benchmarks and shows<br>generalization ability in rainy and snowy scenes.<br>},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)},
PAGES = {16283--16292},
ADDRESS = {New Orleans, LA, USA},
}

Endnote

%0 Conference Proceedings
%A Ruan, Lingyan
%A Chen, Bin
%A Li, Jizhou
%A Lam, Miuling
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Learning to Deblur using Light Field Generated and Real Defocus Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-4354-A
%R 10.1109/CVPR52688.2022.01582
%D 2022
%B 35th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2022-06-19 - 2022-06-24
%C New Orleans, LA, USA
%X   Although considerable progress has been made in semantic scene understanding<br>under clear weather, it is still a tough problem under adverse weather<br>conditions, such as dense fog, due to the uncertainty caused by imperfect<br>observations. Besides, difficulties in collecting and labeling foggy images<br>hinder the progress of this field. Considering the success in semantic scene<br>understanding under clear weather, we think it is reasonable to transfer<br>knowledge learned from clear images to the foggy domain. As such, the problem<br>becomes to bridge the domain gap between clear images and foggy images. Unlike<br>previous methods that mainly focus on closing the domain gap caused by fog --<br>defogging the foggy images or fogging the clear images, we propose to alleviate<br>the domain gap by considering fog influence and style variation simultaneously.<br>The motivation is based on our finding that the style-related gap and the<br>fog-related gap can be divided and closed respectively, by adding an<br>intermediate domain. Thus, we propose a new pipeline to cumulatively adapt<br>style, fog and the dual-factor (style and fog). Specifically, we devise a<br>unified framework to disentangle the style factor and the fog factor<br>separately, and then the dual-factor from images in different domains.<br>Furthermore, we collaborate the disentanglement of three factors with a novel<br>cumulative loss to thoroughly disentangle these three factors. Our method<br>achieves the state-of-the-art performance on three benchmarks and shows<br>generalization ability in rainy and snowy scenes.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 16283 - 16292
%I IEEE
%@ 978-1-6654-6946-3

Article

C. Salaün, I. Georgiev, H.-P. Seidel, and G. Singh

“Scalable Multi-Class Sampling via Filtered Sliced Optimal Transport,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2022), vol. 41, no. 6, 2022.

@article{SalauenSIGGRAPHAsia22,
TITLE = {Scalable Multi-Class Sampling via Filtered Sliced Optimal Transport},
AUTHOR = {Sala{\"u}n, Corentin and Georgiev, Iliyan and Seidel, Hans-Peter and Singh, Gurprit},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3550454.3555484},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {41},
NUMBER = {6},
PAGES = {1--14},
EID = {261},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2022},
}

Endnote

%0 Journal Article
%A Sala&#252;n, Corentin
%A Georgiev, Iliyan
%A Seidel, Hans-Peter
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Scalable Multi-Class Sampling via Filtered Sliced Optimal Transport : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-1716-2
%R 10.1145/3550454.3555484
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 6
%& 1
%P 1 - 14
%Z sequence number: 261
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2022

%O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

Article

C. Salaün, A. Gruson, B.-S. Hua, T. Hachisuka, and G. Singh

“Regression-based Monte Carlo integration,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2022), vol. 41, no. 4, 2022.

@article{Salauen_SIGGRAPH22,
TITLE = {Regression-based {Monte Carlo} integration},
AUTHOR = {Sala{\"u}n, Corentin and Gruson, Adrien and Hua, Binh-Son and Hachisuka, Toshiya and Singh, Gurprit},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3528223.3530095},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {41},
NUMBER = {4},
PAGES = {1--14},
EID = {79},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2022},
}

Endnote

%0 Journal Article
%A Sala&#252;n, Corentin
%A Gruson, Adrien
%A Hua, Binh-Son
%A Hachisuka, Toshiya
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Regression-based Monte Carlo integration : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-1793-4
%R 10.1145/3528223.3530095
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 4
%& 1
%P 1 - 14
%Z sequence number: 79
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2022
%O ACM SIGGRAPH 2022

Conference paper

O. Schneider, B. Fruchard, D. Wittchen, B. R. Joshi, G. Freitag, D. Degraen, and P. Strohmeier

“Sustainable Haptic Design: Improving Collaboration, Sharing, and Reuse in Haptic Design Research,” in CHI ’22, CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 2022.

@inproceedings{Schneider_CHIEA22,
TITLE = {Sustainable Haptic Design: {I}mproving Collaboration, Sharing, and Reuse in Haptic Design Research},
AUTHOR = {Schneider, Oliver and Fruchard, Bruno and Wittchen, Dennis and Joshi, Bibhushan Raj and Freitag, Georg and Degraen, Donald and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {9781450391566},
DOI = {10.1145/3491101.3503734},
PUBLISHER = {ACM},
YEAR = {2022},
BOOKTITLE = {CHI '22, CHI Conference on Human Factors in Computing Systems},
EDITOR = {Barbosa, Simone and Lampe, Cliff and Appert, Caroline and Shamma, David A.},
PAGES = {1--5},
EID = {79},
ADDRESS = {New Orleans, LA, USA},
}

Endnote

%0 Conference Proceedings
%A Schneider, Oliver
%A Fruchard, Bruno
%A Wittchen, Dennis
%A Joshi, Bibhushan Raj
%A Freitag, Georg
%A Degraen, Donald
%A Strohmeier, Paul
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Sustainable Haptic Design: Improving Collaboration, Sharing, and Reuse in Haptic Design Research : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-1748-A
%R 10.1145/3491101.3503734
%D 2022
%B CHI Conference on Human Factors in Computing Systems
%Z date of event: 2022-04-30 - 2022-05-05
%C New Orleans, LA, USA
%B CHI '22
%E Barbosa, Simone; Lampe, Cliff; Appert, Caroline; Shamma, David A.
%P 1 - 5
%Z sequence number: 79
%I ACM
%@ 9781450391566

Conference paper

Z. Shen, C. Lin, K. Liao, L. Nie, Z. Zheng, and Y. Zhao

“PanoFormer: Panorama Transformer for Indoor 360° Depth Estimation,” in Computer Vision -- ECCV 2022, Tel Aviv, Israel, 2022.

@inproceedings{Shen_ECCV2022,
TITLE = {{PanoFormer}: {P}anorama Transformer for Indoor 360$^{\circ}$ Depth Estimation},
AUTHOR = {Shen, Zhijie and Lin, Chunyu and Liao, Kang and Nie, Lang and Zheng, Zishuo and Zhao, Yao},
LANGUAGE = {eng},
ISBN = {978-3-031-19768-0},
DOI = {10.1007/978-3-031-19769-7_12},
PUBLISHER = {Springer},
YEAR = {2022},
DATE = {2022},
BOOKTITLE = {Computer Vision -- ECCV 2022},
EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni Maria and Hassner, Tal},
PAGES = {195--211},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {13661},
ADDRESS = {Tel Aviv, Israel},
}

Endnote

%0 Conference Proceedings
%A Shen, Zhijie
%A Lin, Chunyu
%A Liao, Kang
%A Nie, Lang
%A Zheng, Zishuo
%A Zhao, Yao
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T PanoFormer: Panorama Transformer for Indoor 360&#176;
Depth Estimation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-9585-5
%R 10.1007/978-3-031-19769-7_12
%D 2022
%B 17th European Conference on Computer Vision
%Z date of event: 2022-10-23 - 2022-10-27
%C Tel Aviv, Israel
%B Computer Vision -- ECCV 2022
%E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni Maria; Hassner, Tal
%P 195 - 211
%I Springer
%@ 978-3-031-19768-0
%B Lecture Notes in Computer Science
%N 13661
%U https://rdcu.be/c5Ays

Conference paper

D6D2D4

S. Shimada, V. Golyanik, Z. Li, P. Pérez, W. Xu, and C. Theobalt

“HULC: 3D HUman Motion Capture with Pose Manifold SampLing and Dense Contact Guidance,” in Computer Vision -- ECCV 2022, Tel Aviv, Israel, 2022.

@inproceedings{Shimada_ECCV2022,
TITLE = {{HULC}: {3D} {HU}man Motion Capture with Pose Manifold Samp{Li}ng and Dense {C}ontact Guidance},
AUTHOR = {Shimada, Soshi and Golyanik, Vladislav and Li, Zhi and P{\'e}rez, Patrick and Xu, Weipeng and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-3-031-20046-5},
DOI = {10.1007/978-3-031-20047-2_30},
PUBLISHER = {Springer},
YEAR = {2022},
DATE = {2022},
BOOKTITLE = {Computer Vision -- ECCV 2022},
EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni Maria and Hassner, Tal},
PAGES = {516--533},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {13682},
ADDRESS = {Tel Aviv, Israel},
}

Endnote

%0 Conference Proceedings
%A Shimada, Soshi
%A Golyanik, Vladislav
%A Li, Zhi
%A P&#233;rez, Patrick
%A Xu, Weipeng
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T HULC: 3D HUman Motion Capture with Pose Manifold SampLing and Dense Contact Guidance : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000B-7918-3
%R 10.1007/978-3-031-20047-2_30
%D 2022
%B 17th European Conference on Computer Vision
%Z date of event: 2022-10-23 - 2022-10-27
%C Tel Aviv, Israel
%B Computer Vision -- ECCV 2022
%E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni Maria; Hassner, Tal
%P 516 - 533
%I Springer
%@ 978-3-031-20046-5
%B Lecture Notes in Computer Science
%N 13682
%U https://rdcu.be/c0aoZ

Book chapter / section

P. Strohmeier, A. Mottelson, H. Pohl, J. McIntosh, J. Knibbe, J. Bergström, Y. Jansen, and K. Hornbæk

“Body-based user interfaces,” in The Routledge Handbook of Bodily Awareness, London: Routledge, 2022.

@incollection{strohmeier2022body,
TITLE = {Body-based user interfaces},
AUTHOR = {Strohmeier, Paul and Mottelson, Aske and Pohl, Henning and McIntosh, Jess and Knibbe, Jarrod and Bergstr{\"o}m, Joanna and Jansen, Yvonne and Hornb{\ae}k, Kasper},
LANGUAGE = {eng},
ISBN = {9780429321542},
DOI = {10.4324/9780429321542},
PUBLISHER = {Routledge},
ADDRESS = {London},
YEAR = {2022},
DATE = {2022},
BOOKTITLE = {The Routledge Handbook of Bodily Awareness},
EDITOR = {Alsmith, Adrian J. T. and Longo, Matthew R.},
PAGES = {478--502},
}

Endnote

%0 Book Section
%A Strohmeier, Paul
%A Mottelson, Aske
%A Pohl, Henning
%A McIntosh, Jess
%A Knibbe, Jarrod
%A Bergstr&#246;m, Joanna
%A Jansen, Yvonne
%A Hornb&#230;k, Kasper
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Body-based user interfaces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-A013-9
%D 2022
%B The Routledge Handbook of Bodily Awareness
%E Alsmith, Adrian J. T.; Longo, Matthew R.
%P 478 - 502
%I Routledge
%C London
%@ 9780429321542

Paper

C. Wang, A. Serrano, X. Pan, B. Chen, H.-P. Seidel, C. Theobalt, K. Myszkowski, and T. Leimkühler

“GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild,” 2022. [Online]. Available: https://arxiv.org/abs/2211.12352.

Abstract

Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving
as a partial observation of the High Dynamic Range (HDR) visual world. Despite
limited dynamic range, these LDR images are often captured with different
exposures, implicitly containing information about the underlying HDR image
distribution. Inspired by this intuition, in this work we present, to the best
of our knowledge, the first method for learning a generative model of HDR
images from in-the-wild LDR image collections in a fully unsupervised manner.
The key idea is to train a generative adversarial network (GAN) to generate HDR
images which, when projected to LDR under various exposures, are
indistinguishable from real LDR images. The projection from HDR to LDR is
achieved via a camera model that captures the stochasticity in exposure and
camera response function. Experiments show that our method GlowGAN can
synthesize photorealistic HDR images in many challenging cases such as
landscapes, lightning, or windows, where previous supervised generative models
produce overexposed images. We further demonstrate the new application of
unsupervised inverse tone mapping (ITM) enabled by GlowGAN. Our ITM method does
not need HDR images or paired multi-exposure images for training, yet it
reconstructs more plausible information for overexposed regions than
state-of-the-art supervised learning models trained on such data.

BibTeX

@online{Wang2211.12352,
TITLE = {{GlowGAN}: Unsupervised Learning of {HDR} Images from {LDR} Images in the Wild},
AUTHOR = {Wang, Chao and Serrano, Ana and Pan, X. and Chen, Bin and Seidel, Hans-Peter and Theobalt, Christian and Myszkowski, Karol and Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2211.12352},
EPRINT = {2211.12352},
EPRINTTYPE = {arXiv},
YEAR = {2022},
ABSTRACT = {Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving<br>as a partial observation of the High Dynamic Range (HDR) visual world. Despite<br>limited dynamic range, these LDR images are often captured with different<br>exposures, implicitly containing information about the underlying HDR image<br>distribution. Inspired by this intuition, in this work we present, to the best<br>of our knowledge, the first method for learning a generative model of HDR<br>images from in-the-wild LDR image collections in a fully unsupervised manner.<br>The key idea is to train a generative adversarial network (GAN) to generate HDR<br>images which, when projected to LDR under various exposures, are<br>indistinguishable from real LDR images. The projection from HDR to LDR is<br>achieved via a camera model that captures the stochasticity in exposure and<br>camera response function. Experiments show that our method GlowGAN can<br>synthesize photorealistic HDR images in many challenging cases such as<br>landscapes, lightning, or windows, where previous supervised generative models<br>produce overexposed images. We further demonstrate the new application of<br>unsupervised inverse tone mapping (ITM) enabled by GlowGAN. Our ITM method does<br>not need HDR images or paired multi-exposure images for training, yet it<br>reconstructs more plausible information for overexposed regions than<br>state-of-the-art supervised learning models trained on such data.<br>},
}

Endnote

%0 Report
%A Wang, Chao
%A Serrano, Ana
%A Pan, X.
%A Chen, Bin
%A Seidel, Hans-Peter
%A Theobalt, Christian
%A Myszkowski, Karol
%A Leimk&#252;hler, Thomas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T GlowGAN: Unsupervised Learning of HDR Images from LDR Images in the Wild : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000B-9D08-C
%U https://arxiv.org/abs/2211.12352
%D 2022
%X   Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving<br>as a partial observation of the High Dynamic Range (HDR) visual world. Despite<br>limited dynamic range, these LDR images are often captured with different<br>exposures, implicitly containing information about the underlying HDR image<br>distribution. Inspired by this intuition, in this work we present, to the best<br>of our knowledge, the first method for learning a generative model of HDR<br>images from in-the-wild LDR image collections in a fully unsupervised manner.<br>The key idea is to train a generative adversarial network (GAN) to generate HDR<br>images which, when projected to LDR under various exposures, are<br>indistinguishable from real LDR images. The projection from HDR to LDR is<br>achieved via a camera model that captures the stochasticity in exposure and<br>camera response function. Experiments show that our method GlowGAN can<br>synthesize photorealistic HDR images in many challenging cases such as<br>landscapes, lightning, or windows, where previous supervised generative models<br>produce overexposed images. We further demonstrate the new application of<br>unsupervised inverse tone mapping (ITM) enabled by GlowGAN. Our ITM method does<br>not need HDR images or paired multi-exposure images for training, yet it<br>reconstructs more plausible information for overexposed regions than<br>state-of-the-art supervised learning models trained on such data.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

Article

C. Wang, B. Chen, H.-P. Seidel, K. Myszkowski, and A. Serrano

“Learning a self-supervised tone mapping operator via feature contrast masking loss,” Computer Graphics Forum (Proc. EUROGRAPHICS 2022), vol. 41, no. 2, 2022.

@article{Wang2022,
TITLE = {Learning a self-supervised tone mapping operator via feature contrast masking loss},
AUTHOR = {Wang, Chao and Chen, Bin and Seidel, Hans-Peter and Myszkowski, Karol and Serrano, Ana},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.14459},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2022},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {41},
NUMBER = {2},
PAGES = {71--84},
BOOKTITLE = {The European Association for Computer Graphics 43rdAnnual Conference (EUROGRAPHICS 2022)},
EDITOR = {Caine, Rapha{\"e}lle and Kim, Min H.},
}

Endnote

%0 Journal Article
%A Wang, Chao
%A Chen, Bin
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%A Serrano, Ana
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Learning a self-supervised tone mapping operator via feature contrast masking loss : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-BA09-B
%R 10.1111/cgf.14459
%7 2022
%D 2022
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 41
%N 2
%& 71
%P 71 - 84
%I Blackwell-Wiley
%C Oxford
%@ false
%B The European Association for Computer Graphics 43rdAnnual Conference
%O EUROGRAPHICS 2022 EG 2022 Reims, France, April 25 - 29, 2022

Conference paper

D4D2

Y. Wang, H. Chen, Y. Fan, W. Sun, R. Tao, W. Hou, R. Wang, L. Yang, Z. Zhou, L.-Z. Guo, H. Qi, Z. Wu, Y.-F. Li, S. Nakamura, W. Ye, M. Savvides, B. Raj, T. Shinozaki, B. Schiele, J. Wang, X. Xie, and Y. Zhang

“USB: A Unified Semi-supervised Learning Benchmark for Classification,” in Advances in Neural Information Processing Systems 35 (NeurIPS 2022), New Orleans, LA, USA, 2022.

@inproceedings{Wang_Neurips22,
TITLE = {{USB}: {A} Unified Semi-supervised Learning Benchmark for Classification},
AUTHOR = {Wang, Yidong and Chen, Hao and Fan, Yue and Sun, Wang and Tao, Ran and Hou, Wenxin and Wang, Renjie and Yang, Linyi and Zhou, Zhi and Guo, Lan-Zhe and Qi, Heli and Wu, Zhen and Li, Yu-Feng and Nakamura, Satoshi and Ye, Wei and Savvides, Marios and Raj, Bhiksha and Shinozaki, Takahiro and Schiele, Bernt and Wang, Jindong and Xie, Xing and Zhang, Yue},
LANGUAGE = {eng},
PUBLISHER = {Curran Associates, Inc.},
YEAR = {2022},
BOOKTITLE = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)},
EDITOR = {Koyejo, S. and Mohamed, S. and Agarwal, A. and Belgrave, D. and Cho, K. and Oh, A.},
PAGES = {3938--3961},
ADDRESS = {New Orleans, LA, USA},
}

Endnote

%0 Conference Proceedings
%A Wang, Yidong
%A Chen, Hao
%A Fan, Yue
%A Sun, Wang
%A Tao, Ran
%A Hou, Wenxin
%A Wang, Renjie
%A Yang, Linyi
%A Zhou, Zhi
%A Guo, Lan-Zhe
%A Qi, Heli
%A Wu, Zhen
%A Li, Yu-Feng
%A Nakamura, Satoshi
%A Ye, Wei
%A Savvides, Marios
%A Raj, Bhiksha
%A Shinozaki, Takahiro
%A Schiele, Bernt
%A Wang, Jindong
%A Xie, Xing
%A Zhang, Yue
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T USB: A Unified Semi-supervised Learning Benchmark for Classification : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-184A-7
%D 2022
%B 36th Conference on Neural Information Processing Systems
%Z date of event: 2022-11-28 - 2022-12-09
%C New Orleans, LA, USA
%B Advances in Neural Information Processing Systems 35
%E Koyejo, S.; Mohamed, S.; Agarwal, A.; Belgrave, D.; Cho, K.; Oh, A.
%P 3938 - 3961
%I Curran Associates, Inc.
%U https://openreview.net/pdf?id=QeuwINa96C

Conference paper

D. Wittchen, K. Spiel, B. Fruchard, D. Degraen, O. Schneider, G. Freitag, and P. Strohmeier

“TactJam: An End-to-End Prototyping Suite for Collaborative Design of On-Body Vibrotactile Feedback,” in TEI ’22, Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction, Daejeon, Republic of Korea (Online), 2022.

@inproceedings{Wittchen_TEI22,
TITLE = {{TactJam}: {A}n End-to-End Prototyping Suite for Collaborative Design of On-Body Vibrotactile Feedback},
AUTHOR = {Wittchen, Dennis and Spiel, Katta and Fruchard, Bruno and Degraen, Donald and Schneider, Oliver and Freitag, Georg and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {978-1-4503-9147-4},
DOI = {10.1145/3490149.3501307},
PUBLISHER = {ACM},
YEAR = {2022},
BOOKTITLE = {TEI '22, Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction},
PAGES = {1--13},
EID = {1},
ADDRESS = {Daejeon, Republic of Korea (Online)},
}

Endnote

%0 Conference Proceedings
%A Wittchen, Dennis
%A Spiel, Katta
%A Fruchard, Bruno
%A Degraen, Donald
%A Schneider, Oliver
%A Freitag, Georg
%A Strohmeier, Paul
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T TactJam: An End-to-End Prototyping Suite for Collaborative Design of On-Body Vibrotactile Feedback : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-20B9-1
%R 10.1145/3490149.3501307
%D 2022
%B Sixteenth International Conference on Tangible, Embedded, and Embodied Interaction
%Z date of event: 2022-02-13 - 2022-02-16
%C Daejeon, Republic of Korea (Online)
%B TEI '22
%P 1 - 13
%Z sequence number: 1
%I ACM
%@ 978-1-4503-9147-4

Article

K. Wolski, L. Trutoiu, Z. Dong, Z. Shen, K. Mackenzie, and A. Chapiro

“Geo-Metric: A Perceptual Dataset of Distortions on Faces,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2022), vol. 41, no. 6, 2022.

@article{WolskiSIGGRAPHAsia22,
TITLE = {Geo-Metric: {A} Perceptual Dataset of Distortions on Faces},
AUTHOR = {Wolski, Krzysztof and Trutoiu, Laura and Dong, Zhao and Shen, Zhengyang and Mackenzie, Kevin and Chapiro, Alexandre},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3550454.3555475},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {41},
NUMBER = {6},
PAGES = {1--13},
EID = {215},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2022},
}

Endnote

%0 Journal Article
%A Wolski, Krzysztof
%A Trutoiu, Laura
%A Dong, Zhao
%A Shen, Zhengyang
%A Mackenzie, Kevin
%A Chapiro, Alexandre
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Geo-Metric: A Perceptual Dataset of Distortions on Faces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-209F-D
%R 10.1145/3550454.3555475
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 6
%& 1
%P 1 - 13
%Z sequence number: 215
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2022

%O ACM SIGGRAPH Asia 2022 SA '22 SA 2022

Article

K. Wolski, F. Zhong, K. Myszkowski, and R. K. Mantiuk

“Dark Stereo: Improving Depth Perception Under Low Luminance,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2022), vol. 41, no. 4, 2022.

@article{Wolski_SIGGRAPH22,
TITLE = {Dark Stereo: {I}mproving Depth Perception Under Low Luminance},
AUTHOR = {Wolski, Krzysztof and Zhong, Fangcheng and Myszkowski, Karol and Mantiuk, Rafa{\l} K.},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3528223.3530136},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2022},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {41},
NUMBER = {4},
PAGES = {1--12},
EID = {146},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2022},
}

Endnote

%0 Journal Article
%A Wolski, Krzysztof
%A Zhong, Fangcheng
%A Myszkowski, Karol
%A Mantiuk, Rafa&#322; K.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Dark Stereo: Improving Depth Perception Under Low Luminance : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-BA6D-B
%R 10.1145/3528223.3530136
%7 2022
%D 2022
%J ACM Transactions on Graphics
%V 41
%N 4
%& 1
%P 1 - 12
%Z sequence number: 146
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2022
%O ACM SIGGRAPH 2022

2021

Conference paper

D4D5

J. Ali, P. Lahoti, and K. P. Gummadi

“Accounting for Model Uncertainty in Algorithmic Discrimination,” in AIES ’21, Fourth AAAI/ACM Conference on Artificial Intelligence, Ethics and Society, Virtual Conference, 2021.

@inproceedings{Ali_AIES2021,
TITLE = {Accounting for Model Uncertainty in Algorithmic Discrimination},
AUTHOR = {Ali, Junaid and Lahoti, Preethi and Gummadi, Krishna P.},
LANGUAGE = {eng},
ISBN = {978-1-4503-8473-5},
DOI = {10.1145/3461702.3462630},
PUBLISHER = {ACM},
YEAR = {2021},
BOOKTITLE = {AIES '21, Fourth AAAI/ACM Conference on Artificial Intelligence, Ethics and Society},
EDITOR = {Fourcade, Marion and Kuipers, Benjamin and Lazar, Seth and Mulligan, Deirdre},
PAGES = {336--345},
ADDRESS = {Virtual Conference},
}

Endnote

%0 Conference Proceedings
%A Ali, Junaid
%A Lahoti, Preethi
%A Gummadi, Krishna P.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Databases and Information Systems, MPI for Informatics, Max Planck Society
External Organizations
%T Accounting for Model Uncertainty in Algorithmic Discrimination : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-72E3-7
%R 10.1145/3461702.3462630
%D 2021
%B Fourth AAAI/ACM Conference on Artificial Intelligence, Ethics and Society
%Z date of event: 2021-05-19 - 2021-05-21
%C Virtual Conference
%B AIES '21
%E Fourcade, Marion; Kuipers, Benjamin; Lazar, Seth; Mulligan, Deirdre
%P 336 - 345
%I ACM
%@ 978-1-4503-8473-5

Paper

N. Ansari, H.-P. Seidel, and V. Babaei

“Mixed Integer Neural Inverse Design,” 2021. [Online]. Available: https://arxiv.org/abs/2109.12888.

Abstract

In computational design and fabrication, neural networks are becoming
important surrogates for bulky forward simulations. A long-standing,
intertwined question is that of inverse design: how to compute a design that
satisfies a desired target performance? Here, we show that the piecewise linear
property, very common in everyday neural networks, allows for an inverse design
formulation based on mixed-integer linear programming. Our mixed-integer
inverse design uncovers globally optimal or near optimal solutions in a
principled manner. Furthermore, our method significantly facilitates emerging,
but challenging, combinatorial inverse design tasks, such as material
selection. For problems where finding the optimal solution is not desirable or
tractable, we develop an efficient yet near-optimal hybrid optimization.
Eventually, our method is able to find solutions provably robust to possible
fabrication perturbations among multiple designs with similar performances.

BibTeX

@online{Ansari_2109.12888,
TITLE = {Mixed Integer Neural Inverse Design},
AUTHOR = {Ansari, Navid and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2109.12888},
EPRINT = {2109.12888},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {In computational design and fabrication, neural networks are becoming<br>important surrogates for bulky forward simulations. A long-standing,<br>intertwined question is that of inverse design: how to compute a design that<br>satisfies a desired target performance? Here, we show that the piecewise linear<br>property, very common in everyday neural networks, allows for an inverse design<br>formulation based on mixed-integer linear programming. Our mixed-integer<br>inverse design uncovers globally optimal or near optimal solutions in a<br>principled manner. Furthermore, our method significantly facilitates emerging,<br>but challenging, combinatorial inverse design tasks, such as material<br>selection. For problems where finding the optimal solution is not desirable or<br>tractable, we develop an efficient yet near-optimal hybrid optimization.<br>Eventually, our method is able to find solutions provably robust to possible<br>fabrication perturbations among multiple designs with similar performances.<br>},
}

Endnote

%0 Report
%A Ansari, Navid
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Mixed Integer Neural Inverse Design : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-7104-3
%U https://arxiv.org/abs/2109.12888
%D 2021
%X   In computational design and fabrication, neural networks are becoming<br>important surrogates for bulky forward simulations. A long-standing,<br>intertwined question is that of inverse design: how to compute a design that<br>satisfies a desired target performance? Here, we show that the piecewise linear<br>property, very common in everyday neural networks, allows for an inverse design<br>formulation based on mixed-integer linear programming. Our mixed-integer<br>inverse design uncovers globally optimal or near optimal solutions in a<br>principled manner. Furthermore, our method significantly facilitates emerging,<br>but challenging, combinatorial inverse design tasks, such as material<br>selection. For problems where finding the optimal solution is not desirable or<br>tractable, we develop an efficient yet near-optimal hybrid optimization.<br>Eventually, our method is able to find solutions provably robust to possible<br>fabrication perturbations among multiple designs with similar performances.<br>
%K Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Article

B. Chen, C. Wang, M. Piovarči, H.-P. Seidel, P. Didyk, K. Myszkowski, and A. Serrano

“The Effect of Geometry and Illumination on Appearance Perception of Different Material Categories,” The Visual Computer, vol. 37, 2021.

@article{Chen2021b,
TITLE = {The Effect of Geometry and Illumination on Appearance Perception of Different Material Categories},
AUTHOR = {Chen, Bin and Wang, Chao and Piovar{\v c}i, Michal and Seidel, Hans-Peter and Didyk, Piotr and Myszkowski, Karol and Serrano, Ana},
LANGUAGE = {eng},
ISSN = {0178-2789},
DOI = {10.1007/s00371-021-02227-x},
PUBLISHER = {Springer},
ADDRESS = {Berlin},
YEAR = {2021},
JOURNAL = {The Visual Computer},
VOLUME = {37},
PAGES = {2975--2987},
}

Endnote

%0 Journal Article
%A Chen, Bin
%A Wang, Chao
%A Piovar&#269;i, Michal
%A Seidel, Hans-Peter
%A Didyk, Piotr
%A Myszkowski, Karol
%A Serrano, Ana
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T The Effect of Geometry and Illumination on Appearance Perception of Different Material Categories : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-F05C-2
%R 10.1007/s00371-021-02227-x
%7 2021
%D 2021
%J The Visual Computer
%V 37
%& 2975
%P 2975 - 2987
%I Springer
%C Berlin
%@ false

Article

D6D4

M. Chu, N. Thuerey, H.-P. Seidel, C. Theobalt, and R. Zayer

“Learning Meaningful Controls for Fluids,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2021), vol. 40, no. 4, 2021.

@article{Chu2021,
TITLE = {Learning Meaningful Controls for Fluids},
AUTHOR = {Chu, Mengyu and Thuerey, Nils and Seidel, Hans-Peter and Theobalt, Christian and Zayer, Rhaleb},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3450626.3459845},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2021},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {40},
NUMBER = {4},
PAGES = {1--13},
EID = {100},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2021},
}

Endnote

%0 Journal Article
%A Chu, Mengyu
%A Thuerey, Nils
%A Seidel, Hans-Peter
%A Theobalt, Christian
%A Zayer, Rhaleb
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Learning Meaningful Controls for Fluids : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-4B91-F
%R 10.1145/3450626.3459845
%7 2021
%D 2021
%J ACM Transactions on Graphics
%V 40
%N 4
%& 1
%P 1 - 13
%Z sequence number: 100
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2021
%O ACM SIGGRAPH 2021

Article

J. Delanoy, A. Serrano, B. Masia, and D. Gutierrez

“Perception of Material Appearance: A Comparison between Painted and Rendered Images,” Journal of Vision, vol. 21, no. 5, 2021.

@article{Delanoy2021,
TITLE = {Perception of Material Appearance: {A} Comparison between Painted and Rendered Images},
AUTHOR = {Delanoy, Johanna and Serrano, Ana and Masia, Belen and Gutierrez, Diego},
LANGUAGE = {eng},
ISSN = {1534-7362},
DOI = {10.1167/jov.21.5.16},
PUBLISHER = {Scholar One, Inc.},
ADDRESS = {Charlottesville, VA},
YEAR = {2021},
JOURNAL = {Journal of Vision},
VOLUME = {21},
NUMBER = {5},
EID = {16},
}

Endnote

%0 Journal Article
%A Delanoy, Johanna
%A Serrano, Ana
%A Masia, Belen
%A Gutierrez, Diego
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Perception of Material Appearance: A Comparison between Painted and Rendered Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-A6CC-7
%R 10.1167/jov.21.5.16
%2 PMC8131993
%7 2021
%D 2021
%J Journal of Vision
%V 21
%N 5
%Z sequence number: 16
%I Scholar One, Inc.
%C Charlottesville, VA
%@ false

Conference paper

N. Doosti, J. Panetta, and V. Babaei

“Topology Optimization via Frequency Tuning of Neural Design Representations,” in Proceedings SCF 2021, Virtual Event, 2021.

@inproceedings{Doosti21,
TITLE = {Topology Optimization via Frequency Tuning of Neural Design Representations},
AUTHOR = {Doosti, Nikan and Panetta, Julian and Babaei, Vahid},
LANGUAGE = {eng},
ISBN = {978-1-4503-9090-3},
DOI = {10.1145/3485114.3485124},
PUBLISHER = {ACM},
YEAR = {2021},
BOOKTITLE = {Proceedings SCF 2021},
EDITOR = {Whiting, Emily and Hart, John and Sung, Cynthia and McCann, James and Peek, Nadya},
PAGES = {1--9},
EID = {1},
ADDRESS = {Virtual Event},
}

Endnote

%0 Conference Proceedings
%A Doosti, Nikan
%A Panetta, Julian
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Topology Optimization via Frequency Tuning of Neural Design Representations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-C159-6
%R 10.1145/3485114.3485124
%D 2021
%B ACM Symposium on Computational Fabrication
%Z date of event: 2021-10-28 - 2021-10-29
%C Virtual Event
%B Proceedings SCF 2021
%E Whiting, Emily; Hart, John; Sung, Cynthia; McCann, James; Peek, Nadya
%P 1 - 9
%Z sequence number: 1
%I ACM
%@ 978-1-4503-9090-3

Article

O. Elek, R. Zhang, D. Sumin, K. Myszkowski, B. Bickel, A. Wilkie, J. Krivanek, and T. Weyrich

“Robust and Practical Measurement of Volume Transport Parameters in Solid Photo-polymer Materials for 3D Printing,” Optics Express, vol. 29, no. 5, 2021.

@article{Elek2021,
TITLE = {Robust and Practical Measurement of Volume Transport Parameters in Solid Photo-polymer Materials for {3D} Printing},
AUTHOR = {Elek, Oskar and Zhang, Ran and Sumin, Denis and Myszkowski, Karol and Bickel, Bernd and Wilkie, Alexander and Krivanek, Jaroslav and Weyrich, Tim},
LANGUAGE = {eng},
ISSN = {1094-4087},
DOI = {10.1364/OE.406095},
PUBLISHER = {Optical Society of America},
ADDRESS = {Washington, DC},
YEAR = {2021},
DATE = {2021},
JOURNAL = {Optics Express},
VOLUME = {29},
NUMBER = {5},
PAGES = {7568--7588},
}

Endnote

%0 Journal Article
%A Elek, Oskar
%A Zhang, Ran
%A Sumin, Denis
%A Myszkowski, Karol
%A Bickel, Bernd
%A Wilkie, Alexander
%A Krivanek, Jaroslav
%A Weyrich, Tim
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
%T Robust and Practical Measurement of Volume Transport Parameters in Solid Photo-polymer Materials for 3D Printing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E013-6
%R 10.1364/OE.406095
%7 2021
%D 2021
%J Optics Express
%O Opt. Express
%V 29
%N 5
%& 7568
%P 7568 - 7588
%I Optical Society of America
%C Washington, DC
%@ false

Thesis

D4IMPR-CS

M. Habermann

“Real-time human performance capture and synthesis,” Universität des Saarlandes, Saarbrücken, 2021.

Abstract

Most of the images one finds in the media, such as on the Internet or in textbooks and magazines, contain humans as the main point of attention. Thus, there is an inherent necessity for industry, society, and private persons to be able to thoroughly analyze and synthesize the human-related content in these images. One aspect of this analysis and subject of this thesis is to infer the 3D pose and surface deformation, using only visual information, which is also known as human performance capture. Human performance capture enables the tracking of virtual characters from real-world observations, and this is key for visual effects, games, VR, and AR, to name just a few application areas. However, traditional capture methods usually rely on expensive multi-view (marker-based) systems that are prohibitively expensive for the vast majority of people, or they use depth sensors, which are still not as common as single color cameras. Recently, some approaches have attempted to solve the task by assuming only a single RGB image is given. Nonetheless, they can either not track the dense deforming geometry of the human, such as the clothing layers, or they are far from real time, which is indispensable for many applications. To overcome these shortcomings, this thesis proposes two monocular human performance capture methods, which for the first time allow the real-time capture of the dense deforming geometry as well as an unseen 3D accuracy for pose and surface deformations. At the technical core, this work introduces novel GPU-based and data-parallel optimization strategies in conjunction with other algorithmic design choices that are all geared towards real-time performance at high accuracy. Moreover, this thesis presents a new weakly supervised multiview training strategy combined with a fully differentiable character representation that shows superior 3D accuracy. However, there is more to human-related Computer Vision than only the analysis of people in images. It is equally important to synthesize new images of humans in unseen poses and also from camera viewpoints that have not been observed in the real world. Such tools are essential for the movie industry because they, for example, allow the synthesis of photo-realistic virtual worlds with real-looking humans or of contents that are too dangerous for actors to perform on set. But also video conferencing and telepresence applications can benefit from photo-real 3D characters, as they can enhance the immersive experience of these applications. Here, the traditional Computer Graphics pipeline for rendering photo-realistic images involves many tedious and time-consuming steps that require expert knowledge and are far from real time. Traditional rendering involves character rigging and skinning, the modeling of the surface appearance properties, and physically based ray tracing. Recent learning-based methods attempt to simplify the traditional rendering pipeline and instead learn the rendering function from data resulting in methods that are easier accessible to non-experts. However, most of them model the synthesis task entirely in image space such that 3D consistency cannot be achieved, and/or they fail to model motion- and view-dependent appearance effects. To this end, this thesis presents a method and ongoing work on character synthesis, which allow the synthesis of controllable photoreal characters that achieve motion- and view-dependent appearance effects as well as 3D consistency and which run in real time. This is technically achieved by a novel coarse-to-fine geometric character representation for efficient synthesis, which can be solely supervised on multi-view imagery. Furthermore, this work shows how such a geometric representation can be combined with an implicit surface representation to boost synthesis and geometric quality.

BibTeX

@phdthesis{Habermannphd2021,
TITLE = {Real-time human performance capture and synthesis},
AUTHOR = {Habermann, Marc},
LANGUAGE = {eng},
URL = {nbn:de:bsz:291--ds-349617},
DOI = {10.22028/D291-34961},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2021},
DATE = {2021},
ABSTRACT = {Most of the images one finds in the media, such as on the Internet or in textbooks and magazines, contain humans as the main point of attention. Thus, there is an inherent necessity for industry, society, and private persons to be able to thoroughly analyze and synthesize the human-related content in these images. One aspect of this analysis and subject of this thesis is to infer the 3D pose and surface deformation, using only visual information, which is also known as human performance capture. Human performance capture enables the tracking of virtual characters from real-world observations, and this is key for visual effects, games, VR, and AR, to name just a few application areas. However, traditional capture methods usually rely on expensive multi-view (marker-based) systems that are prohibitively expensive for the vast majority of people, or they use depth sensors, which are still not as common as single color cameras. Recently, some approaches have attempted to solve the task by assuming only a single RGB image is given. Nonetheless, they can either not track the dense deforming geometry of the human, such as the clothing layers, or they are far from real time, which is indispensable for many applications. To overcome these shortcomings, this thesis proposes two monocular human performance capture methods, which for the first time allow the real-time capture of the dense deforming geometry as well as an unseen 3D accuracy for pose and surface deformations. At the technical core, this work introduces novel GPU-based and data-parallel optimization strategies in conjunction with other algorithmic design choices that are all geared towards real-time performance at high accuracy. Moreover, this thesis presents a new weakly supervised multiview training strategy combined with a fully differentiable character representation that shows superior 3D accuracy. However, there is more to human-related Computer Vision than only the analysis of people in images. It is equally important to synthesize new images of humans in unseen poses and also from camera viewpoints that have not been observed in the real world. Such tools are essential for the movie industry because they, for example, allow the synthesis of photo-realistic virtual worlds with real-looking humans or of contents that are too dangerous for actors to perform on set. But also video conferencing and telepresence applications can benefit from photo-real 3D characters, as they can enhance the immersive experience of these applications. Here, the traditional Computer Graphics pipeline for rendering photo-realistic images involves many tedious and time-consuming steps that require expert knowledge and are far from real time. Traditional rendering involves character rigging and skinning, the modeling of the surface appearance properties, and physically based ray tracing. Recent learning-based methods attempt to simplify the traditional rendering pipeline and instead learn the rendering function from data resulting in methods that are easier accessible to non-experts. However, most of them model the synthesis task entirely in image space such that 3D consistency cannot be achieved, and/or they fail to model motion- and view-dependent appearance effects. To this end, this thesis presents a method and ongoing work on character synthesis, which allow the synthesis of controllable photoreal characters that achieve motion- and view-dependent appearance effects as well as 3D consistency and which run in real time. This is technically achieved by a novel coarse-to-fine geometric character representation for efficient synthesis, which can be solely supervised on multi-view imagery. Furthermore, this work shows how such a geometric representation can be combined with an implicit surface representation to boost synthesis and geometric quality.},
}

Endnote

%0 Thesis
%A Habermann, Marc
%Y Theobalt, Christian
%A referee: Seidel, Hans-Peter
%A referee: Hilton, Adrian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Real-time human performance capture and synthesis : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-7D87-3
%R 10.22028/D291-34961
%U nbn:de:bsz:291--ds-349617
%F OTHER: hdl:20.500.11880/31986
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2021
%P 153 p.
%V phd
%9 phd
%X Most of the images one finds in the media, such as on the Internet or in textbooks and magazines, contain humans as the main point of attention. Thus, there is an inherent necessity for industry, society, and private persons to be able to thoroughly analyze and synthesize the human-related content in these images. One aspect of this analysis and subject of this thesis is to infer the 3D pose and surface deformation, using only visual information, which is also known as human performance capture. Human performance capture enables the tracking of virtual characters from real-world observations, and this is key for visual effects, games, VR, and AR, to name just a few application areas. However, traditional capture methods usually rely on expensive multi-view (marker-based) systems that are prohibitively expensive for the vast majority of people, or they use depth sensors, which are still not as common as single color cameras. Recently, some approaches have attempted to solve the task by assuming only a single RGB image is given. Nonetheless, they can either not track the dense deforming geometry of the human, such as the clothing layers, or they are far from real time, which is indispensable for many applications. To overcome these shortcomings, this thesis proposes two monocular human performance capture methods, which for the first time allow the real-time capture of the dense deforming geometry as well as an unseen 3D accuracy for pose and surface deformations. At the technical core, this work introduces novel GPU-based and data-parallel optimization strategies in conjunction with other algorithmic design choices that are all geared towards real-time performance at high accuracy. Moreover, this thesis presents a new weakly supervised multiview training strategy combined with a fully differentiable character representation that shows superior 3D accuracy. However, there is more to human-related Computer Vision than only the analysis of people in images. It is equally important to synthesize new images of humans in unseen poses and also from camera viewpoints that have not been observed in the real world. Such tools are essential for the movie industry because they, for example, allow the synthesis of photo-realistic virtual worlds with real-looking humans or of contents that are too dangerous for actors to perform on set. But also video conferencing and telepresence applications can benefit from photo-real 3D characters, as they can enhance the immersive experience of these applications. Here, the traditional Computer Graphics pipeline for rendering photo-realistic images involves many tedious and time-consuming steps that require expert knowledge and are far from real time. Traditional rendering involves character rigging and skinning, the modeling of the surface appearance properties, and physically based ray tracing. Recent learning-based methods attempt to simplify the traditional rendering pipeline and instead learn the rendering function from data resulting in methods that are easier accessible to non-experts. However, most of them model the synthesis task entirely in image space such that 3D consistency cannot be achieved, and/or they fail to model motion- and view-dependent appearance effects. To this end, this thesis presents a method and ongoing work on character synthesis, which allow the synthesis of controllable photoreal characters that achieve motion- and view-dependent appearance effects as well as 3D consistency and which run in real time. This is technically achieved by a novel coarse-to-fine geometric character representation for efficient synthesis, which can be solely supervised on multi-view imagery. Furthermore, this work shows how such a geometric representation can be combined with an implicit surface representation to boost synthesis and geometric quality.
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/31986

Paper

D4D6

I. Habibie, W. Xu, D. Mehta, L. Liu, H.-P. Seidel, G. Pons-Moll, M. Elgharib, and C. Theobalt

“Learning Speech-driven 3D Conversational Gestures from Video,” 2021. [Online]. Available: https://arxiv.org/abs/2102.06837.

Abstract

We propose the first approach to automatically and jointly synthesize both
the synchronous 3D conversational body and hand gestures, as well as 3D face
and head animations, of a virtual character from speech input. Our algorithm
uses a CNN architecture that leverages the inherent correlation between facial
expression and hand gestures. Synthesis of conversational body gestures is a
multi-modal problem since many similar gestures can plausibly accompany the
same input speech. To synthesize plausible body gestures in this setting, we
train a Generative Adversarial Network (GAN) based model that measures the
plausibility of the generated sequences of 3D body motion when paired with the
input audio features. We also contribute a new way to create a large corpus of
more than 33 hours of annotated body, hand, and face data from in-the-wild
videos of talking people. To this end, we apply state-of-the-art monocular
approaches for 3D body and hand pose estimation as well as dense 3D face
performance capture to the video corpus. In this way, we can train on orders of
magnitude more data than previous algorithms that resort to complex in-studio
motion capture solutions, and thereby train more expressive synthesis
algorithms. Our experiments and user study show the state-of-the-art quality of
our speech-synthesized full 3D character animations.

BibTeX

@online{Habibie_2102.06837,
TITLE = {Learning Speech-driven {3D} Conversational Gestures from Video},
AUTHOR = {Habibie, Ikhsanul and Xu, Weipeng and Mehta, Dushyant and Liu, Lingjie and Seidel, Hans-Peter and Pons-Moll, Gerard and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2102.06837},
EPRINT = {2102.06837},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {We propose the first approach to automatically and jointly synthesize both<br>the synchronous 3D conversational body and hand gestures, as well as 3D face<br>and head animations, of a virtual character from speech input. Our algorithm<br>uses a CNN architecture that leverages the inherent correlation between facial<br>expression and hand gestures. Synthesis of conversational body gestures is a<br>multi-modal problem since many similar gestures can plausibly accompany the<br>same input speech. To synthesize plausible body gestures in this setting, we<br>train a Generative Adversarial Network (GAN) based model that measures the<br>plausibility of the generated sequences of 3D body motion when paired with the<br>input audio features. We also contribute a new way to create a large corpus of<br>more than 33 hours of annotated body, hand, and face data from in-the-wild<br>videos of talking people. To this end, we apply state-of-the-art monocular<br>approaches for 3D body and hand pose estimation as well as dense 3D face<br>performance capture to the video corpus. In this way, we can train on orders of<br>magnitude more data than previous algorithms that resort to complex in-studio<br>motion capture solutions, and thereby train more expressive synthesis<br>algorithms. Our experiments and user study show the state-of-the-art quality of<br>our speech-synthesized full 3D character animations.<br>},
}

Endnote

%0 Report
%A Habibie, Ikhsanul
%A Xu, Weipeng
%A Mehta, Dushyant
%A Liu, Lingjie
%A Seidel, Hans-Peter
%A Pons-Moll, Gerard
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Learning Speech-driven 3D Conversational Gestures from Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-70C7-8
%U https://arxiv.org/abs/2102.06837
%D 2021
%X   We propose the first approach to automatically and jointly synthesize both<br>the synchronous 3D conversational body and hand gestures, as well as 3D face<br>and head animations, of a virtual character from speech input. Our algorithm<br>uses a CNN architecture that leverages the inherent correlation between facial<br>expression and hand gestures. Synthesis of conversational body gestures is a<br>multi-modal problem since many similar gestures can plausibly accompany the<br>same input speech. To synthesize plausible body gestures in this setting, we<br>train a Generative Adversarial Network (GAN) based model that measures the<br>plausibility of the generated sequences of 3D body motion when paired with the<br>input audio features. We also contribute a new way to create a large corpus of<br>more than 33 hours of annotated body, hand, and face data from in-the-wild<br>videos of talking people. To this end, we apply state-of-the-art monocular<br>approaches for 3D body and hand pose estimation as well as dense 3D face<br>performance capture to the video corpus. In this way, we can train on orders of<br>magnitude more data than previous algorithms that resort to complex in-studio<br>motion capture solutions, and thereby train more expressive synthesis<br>algorithms. Our experiments and user study show the state-of-the-art quality of<br>our speech-synthesized full 3D character animations.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

D6D4

I. Habibie, W. Xu, D. Mehta, L. Liu, H.-P. Seidel, G. Pons-Moll, M. Elgharib, and C. Theobalt

“Learning Speech-driven 3D Conversational Gestures from Video,” in Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents (IVA 2021), Virtual Event, Japan, 2021.

@inproceedings{Habibie_IVA2021,
TITLE = {Learning Speech-driven {3D} Conversational Gestures from Video},
AUTHOR = {Habibie, Ikhsanul and Xu, Weipeng and Mehta, Dushyant and Liu, Lingjie and Seidel, Hans-Peter and Pons-Moll, Gerard and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {9781450386197},
DOI = {10.1145/3472306.3478335},
PUBLISHER = {ACM},
YEAR = {2021},
BOOKTITLE = {Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents (IVA 2021)},
PAGES = {101--108},
ADDRESS = {Virtual Event, Japan},
}

Endnote

%0 Conference Proceedings
%A Habibie, Ikhsanul
%A Xu, Weipeng
%A Mehta, Dushyant
%A Liu, Lingjie
%A Seidel, Hans-Peter
%A Pons-Moll, Gerard
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Learning Speech-driven 3D Conversational Gestures from Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-4D19-6
%R 10.1145/3472306.3478335
%D 2021
%B 21st ACM International Conference on Intelligent Virtual 
Agents 
%Z date of event: 2021-09-14 - 2021-09-17
%C Virtual Event, Japan
%B Proceedings of the 21st ACM International Conference on Intelligent Virtual Agents
%P 101 - 108
%I ACM
%@ 9781450386197

Article

J. Hladký, H.-P. Seidel, and M. Steinberger

“SnakeBinning: Efficient Temporally Coherent Triangle Packing for Shading Streaming,” Computer Graphics Forum (Proc. EUROGRAPHICS 2021), vol. 40, no. 2, 2021.

@article{10.1111:cgf.142648,
TITLE = {{SnakeBinning}: {E}fficient Temporally Coherent Triangle Packing for Shading Streaming},
AUTHOR = {Hladk{\'y}, Jozef and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.142648},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2021},
DATE = {2021},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {40},
NUMBER = {2},
PAGES = {475--488},
BOOKTITLE = {42nd Annual Conference of the European Association for Computer Graphics (EUROGRAPHICS 2021)},
EDITOR = {Mitra, Niloy and Viola, Ivan},
}

Endnote

%0 Journal Article
%A Hladk&#253;, Jozef
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T SnakeBinning: Efficient Temporally Coherent Triangle Packing 
for Shading Streaming : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-7AFD-3
%R 10.1111/cgf.142648
%7 2021
%D 2021
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 40
%N 2
%& 475
%P 475 - 488
%I Blackwell-Wiley
%C Oxford
%@ false
%B 42nd Annual Conference of the European Association for Computer Graphics
%O EUROGRAPHICS 2021 EG 2021

Article

C. Jiang, C. Tang, H.-P. Seidel, R. Chen, and P. Wonka

“Computational Design of Lightweight Trusses,” Computer-Aided Design, vol. 141, 2021.

@article{Jiang2021,
TITLE = {Computational Design of Lightweight Trusses},
AUTHOR = {Jiang, Caigui and Tang, Chengcheng and Seidel, Hans-Peter and Chen, Renjie and Wonka, Peter},
ISSN = {0010-4485},
DOI = {10.1016/j.cad.2021.103076},
PUBLISHER = {Elsevier},
ADDRESS = {Amsterdam},
YEAR = {2021},
JOURNAL = {Computer-Aided Design},
VOLUME = {141},
PAGES = {1--11},
EID = {103076},
}

Endnote

%0 Journal Article
%A Jiang, Caigui
%A Tang, Chengcheng
%A Seidel, Hans-Peter
%A Chen, Renjie
%A Wonka, Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Computational Design of Lightweight Trusses : 
%U http://hdl.handle.net/21.11116/0000-0009-70C2-D
%R 10.1016/j.cad.2021.103076
%7 2021
%D 2021
%J Computer-Aided Design
%V 141
%& 1
%P 1 - 11
%Z sequence number: 103076
%I Elsevier
%C Amsterdam
%@ false

Article

A. Jindal, K. Wolski, R. K. Mantiuk, and K. Myszkowski

“Perceptual Model for Adaptive Local Shading and Refresh Rate,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2021), vol. 40, no. 6, 2021.

@article{JindalSIGGRAPHAsia21,
TITLE = {Perceptual Model for Adaptive Local Shading and Refresh Rate},
AUTHOR = {Jindal, Akshay and Wolski, Krzysztof and Mantiuk, Rafa{\l} K. and Myszkowski, Karol},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3478513.3480514},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2021},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {40},
NUMBER = {6},
PAGES = {1--18},
EID = {281},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2021},
}

Endnote

%0 Journal Article
%A Jindal, Akshay
%A Wolski, Krzysztof
%A Mantiuk, Rafa&#322; K.
%A Myszkowski, Karol
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perceptual Model for Adaptive Local Shading and Refresh Rate : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-9B45-B
%R 10.1145/3478513.3480514
%7 2021
%D 2021
%J ACM Transactions on Graphics
%V 40
%N 6
%& 1
%P 1 - 18
%Z sequence number: 281
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2021

%O ACM SIGGRAPH Asia 2021 SA '21 SA 2021

Conference paper

D6D4

M. Kappel, V. Golyanik, M. Elgharib, J.-O. Henningson, H.-P. Seidel, S. Castillo, C. Theobalt, and M. A. Magnor

“High-Fidelity Neural Human Motion Transfer from Monocular Video,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), Virtual Conference, 2021.

@inproceedings{Kappel_CVPR2021,
TITLE = {High-Fidelity Neural Human Motion Transfer from Monocular Video},
AUTHOR = {Kappel, Moritz and Golyanik, Vladislav and Elgharib, Mohamed and Henningson, Jann-Ole and Seidel, Hans-Peter and Castillo, Susana and Theobalt, Christian and Magnor, Marcus A.},
LANGUAGE = {eng},
ISBN = {978-1-6654-4509-2},
DOI = {10.1109/CVPR46437.2021.00159},
PUBLISHER = {IEEE},
YEAR = {2021},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)},
PAGES = {1541--1550},
ADDRESS = {Virtual Conference},
}

Endnote

%0 Conference Proceedings
%A Kappel, Moritz
%A Golyanik, Vladislav
%A Elgharib, Mohamed
%A Henningson, Jann-Ole
%A Seidel, Hans-Peter
%A Castillo, Susana
%A Theobalt, Christian
%A Magnor, Marcus A.
%+ External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
%T High-Fidelity Neural Human Motion Transfer from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-8947-E
%R 10.1109/CVPR46437.2021.00159
%D 2021
%B 34th IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2021-06-19 - 2021-06-25
%C Virtual Conference
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 1541 - 1550
%I IEEE
%@ 978-1-6654-4509-2
%U https://gvv.mpi-inf.mpg.de/projects/NHMT/

Conference paper

J. Knibbe, R. Freire, M. Koelle, and P. Strohmeier

“Skill-Sleeves: Designing Electrode Garments for Wearability,” in TEI ’21, Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, Salzburg, Austria, 2021.

@inproceedings{Knibbe_TEI21,
TITLE = {Skill-Sleeves: {D}esigning Electrode Garments for Wearability},
AUTHOR = {Knibbe, Jarrod and Freire, Rachel and Koelle, Marion and Strohmeier, Paul},
LANGUAGE = {eng},
ISBN = {978-1-4503-8213-7},
DOI = {10.1145/3430524.3440652},
PUBLISHER = {ACM},
YEAR = {2021},
BOOKTITLE = {TEI '21, Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction},
PAGES = {1--16},
EID = {33},
ADDRESS = {Salzburg, Austria},
}

Endnote

%0 Conference Proceedings
%A Knibbe, Jarrod
%A Freire, Rachel
%A Koelle, Marion
%A Strohmeier, Paul
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Skill-Sleeves: Designing Electrode Garments for Wearability : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-BDF0-0
%R 10.1145/3430524.3440652
%D 2021
%B Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction
%Z date of event: 2021-02-14 - 2021-02-17
%C Salzburg, Austria
%B TEI '21
%P 1 - 16
%Z sequence number: 33
%I ACM
%@ 978-1-4503-8213-7

Article

M. Lagunas, A. Serrano, D. Gutierrez, and B. Masia

“The Joint Role of Geometry and Illumination on Material Recognition,” Journal of Vision, vol. 21, no. 2, 2021.

@article{Lagunas2021_MatRecog,
TITLE = {The Joint Role of Geometry and Illumination on Material Recognition},
AUTHOR = {Lagunas, Manuel and Serrano, Ana and Gutierrez, Diego and Masia, Belen},
LANGUAGE = {eng},
ISSN = {1534-7362},
DOI = {10.1167/jov.21.2.2},
PUBLISHER = {Scholar One, Inc.},
ADDRESS = {Charlottesville, VA},
YEAR = {2021},
JOURNAL = {Journal of Vision},
VOLUME = {21},
NUMBER = {2},
PAGES = {1--18},
}

Endnote

%0 Journal Article
%A Lagunas, Manuel
%A Serrano, Ana
%A Gutierrez, Diego
%A Masia, Belen
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T The Joint Role of Geometry and Illumination on Material Recognition  : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-EAF9-9
%R 10.1167/jov.21.2.2
%7 2021
%D 2021
%J Journal of Vision
%V 21
%N 2
%& 1
%P 1 - 18
%I Scholar One, Inc.
%C Charlottesville, VA
%@ false

Article

T. Leimkühler and G. Drettakis

“FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2021), vol. 40, no. 6, 2021.

@article{Leimkuehler_SIGGRAPHAsia21,
TITLE = {{FreeStyleGAN}: {F}ree-view editable portrait rendering with the camera manifold},
AUTHOR = {Leimk{\"u}hler, Thomas and Drettakis, George},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3478513.3480538},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2021},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {40},
NUMBER = {6},
PAGES = {1--15},
EID = {224},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2021},
}

Endnote

%0 Journal Article
%A Leimk&#252;hler, Thomas
%A Drettakis, George
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T FreeStyleGAN: Free-view Editable Portrait Rendering with the Camera Manifold : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000C-AFF0-0
%R 10.1145/3478513.3480538
%7 2021
%D 2021
%J ACM Transactions on Graphics
%V 40
%N 6
%& 1
%P 1 - 15
%Z sequence number: 224
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2021

%O ACM SIGGRAPH Asia 2021 SA '21 SA 2021

Article

L. Liu, W. Xu, M. Habermann, M. Zollhöfer, F. Bernard, H. Kim, W. Wang, and C. Theobalt

“Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 10, 2021.

@article{liu2020NeuralHumanRendering,
TITLE = {Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation},
AUTHOR = {Liu, Lingjie and Xu, Weipeng and Habermann, Marc and Zollh{\"o}fer, Michael and Bernard, Florian and Kim, Hyeongwoo and Wang, Wenping and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2020.2996594},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2021},
DATE = {2021},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics},
VOLUME = {27},
NUMBER = {10},
PAGES = {4009--4022},
}

Endnote

%0 Journal Article
%A Liu, Lingjie
%A Xu, Weipeng
%A Habermann, Marc
%A Zollh&#246;fer, Michael
%A Bernard, Florian
%A Kim, Hyeongwoo
%A Wang, Wenping
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-0369-F
%R 10.1109/TVCG.2020.2996594
%7 2020
%D 2021
%J IEEE Transactions on Visualization and Computer Graphics
%V 27
%N 10
%& 4009
%P 4009 - 4022
%I IEEE
%C Piscataway, NJ
%@ false

Conference paper

D4D6

Y. Li, M. Habermann, B. Thomaszewski, S. Coros, T. Beeler, and C. Theobalt

“Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture,” in 2021 International Conference on 3D Vision, Virtual Conference, 2021.

@inproceedings{Li_3DV21,
TITLE = {Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture},
AUTHOR = {Li, Yue and Habermann, Marc and Thomaszewski,, Bernhard and Coros, Stelian and Beeler, Thabo and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-6654-2688-6},
DOI = {10.1109/3DV53792.2021.00047},
PUBLISHER = {IEEE},
YEAR = {2021},
DATE = {2021},
BOOKTITLE = {2021 International Conference on 3D Vision},
PAGES = {373--384},
ADDRESS = {Virtual Conference},
}

Endnote

%0 Conference Proceedings
%A Li, Yue
%A Habermann, Marc
%A Thomaszewski,, Bernhard
%A Coros, Stelian
%A Beeler, Thabo
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E9D0-7
%R 10.1109/3DV53792.2021.00047
%D 2021
%B International Conference on 3D Vision
%Z date of event: 2021-12-01 - 2021-12-03
%C Virtual Conference
%B 2021 International Conference on 3D Vision
%P 373 - 384
%I IEEE
%@ 978-1-6654-2688-6

Conference paper

D6D4

Mallikarjun B R, A. Tewari, H.-P. Seidel, M. Elgharib, and C. Theobalt, “Learning Complete 3D Morphable Face Models from Images and Videos,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), Virtual Conference, 2021.

@inproceedings{Mallikarjun_CVPR2021b,
TITLE = {Learning Complete {3D} Morphable Face Models from Images and Videos},
AUTHOR = {Mallikarjun B R and Tewari, Ayush and Seidel, Hans-Peter and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-6654-4509-2},
DOI = {10.1109/CVPR46437.2021.00337},
PUBLISHER = {IEEE},
YEAR = {2021},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)},
PAGES = {3361--3371},
ADDRESS = {Virtual Conference},
}

Endnote

%0 Conference Proceedings
%A Mallikarjun B R, 
%A Tewari, Ayush
%A Seidel, Hans-Peter
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Learning Complete 3D Morphable Face Models from Images and Videos  : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-8926-3
%R 10.1109/CVPR46437.2021.00337
%D 2021
%B 34th IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2021-06-19 - 2021-06-25
%C Virtual Conference
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 3361 - 3371
%I IEEE
%@ 978-1-6654-4509-2
%U https://gvv.mpi-inf.mpg.de/projects/LeMoMo/

Conference paper

D6D4

Mallikarjun B R, A. Tewari, T.-H. Oh, T. Weyrich, B. Bickel, H.-P. Seidel, H. Pfister, W. Matusik, M. Elgharib, and C. Theobalt, “Monocular Reconstruction of Neural Face Reflectance Fields,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), Virtual Conference, 2021.

@inproceedings{Mallikarjun_CVPR2021,
TITLE = {Monocular Reconstruction of Neural Face Reflectance Fields},
AUTHOR = {Mallikarjun B R and Tewari, Ayush and Oh, Tae-Hyun and Weyrich, Tim and Bickel, Bernd and Seidel, Hans-Peter and Pfister, Hanspeter and Matusik, Wojciech and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-6654-4509-2},
DOI = {10.1109/CVPR46437.2021.00476},
PUBLISHER = {IEEE},
YEAR = {2021},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)},
PAGES = {4791--4800},
ADDRESS = {Virtual Conference},
}

Endnote

%0 Conference Proceedings
%A Mallikarjun B R, 
%A Tewari, Ayush
%A Oh, Tae-Hyun
%A Weyrich, Tim
%A Bickel, Bernd
%A Seidel, Hans-Peter
%A Pfister, Hanspeter
%A Matusik, Wojciech
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Monocular Reconstruction of Neural Face Reflectance Fields : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-88FB-4
%R 10.1109/CVPR46437.2021.00476
%D 2021
%B 34th IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2021-06-19 - 2021-06-25
%C Virtual Conference
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 4791 - 4800
%I IEEE
%@ 978-1-6654-4509-2
%U https://gvv.mpi-inf.mpg.de/projects/FaceReflectanceFields/

Article

D6D4

Mallikarjun B R, A. Tewari, A. Dib, T. Weyrich, B. Bickel, H.-P. Seidel, H. Pfister, W. Matusik, L. Chevallier, M. Elgharib, and C. Theobalt, “PhotoApp: Photorealistic Appearance Editing of Head Portraits,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2021), vol. 40, no. 4, 2021.

@article{MallikarjunBR2021,
TITLE = {{PhotoApp}: {P}hotorealistic Appearance Editing of Head Portraits},
AUTHOR = {Mallikarjun B R and Tewari, Ayush and Dib, Abdallah and Weyrich, Tim and Bickel, Bernd and Seidel, Hans-Peter and Pfister, Hanspeter and Matusik, Wojciech and Chevallier, Louis and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3450626.3459765},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2021},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {40},
NUMBER = {4},
PAGES = {1--16},
EID = {44},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2021},
}

Endnote

%0 Journal Article
%A Mallikarjun B R, 
%A Tewari, Ayush
%A Dib, Abdallah
%A Weyrich, Tim
%A Bickel, Bernd
%A Seidel, Hans-Peter
%A Pfister, Hanspeter
%A Matusik, Wojciech
%A Chevallier, Louis
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T PhotoApp: Photorealistic Appearance Editing of Head Portraits : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-2A9B-A
%R 10.1145/3450626.3459765
%7 2021
%D 2021
%J ACM Transactions on Graphics
%V 40
%N 4
%& 1
%P 1 - 16
%Z sequence number: 44
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2021
%O ACM SIGGRAPH 2021

Paper

D. Martin, S. Malpica, D. Gutierrez, B. Masia, and A. Serrano

“Multimodality in VR: A Survey,” 2021. [Online]. Available: https://arxiv.org/abs/2101.07906.

Abstract

Virtual reality has the potential to change the way we create and consume
content in our everyday life. Entertainment, training, design and
manufacturing, communication, or advertising are all applications that already
benefit from this new medium reaching consumer level. VR is inherently
different from traditional media: it offers a more immersive experience, and
has the ability to elicit a sense of presence through the place and
plausibility illusions. It also gives the user unprecedented capabilities to
explore their environment, in contrast with traditional media. In VR, like in
the real world, users integrate the multimodal sensory information they receive
to create a unified perception of the virtual world. Therefore, the sensory
cues that are available in a virtual environment can be leveraged to enhance
the final experience. This may include increasing realism, or the sense of
presence; predicting or guiding the attention of the user through the
experience; or increasing their performance if the experience involves the
completion of certain tasks. In this state-of-the-art report, we survey the
body of work addressing multimodality in virtual reality, its role and benefits
in the final user experience. The works here reviewed thus encompass several
fields of research, including computer graphics, human computer interaction, or
psychology and perception. Additionally, we give an overview of different
applications that leverage multimodal input in areas such as medicine, training
and education, or entertainment; we include works in which the integration of
multiple sensory information yields significant improvements, demonstrating how
multimodality can play a fundamental role in the way VR systems are designed,
and VR experiences created and consumed.

BibTeX

@online{Martin2021_VRsurvey,
TITLE = {Multimodality in {VR}: {A} Survey},
AUTHOR = {Martin, Daniel and Malpica, Sandra and Gutierrez, Diego and Masia, Belen and Serrano, Ana},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2101.07906},
EPRINT = {2101.07906},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {Virtual reality has the potential to change the way we create and consume<br>content in our everyday life. Entertainment, training, design and<br>manufacturing, communication, or advertising are all applications that already<br>benefit from this new medium reaching consumer level. VR is inherently<br>different from traditional media: it offers a more immersive experience, and<br>has the ability to elicit a sense of presence through the place and<br>plausibility illusions. It also gives the user unprecedented capabilities to<br>explore their environment, in contrast with traditional media. In VR, like in<br>the real world, users integrate the multimodal sensory information they receive<br>to create a unified perception of the virtual world. Therefore, the sensory<br>cues that are available in a virtual environment can be leveraged to enhance<br>the final experience. This may include increasing realism, or the sense of<br>presence; predicting or guiding the attention of the user through the<br>experience; or increasing their performance if the experience involves the<br>completion of certain tasks. In this state-of-the-art report, we survey the<br>body of work addressing multimodality in virtual reality, its role and benefits<br>in the final user experience. The works here reviewed thus encompass several<br>fields of research, including computer graphics, human computer interaction, or<br>psychology and perception. Additionally, we give an overview of different<br>applications that leverage multimodal input in areas such as medicine, training<br>and education, or entertainment; we include works in which the integration of<br>multiple sensory information yields significant improvements, demonstrating how<br>multimodality can play a fundamental role in the way VR systems are designed,<br>and VR experiences created and consumed.<br>},
}

Endnote

%0 Report
%A Martin, Daniel
%A Malpica, Sandra
%A Gutierrez, Diego
%A Masia, Belen
%A Serrano, Ana
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Multimodality in VR: A Survey : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-EB00-0
%U https://arxiv.org/abs/2101.07906
%D 2021
%X   Virtual reality has the potential to change the way we create and consume<br>content in our everyday life. Entertainment, training, design and<br>manufacturing, communication, or advertising are all applications that already<br>benefit from this new medium reaching consumer level. VR is inherently<br>different from traditional media: it offers a more immersive experience, and<br>has the ability to elicit a sense of presence through the place and<br>plausibility illusions. It also gives the user unprecedented capabilities to<br>explore their environment, in contrast with traditional media. In VR, like in<br>the real world, users integrate the multimodal sensory information they receive<br>to create a unified perception of the virtual world. Therefore, the sensory<br>cues that are available in a virtual environment can be leveraged to enhance<br>the final experience. This may include increasing realism, or the sense of<br>presence; predicting or guiding the attention of the user through the<br>experience; or increasing their performance if the experience involves the<br>completion of certain tasks. In this state-of-the-art report, we survey the<br>body of work addressing multimodality in virtual reality, its role and benefits<br>in the final user experience. The works here reviewed thus encompass several<br>fields of research, including computer graphics, human computer interaction, or<br>psychology and perception. Additionally, we give an overview of different<br>applications that leverage multimodal input in areas such as medicine, training<br>and education, or entertainment; we include works in which the integration of<br>multiple sensory information yields significant improvements, demonstrating how<br>multimodality can play a fundamental role in the way VR systems are designed,<br>and VR experiences created and consumed.<br>
%K Computer Science, Human-Computer Interaction, cs.HC,Computer Science, Graphics, cs.GR

Article

B. Masia, J. Camon, D. Gutierrez, and A. Serrano

“Influence of Directional Sound Cues on Users’ Exploration Across 360° Movie Cuts,” IEEE Computer Graphics and Applications, vol. 41, no. 4, 2021.

@article{Masia2021,
TITLE = {Influence of Directional Sound Cues on Users' Exploration Across 360{\textdegree} Movie Cuts},
AUTHOR = {Masia, Belen and Camon, Javier and Gutierrez,, Diego and Serrano, Ana},
LANGUAGE = {eng},
ISSN = {0272-1716},
DOI = {10.1109/MCG.2021.3064688},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2021},
DATE = {2021},
JOURNAL = {IEEE Computer Graphics and Applications},
VOLUME = {41},
NUMBER = {4},
PAGES = {64--75},
}

Endnote

%0 Journal Article
%A Masia, Belen
%A Camon, Javier
%A Gutierrez,, Diego
%A Serrano, Ana
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Influence of Directional Sound Cues on Users' Exploration Across 360&#176; Movie Cuts : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-F077-3
%R 10.1109/MCG.2021.3064688
%7 2021
%D 2021
%J IEEE Computer Graphics and Applications
%V 41
%N 4
%& 64
%P 64 - 75
%I IEEE
%C Piscataway, NJ
%@ false

Article

D4D6

A. Meka, M. Shafiei, M. Zollhöfer, C. Richardt, and C. Theobalt

“Real-time Global Illumination Decomposition of Videos,” ACM Transactions on Graphics, vol. 40, no. 3, 2021.

@article{Meka:2021,
TITLE = {Real-time Global Illumination Decomposition of Videos},
AUTHOR = {Meka, Abhimitra and Shafiei, Mohammad and Zollh{\"o}fer, Michael and Richardt, Christian and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3374753},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2021},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {40},
NUMBER = {3},
PAGES = {1--16},
EID = {22},
}

Endnote

%0 Journal Article
%A Meka, Abhimitra
%A Shafiei, Mohammad
%A Zollh&#246;fer, Michael
%A Richardt, Christian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Real-time Global Illumination Decomposition of Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-EE07-6
%R 10.1145/3374753
%7 2021
%D 2021
%J ACM Transactions on Graphics
%V 40
%N 3
%& 1
%P 1 - 16
%Z sequence number: 22
%I ACM
%C New York, NY
%@ false
%U http://gvv.mpi-inf.mpg.de/projects/LiveIlluminationDecomposition/

Conference paper

D6D4

J. Nehvi, V. Golyanik, F. Mueller, H.-P. Seidel, M. Elgharib, and C. Theobalt

“Differentiable Event Stream Simulator for Non-Rigid 3D Tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR 2021), Nashville, TN, USA, 2021.

@inproceedings{Nehvi_CVPR2021Workshop,
TITLE = {Differentiable Event Stream Simulator for Non-Rigid {3D} Tracking},
AUTHOR = {Nehvi, Jalees and Golyanik, Vladislav and Mueller, Franziska and Seidel, Hans-Peter and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-6654-4899-4},
DOI = {10.1109/CVPRW53098.2021.00143},
PUBLISHER = {IEEE},
YEAR = {2021},
BOOKTITLE = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR 2021)},
PAGES = {1302--1311},
ADDRESS = {Nashville, TN, USA},
}

Endnote

%0 Conference Proceedings
%A Nehvi, Jalees
%A Golyanik, Vladislav
%A Mueller, Franziska
%A Seidel, Hans-Peter
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Differentiable Event Stream Simulator for Non-Rigid 3D Tracking : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-8957-C
%R 10.1109/CVPRW53098.2021.00143
%D 2021
%B 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
%Z date of event: 2021-06-19 - 2021-06-25
%C Nashville, TN, USA
%B Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
%P 1302 - 1311
%I IEEE
%@ 978-1-6654-4899-4
%U https://gvv.mpi-inf.mpg.de/projects/Event-based_Non-rigid_3D_Tracking/

Conference paper

D4D2

S. Rao, D. Stutz, and B. Schiele

“Adversarial Training Against Location-Optimized Adversarial Patches,” in Computer Vision -- ECCV Workshops 2020, Glasgow, UK, 2021.

@inproceedings{DBLP:conf/eccv/RaoSS20,
TITLE = {Adversarial Training Against Location-Optimized Adversarial Patches},
AUTHOR = {Rao, Sukrut and Stutz, David and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {978-3-030-68237-8},
DOI = {10.1007/978-3-030-68238-5_32},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2021},
BOOKTITLE = {Computer Vision -- ECCV Workshops 2020},
EDITOR = {Bartoli, Adrian and Fusiello, Andrea},
PAGES = {429--448},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12539},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Rao, Sukrut
%A Stutz, David
%A Schiele, Bernt
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Adversarial Training Against Location-Optimized Adversarial Patches : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-1662-1
%R 10.1007/978-3-030-68238-5_32
%D 2021
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV Workshops 2020 
%E Bartoli, Adrian; Fusiello, Andrea
%P 429 - 448
%I Springer
%@ 978-3-030-68237-8
%B Lecture Notes in Computer Science
%N 12539

Article

T. Rittig, D. Sumin, V. Babaei, P. Didyk, A. Voloboy, A. Wilkie, B. Bickel, K. Myszkowski, T. Weyrich, and J. Krivanek

“Neural Acceleration of Scattering-Aware Color 3D Printing,” Computer Graphics Forum (Proc. EUROGRAPHICS 2021), vol. 40, no. 2, 2021.

@article{rittig2021neural,
TITLE = {Neural Acceleration of Scattering-Aware Color {3D} Printing},
AUTHOR = {Rittig, Tobias and Sumin, Denis and Babaei, Vahid and Didyk, Piotr and Voloboy, Alexei and Wilkie, Alexander and Bickel, Bernd and Myszkowski, Karol and Weyrich, Tim and Krivanek, Jaroslav},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.142626},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2021},
DATE = {2021},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
EDITOR = {Mitra, Niloy and Violoa, Ivan},
VOLUME = {40},
NUMBER = {2},
PAGES = {205--219},
BOOKTITLE = {42nd Annual Conference of the European Association for Computer Graphics (EUROGRAPHICS 2021)},
}

Endnote

%0 Journal Article
%A Rittig, Tobias
%A Sumin, Denis
%A Babaei, Vahid
%A Didyk, Piotr
%A Voloboy, Alexei
%A Wilkie, Alexander
%A Bickel, Bernd
%A Myszkowski, Karol
%A Weyrich, Tim
%A Krivanek, Jaroslav
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Neural Acceleration of Scattering-Aware Color 3D Printing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-F073-8
%R 10.1111/cgf.142626
%7 2021
%D 2021
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 40
%N 2
%& 205
%P 205 - 219
%I Blackwell-Wiley
%C Oxford
%@ false
%B 42nd Annual Conference of the European Association for Computer Graphics
%O EUROGRAPHICS 2021 EG 2021

Article

L. Ruan, B. Chen, J. Li, and M.-L. Lam

“AIFNet: All-in-Focus Image Restoration Network Using a Light Field-Based Dataset,” IEEE Transactions on Computational Imaging, vol. 7, 2021.

@article{Ruan2021,
TITLE = {{AIFNet}: {A}ll-in-Focus Image Restoration Network Using a Light Field-Based Dataset},
AUTHOR = {Ruan, Lingyan and Chen, Bin and Li, Jizhou and Lam, Miu-Ling},
LANGUAGE = {eng},
ISSN = {2573-0436},
DOI = {10.1109/TCI.2021.3092891},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2021},
DATE = {2021},
JOURNAL = {IEEE Transactions on Computational Imaging},
VOLUME = {7},
PAGES = {675--688},
}

Endnote

%0 Journal Article
%A Ruan, Lingyan
%A Chen, Bin
%A Li, Jizhou
%A Lam, Miu-Ling
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T AIFNet: All-in-Focus Image Restoration Network Using a Light Field-Based Dataset : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-4795-F
%R 10.1109/TCI.2021.3092891
%7 2021
%D 2021
%J IEEE Transactions on Computational Imaging
%V 7
%& 675
%P 675 - 688
%I IEEE
%C Piscataway, NJ
%@ false

Conference paper

D6D4

V. Rudnev, V. Golyanik, J. Wang, H.-P. Seidel, F. Mueller, M. Elgharib, and C. Theobalt

“EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream,” in ICCV 2021, Montreal, QC, Canada, 2021.

@inproceedings{Rudnev_2021_ICCV,
TITLE = {{EventHands}: {R}eal-Time Neural {3D} Hand Pose Estimation from an Event Stream},
AUTHOR = {Rudnev, Viktor and Golyanik, Vladislav and Wang, Jiayi and Seidel, Hans-Peter and Mueller, Franziska and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-6654-2812-5},
DOI = {10.1109/ICCV48922.2021.01216},
PUBLISHER = {IEEE},
YEAR = {2021},
BOOKTITLE = {ICCV 2021},
PAGES = {12365--12375},
ADDRESS = {Montreal, QC, Canada},
}

Endnote

%0 Conference Proceedings
%A Rudnev, Viktor
%A Golyanik, Vladislav
%A Wang, Jiayi
%A Seidel, Hans-Peter
%A Mueller, Franziska
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T EventHands: Real-Time Neural 3D Hand Pose Estimation from an Event Stream : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B709-1
%R 10.1109/ICCV48922.2021.01216
%D 2021
%B IEEE/CVF International Conference on Computer Vision
%Z date of event: 2021-10-10 - 2021-10-17
%C Montreal, QC, Canada
%B ICCV 2021
%P 12365 - 12375
%I IEEE
%@ 978-1-6654-2812-5

Paper

D6D4

K. Sarkar, D. Mehta, W. Xu, V. Golyanik, and C. Theobalt

“Neural Re-Rendering of Humans from a Single Image,” 2021. [Online]. Available: https://arxiv.org/abs/2101.04104.

Abstract

Human re-rendering from a single image is a starkly under-constrained
problem, and state-of-the-art algorithms often exhibit undesired artefacts,
such as over-smoothing, unrealistic distortions of the body parts and garments,
or implausible changes of the texture. To address these challenges, we propose
a new method for neural re-rendering of a human under a novel user-defined pose
and viewpoint, given one input image. Our algorithm represents body pose and
shape as a parametric mesh which can be reconstructed from a single image and
easily reposed. Instead of a colour-based UV texture map, our approach further
employs a learned high-dimensional UV feature map to encode appearance. This
rich implicit representation captures detailed appearance variation across
poses, viewpoints, person identities and clothing styles better than learned
colour texture maps. The body model with the rendered feature maps is fed
through a neural image-translation network that creates the final rendered
colour image. The above components are combined in an end-to-end-trained neural
network architecture that takes as input a source person image, and images of
the parametric body model in the source pose and desired target pose.
Experimental evaluation demonstrates that our approach produces higher quality
single image re-rendering results than existing methods.

BibTeX

@online{Sarkar_arXiv2101.04104,
TITLE = {Neural Re-Rendering of Humans from a Single Image},
AUTHOR = {Sarkar, Kripasindhu and Mehta, Dushyant and Xu, Weipeng and Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2101.04104},
EPRINT = {2101.04104},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {Human re-rendering from a single image is a starkly under-constrained<br>problem, and state-of-the-art algorithms often exhibit undesired artefacts,<br>such as over-smoothing, unrealistic distortions of the body parts and garments,<br>or implausible changes of the texture. To address these challenges, we propose<br>a new method for neural re-rendering of a human under a novel user-defined pose<br>and viewpoint, given one input image. Our algorithm represents body pose and<br>shape as a parametric mesh which can be reconstructed from a single image and<br>easily reposed. Instead of a colour-based UV texture map, our approach further<br>employs a learned high-dimensional UV feature map to encode appearance. This<br>rich implicit representation captures detailed appearance variation across<br>poses, viewpoints, person identities and clothing styles better than learned<br>colour texture maps. The body model with the rendered feature maps is fed<br>through a neural image-translation network that creates the final rendered<br>colour image. The above components are combined in an end-to-end-trained neural<br>network architecture that takes as input a source person image, and images of<br>the parametric body model in the source pose and desired target pose.<br>Experimental evaluation demonstrates that our approach produces higher quality<br>single image re-rendering results than existing methods.<br>},
}

Endnote

%0 Report
%A Sarkar, Kripasindhu
%A Mehta, Dushyant
%A Xu, Weipeng
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Neural Re-Rendering of Humans from a Single Image : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CF05-B
%U https://arxiv.org/abs/2101.04104
%D 2021
%X   Human re-rendering from a single image is a starkly under-constrained<br>problem, and state-of-the-art algorithms often exhibit undesired artefacts,<br>such as over-smoothing, unrealistic distortions of the body parts and garments,<br>or implausible changes of the texture. To address these challenges, we propose<br>a new method for neural re-rendering of a human under a novel user-defined pose<br>and viewpoint, given one input image. Our algorithm represents body pose and<br>shape as a parametric mesh which can be reconstructed from a single image and<br>easily reposed. Instead of a colour-based UV texture map, our approach further<br>employs a learned high-dimensional UV feature map to encode appearance. This<br>rich implicit representation captures detailed appearance variation across<br>poses, viewpoints, person identities and clothing styles better than learned<br>colour texture maps. The body model with the rendered feature maps is fed<br>through a neural image-translation network that creates the final rendered<br>colour image. The above components are combined in an end-to-end-trained neural<br>network architecture that takes as input a source person image, and images of<br>the parametric body model in the source pose and desired target pose.<br>Experimental evaluation demonstrates that our approach produces higher quality<br>single image re-rendering results than existing methods.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

A. Serrano, B. Chen, C. Wang, M. Piovarči, H.-P. Seidel, P. Didyk, and K. Myszkowski

“The Effect of Shape and Illumination on Material Perception: Model and Applications,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2021), vol. 40, no. 4, 2021.

@article{SIGG2021_Materials,
TITLE = {The Effect of Shape and Illumination on Material Perception: Model and Applications},
AUTHOR = {Serrano, Ana and Chen, Bin and Wang, Chao and Piovar{\v c}i, Michal and Seidel, Hans-Peter and Didyk, Piotr and Myszkowski, Karol},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3450626.3459813},
PUBLISHER = {Association for Computing Machinery},
ADDRESS = {New York, NY},
YEAR = {2021},
DATE = {2021},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {40},
NUMBER = {4},
PAGES = {1--16},
EID = {125},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2021},
}

Endnote

%0 Journal Article
%A Serrano, Ana
%A Chen, Bin
%A Wang, Chao
%A Piovar&#269;i, Michal
%A Seidel, Hans-Peter
%A Didyk, Piotr
%A Myszkowski, Karol
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T The Effect of Shape and Illumination on Material Perception: Model and Applications : Model and Applications
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-0565-0
%R 10.1145/3450626.3459813
%7 2021
%D 2021
%J ACM Transactions on Graphics
%V 40
%N 4
%& 1
%P 1 - 16
%Z sequence number: 125
%I Association for Computing Machinery
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2021
%O ACM SIGGRAPH 2021

Paper

L. Surace, M. Wernikowski, O. T. Tursun, K. Myszkowski, R. Mantiuk, and P. Didyk

“Learning Foveated Reconstruction to Preserve Perceived Image Statistics,” 2021. [Online]. Available: https://arxiv.org/abs/2108.03499.

Abstract

Foveated image reconstruction recovers full image from a sparse set of
samples distributed according to the human visual system's retinal sensitivity
that rapidly drops with eccentricity. Recently, the use of Generative
Adversarial Networks was shown to be a promising solution for such a task as
they can successfully hallucinate missing image information. Like for other
supervised learning approaches, also for this one, the definition of the loss
function and training strategy heavily influences the output quality. In this
work, we pose the question of how to efficiently guide the training of foveated
reconstruction techniques such that they are fully aware of the human visual
system's capabilities and limitations, and therefore, reconstruct visually
important image features. Due to the nature of GAN-based solutions, we
concentrate on the human's sensitivity to hallucination for different input
sample densities. We present new psychophysical experiments, a dataset, and a
procedure for training foveated image reconstruction. The strategy provides
flexibility to the generator network by penalizing only perceptually important
deviations in the output. As a result, the method aims to preserve perceived
image statistics rather than natural image statistics. We evaluate our strategy
and compare it to alternative solutions using a newly trained objective metric
and user experiments.

BibTeX

@online{Surace2108.03499,
TITLE = {Learning Foveated Reconstruction to Preserve Perceived Image Statistics},
AUTHOR = {Surace, Luca and Wernikowski, Marek and Tursun, Okan Tarhan and Myszkowski, Karol and Mantiuk, Rados{\l}aw and Didyk, Piotr},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2108.03499},
EPRINT = {2108.03499},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {Foveated image reconstruction recovers full image from a sparse set of<br>samples distributed according to the human visual system's retinal sensitivity<br>that rapidly drops with eccentricity. Recently, the use of Generative<br>Adversarial Networks was shown to be a promising solution for such a task as<br>they can successfully hallucinate missing image information. Like for other<br>supervised learning approaches, also for this one, the definition of the loss<br>function and training strategy heavily influences the output quality. In this<br>work, we pose the question of how to efficiently guide the training of foveated<br>reconstruction techniques such that they are fully aware of the human visual<br>system's capabilities and limitations, and therefore, reconstruct visually<br>important image features. Due to the nature of GAN-based solutions, we<br>concentrate on the human's sensitivity to hallucination for different input<br>sample densities. We present new psychophysical experiments, a dataset, and a<br>procedure for training foveated image reconstruction. The strategy provides<br>flexibility to the generator network by penalizing only perceptually important<br>deviations in the output. As a result, the method aims to preserve perceived<br>image statistics rather than natural image statistics. We evaluate our strategy<br>and compare it to alternative solutions using a newly trained objective metric<br>and user experiments.<br>},
}

Endnote

%0 Report
%A Surace, Luca
%A Wernikowski, Marek
%A Tursun, Okan Tarhan
%A Myszkowski, Karol
%A Mantiuk, Rados&#322;aw
%A Didyk, Piotr
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Learning Foveated Reconstruction to Preserve Perceived Image Statistics : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-73D9-1
%U https://arxiv.org/abs/2108.03499
%D 2021
%X   Foveated image reconstruction recovers full image from a sparse set of<br>samples distributed according to the human visual system's retinal sensitivity<br>that rapidly drops with eccentricity. Recently, the use of Generative<br>Adversarial Networks was shown to be a promising solution for such a task as<br>they can successfully hallucinate missing image information. Like for other<br>supervised learning approaches, also for this one, the definition of the loss<br>function and training strategy heavily influences the output quality. In this<br>work, we pose the question of how to efficiently guide the training of foveated<br>reconstruction techniques such that they are fully aware of the human visual<br>system's capabilities and limitations, and therefore, reconstruct visually<br>important image features. Due to the nature of GAN-based solutions, we<br>concentrate on the human's sensitivity to hallucination for different input<br>sample densities. We present new psychophysical experiments, a dataset, and a<br>procedure for training foveated image reconstruction. The strategy provides<br>flexibility to the generator network by penalizing only perceptually important<br>deviations in the output. As a result, the method aims to preserve perceived<br>image statistics rather than natural image statistics. We evaluate our strategy<br>and compare it to alternative solutions using a newly trained objective metric<br>and user experiments.<br>
%K Computer Science, Graphics, cs.GR,Computer Science, Computer Vision and Pattern Recognition, cs.CV

Thesis

D4IMPR-CS

A. Tewari

“Self-supervised reconstruction and synthesis of faces,” Universität des Saarlandes, Saarbrücken, 2021.

@phdthesis{Tewariphd2021,
TITLE = {Self-supervised reconstruction and synthesis of faces},
AUTHOR = {Tewari, Ayush},
LANGUAGE = {eng},
URL = {nbn:de:bsz:291--ds-345982},
DOI = {10.22028/D291-34598},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2021},
DATE = {2021},
}

Endnote

%0 Thesis
%A Tewari, Ayush
%Y Theobalt, Christian
%A referee: Zollh&#246;fer, Michael
%A referee: Wonka, Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Self-supervised reconstruction and synthesis of faces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-9CD2-A
%R 10.22028/D291-34598
%U nbn:de:bsz:291--ds-345982
%F OTHER: hdl:20.500.11880/31754
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2021
%P 173 p.
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/31754

Paper

C. van Onzenoodt, G. Singh, T. Ropinski, and T. Ritschel

“Blue Noise Plots,” 2021. [Online]. Available: https://arxiv.org/abs/2102.04072.

Abstract

We propose Blue Noise Plots, two-dimensional dot plots that depict data
points of univariate data sets. While often one-dimensional strip plots are
used to depict such data, one of their main problems is visual clutter which
results from overlap. To reduce this overlap, jitter plots were introduced,
whereby an additional, non-encoding plot dimension is introduced, along which
the data point representing dots are randomly perturbed. Unfortunately, this
randomness can suggest non-existent clusters, and often leads to visually
unappealing plots, in which overlap might still occur. To overcome these
shortcomings, we introduce BlueNoise Plots where random jitter along the
non-encoding plot dimension is replaced by optimizing all dots to keep a
minimum distance in 2D i. e., Blue Noise. We evaluate the effectiveness as well
as the aesthetics of Blue Noise Plots through both, a quantitative and a
qualitative user study.

BibTeX

@online{Onzenoodt_2102.04072,
TITLE = {Blue Noise Plots},
AUTHOR = {van Onzenoodt, Christian and Singh, Gurprit and Ropinski, Timo and Ritschel, Tobias},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2102.04072},
EPRINT = {2102.04072},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {We propose Blue Noise Plots, two-dimensional dot plots that depict data<br>points of univariate data sets. While often one-dimensional strip plots are<br>used to depict such data, one of their main problems is visual clutter which<br>results from overlap. To reduce this overlap, jitter plots were introduced,<br>whereby an additional, non-encoding plot dimension is introduced, along which<br>the data point representing dots are randomly perturbed. Unfortunately, this<br>randomness can suggest non-existent clusters, and often leads to visually<br>unappealing plots, in which overlap might still occur. To overcome these<br>shortcomings, we introduce BlueNoise Plots where random jitter along the<br>non-encoding plot dimension is replaced by optimizing all dots to keep a<br>minimum distance in 2D i. e., Blue Noise. We evaluate the effectiveness as well<br>as the aesthetics of Blue Noise Plots through both, a quantitative and a<br>qualitative user study.<br>},
}

Endnote

%0 Report
%A van Onzenoodt, Christian
%A Singh, Gurprit
%A Ropinski, Timo
%A Ritschel, Tobias
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Blue Noise Plots : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-01ED-C
%U https://arxiv.org/abs/2102.04072
%D 2021
%X   We propose Blue Noise Plots, two-dimensional dot plots that depict data<br>points of univariate data sets. While often one-dimensional strip plots are<br>used to depict such data, one of their main problems is visual clutter which<br>results from overlap. To reduce this overlap, jitter plots were introduced,<br>whereby an additional, non-encoding plot dimension is introduced, along which<br>the data point representing dots are randomly perturbed. Unfortunately, this<br>randomness can suggest non-existent clusters, and often leads to visually<br>unappealing plots, in which overlap might still occur. To overcome these<br>shortcomings, we introduce BlueNoise Plots where random jitter along the<br>non-encoding plot dimension is replaced by optimizing all dots to keep a<br>minimum distance in 2D i. e., Blue Noise. We evaluate the effectiveness as well<br>as the aesthetics of Blue Noise Plots through both, a quantitative and a<br>qualitative user study.<br>
%K Computer Science, Graphics, cs.GR

Article

C. van Onzenoodt, G. Singh, T. Ropinski, and T. Ritschel

“Blue Noise Plots,” Computer Graphics Forum (Proc. EUROGRAPHICS 2021), vol. 40, no. 2, 2021.

@article{onzenoodt2021blue,
TITLE = {Blue Noise Plots},
AUTHOR = {van Onzenoodt, Christian and Singh, Gurprit and Ropinski, Timo and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.142644},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2021},
DATE = {2021},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {40},
NUMBER = {2},
PAGES = {425--433},
BOOKTITLE = {42nd Annual Conference of the European Association for Computer Graphics (EUROGRAPHICS 2021)},
EDITOR = {Mitra, Niloy and Viola, Ivan},
}

Endnote

%0 Journal Article
%A van Onzenoodt, Christian
%A Singh, Gurprit
%A Ropinski, Timo
%A Ritschel, Tobias
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Blue Noise Plots : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-010F-7
%R 10.1111/cgf.142644
%7 2021
%D 2021
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 40
%N 2
%& 425
%P 425 - 433
%I Blackwell-Wiley
%C Oxford
%@ false
%B 42nd Annual Conference of the European Association for Computer Graphics
%O EUROGRAPHICS 2021 EG 2021

Paper

C. Wang, B. Chen, H.-P. Seidel, K. Myszkowski, and A. Serrano

“Learning a self-supervised tone mapping operator via feature contrast masking loss,” 2021. [Online]. Available: https://arxiv.org/abs/2110.09866.

Abstract

High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid
development of capture technologies. Nevertheless, the dynamic range of common
display devices is still limited, therefore tone mapping (TM) remains a key
challenge for image visualization. Recent work has demonstrated that neural
networks can achieve remarkable performance in this task when compared to
traditional methods, however, the quality of the results of these
learning-based methods is limited by the training data. Most existing works use
as training set a curated selection of best-performing results from existing
traditional tone mapping operators (often guided by a quality metric),
therefore, the quality of newly generated results is fundamentally limited by
the performance of such operators. This quality might be even further limited
by the pool of HDR content that is used for training. In this work we propose a
learning-based self-supervised tone mapping operator that is trained at test
time specifically for each HDR image and does not need any data labeling. The
key novelty of our approach is a carefully designed loss function built upon
fundamental knowledge on contrast perception that allows for directly comparing
the content in the HDR and tone mapped images. We achieve this goal by
reformulating classic VGG feature maps into feature contrast maps that
normalize local feature differences by their average magnitude in a local
neighborhood, allowing our loss to account for contrast masking effects. We
perform extensive ablation studies and exploration of parameters and
demonstrate that our solution outperforms existing approaches with a single set
of fixed parameters, as confirmed by both objective and subjective metrics.

BibTeX

@online{Wang_2110.09866,
TITLE = {Learning a self-supervised tone mapping operator via feature contrast masking loss},
AUTHOR = {Wang, Chao and Chen, Bin and Seidel, Hans-Peter and Myszkowski, Karol and Serrano, Ana},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2110.09866},
EPRINT = {2110.09866},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid<br>development of capture technologies. Nevertheless, the dynamic range of common<br>display devices is still limited, therefore tone mapping (TM) remains a key<br>challenge for image visualization. Recent work has demonstrated that neural<br>networks can achieve remarkable performance in this task when compared to<br>traditional methods, however, the quality of the results of these<br>learning-based methods is limited by the training data. Most existing works use<br>as training set a curated selection of best-performing results from existing<br>traditional tone mapping operators (often guided by a quality metric),<br>therefore, the quality of newly generated results is fundamentally limited by<br>the performance of such operators. This quality might be even further limited<br>by the pool of HDR content that is used for training. In this work we propose a<br>learning-based self-supervised tone mapping operator that is trained at test<br>time specifically for each HDR image and does not need any data labeling. The<br>key novelty of our approach is a carefully designed loss function built upon<br>fundamental knowledge on contrast perception that allows for directly comparing<br>the content in the HDR and tone mapped images. We achieve this goal by<br>reformulating classic VGG feature maps into feature contrast maps that<br>normalize local feature differences by their average magnitude in a local<br>neighborhood, allowing our loss to account for contrast masking effects. We<br>perform extensive ablation studies and exploration of parameters and<br>demonstrate that our solution outperforms existing approaches with a single set<br>of fixed parameters, as confirmed by both objective and subjective metrics.<br>},
}

Endnote

%0 Report
%A Wang, Chao
%A Chen, Bin
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%A Serrano, Ana
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Learning a self-supervised tone mapping operator via feature contrast masking loss : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-710E-9
%U https://arxiv.org/abs/2110.09866
%D 2021
%X   High Dynamic Range (HDR) content is becoming ubiquitous due to the rapid<br>development of capture technologies. Nevertheless, the dynamic range of common<br>display devices is still limited, therefore tone mapping (TM) remains a key<br>challenge for image visualization. Recent work has demonstrated that neural<br>networks can achieve remarkable performance in this task when compared to<br>traditional methods, however, the quality of the results of these<br>learning-based methods is limited by the training data. Most existing works use<br>as training set a curated selection of best-performing results from existing<br>traditional tone mapping operators (often guided by a quality metric),<br>therefore, the quality of newly generated results is fundamentally limited by<br>the performance of such operators. This quality might be even further limited<br>by the pool of HDR content that is used for training. In this work we propose a<br>learning-based self-supervised tone mapping operator that is trained at test<br>time specifically for each HDR image and does not need any data labeling. The<br>key novelty of our approach is a carefully designed loss function built upon<br>fundamental knowledge on contrast perception that allows for directly comparing<br>the content in the HDR and tone mapped images. We achieve this goal by<br>reformulating classic VGG feature maps into feature contrast maps that<br>normalize local feature differences by their average magnitude in a local<br>neighborhood, allowing our loss to account for contrast masking effects. We<br>perform extensive ablation studies and exploration of parameters and<br>demonstrate that our solution outperforms existing approaches with a single set<br>of fixed parameters, as confirmed by both objective and subjective metrics.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

Paper

A. Weinrauch, H.-P. Seidel, D. Mlakar, M. Steinberger, and R. Zayer

“A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and Reeb Graph Construction on Surfaces,” 2021. [Online]. Available: https://arxiv.org/abs/2105.13168.

Abstract

The humble loop shrinking property played a central role in the inception of
modern topology but it has been eclipsed by more abstract algebraic formalism.
This is particularly true in the context of detecting relevant non-contractible
loops on surfaces where elaborate homological and/or graph theoretical
constructs are favored in algorithmic solutions. In this work, we devise a
variational analogy to the loop shrinking property and show that it yields a
simple, intuitive, yet powerful solution allowing a streamlined treatment of
the problem of handle and tunnel loop detection. Our formalization tracks the
evolution of a diffusion front randomly initiated on a single location on the
surface. Capitalizing on a diffuse interface representation combined with a set
of rules for concurrent front interactions, we develop a dynamic data structure
for tracking the evolution on the surface encoded as a sparse matrix which
serves for performing both diffusion numerics and loop detection and acts as
the workhorse of our fully parallel implementation. The substantiated results
suggest our approach outperforms state of the art and robustly copes with
highly detailed geometric models. As a byproduct, our approach can be used to
construct Reeb graphs by diffusion thus avoiding commonly encountered issues
when using Morse functions.

BibTeX

@online{Weinrauch_2105.13168,
TITLE = {A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and {Reeb} Graph Construction on Surfaces},
AUTHOR = {Weinrauch, Alexander and Seidel, Hans-Peter and Mlakar, Daniel and Steinberger, Markus and Zayer, Rhaleb},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2105.13168},
EPRINT = {2105.13168},
EPRINTTYPE = {arXiv},
YEAR = {2021},
ABSTRACT = {The humble loop shrinking property played a central role in the inception of<br>modern topology but it has been eclipsed by more abstract algebraic formalism.<br>This is particularly true in the context of detecting relevant non-contractible<br>loops on surfaces where elaborate homological and/or graph theoretical<br>constructs are favored in algorithmic solutions. In this work, we devise a<br>variational analogy to the loop shrinking property and show that it yields a<br>simple, intuitive, yet powerful solution allowing a streamlined treatment of<br>the problem of handle and tunnel loop detection. Our formalization tracks the<br>evolution of a diffusion front randomly initiated on a single location on the<br>surface. Capitalizing on a diffuse interface representation combined with a set<br>of rules for concurrent front interactions, we develop a dynamic data structure<br>for tracking the evolution on the surface encoded as a sparse matrix which<br>serves for performing both diffusion numerics and loop detection and acts as<br>the workhorse of our fully parallel implementation. The substantiated results<br>suggest our approach outperforms state of the art and robustly copes with<br>highly detailed geometric models. As a byproduct, our approach can be used to<br>construct Reeb graphs by diffusion thus avoiding commonly encountered issues<br>when using Morse functions.<br>},
}

Endnote

%0 Report
%A Weinrauch, Alexander
%A Seidel, Hans-Peter
%A Mlakar, Daniel
%A Steinberger, Markus
%A Zayer, Rhaleb
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T A Variational Loop Shrinking Analogy for Handle and Tunnel Detection and Reeb Graph Construction on Surfaces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-70EE-D
%U https://arxiv.org/abs/2105.13168
%D 2021
%X   The humble loop shrinking property played a central role in the inception of<br>modern topology but it has been eclipsed by more abstract algebraic formalism.<br>This is particularly true in the context of detecting relevant non-contractible<br>loops on surfaces where elaborate homological and/or graph theoretical<br>constructs are favored in algorithmic solutions. In this work, we devise a<br>variational analogy to the loop shrinking property and show that it yields a<br>simple, intuitive, yet powerful solution allowing a streamlined treatment of<br>the problem of handle and tunnel loop detection. Our formalization tracks the<br>evolution of a diffusion front randomly initiated on a single location on the<br>surface. Capitalizing on a diffuse interface representation combined with a set<br>of rules for concurrent front interactions, we develop a dynamic data structure<br>for tracking the evolution on the surface encoded as a sparse matrix which<br>serves for performing both diffusion numerics and loop detection and acts as<br>the workhorse of our fully parallel implementation. The substantiated results<br>suggest our approach outperforms state of the art and robustly copes with<br>highly detailed geometric models. As a byproduct, our approach can be used to<br>construct Reeb graphs by diffusion thus avoiding commonly encountered issues<br>when using Morse functions.<br>
%K Computer Science, Graphics, cs.GR,Computer Science, Computational Geometry, cs.CG,Mathematics, Algebraic Topology, math.AT

Conference paper

D6D4

T. Yenamandra, A. Tewari, F. Bernard, H.-P. Seidel, M. Elgharib, D. Cremers, and C. Theobalt

“i3DMM: Deep Implicit 3D Morphable Model of Human Heads,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021), Virtual Conference, 2021.

@inproceedings{Yenamandra_CVPR2021,
TITLE = {{i3DMM}: {D}eep Implicit {3D} Morphable Model of Human Heads},
AUTHOR = {Yenamandra, Tarun and Tewari, Ayush and Bernard, Florian and Seidel, Hans-Peter and Elgharib, Mohamed and Cremers, Daniel and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-6654-4509-2},
DOI = {10.1109/CVPR46437.2021.01261},
PUBLISHER = {IEEE},
YEAR = {2021},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)},
PAGES = {12803--12813},
ADDRESS = {Virtual Conference},
}

Endnote

%0 Conference Proceedings
%A Yenamandra, Tarun
%A Tewari, Ayush
%A Bernard, Florian
%A Seidel, Hans-Peter
%A Elgharib, Mohamed
%A Cremers, Daniel
%A Theobalt, Christian
%+ External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T i3DMM: Deep Implicit 3D Morphable Model of Human Heads  : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-8966-B
%R 10.1109/CVPR46437.2021.01261
%D 2021
%B 34th IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2021-06-19 - 2021-06-25
%C Virtual Conference
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 12803 - 12813
%I IEEE
%@ 978-1-6654-4509-2
%U https://gvv.mpi-inf.mpg.de/projects/i3DMM/

Conference paper

Q. Zheng, G. Singh, and H.-P. Seidel

“Neural Relightable Participating Media Rendering,” in Advances in Neural Information Processing Systems 34 (NeurIPS 2021), Virtual, 2021.

@inproceedings{Zheng_Neurips2021,
TITLE = {Neural Relightable Participating Media Rendering},
AUTHOR = {Zheng, Quan and Singh, Gurprit and Seidel, Hans-Peter},
LANGUAGE = {eng},
ISBN = {9781713845393},
PUBLISHER = {Curran Associates, Inc.},
YEAR = {2021},
BOOKTITLE = {Advances in Neural Information Processing Systems 34 (NeurIPS 2021)},
EDITOR = {Ranzato, M. and Beygelzimer, A. and Liang, P. S. and Vaughan, J. W. and Dauphin, Y.},
PAGES = {15203--15215},
ADDRESS = {Virtual},
}

Endnote

%0 Conference Proceedings
%A Zheng, Quan
%A Singh, Gurprit
%A Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Relightable Participating Media Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-7117-E
%D 2021
%B 35th Conference on Neural Information Processing Systems
%Z date of event: 2021-12-06 - 2021-12-14
%C Virtual
%B Advances in Neural Information Processing Systems 34
%E Ranzato, M.; Beygelzimer, A.; Liang, P. S.; Vaughan, J. W.; Dauphin, Y.
%P 15203 - 15215
%I Curran Associates, Inc.
%@ 9781713845393

2020

Paper

S. A. Ali, K. Kahraman, C. Theobalt, D. Stricker, and V. Golyanik

“Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations,” 2020. [Online]. Available: https://arxiv.org/abs/2009.14005.

Abstract

This article introduces a new physics-based method for rigid point set
alignment called Fast Gravitational Approach (FGA). In FGA, the source and
target point sets are interpreted as rigid particle swarms with masses
interacting in a globally multiply-linked manner while moving in a simulated
gravitational force field. The optimal alignment is obtained by explicit
modeling of forces acting on the particles as well as their velocities and
displacements with second-order ordinary differential equations of motion.
Additional alignment cues (point-based or geometric features, and other
boundary conditions) can be integrated into FGA through particle masses. We
propose a smooth-particle mass function for point mass initialization, which
improves robustness to noise and structural discontinuities. To avoid
prohibitive quadratic complexity of all-to-all point interactions, we adapt a
Barnes-Hut tree for accelerated force computation and achieve quasilinear
computational complexity. We show that the new method class has characteristics
not found in previous alignment methods such as efficient handling of partial
overlaps, inhomogeneous point sampling densities, and coping with large point
clouds with reduced runtime compared to the state of the art. Experiments show
that our method performs on par with or outperforms all compared competing
non-deep-learning-based and general-purpose techniques (which do not assume the
availability of training data and a scene prior) in resolving transformations
for LiDAR data and gains state-of-the-art accuracy and speed when coping with
different types of data disturbances.

BibTeX

@online{Ali_2009.14005,
TITLE = {Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations},
AUTHOR = {Ali, Sk Aziz and Kahraman, Kerem and Theobalt, Christian and Stricker, Didier and Golyanik, Vladislav},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2009.14005},
EPRINT = {2009.14005},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {This article introduces a new physics-based method for rigid point set<br>alignment called Fast Gravitational Approach (FGA). In FGA, the source and<br>target point sets are interpreted as rigid particle swarms with masses<br>interacting in a globally multiply-linked manner while moving in a simulated<br>gravitational force field. The optimal alignment is obtained by explicit<br>modeling of forces acting on the particles as well as their velocities and<br>displacements with second-order ordinary differential equations of motion.<br>Additional alignment cues (point-based or geometric features, and other<br>boundary conditions) can be integrated into FGA through particle masses. We<br>propose a smooth-particle mass function for point mass initialization, which<br>improves robustness to noise and structural discontinuities. To avoid<br>prohibitive quadratic complexity of all-to-all point interactions, we adapt a<br>Barnes-Hut tree for accelerated force computation and achieve quasilinear<br>computational complexity. We show that the new method class has characteristics<br>not found in previous alignment methods such as efficient handling of partial<br>overlaps, inhomogeneous point sampling densities, and coping with large point<br>clouds with reduced runtime compared to the state of the art. Experiments show<br>that our method performs on par with or outperforms all compared competing<br>non-deep-learning-based and general-purpose techniques (which do not assume the<br>availability of training data and a scene prior) in resolving transformations<br>for LiDAR data and gains state-of-the-art accuracy and speed when coping with<br>different types of data disturbances.<br>},
}

Endnote

%0 Report
%A Ali, Sk Aziz
%A Kahraman, Kerem
%A Theobalt, Christian
%A Stricker, Didier
%A Golyanik, Vladislav
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Fast Gravitational Approach for Rigid Point Set Registration with
  Ordinary Differential Equations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E8FA-A
%U https://arxiv.org/abs/2009.14005
%D 2020
%X   This article introduces a new physics-based method for rigid point set<br>alignment called Fast Gravitational Approach (FGA). In FGA, the source and<br>target point sets are interpreted as rigid particle swarms with masses<br>interacting in a globally multiply-linked manner while moving in a simulated<br>gravitational force field. The optimal alignment is obtained by explicit<br>modeling of forces acting on the particles as well as their velocities and<br>displacements with second-order ordinary differential equations of motion.<br>Additional alignment cues (point-based or geometric features, and other<br>boundary conditions) can be integrated into FGA through particle masses. We<br>propose a smooth-particle mass function for point mass initialization, which<br>improves robustness to noise and structural discontinuities. To avoid<br>prohibitive quadratic complexity of all-to-all point interactions, we adapt a<br>Barnes-Hut tree for accelerated force computation and achieve quasilinear<br>computational complexity. We show that the new method class has characteristics<br>not found in previous alignment methods such as efficient handling of partial<br>overlaps, inhomogeneous point sampling densities, and coping with large point<br>clouds with reduced runtime compared to the state of the art. Experiments show<br>that our method performs on par with or outperforms all compared competing<br>non-deep-learning-based and general-purpose techniques (which do not assume the<br>availability of training data and a scene prior) in resolving transformations<br>for LiDAR data and gains state-of-the-art accuracy and speed when coping with<br>different types of data disturbances.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

Article

N. Ansari, O. Alizadeh-Mousavi, H.-P. Seidel, and V. Babaei

“Mixed Integer Ink Selection for Spectral Reproduction,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Ansari_ToG2020,
TITLE = {Mixed Integer Ink Selection for Spectral Reproduction},
AUTHOR = {Ansari, Navid and Alizadeh-Mousavi, Omid and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417761},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {255},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Ansari, Navid
%A Alizadeh-Mousavi, Omid
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Mixed Integer Ink Selection for Spectral Reproduction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-9B23-3
%R 10.1145/3414685.3417761
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 255
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Article

M. Bemana, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Bemana2020,
TITLE = {X-{F}ields: {I}mplicit Neural View-, Light- and Time-Image Interpolation},
AUTHOR = {Bemana, Mojtaba and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417827},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {257},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Bemana, Mojtaba
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-FBF0-0
%R 10.1145/3414685.3417827
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 257
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Paper

M. Bemana, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation,” 2020. [Online]. Available: https://arxiv.org/abs/2010.00450.

Abstract

We suggest to represent an X-Field -a set of 2D images taken across different
view, time or illumination conditions, i.e., video, light field, reflectance
fields or combinations thereof-by learning a neural network (NN) to map their
view, time or light coordinates to 2D images. Executing this NN at new
coordinates results in joint view, time or light interpolation. The key idea to
make this workable is a NN that already knows the "basic tricks" of graphics
(lighting, 3D projection, occlusion) in a hard-coded and differentiable form.
The NN represents the input to that rendering as an implicit map, that for any
view, time, or light coordinate and for any pixel can quantify how it will move
if view, time or light coordinates change (Jacobian of pixel position with
respect to view, time, illumination, etc.). Our X-Field representation is
trained for one scene within minutes, leading to a compact set of trainable
parameters and hence real-time navigation in view, time and illumination.

BibTeX

@online{Bemana_arXiv2010.00450,
TITLE = {X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation},
AUTHOR = {Bemana, Mojtaba and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2010.00450},
EPRINT = {2010.00450},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We suggest to represent an X-Field -a set of 2D images taken across different<br>view, time or illumination conditions, i.e., video, light field, reflectance<br>fields or combinations thereof-by learning a neural network (NN) to map their<br>view, time or light coordinates to 2D images. Executing this NN at new<br>coordinates results in joint view, time or light interpolation. The key idea to<br>make this workable is a NN that already knows the "basic tricks" of graphics<br>(lighting, 3D projection, occlusion) in a hard-coded and differentiable form.<br>The NN represents the input to that rendering as an implicit map, that for any<br>view, time, or light coordinate and for any pixel can quantify how it will move<br>if view, time or light coordinates change (Jacobian of pixel position with<br>respect to view, time, illumination, etc.). Our X-Field representation is<br>trained for one scene within minutes, leading to a compact set of trainable<br>parameters and hence real-time navigation in view, time and illumination.<br>},
}

Endnote

%0 Report
%A Bemana, Mojtaba
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B6EC-2
%U https://arxiv.org/abs/2010.00450
%D 2020
%X   We suggest to represent an X-Field -a set of 2D images taken across different<br>view, time or illumination conditions, i.e., video, light field, reflectance<br>fields or combinations thereof-by learning a neural network (NN) to map their<br>view, time or light coordinates to 2D images. Executing this NN at new<br>coordinates results in joint view, time or light interpolation. The key idea to<br>make this workable is a NN that already knows the "basic tricks" of graphics<br>(lighting, 3D projection, occlusion) in a hard-coded and differentiable form.<br>The NN represents the input to that rendering as an implicit map, that for any<br>view, time, or light coordinate and for any pixel can quantify how it will move<br>if view, time or light coordinates change (Jacobian of pixel position with<br>respect to view, time, illumination, etc.). Our X-Field representation is<br>trained for one scene within minutes, leading to a compact set of trainable<br>parameters and hence real-time navigation in view, time and illumination.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Conference paper

F. Bernard, Z. K. Suri, and C. Theobalt

“MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{Bernard_CVPR2020,
TITLE = {{MINA}: {C}onvex Mixed-Integer Programming for Non-Rigid Shape Alignment},
AUTHOR = {Bernard, Florian and Suri, Zeeshan Khan and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.01384},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {13823--13832},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Bernard, Florian
%A Suri, Zeeshan Khan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D059-A
%R 10.1109/CVPR42600.2020.01384
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 13823 - 13832
%I IEEE
%@ 978-1-7281-7168-5

Paper

F. Bernard, Z. K. Suri, and C. Theobalt

“MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment,” 2020. [Online]. Available: https://arxiv.org/abs/2002.12623.

Abstract

We present a convex mixed-integer programming formulation for non-rigid shape
matching. To this end, we propose a novel shape deformation model based on an
efficient low-dimensional discrete model, so that finding a globally optimal
solution is tractable in (most) practical cases. Our approach combines several
favourable properties: it is independent of the initialisation, it is much more
efficient to solve to global optimality compared to analogous quadratic
assignment problem formulations, and it is highly flexible in terms of the
variants of matching problems it can handle. Experimentally we demonstrate that
our approach outperforms existing methods for sparse shape matching, that it
can be used for initialising dense shape matching methods, and we showcase its
flexibility on several examples.

BibTeX

@online{Bernard_arXiv2002.12623,
TITLE = {MINA: {C}onvex Mixed-Integer Programming for Non-Rigid Shape Alignment},
AUTHOR = {Bernard, Florian and Suri, Zeeshan Khan and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2002.12623},
EPRINT = {2002.12623},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present a convex mixed-integer programming formulation for non-rigid shape<br>matching. To this end, we propose a novel shape deformation model based on an<br>efficient low-dimensional discrete model, so that finding a globally optimal<br>solution is tractable in (most) practical cases. Our approach combines several<br>favourable properties: it is independent of the initialisation, it is much more<br>efficient to solve to global optimality compared to analogous quadratic<br>assignment problem formulations, and it is highly flexible in terms of the<br>variants of matching problems it can handle. Experimentally we demonstrate that<br>our approach outperforms existing methods for sparse shape matching, that it<br>can be used for initialising dense shape matching methods, and we showcase its<br>flexibility on several examples.<br>},
}

Endnote

%0 Report
%A Bernard, Florian
%A Suri, Zeeshan Khan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E00C-F
%U https://arxiv.org/abs/2002.12623
%D 2020
%X   We present a convex mixed-integer programming formulation for non-rigid shape<br>matching. To this end, we propose a novel shape deformation model based on an<br>efficient low-dimensional discrete model, so that finding a globally optimal<br>solution is tractable in (most) practical cases. Our approach combines several<br>favourable properties: it is independent of the initialisation, it is much more<br>efficient to solve to global optimality compared to analogous quadratic<br>assignment problem formulations, and it is highly flexible in terms of the<br>variants of matching problems it can handle. Experimentally we demonstrate that<br>our approach outperforms existing methods for sparse shape matching, that it<br>can be used for initialising dense shape matching methods, and we showcase its<br>flexibility on several examples.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG,Mathematics, Optimization and Control, math.OC

Paper

B. L. Bhatnagar, C. Sminchisescu, C. Theobalt, and G. Pons-Moll

“LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration,” 2020. [Online]. Available: https://arxiv.org/abs/2010.12447.

Abstract

We address the problem of fitting 3D human models to 3D scans of dressed
humans. Classical methods optimize both the data-to-model correspondences and
the human model parameters (pose and shape), but are reliable only when
initialized close to the solution. Some methods initialize the optimization
based on fully supervised correspondence predictors, which is not
differentiable end-to-end, and can only process a single scan at a time. Our
main contribution is LoopReg, an end-to-end learning framework to register a
corpus of scans to a common 3D human model. The key idea is to create a
self-supervised loop. A backward map, parameterized by a Neural Network,
predicts the correspondence from every scan point to the surface of the human
model. A forward map, parameterized by a human model, transforms the
corresponding points back to the scan based on the model parameters (pose and
shape), thus closing the loop. Formulating this closed loop is not
straightforward because it is not trivial to force the output of the NN to be
on the surface of the human model - outside this surface the human model is not
even defined. To this end, we propose two key innovations. First, we define the
canonical surface implicitly as the zero level set of a distance field in R3,
which in contrast to morecommon UV parameterizations, does not require cutting
the surface, does not have discontinuities, and does not induce distortion.
Second, we diffuse the human model to the 3D domain R3. This allows to map the
NN predictions forward,even when they slightly deviate from the zero level set.
Results demonstrate that we can train LoopRegmainly self-supervised - following
a supervised warm-start, the model becomes increasingly more accurate as
additional unlabelled raw scans are processed. Our code and pre-trained models
can be downloaded for research.

BibTeX

@online{Bhatnagar_2010.12447,
TITLE = {{LoopReg}: {S}elf-supervised Learning of Implicit Surface Correspondences, Pose and Shape for {3D} Human Mesh Registration},
AUTHOR = {Bhatnagar, Bharat Lal and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2010.12447},
EPRINT = {2010.12447},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We address the problem of fitting 3D human models to 3D scans of dressed<br>humans. Classical methods optimize both the data-to-model correspondences and<br>the human model parameters (pose and shape), but are reliable only when<br>initialized close to the solution. Some methods initialize the optimization<br>based on fully supervised correspondence predictors, which is not<br>differentiable end-to-end, and can only process a single scan at a time. Our<br>main contribution is LoopReg, an end-to-end learning framework to register a<br>corpus of scans to a common 3D human model. The key idea is to create a<br>self-supervised loop. A backward map, parameterized by a Neural Network,<br>predicts the correspondence from every scan point to the surface of the human<br>model. A forward map, parameterized by a human model, transforms the<br>corresponding points back to the scan based on the model parameters (pose and<br>shape), thus closing the loop. Formulating this closed loop is not<br>straightforward because it is not trivial to force the output of the NN to be<br>on the surface of the human model -- outside this surface the human model is not<br>even defined. To this end, we propose two key innovations. First, we define the<br>canonical surface implicitly as the zero level set of a distance field in R3,<br>which in contrast to morecommon UV parameterizations, does not require cutting<br>the surface, does not have discontinuities, and does not induce distortion.<br>Second, we diffuse the human model to the 3D domain R3. This allows to map the<br>NN predictions forward,even when they slightly deviate from the zero level set.<br>Results demonstrate that we can train LoopRegmainly self-supervised -- following<br>a supervised warm-start, the model becomes increasingly more accurate as<br>additional unlabelled raw scans are processed. Our code and pre-trained models<br>can be downloaded for research.<br>},
}

Endnote

%0 Report
%A Bhatnagar, Bharat Lal
%A Sminchisescu, Cristian
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E91C-4
%U https://arxiv.org/abs/2010.12447
%D 2020
%X   We address the problem of fitting 3D human models to 3D scans of dressed<br>humans. Classical methods optimize both the data-to-model correspondences and<br>the human model parameters (pose and shape), but are reliable only when<br>initialized close to the solution. Some methods initialize the optimization<br>based on fully supervised correspondence predictors, which is not<br>differentiable end-to-end, and can only process a single scan at a time. Our<br>main contribution is LoopReg, an end-to-end learning framework to register a<br>corpus of scans to a common 3D human model. The key idea is to create a<br>self-supervised loop. A backward map, parameterized by a Neural Network,<br>predicts the correspondence from every scan point to the surface of the human<br>model. A forward map, parameterized by a human model, transforms the<br>corresponding points back to the scan based on the model parameters (pose and<br>shape), thus closing the loop. Formulating this closed loop is not<br>straightforward because it is not trivial to force the output of the NN to be<br>on the surface of the human model - outside this surface the human model is not<br>even defined. To this end, we propose two key innovations. First, we define the<br>canonical surface implicitly as the zero level set of a distance field in R3,<br>which in contrast to morecommon UV parameterizations, does not require cutting<br>the surface, does not have discontinuities, and does not induce distortion.<br>Second, we diffuse the human model to the 3D domain R3. This allows to map the<br>NN predictions forward,even when they slightly deviate from the zero level set.<br>Results demonstrate that we can train LoopRegmainly self-supervised - following<br>a supervised warm-start, the model becomes increasingly more accurate as<br>additional unlabelled raw scans are processed. Our code and pre-trained models<br>can be downloaded for research.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

D2D4

B. L. Bhatnagar, C. Sminchisescu, C. Theobalt, and G. Pons-Moll

“LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration,” in Advances in Neural Information Processing Systems 33 (NeurIPS 2020), Virtual Event, 2020.

@inproceedings{bhatnagar2020loopreg,
TITLE = {{LoopReg}: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for {3D} Human Mesh Registration},
AUTHOR = {Bhatnagar, Bharat Lal and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
ISBN = {9781713829546},
PUBLISHER = {Curran Associates, Inc.},
YEAR = {2020},
BOOKTITLE = {Advances in Neural Information Processing Systems 33 (NeurIPS 2020)},
EDITOR = {Larochelle, H. and Ranzato, M. and Hadsell, R. and Balcan, M. F. and Lin, H.},
PAGES = {12909--12922},
ADDRESS = {Virtual Event},
}

Endnote

%0 Conference Proceedings
%A Bhatnagar, Bharat Lal
%A Sminchisescu, Cristian
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-6FD1-1
%D 2020
%B 34th Conference on Neural Information Processing Systems
%Z date of event: 2020-12-06 - 2020-12-12
%C Virtual Event
%B Advances in Neural Information Processing Systems 33
%E Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.; Lin, H.
%P 12909 - 12922
%I Curran Associates, Inc.
%@ 9781713829546
%U https://papers.nips.cc/paper/2020/file/970af30e481057c48f87e101b61e6994-Paper.pdf

Conference paper

D2D4

B. L. Bhatnagar, C. Sminchisescu, C. Theobalt, and G. Pons-Moll

“Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{bhatnagar2020ipnet,
TITLE = {Combining Implicit Function Learning and Parametric Models for {3D} Human Reconstruction},
AUTHOR = {Bhatnagar, Bharat Lal and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
ISBN = {978-3-030-58535-8},
DOI = {10.1007/978-3-030-58536-5_19},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {311--329},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12347},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Bhatnagar, Bharat Lal
%A Sminchisescu, Cristian
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-E89E-3
%R 10.1007/978-3-030-58536-5_19
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 311 - 329
%I Springer
%@ 978-3-030-58535-8
%B Lecture Notes in Computer Science
%N 12347

Paper

B. L. Bhatnagar, C. Sminchisescu, C. Theobalt, and G. Pons-Moll

“Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction,” 2020. [Online]. Available: https://arxiv.org/abs/2007.11432.

Abstract

Implicit functions represented as deep learning approximations are powerful
for reconstructing 3D surfaces. However, they can only produce static surfaces
that are not controllable, which provides limited ability to modify the
resulting model by editing its pose or shape parameters. Nevertheless, such
features are essential in building flexible models for both computer graphics
and computer vision. In this work, we present methodology that combines
detail-rich implicit functions and parametric representations in order to
reconstruct 3D models of people that remain controllable and accurate even in
the presence of clothing. Given sparse 3D point clouds sampled on the surface
of a dressed person, we use an Implicit Part Network (IP-Net)to jointly predict
the outer 3D surface of the dressed person, the and inner body surface, and the
semantic correspondences to a parametric body model. We subsequently use
correspondences to fit the body model to our inner surface and then non-rigidly
deform it (under a parametric body + displacement model) to the outer surface
in order to capture garment, face and hair detail. In quantitative and
qualitative experiments with both full body data and hand scans we show that
the proposed methodology generalizes, and is effective even given incomplete
point clouds collected from single-view depth images. Our models and code can
be downloaded from virtualhumans.mpi-inf.mpg.de/ipnet.

BibTeX

@online{Bhatnagar_2007.11432,
TITLE = {Combining Implicit Function Learning and Parametric Models for {3D} Human Reconstruction},
AUTHOR = {Bhatnagar, Bharat Lal and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2007.11432},
EPRINT = {2007.11432},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Implicit functions represented as deep learning approximations are powerful<br>for reconstructing 3D surfaces. However, they can only produce static surfaces<br>that are not controllable, which provides limited ability to modify the<br>resulting model by editing its pose or shape parameters. Nevertheless, such<br>features are essential in building flexible models for both computer graphics<br>and computer vision. In this work, we present methodology that combines<br>detail-rich implicit functions and parametric representations in order to<br>reconstruct 3D models of people that remain controllable and accurate even in<br>the presence of clothing. Given sparse 3D point clouds sampled on the surface<br>of a dressed person, we use an Implicit Part Network (IP-Net)to jointly predict<br>the outer 3D surface of the dressed person, the and inner body surface, and the<br>semantic correspondences to a parametric body model. We subsequently use<br>correspondences to fit the body model to our inner surface and then non-rigidly<br>deform it (under a parametric body + displacement model) to the outer surface<br>in order to capture garment, face and hair detail. In quantitative and<br>qualitative experiments with both full body data and hand scans we show that<br>the proposed methodology generalizes, and is effective even given incomplete<br>point clouds collected from single-view depth images. Our models and code can<br>be downloaded from http://virtualhumans.mpi-inf.mpg.de/ipnet.<br>},
}

Endnote

%0 Report
%A Bhatnagar, Bharat Lal
%A Sminchisescu, Cristian
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E8A0-E
%U https://arxiv.org/abs/2007.11432
%D 2020
%X   Implicit functions represented as deep learning approximations are powerful<br>for reconstructing 3D surfaces. However, they can only produce static surfaces<br>that are not controllable, which provides limited ability to modify the<br>resulting model by editing its pose or shape parameters. Nevertheless, such<br>features are essential in building flexible models for both computer graphics<br>and computer vision. In this work, we present methodology that combines<br>detail-rich implicit functions and parametric representations in order to<br>reconstruct 3D models of people that remain controllable and accurate even in<br>the presence of clothing. Given sparse 3D point clouds sampled on the surface<br>of a dressed person, we use an Implicit Part Network (IP-Net)to jointly predict<br>the outer 3D surface of the dressed person, the and inner body surface, and the<br>semantic correspondences to a parametric body model. We subsequently use<br>correspondences to fit the body model to our inner surface and then non-rigidly<br>deform it (under a parametric body + displacement model) to the outer surface<br>in order to capture garment, face and hair detail. In quantitative and<br>qualitative experiments with both full body data and hand scans we show that<br>the proposed methodology generalizes, and is effective even given incomplete<br>point clouds collected from single-view depth images. Our models and code can<br>be downloaded from http://virtualhumans.mpi-inf.mpg.de/ipnet.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

A. Božič, M. Zollhöfer, C. Theobalt, and M. Nießner

“DeepDeform: Learning Non-Rigid RGB-D Reconstruction With Semi-Supervised Data,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{Bozic_CVPR2020,
TITLE = {{DeepDeform}: {L}earning Non-Rigid {RGB}-{D} Reconstruction With Semi-Supervised Data},
AUTHOR = {Bo{\v z}i{\v c}, Alja{\v z} and Zollh{\"o}fer, Michael and Theobalt, Christian and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00703},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {7000--7010},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Bo&#382;i&#269;, Alja&#382;
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T DeepDeform: Learning Non-Rigid RGB-D Reconstruction With Semi-Supervised Data : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CF63-1
%R 10.1109/CVPR42600.2020.00703
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 7000 - 7010
%I IEEE
%@ 978-1-7281-7168-5

Paper

V. Chizhov, I. Georgiev, K. Myszkowski, and G. Singh

“Perceptual Error Optimization for Monte Carlo Rendering,” 2020. [Online]. Available: https://arxiv.org/abs/2012.02344.

Abstract

Realistic image synthesis involves computing high-dimensional light transport
integrals which in practice are numerically estimated using Monte Carlo
integration. The error of this estimation manifests itself in the image as
visually displeasing aliasing or noise. To ameliorate this, we develop a
theoretical framework for optimizing screen-space error distribution. Our model
is flexible and works for arbitrary target error power spectra. We focus on
perceptual error optimization by leveraging models of the human visual system's
(HVS) point spread function (PSF) from halftoning literature. This results in a
specific optimization problem whose solution distributes the error as visually
pleasing blue noise in image space. We develop a set of algorithms that provide
a trade-off between quality and speed, showing substantial improvements over
prior state of the art. We perform evaluations using both quantitative and
perceptual error metrics to support our analysis, and provide extensive
supplemental material to help evaluate the perceptual improvements achieved by
our methods.

BibTeX

@online{Chizhov_arXiv2012.02344,
TITLE = {Perceptual Error Optimization for {Monte Carlo} Rendering},
AUTHOR = {Chizhov, Vassillen and Georgiev, Iliyan and Myszkowski, Karol and Singh, Gurprit},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2012.02344},
EPRINT = {2012.02344},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Realistic image synthesis involves computing high-dimensional light transport<br>integrals which in practice are numerically estimated using Monte Carlo<br>integration. The error of this estimation manifests itself in the image as<br>visually displeasing aliasing or noise. To ameliorate this, we develop a<br>theoretical framework for optimizing screen-space error distribution. Our model<br>is flexible and works for arbitrary target error power spectra. We focus on<br>perceptual error optimization by leveraging models of the human visual system's<br>(HVS) point spread function (PSF) from halftoning literature. This results in a<br>specific optimization problem whose solution distributes the error as visually<br>pleasing blue noise in image space. We develop a set of algorithms that provide<br>a trade-off between quality and speed, showing substantial improvements over<br>prior state of the art. We perform evaluations using both quantitative and<br>perceptual error metrics to support our analysis, and provide extensive<br>supplemental material to help evaluate the perceptual improvements achieved by<br>our methods.<br>},
}

Endnote

%0 Report
%A Chizhov, Vassillen
%A Georgiev, Iliyan
%A Myszkowski, Karol
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perceptual Error Optimization for Monte Carlo Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CEB7-3
%U https://arxiv.org/abs/2012.02344
%D 2020
%X   Realistic image synthesis involves computing high-dimensional light transport<br>integrals which in practice are numerically estimated using Monte Carlo<br>integration. The error of this estimation manifests itself in the image as<br>visually displeasing aliasing or noise. To ameliorate this, we develop a<br>theoretical framework for optimizing screen-space error distribution. Our model<br>is flexible and works for arbitrary target error power spectra. We focus on<br>perceptual error optimization by leveraging models of the human visual system's<br>(HVS) point spread function (PSF) from halftoning literature. This results in a<br>specific optimization problem whose solution distributes the error as visually<br>pleasing blue noise in image space. We develop a set of algorithms that provide<br>a trade-off between quality and speed, showing substantial improvements over<br>prior state of the art. We perform evaluations using both quantitative and<br>perceptual error metrics to support our analysis, and provide extensive<br>supplemental material to help evaluate the perceptual improvements achieved by<br>our methods.<br>
%K Computer Science, Graphics, cs.GR

Article

U. Çoğalan and A. O. Akyüz

“Deep Joint Deinterlacing and Denoising for Single Shot Dual-ISO HDR Reconstruction,” IEEE Transactions on Image Processing, vol. 29, 2020.

@article{Cogalan2020,
TITLE = {Deep Joint Deinterlacing and Denoising for Single Shot Dual-{ISO HDR} Reconstruction},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur and Aky{\"u}z, Ahmet O{\u g}uz},
LANGUAGE = {eng},
ISSN = {1057-7149},
DOI = {10.1109/TIP.2020.3004014},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2020},
DATE = {2020},
JOURNAL = {IEEE Transactions on Image Processing},
VOLUME = {29},
PAGES = {7511--7524},
}

Endnote

%0 Journal Article
%A &#199;o&#287;alan, U&#287;ur
%A Aky&#252;z, Ahmet O&#287;uz
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Deep Joint Deinterlacing and Denoising for Single Shot Dual-ISO HDR Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-DCA7-6
%R 10.1109/TIP.2020.3004014
%7 2020
%D 2020
%J IEEE Transactions on Image Processing
%V 29
%& 7511
%P 7511 - 7524
%I IEEE
%C Piscataway, NJ
%@ false

Paper

U. Çoğalan, M. Bemana, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models,” 2020. [Online]. Available: https://arxiv.org/abs/2012.12009.

Abstract

We seek to reconstruct sharp and noise-free high-dynamic range (HDR) video
from a dual-exposure sensor that records different low-dynamic range (LDR)
information in different pixel columns: Odd columns provide low-exposure,
sharp, but noisy information; even columns complement this with less noisy,
high-exposure, but motion-blurred data. Previous LDR work learns to deblur and
denoise (DISTORTED->CLEAN) supervised by pairs of CLEAN and DISTORTED images.
Regrettably, capturing DISTORTED sensor readings is time-consuming; as well,
there is a lack of CLEAN HDR videos. We suggest a method to overcome those two
limitations. First, we learn a different function instead: CLEAN->DISTORTED,
which generates samples containing correlated pixel noise, and row and column
noise, as well as motion blur from a low number of CLEAN sensor readings.
Second, as there is not enough CLEAN HDR video available, we devise a method to
learn from LDR video in-stead. Our approach compares favorably to several
strong baselines, and can boost existing methods when they are re-trained on
our data. Combined with spatial and temporal super-resolution, it enables
applications such as re-lighting with low noise or blur.

BibTeX

@online{Cogalan_arXiv2012.12009,
TITLE = {{HDR} Denoising and Deblurring by Learning Spatio-temporal Distortion Model},
AUTHOR = {{\c C}o{\u g}alan, U{\u g}ur and Bemana, Mojtaba and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2012.12009},
EPRINT = {2012.12009},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We seek to reconstruct sharp and noise-free high-dynamic range (HDR) video<br>from a dual-exposure sensor that records different low-dynamic range (LDR)<br>information in different pixel columns: Odd columns provide low-exposure,<br>sharp, but noisy information; even columns complement this with less noisy,<br>high-exposure, but motion-blurred data. Previous LDR work learns to deblur and<br>denoise (DISTORTED->CLEAN) supervised by pairs of CLEAN and DISTORTED images.<br>Regrettably, capturing DISTORTED sensor readings is time-consuming; as well,<br>there is a lack of CLEAN HDR videos. We suggest a method to overcome those two<br>limitations. First, we learn a different function instead: CLEAN->DISTORTED,<br>which generates samples containing correlated pixel noise, and row and column<br>noise, as well as motion blur from a low number of CLEAN sensor readings.<br>Second, as there is not enough CLEAN HDR video available, we devise a method to<br>learn from LDR video in-stead. Our approach compares favorably to several<br>strong baselines, and can boost existing methods when they are re-trained on<br>our data. Combined with spatial and temporal super-resolution, it enables<br>applications such as re-lighting with low noise or blur.<br>},
}

Endnote

%0 Report
%A &#199;o&#287;alan, U&#287;ur
%A Bemana, Mojtaba
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T HDR Denoising and Deblurring by Learning Spatio-temporal Distortion Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B721-5
%U https://arxiv.org/abs/2012.12009
%D 2020
%X   We seek to reconstruct sharp and noise-free high-dynamic range (HDR) video<br>from a dual-exposure sensor that records different low-dynamic range (LDR)<br>information in different pixel columns: Odd columns provide low-exposure,<br>sharp, but noisy information; even columns complement this with less noisy,<br>high-exposure, but motion-blurred data. Previous LDR work learns to deblur and<br>denoise (DISTORTED->CLEAN) supervised by pairs of CLEAN and DISTORTED images.<br>Regrettably, capturing DISTORTED sensor readings is time-consuming; as well,<br>there is a lack of CLEAN HDR videos. We suggest a method to overcome those two<br>limitations. First, we learn a different function instead: CLEAN->DISTORTED,<br>which generates samples containing correlated pixel noise, and row and column<br>noise, as well as motion blur from a low number of CLEAN sensor readings.<br>Second, as there is not enough CLEAN HDR video available, we devise a method to<br>learn from LDR video in-stead. Our approach compares favorably to several<br>strong baselines, and can boost existing methods when they are re-trained on<br>our data. Combined with spatial and temporal super-resolution, it enables<br>applications such as re-lighting with low noise or blur.<br>
%K eess.IV,Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

S. Cucerca, P. Didyk, H.-P. Seidel, and V. Babaei

“Computational Image Marking on Metals via Laser Induced Heating,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2020), vol. 39, no. 4, 2020.

@article{Cucerca_SIGGRAPH2020,
TITLE = {Computational Image Marking on Metals via Laser Induced Heating},
AUTHOR = {Cucerca, Sebastian and Didyk, Piotr and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3386569.3392423},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {39},
NUMBER = {4},
EID = {70},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2020},
}

Endnote

%0 Journal Article
%A Cucerca, Sebastian
%A Didyk, Piotr
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Computational Image Marking on Metals via Laser Induced Heating : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-9664-F
%R 10.1145/3386569.3392423
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 4
%Z sequence number: 70
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2020
%O ACM SIGGRAPH 2020 Virtual Conference ; 2020, 17-28  August

Conference paper

D. Dunn, O. Tursun, H. Yu, P. Didyk, K. Myszkowski, and H. Fuchs

“Stimulating the Human Visual System Beyond Real World Performance in Future Augmented Reality Displays,” in IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2020), Recife/Porto de Galinhas, Brazil (Virtual Conference), 2020.

@inproceedings{Dunn2020,
TITLE = {Stimulating the Human Visual System Beyond Real World Performance in Future Augmented Reality Displays},
AUTHOR = {Dunn, David and Tursun, Okan and Yu, Hyeonseung and Didyk, Piotr and Myszkowski, Karol and Fuchs, Henry},
LANGUAGE = {eng},
ISBN = {978-1-7281-8508-8},
DOI = {10.1109/ISMAR50242.2020.00029},
PUBLISHER = {IEEE},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2020)},
PAGES = {90--100},
ADDRESS = {Recife/Porto de Galinhas, Brazil (Virtual Conference)},
}

Endnote

%0 Conference Proceedings
%A Dunn, David
%A Tursun, Okan
%A Yu, Hyeonseung
%A Didyk, Piotr
%A Myszkowski, Karol
%A Fuchs, Henry
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Stimulating the Human Visual System Beyond Real World Performance in Future Augmented Reality Displays : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-FBDF-5
%R 10.1109/ISMAR50242.2020.00029
%D 2020
%B International Symposium on Mixed and Augmented Reality
%Z date of event: 2020-11-09 - 2020-11-13
%C Recife/Porto de Galinhas, Brazil (Virtual Conference)
%B IEEE International Symposium on Mixed and Augmented Reality 
%P 90 - 100
%I IEEE
%@ 978-1-7281-8508-8

Article

B. Egger, W. A. P. Smith, A. Tewari, S. Wuhrer, M. Zollhöfer, T. Beeler, F. Bernard, T. Bolkart, A. Kortylewski, S. Romdhani, C. Theobalt, V. Blanz, and T. Vetter

“3D Morphable Face Models -Past, Present and Future,” ACM Transactions on Graphics, vol. 39, no. 5, 2020.

@article{Egger_TOG2020,
TITLE = {{3D} Morphable Face Models -- Past, Present and Future},
AUTHOR = {Egger, Bernhard and Smith, William A. P. and Tewari, Ayush and Wuhrer, Stefanie and Zollh{\"o}fer, Michael and Beeler, Thabo and Bernard, Florian and Bolkart, Timo and Kortylewski, Adam and Romdhani, Sami and Theobalt, Christian and Blanz, Volker and Vetter, Thomas},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3395208},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {39},
NUMBER = {5},
EID = {157},
}

Endnote

%0 Journal Article
%A Egger, Bernhard
%A Smith, William A. P.
%A Tewari, Ayush
%A Wuhrer, Stefanie
%A Zollh&#246;fer, Michael
%A Beeler, Thabo
%A Bernard, Florian
%A Bolkart, Timo
%A Kortylewski, Adam
%A Romdhani, Sami
%A Theobalt, Christian
%A Blanz, Volker
%A Vetter, Thomas
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T 3D Morphable Face Models -Past, Present and Future : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-1CF5-6
%R 10.1145/3395208
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 5
%Z sequence number: 157
%I ACM
%C New York, NY
%@ false

Article

M. Elgharib, M. Mendiratta, J. Thies, M. Nießner, H.-P. Seidel, A. Tewari, V. Golyanik, and C. Theobalt

“Egocentric Videoconferencing,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Elgharib_ToG2020,
TITLE = {Egocentric Videoconferencing},
AUTHOR = {Elgharib, Mohamed and Mendiratta, Mohit and Thies, Justus and Nie{\ss}ner, Matthias and Seidel, Hans-Peter and Tewari, Ayush and Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417808},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {268},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Elgharib, Mohamed
%A Mendiratta, Mohit
%A Thies, Justus
%A Nie&#223;ner, Matthias
%A Seidel, Hans-Peter
%A Tewari, Ayush
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Egocentric Videoconferencing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-9B36-E
%R 10.1145/3414685.3417808
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 268
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Paper

G. Fox, W. Liu, H. Kim, H.-P. Seidel, M. Elgharib, and C. Theobalt

“VideoForensicsHQ: Detecting High-quality Manipulated Face Videos,” 2020. [Online]. Available: https://arxiv.org/abs/2005.10360.

Abstract

New approaches to synthesize and manipulate face videos at very high quality
have paved the way for new applications in computer animation, virtual and
augmented reality, or face video analysis. However, there are concerns that
they may be used in a malicious way, e.g. to manipulate videos of public
figures, politicians or reporters, to spread false information. The research
community therefore developed techniques for automated detection of modified
imagery, and assembled benchmark datasets showing manipulatons by
state-of-the-art techniques. In this paper, we contribute to this initiative in
two ways: First, we present a new audio-visual benchmark dataset. It shows some
of the highest quality visual manipulations available today. Human observers
find them significantly harder to identify as forged than videos from other
benchmarks. Furthermore we propose new family of deep-learning-based fake
detectors, demonstrating that existing detectors are not well-suited for
detecting fakes of a quality as high as presented in our dataset. Our detectors
examine spatial and temporal features. This allows them to outperform existing
approaches both in terms of high detection accuracy and generalization to
unseen fake generation methods and unseen identities.

BibTeX

@online{Fox_2005.10360,
TITLE = {{Video\-Foren\-sics\-HQ}: {D}etecting High-quality Manipulated Face Videos},
AUTHOR = {Fox, Gereon and Liu, Wentao and Kim, Hyeongwoo and Seidel, Hans-Peter and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2005.10360},
EPRINT = {2005.10360},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {New approaches to synthesize and manipulate face videos at very high quality<br>have paved the way for new applications in computer animation, virtual and<br>augmented reality, or face video analysis. However, there are concerns that<br>they may be used in a malicious way, e.g. to manipulate videos of public<br>figures, politicians or reporters, to spread false information. The research<br>community therefore developed techniques for automated detection of modified<br>imagery, and assembled benchmark datasets showing manipulatons by<br>state-of-the-art techniques. In this paper, we contribute to this initiative in<br>two ways: First, we present a new audio-visual benchmark dataset. It shows some<br>of the highest quality visual manipulations available today. Human observers<br>find them significantly harder to identify as forged than videos from other<br>benchmarks. Furthermore we propose new family of deep-learning-based fake<br>detectors, demonstrating that existing detectors are not well-suited for<br>detecting fakes of a quality as high as presented in our dataset. Our detectors<br>examine spatial and temporal features. This allows them to outperform existing<br>approaches both in terms of high detection accuracy and generalization to<br>unseen fake generation methods and unseen identities.<br>},
}

Endnote

%0 Report
%A Fox, Gereon
%A Liu, Wentao
%A Kim, Hyeongwoo
%A Seidel, Hans-Peter
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T VideoForensicsHQ: Detecting High-quality Manipulated Face Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B109-7
%U https://arxiv.org/abs/2005.10360
%D 2020
%X   New approaches to synthesize and manipulate face videos at very high quality<br>have paved the way for new applications in computer animation, virtual and<br>augmented reality, or face video analysis. However, there are concerns that<br>they may be used in a malicious way, e.g. to manipulate videos of public<br>figures, politicians or reporters, to spread false information. The research<br>community therefore developed techniques for automated detection of modified<br>imagery, and assembled benchmark datasets showing manipulatons by<br>state-of-the-art techniques. In this paper, we contribute to this initiative in<br>two ways: First, we present a new audio-visual benchmark dataset. It shows some<br>of the highest quality visual manipulations available today. Human observers<br>find them significantly harder to identify as forged than videos from other<br>benchmarks. Furthermore we propose new family of deep-learning-based fake<br>detectors, demonstrating that existing detectors are not well-suited for<br>detecting fakes of a quality as high as presented in our dataset. Our detectors<br>examine spatial and temporal features. This allows them to outperform existing<br>approaches both in terms of high detection accuracy and generalization to<br>unseen fake generation methods and unseen identities.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

V. Golyanik, S. Shimada, and C. Theobalt

“Fast Simultaneous Gravitational Alignment of Multiple Point Sets,” in International Conference on 3D Vision, Fukuoka, Japan (Virtual Event), 2020.

@inproceedings{Golyanik_MBGA2020,
TITLE = {Fast Simultaneous Gravitational Alignment of Multiple Point Sets},
AUTHOR = {Golyanik, Vladislav and Shimada, Soshi and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-8128-8},
DOI = {10.1109/3DV50981.2020.00019},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {International Conference on 3D Vision},
PAGES = {91--100},
ADDRESS = {Fukuoka, Japan (Virtual Event)},
}

Endnote

%0 Conference Proceedings
%A Golyanik, Vladislav
%A Shimada, Soshi
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Fast Simultaneous Gravitational Alignment of Multiple Point Sets : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-0358-2
%R 10.1109/3DV50981.2020.00019
%D 2020
%B International Conference on 3D Vision
%Z date of event: 2020-11-25 - 2020-11-28
%C Fukuoka, Japan (Virtual Event)
%B International Conference on 3D Vision
%P 91 - 100
%I IEEE
%@ 978-1-7281-8128-8

Conference paper

V. Golyanik and C. Theobalt

“A Quantum Computational Approach to Correspondence Problems on Point Sets,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{Golyanik_CVPR2020,
TITLE = {A Quantum Computational Approach to Correspondence Problems on Point Sets},
AUTHOR = {Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00920},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {9179--9188},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T A Quantum Computational Approach to Correspondence Problems on Point Sets : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D053-0
%R 10.1109/CVPR42600.2020.00920
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 9179 - 9188
%I IEEE
%@ 978-1-7281-7168-5

Conference paper

V. Golyanik, A. Jonas, D. Stricker, and C. Theobalt

“Intrinsic Dynamic Shape Prior for Dense Non-Rigid Structure from Motion,” in International Conference on 3D Vision, Fukuoka, Japan (Virtual Event), 2020.

@inproceedings{Golyanik2020DSPR,
TITLE = {Intrinsic Dynamic Shape Prior for Dense Non-Rigid Structure from Motion},
AUTHOR = {Golyanik, Vladislav and Jonas, Andr{\'e} and Stricker, Didier and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-8128-8},
DOI = {10.1109/3DV50981.2020.00079},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {International Conference on 3D Vision},
PAGES = {692--701},
ADDRESS = {Fukuoka, Japan (Virtual Event)},
}

Endnote

%0 Conference Proceedings
%A Golyanik, Vladislav
%A Jonas, Andr&#233;
%A Stricker, Didier
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Intrinsic Dynamic Shape Prior for Dense Non-Rigid 
Structure from Motion : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-0364-4
%R 10.1109/3DV50981.2020.00079
%D 2020
%B International Conference on 3D Vision
%Z date of event: 2020-11-25 - 2020-11-28
%C Fukuoka, Japan (Virtual Event)
%B International Conference on 3D Vision
%P 692 - 701
%I IEEE
%@ 978-1-7281-8128-8

Article

F. Günther, C. Jiang, and H. Pottmann

“Smooth Polyhedral Surfaces,” Advances in Mathematics, vol. 363, 2020.

Abstract

Polyhedral surfaces are fundamental objects in architectural geometry and industrial design. Whereas closeness of a given mesh to a smooth reference surface and its suitability for numerical simulations were already studied extensively, the aim of our work is to find and to discuss suitable assessments of smoothness of polyhedral surfaces that only take the geometry of the polyhedral surface itself into account. Motivated by analogies to classical differential geometry, we propose a theory of smoothness of polyhedral surfaces including suitable notions of normal vectors, tangent planes, asymptotic directions, and parabolic curves that are invariant under projective transformations. It is remarkable that seemingly mild conditions significantly limit the shapes of faces of a smooth polyhedral surface. Besides being of theoretical interest, we believe that smoothness of polyhedral surfaces is of interest in the architectural context, where vertices and edges of polyhedral surfaces are highly visible.

BibTeX

@article{Guenther2020,
TITLE = {Smooth Polyhedral Surfaces},
AUTHOR = {G{\"u}nther, Felix and Jiang, Caigui and Pottmann, Helmut},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1703.05318},
DOI = {10.1016/j.aim.2020.107004},
EPRINT = {1703.05318},
EPRINTTYPE = {arXiv},
PUBLISHER = {Elsevier},
YEAR = {2020},
DATE = {2020},
ABSTRACT = {Polyhedral surfaces are fundamental objects in architectural geometry and industrial design. Whereas closeness of a given mesh to a smooth reference surface and its suitability for numerical simulations were already studied extensively, the aim of our work is to find and to discuss suitable assessments of smoothness of polyhedral surfaces that only take the geometry of the polyhedral surface itself into account. Motivated by analogies to classical differential geometry, we propose a theory of smoothness of polyhedral surfaces including suitable notions of normal vectors, tangent planes, asymptotic directions, and parabolic curves that are invariant under projective transformations. It is remarkable that seemingly mild conditions significantly limit the shapes of faces of a smooth polyhedral surface. Besides being of theoretical interest, we believe that smoothness of polyhedral surfaces is of interest in the architectural context, where vertices and edges of polyhedral surfaces are highly visible.<br><br><br>},
JOURNAL = {Advances in Mathematics},
VOLUME = {363},
EID = {107004},
}

Endnote

%0 Journal Article
%A G&#252;nther, Felix
%A Jiang, Caigui
%A Pottmann, Helmut
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Smooth Polyhedral Surfaces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-9760-3
%U http://arxiv.org/abs/1703.05318
%R 10.1016/j.aim.2020.107004
%D 2020
%* Review method: peer-reviewed
%X Polyhedral surfaces are fundamental objects in architectural geometry and industrial design. Whereas closeness of a given mesh to a smooth reference surface and its suitability for numerical simulations were already studied extensively, the aim of our work is to find and to discuss suitable assessments of smoothness of polyhedral surfaces that only take the geometry of the polyhedral surface itself into account. Motivated by analogies to classical differential geometry, we propose a theory of smoothness of polyhedral surfaces including suitable notions of normal vectors, tangent planes, asymptotic directions, and parabolic curves that are invariant under projective transformations. It is remarkable that seemingly mild conditions significantly limit the shapes of faces of a smooth polyhedral surface. Besides being of theoretical interest, we believe that smoothness of polyhedral surfaces is of interest in the architectural context, where vertices and edges of polyhedral surfaces are highly visible.<br><br><br>
%K Mathematics, Metric Geometry, Mathematics, Differential Geometry
%J Advances in Mathematics 
%O Adv. Math.
%V 363
%Z sequence number: 107004
%I Elsevier

Paper

M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll, and C. Theobalt

“DeepCap: Monocular Human Performance Capture Using Weak Supervision,” 2020. [Online]. Available: https://arxiv.org/abs/2003.08325.

Abstract

BibTeX

@online{Habermann2003.08325,
TITLE = {{DeepCap}: {M}onocular Human Performance Capture Using Weak Supervision},
AUTHOR = {Habermann, Marc and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2003.08325},
EPRINT = {2003.08325},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Human performance capture is a highly important computer vision problem with<br>many applications in movie production and virtual/augmented reality. Many<br>previous performance capture approaches either required expensive multi-view<br>setups or did not recover dense space-time coherent geometry with<br>frame-to-frame correspondences. We propose a novel deep learning approach for<br>monocular dense human performance capture. Our method is trained in a weakly<br>supervised manner based on multi-view supervision completely removing the need<br>for training data with 3D ground truth annotations. The network architecture is<br>based on two separate networks that disentangle the task into a pose estimation<br>and a non-rigid surface deformation step. Extensive qualitative and<br>quantitative evaluations show that our approach outperforms the state of the<br>art in terms of quality and robustness.<br>},
}

Endnote

%0 Report
%A Habermann, Marc
%A Xu, Weipeng
%A Zollh&#246;fer, Michael
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T DeepCap: Monocular Human Performance Capture Using Weak Supervision : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E010-9
%U https://arxiv.org/abs/2003.08325
%D 2020
%X   Human performance capture is a highly important computer vision problem with<br>many applications in movie production and virtual/augmented reality. Many<br>previous performance capture approaches either required expensive multi-view<br>setups or did not recover dense space-time coherent geometry with<br>frame-to-frame correspondences. We propose a novel deep learning approach for<br>monocular dense human performance capture. Our method is trained in a weakly<br>supervised manner based on multi-view supervision completely removing the need<br>for training data with 3D ground truth annotations. The network architecture is<br>based on two separate networks that disentangle the task into a pose estimation<br>and a non-rigid surface deformation step. Extensive qualitative and<br>quantitative evaluations show that our approach outperforms the state of the<br>art in terms of quality and robustness.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

D4D2

M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll, and C. Theobalt

“DeepCap: Monocular Human Performance Capture Using Weak Supervision,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{deepcap2020,
TITLE = {{DeepCap}: {M}onocular Human Performance Capture Using Weak Supervision},
AUTHOR = {Habermann, Marc and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00510},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {5051--5062},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Habermann, Marc
%A Xu, Weipeng
%A Zollh&#246;fer, Michael
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T DeepCap: Monocular Human Performance Capture Using Weak Supervision : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-A895-4
%R 10.1109/CVPR42600.2020.00510
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 5051 - 5062
%I IEEE
%@ 978-1-7281-7168-5

Conference paper

L. Huang, C. Gao, Y. Zhou, C. Xie, A. Yuille, C. Zou, and N. Liu

“Universal Physical Camouﬂage Attacks on Object Detectors,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{Huang_CVPR2020,
TITLE = {Universal Physical Camou{fl}age Attacks on Object Detectors},
AUTHOR = {Huang, Lifeng and Gao, Chengying and Zhou, Yuyin and Xie, Cihang and Yuille, Alan and Zou, Changqing and Liu, Ning},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00080},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {717--726},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Huang, Lifeng
%A Gao, Chengying
%A Zhou, Yuyin
%A Xie, Cihang
%A Yuille, Alan
%A Zou, Changqing
%A Liu, Ning
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Universal Physical Camou&#64258;age Attacks on Object Detectors : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-09F0-1
%R 10.1109/CVPR42600.2020.00080
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 717 - 726
%I IEEE
%@ 978-1-7281-7168-5

Paper

M. Kappel, V. Golyanik, M. Elgharib, J.-O. Henningson, H.-P. Seidel, S. Castillo, C. Theobalt, and M. A. Magnor

“High-Fidelity Neural Human Motion Transfer from Monocular Video,” 2020. [Online]. Available: https://arxiv.org/abs/2012.10974.

Abstract

Video-based human motion transfer creates video animations of humans
following a source motion. Current methods show remarkable results for
tightly-clad subjects. However, the lack of temporally consistent handling of
plausible clothing dynamics, including fine and high-frequency details,
significantly limits the attainable visual quality. We address these
limitations for the first time in the literature and present a new framework
which performs high-fidelity and temporally-consistent human motion transfer
with natural pose-dependent non-rigid deformations, for several types of loose
garments. In contrast to the previous techniques, we perform image generation
in three subsequent stages, synthesizing human shape, structure, and
appearance. Given a monocular RGB video of an actor, we train a stack of
recurrent deep neural networks that generate these intermediate representations
from 2D poses and their temporal derivatives. Splitting the difficult motion
transfer problem into subtasks that are aware of the temporal motion context
helps us to synthesize results with plausible dynamics and pose-dependent
detail. It also allows artistic control of results by manipulation of
individual framework stages. In the experimental results, we significantly
outperform the state-of-the-art in terms of video realism. Our code and data
will be made publicly available.

BibTeX

@online{Kappel_arXiv2012.10974,
TITLE = {High-Fidelity Neural Human Motion Transfer from Monocular Video},
AUTHOR = {Kappel, Moritz and Golyanik, Vladislav and Elgharib, Mohamed and Henningson, Jann-Ole and Seidel, Hans-Peter and Castillo, Susana and Theobalt, Christian and Magnor, Marcus A.},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2012.10974},
EPRINT = {2012.10974},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Video-based human motion transfer creates video animations of humans<br>following a source motion. Current methods show remarkable results for<br>tightly-clad subjects. However, the lack of temporally consistent handling of<br>plausible clothing dynamics, including fine and high-frequency details,<br>significantly limits the attainable visual quality. We address these<br>limitations for the first time in the literature and present a new framework<br>which performs high-fidelity and temporally-consistent human motion transfer<br>with natural pose-dependent non-rigid deformations, for several types of loose<br>garments. In contrast to the previous techniques, we perform image generation<br>in three subsequent stages, synthesizing human shape, structure, and<br>appearance. Given a monocular RGB video of an actor, we train a stack of<br>recurrent deep neural networks that generate these intermediate representations<br>from 2D poses and their temporal derivatives. Splitting the difficult motion<br>transfer problem into subtasks that are aware of the temporal motion context<br>helps us to synthesize results with plausible dynamics and pose-dependent<br>detail. It also allows artistic control of results by manipulation of<br>individual framework stages. In the experimental results, we significantly<br>outperform the state-of-the-art in terms of video realism. Our code and data<br>will be made publicly available.<br>},
}

Endnote

%0 Report
%A Kappel, Moritz
%A Golyanik, Vladislav
%A Elgharib, Mohamed
%A Henningson, Jann-Ole
%A Seidel, Hans-Peter
%A Castillo, Susana
%A Theobalt, Christian
%A Magnor, Marcus A.
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T High-Fidelity Neural Human Motion Transfer from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B715-3
%U https://arxiv.org/abs/2012.10974
%D 2020
%X   Video-based human motion transfer creates video animations of humans<br>following a source motion. Current methods show remarkable results for<br>tightly-clad subjects. However, the lack of temporally consistent handling of<br>plausible clothing dynamics, including fine and high-frequency details,<br>significantly limits the attainable visual quality. We address these<br>limitations for the first time in the literature and present a new framework<br>which performs high-fidelity and temporally-consistent human motion transfer<br>with natural pose-dependent non-rigid deformations, for several types of loose<br>garments. In contrast to the previous techniques, we perform image generation<br>in three subsequent stages, synthesizing human shape, structure, and<br>appearance. Given a monocular RGB video of an actor, we train a stack of<br>recurrent deep neural networks that generate these intermediate representations<br>from 2D poses and their temporal derivatives. Splitting the difficult motion<br>transfer problem into subtasks that are aware of the temporal motion context<br>helps us to synthesize results with plausible dynamics and pose-dependent<br>detail. It also allows artistic control of results by manipulation of<br>individual framework stages. In the experimental results, we significantly<br>outperform the state-of-the-art in terms of video realism. Our code and data<br>will be made publicly available.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Conference paper

L. Liu, J. Gu, K. Z. Lin, T.-S. Chua, and C. Theobalt

“Neural Sparse Voxel Fields,” in Advances in Neural Information Processing Systems 33 (NeurIPS 2020), Virtual Event, 2020.

@inproceedings{LiuNeural20,
TITLE = {Neural Sparse Voxel Fields},
AUTHOR = {Liu, Lingjie and Gu, Jiatao and Lin, Kyaw Zaw and Chua, Tat-Seng and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {9781713829546},
PUBLISHER = {Curran Associates, Inc.},
YEAR = {2020},
BOOKTITLE = {Advances in Neural Information Processing Systems 33 (NeurIPS 2020)},
EDITOR = {Larochelle, H. and Ranzato, M. and Hadsell, R. and Balcan, M. F. and Lin, H.},
PAGES = {15651--15663},
ADDRESS = {Virtual Event},
}

Endnote

%0 Conference Proceedings
%A Liu, Lingjie
%A Gu, Jiatao
%A Lin, Kyaw Zaw
%A Chua, Tat-Seng
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Sparse Voxel Fields : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D437-C
%D 2020
%B 34th Conference on Neural Information Processing Systems
%Z date of event: 2020-12-06 - 2020-12-12
%C Virtual Event
%B Advances in Neural Information Processing Systems 33
%E Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.; Lin, H.
%P 15651 - 15663
%I Curran Associates, Inc.
%@ 9781713829546
%U https://proceedings.neurips.cc/paper/2020/file/b4b758962f17808746e9bb832a6fa4b8-Paper.pdf

Article

D4D6

L. Liu, W. Xu, M. Habermann, M. Zollhöfer, F. Bernard, H. Kim, W. Wang, and C. Theobalt

“Learning Dynamic Textures for Neural Rendering of Human Actors,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 10, 2020.

@article{Liu2020,
TITLE = {Learning Dynamic Textures for Neural Rendering of Human Actors},
AUTHOR = {Liu, Lingjie and Xu, Weipeng and Habermann, Marc and Zollh{\"o}fer, Michael and Bernard, Florian and Kim, Hyeongwoo and Wang, Wenping and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2020.2996594},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2020},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics},
VOLUME = {27},
NUMBER = {10},
PAGES = {4009--4022},
}

Endnote

%0 Journal Article
%A Liu, Lingjie
%A Xu, Weipeng
%A Habermann, Marc
%A Zollh&#246;fer, Michael
%A Bernard, Florian
%A Kim, Hyeongwoo
%A Wang, Wenping
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
%T Learning Dynamic Textures for Neural Rendering of Human Actors : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0009-4C96-9
%R 10.1109/TVCG.2020.2996594
%7 2020
%D 2020
%J IEEE Transactions on Visualization and Computer Graphics
%V 27
%N 10
%& 4009
%P 4009 - 4022
%I IEEE
%C Piscataway, NJ
%@ false

Paper

L. Liu, J. Gu, K. Z. Lin, T.-S. Chua, and C. Theobalt

“Neural Sparse Voxel Fields,” 2020. [Online]. Available: https://arxiv.org/abs/2007.11571.

Abstract

Photo-realistic free-viewpoint rendering of real-world scenes using classical
computer graphics techniques is challenging, because it requires the difficult
step of capturing detailed appearance and geometry models. Recent studies have
demonstrated promising results by learning scene representations that
implicitly encode both geometry and appearance without 3D supervision. However,
existing approaches in practice often show blurry renderings caused by the
limited network capacity or the difficulty in finding accurate intersections of
camera rays with the scene geometry. Synthesizing high-resolution imagery from
these representations often requires time-consuming optical ray marching. In
this work, we introduce Neural Sparse Voxel Fields (NSVF), a new neural scene
representation for fast and high-quality free-viewpoint rendering. NSVF defines
a set of voxel-bounded implicit fields organized in a sparse voxel octree to
model local properties in each cell. We progressively learn the underlying
voxel structures with a differentiable ray-marching operation from only a set
of posed RGB images. With the sparse voxel octree structure, rendering novel
views can be accelerated by skipping the voxels containing no relevant scene
content. Our method is typically over 10 times faster than the state-of-the-art
(namely, NeRF(Mildenhall et al., 2020)) at inference time while achieving
higher quality results. Furthermore, by utilizing an explicit sparse voxel
representation, our method can easily be applied to scene editing and scene
composition. We also demonstrate several challenging tasks, including
multi-scene learning, free-viewpoint rendering of a moving human, and
large-scale scene rendering. Code and data are available at our website:
github.com/facebookresearch/NSVF.

BibTeX

@online{Liu_2007.11571,
TITLE = {Neural Sparse Voxel Fields},
AUTHOR = {Liu, Lingjie and Gu, Jiatao and Lin, Kyaw Zaw and Chua, Tat-Seng and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2007.11571},
EPRINT = {2007.11571},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Photo-realistic free-viewpoint rendering of real-world scenes using classical<br>computer graphics techniques is challenging, because it requires the difficult<br>step of capturing detailed appearance and geometry models. Recent studies have<br>demonstrated promising results by learning scene representations that<br>implicitly encode both geometry and appearance without 3D supervision. However,<br>existing approaches in practice often show blurry renderings caused by the<br>limited network capacity or the difficulty in finding accurate intersections of<br>camera rays with the scene geometry. Synthesizing high-resolution imagery from<br>these representations often requires time-consuming optical ray marching. In<br>this work, we introduce Neural Sparse Voxel Fields (NSVF), a new neural scene<br>representation for fast and high-quality free-viewpoint rendering. NSVF defines<br>a set of voxel-bounded implicit fields organized in a sparse voxel octree to<br>model local properties in each cell. We progressively learn the underlying<br>voxel structures with a differentiable ray-marching operation from only a set<br>of posed RGB images. With the sparse voxel octree structure, rendering novel<br>views can be accelerated by skipping the voxels containing no relevant scene<br>content. Our method is typically over 10 times faster than the state-of-the-art<br>(namely, NeRF(Mildenhall et al., 2020)) at inference time while achieving<br>higher quality results. Furthermore, by utilizing an explicit sparse voxel<br>representation, our method can easily be applied to scene editing and scene<br>composition. We also demonstrate several challenging tasks, including<br>multi-scene learning, free-viewpoint rendering of a moving human, and<br>large-scale scene rendering. Code and data are available at our website:<br>https://github.com/facebookresearch/NSVF.<br>},
}

Endnote

%0 Report
%A Liu, Lingjie
%A Gu, Jiatao
%A Lin, Kyaw Zaw
%A Chua, Tat-Seng
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Sparse Voxel Fields : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E8B2-A
%U https://arxiv.org/abs/2007.11571
%D 2020
%X   Photo-realistic free-viewpoint rendering of real-world scenes using classical<br>computer graphics techniques is challenging, because it requires the difficult<br>step of capturing detailed appearance and geometry models. Recent studies have<br>demonstrated promising results by learning scene representations that<br>implicitly encode both geometry and appearance without 3D supervision. However,<br>existing approaches in practice often show blurry renderings caused by the<br>limited network capacity or the difficulty in finding accurate intersections of<br>camera rays with the scene geometry. Synthesizing high-resolution imagery from<br>these representations often requires time-consuming optical ray marching. In<br>this work, we introduce Neural Sparse Voxel Fields (NSVF), a new neural scene<br>representation for fast and high-quality free-viewpoint rendering. NSVF defines<br>a set of voxel-bounded implicit fields organized in a sparse voxel octree to<br>model local properties in each cell. We progressively learn the underlying<br>voxel structures with a differentiable ray-marching operation from only a set<br>of posed RGB images. With the sparse voxel octree structure, rendering novel<br>views can be accelerated by skipping the voxels containing no relevant scene<br>content. Our method is typically over 10 times faster than the state-of-the-art<br>(namely, NeRF(Mildenhall et al., 2020)) at inference time while achieving<br>higher quality results. Furthermore, by utilizing an explicit sparse voxel<br>representation, our method can easily be applied to scene editing and scene<br>composition. We also demonstrate several challenging tasks, including<br>multi-scene learning, free-viewpoint rendering of a moving human, and<br>large-scale scene rendering. Code and data are available at our website:<br>https://github.com/facebookresearch/NSVF.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Paper

X. Long, L. Liu, W. Li, C. Theobalt, and W. Wang

“Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks,” 2020. [Online]. Available: https://arxiv.org/abs/2011.13118.

Abstract

We present a novel method for multi-view depth estimation from a single
video, which is a critical task in various applications, such as perception,
reconstruction and robot navigation. Although previous learning-based methods
have demonstrated compelling results, most works estimate depth maps of
individual video frames independently, without taking into consideration the
strong geometric and temporal coherence among the frames. Moreover, current
state-of-the-art (SOTA) models mostly adopt a fully 3D convolution network for
cost regularization and therefore require high computational cost, thus
limiting their deployment in real-world applications. Our method achieves
temporally coherent depth estimation results by using a novel Epipolar
Spatio-Temporal (EST) transformer to explicitly associate geometric and
temporal correlation with multiple estimated depth maps. Furthermore, to reduce
the computational cost, inspired by recent Mixture-of-Experts models, we design
a compact hybrid network consisting of a 2D context-aware network and a 3D
matching network which learn 2D context information and 3D disparity cues
separately. Extensive experiments demonstrate that our method achieves higher
accuracy in depth estimation and significant speedup than the SOTA methods.

BibTeX

@online{Long_2011.13118,
TITLE = {Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks},
AUTHOR = {Long, Xiaoxiao and Liu, Lingjie and Li, Wei and Theobalt, Christian and Wang, Wenping},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2011.13118},
EPRINT = {2011.13118},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present a novel method for multi-view depth estimation from a single<br>video, which is a critical task in various applications, such as perception,<br>reconstruction and robot navigation. Although previous learning-based methods<br>have demonstrated compelling results, most works estimate depth maps of<br>individual video frames independently, without taking into consideration the<br>strong geometric and temporal coherence among the frames. Moreover, current<br>state-of-the-art (SOTA) models mostly adopt a fully 3D convolution network for<br>cost regularization and therefore require high computational cost, thus<br>limiting their deployment in real-world applications. Our method achieves<br>temporally coherent depth estimation results by using a novel Epipolar<br>Spatio-Temporal (EST) transformer to explicitly associate geometric and<br>temporal correlation with multiple estimated depth maps. Furthermore, to reduce<br>the computational cost, inspired by recent Mixture-of-Experts models, we design<br>a compact hybrid network consisting of a 2D context-aware network and a 3D<br>matching network which learn 2D context information and 3D disparity cues<br>separately. Extensive experiments demonstrate that our method achieves higher<br>accuracy in depth estimation and significant speedup than the SOTA methods.<br>},
}

Endnote

%0 Report
%A Long, Xiaoxiao
%A Liu, Lingjie
%A Li, Wei
%A Theobalt, Christian
%A Wang, Wenping
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E9EA-B
%U https://arxiv.org/abs/2011.13118
%D 2020
%X   We present a novel method for multi-view depth estimation from a single<br>video, which is a critical task in various applications, such as perception,<br>reconstruction and robot navigation. Although previous learning-based methods<br>have demonstrated compelling results, most works estimate depth maps of<br>individual video frames independently, without taking into consideration the<br>strong geometric and temporal coherence among the frames. Moreover, current<br>state-of-the-art (SOTA) models mostly adopt a fully 3D convolution network for<br>cost regularization and therefore require high computational cost, thus<br>limiting their deployment in real-world applications. Our method achieves<br>temporally coherent depth estimation results by using a novel Epipolar<br>Spatio-Temporal (EST) transformer to explicitly associate geometric and<br>temporal correlation with multiple estimated depth maps. Furthermore, to reduce<br>the computational cost, inspired by recent Mixture-of-Experts models, we design<br>a compact hybrid network consisting of a 2D context-aware network and a 3D<br>matching network which learn 2D context information and 3D disparity cues<br>separately. Extensive experiments demonstrate that our method achieves higher<br>accuracy in depth estimation and significant speedup than the SOTA methods.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Paper

X. Long, L. Liu, C. Theobalt, and W. Wang

“Occlusion-Aware Depth Estimation with Adaptive Normal Constraints,” ECCV 2020. Lecture Notes in Computer Science, vol 12354. Springer, Cham, 2020. [Online]. Available: https://arxiv.org/abs/2004.00845.

Abstract

We present a new learning-based method for multi-frame depth estimation from
a color video, which is a fundamental problem in scene understanding, robot
navigation or handheld 3D reconstruction. While recent learning-based methods
estimate depth at high accuracy, 3D point clouds exported from their depth maps
often fail to preserve important geometric feature (e.g., corners, edges,
planes) of man-made scenes. Widely-used pixel-wise depth errors do not
specifically penalize inconsistency on these features. These inaccuracies are
particularly severe when subsequent depth reconstructions are accumulated in an
attempt to scan a full environment with man-made objects with this kind of
features. Our depth estimation algorithm therefore introduces a Combined Normal
Map (CNM) constraint, which is designed to better preserve high-curvature
features and global planar regions. In order to further improve the depth
estimation accuracy, we introduce a new occlusion-aware strategy that
aggregates initial depth predictions from multiple adjacent views into one
final depth map and one occlusion probability map for the current reference
view. Our method outperforms the state-of-the-art in terms of depth estimation
accuracy, and preserves essential geometric features of man-made indoor scenes
much better than other algorithms.

BibTeX

@online{Long2004.00845,
TITLE = {Occlusion-Aware Depth Estimation with Adaptive Normal Constraints},
AUTHOR = {Long, Xiaoxiao and Liu, Lingjie and Theobalt, Christian and Wang, Wenping},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2004.00845},
EPRINT = {2004.00845},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present a new learning-based method for multi-frame depth estimation from<br>a color video, which is a fundamental problem in scene understanding, robot<br>navigation or handheld 3D reconstruction. While recent learning-based methods<br>estimate depth at high accuracy, 3D point clouds exported from their depth maps<br>often fail to preserve important geometric feature (e.g., corners, edges,<br>planes) of man-made scenes. Widely-used pixel-wise depth errors do not<br>specifically penalize inconsistency on these features. These inaccuracies are<br>particularly severe when subsequent depth reconstructions are accumulated in an<br>attempt to scan a full environment with man-made objects with this kind of<br>features. Our depth estimation algorithm therefore introduces a Combined Normal<br>Map (CNM) constraint, which is designed to better preserve high-curvature<br>features and global planar regions. In order to further improve the depth<br>estimation accuracy, we introduce a new occlusion-aware strategy that<br>aggregates initial depth predictions from multiple adjacent views into one<br>final depth map and one occlusion probability map for the current reference<br>view. Our method outperforms the state-of-the-art in terms of depth estimation<br>accuracy, and preserves essential geometric features of man-made indoor scenes<br>much better than other algorithms.<br>},
JOURNAL = {ECCV 2020. Lecture Notes in Computer Science, vol 12354. Springer, Cham},
}

Endnote

%0 Report
%A Long, Xiaoxiao
%A Liu, Lingjie
%A Theobalt, Christian
%A Wang, Wenping
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Occlusion-Aware Depth Estimation with Adaptive Normal Constraints : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E0E9-5
%U https://arxiv.org/abs/2004.00845
%D 2020
%X   We present a new learning-based method for multi-frame depth estimation from<br>a color video, which is a fundamental problem in scene understanding, robot<br>navigation or handheld 3D reconstruction. While recent learning-based methods<br>estimate depth at high accuracy, 3D point clouds exported from their depth maps<br>often fail to preserve important geometric feature (e.g., corners, edges,<br>planes) of man-made scenes. Widely-used pixel-wise depth errors do not<br>specifically penalize inconsistency on these features. These inaccuracies are<br>particularly severe when subsequent depth reconstructions are accumulated in an<br>attempt to scan a full environment with man-made objects with this kind of<br>features. Our depth estimation algorithm therefore introduces a Combined Normal<br>Map (CNM) constraint, which is designed to better preserve high-curvature<br>features and global planar regions. In order to further improve the depth<br>estimation accuracy, we introduce a new occlusion-aware strategy that<br>aggregates initial depth predictions from multiple adjacent views into one<br>final depth map and one occlusion probability map for the current reference<br>view. Our method outperforms the state-of-the-art in terms of depth estimation<br>accuracy, and preserves essential geometric features of man-made indoor scenes<br>much better than other algorithms.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%J ECCV 2020. Lecture Notes in Computer Science, vol 12354. Springer,
  Cham

Conference paper

X. Long, L. Liu, C. Theobalt, and W. Wang

“Occlusion-Aware Depth Estimation with Adaptive Normal Constraints,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Long_ECCV20,
TITLE = {Occlusion-Aware Depth Estimation with Adaptive Normal Constraints},
AUTHOR = {Long, Xiaoxiao and Liu, Lingjie and Theobalt, Christian and Wang, Wenping},
LANGUAGE = {eng},
ISBN = {978-3-030-58544-0},
DOI = {10.1007/978-3-030-58545-7_37},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {640--657},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12354},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Long, Xiaoxiao
%A Liu, Lingjie
%A Theobalt, Christian
%A Wang, Wenping
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Occlusion-Aware Depth Estimation with Adaptive Normal Constraints : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D42B-A
%R 10.1007/978-3-030-58545-7_37
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 640 - 657
%I Springer
%@ 978-3-030-58544-0
%B Lecture Notes in Computer Science
%N 12354

Paper

J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt, and D. Stricker

“HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map,” 2020. [Online]. Available: https://arxiv.org/abs/2004.01588.

Abstract

3D hand shape and pose estimation from a single depth map is a new and
challenging computer vision problem with many applications. The
state-of-the-art methods directly regress 3D hand meshes from 2D depth images
via 2D convolutional neural networks, which leads to artefacts in the
estimations due to perspective distortions in the images. In contrast, we
propose a novel architecture with 3D convolutions trained in a
weakly-supervised manner. The input to our method is a 3D voxelized depth map,
and we rely on two hand shape representations. The first one is the 3D
voxelized grid of the shape which is accurate but does not preserve the mesh
topology and the number of mesh vertices. The second representation is the 3D
hand surface which is less accurate but does not suffer from the limitations of
the first representation. We combine the advantages of these two
representations by registering the hand surface to the voxelized hand shape. In
the extensive experiments, the proposed approach improves over the state of the
art by 47.8% on the SynHand5M dataset. Moreover, our augmentation policy for
voxelized depth maps further enhances the accuracy of 3D hand pose estimation
on real data. Our method produces visually more reasonable and realistic hand
shapes on NYU and BigHand2.2M datasets compared to the existing approaches.

BibTeX

@online{Malik2004.01588,
TITLE = {{HandVoxNet}: {D}eep Voxel-Based Network for {3D} Hand Shape and Pose Estimation from a Single Depth Map},
AUTHOR = {Malik, Jameel and Abdelaziz, Ibrahim and Elhayek, Ahmed and Shimada, Soshi and Ali, Sk Aziz and Golyanik, Vladislav and Theobalt, Christian and Stricker, Didier},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2004.01588},
EPRINT = {2004.01588},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {3D hand shape and pose estimation from a single depth map is a new and<br>challenging computer vision problem with many applications. The<br>state-of-the-art methods directly regress 3D hand meshes from 2D depth images<br>via 2D convolutional neural networks, which leads to artefacts in the<br>estimations due to perspective distortions in the images. In contrast, we<br>propose a novel architecture with 3D convolutions trained in a<br>weakly-supervised manner. The input to our method is a 3D voxelized depth map,<br>and we rely on two hand shape representations. The first one is the 3D<br>voxelized grid of the shape which is accurate but does not preserve the mesh<br>topology and the number of mesh vertices. The second representation is the 3D<br>hand surface which is less accurate but does not suffer from the limitations of<br>the first representation. We combine the advantages of these two<br>representations by registering the hand surface to the voxelized hand shape. In<br>the extensive experiments, the proposed approach improves over the state of the<br>art by 47.8% on the SynHand5M dataset. Moreover, our augmentation policy for<br>voxelized depth maps further enhances the accuracy of 3D hand pose estimation<br>on real data. Our method produces visually more reasonable and realistic hand<br>shapes on NYU and BigHand2.2M datasets compared to the existing approaches.<br>},
}

Endnote

%0 Report
%A Malik, Jameel
%A Abdelaziz, Ibrahim
%A Elhayek, Ahmed
%A Shimada, Soshi
%A Ali, Sk Aziz
%A Golyanik, Vladislav
%A Theobalt, Christian
%A Stricker, Didier
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose
  Estimation from a Single Depth Map : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E0FF-D
%U https://arxiv.org/abs/2004.01588
%D 2020
%X   3D hand shape and pose estimation from a single depth map is a new and<br>challenging computer vision problem with many applications. The<br>state-of-the-art methods directly regress 3D hand meshes from 2D depth images<br>via 2D convolutional neural networks, which leads to artefacts in the<br>estimations due to perspective distortions in the images. In contrast, we<br>propose a novel architecture with 3D convolutions trained in a<br>weakly-supervised manner. The input to our method is a 3D voxelized depth map,<br>and we rely on two hand shape representations. The first one is the 3D<br>voxelized grid of the shape which is accurate but does not preserve the mesh<br>topology and the number of mesh vertices. The second representation is the 3D<br>hand surface which is less accurate but does not suffer from the limitations of<br>the first representation. We combine the advantages of these two<br>representations by registering the hand surface to the voxelized hand shape. In<br>the extensive experiments, the proposed approach improves over the state of the<br>art by 47.8% on the SynHand5M dataset. Moreover, our augmentation policy for<br>voxelized depth maps further enhances the accuracy of 3D hand pose estimation<br>on real data. Our method produces visually more reasonable and realistic hand<br>shapes on NYU and BigHand2.2M datasets compared to the existing approaches.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

J. Malik, I. Abdelaziz, A. Elhayek, S. Shimada, S. A. Ali, V. Golyanik, C. Theobalt, and D. Stricker

“HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation From a Single Depth Map,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{Malik_CVPR2020,
TITLE = {{HandVoxNet}: {D}eep Voxel-Based Network for {3D} Hand Shape and Pose Estimation From a Single Depth Map},
AUTHOR = {Malik, Jameel and Abdelaziz, Ibrahim and Elhayek, Ahmed and Shimada, Soshi and Ali, Sk Aziz and Golyanik, Vladislav and Theobalt, Christian and Stricker, Didier},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00714},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {7111--7120},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Malik, Jameel
%A Abdelaziz, Ibrahim
%A Elhayek, Ahmed
%A Shimada, Soshi
%A Ali, Sk Aziz
%A Golyanik, Vladislav
%A Theobalt, Christian
%A Stricker, Didier
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation From a Single Depth Map : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CFCA-D
%R 10.1109/CVPR42600.2020.00714
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 7111 - 7120
%I IEEE
%@ 978-1-7281-7168-5

Paper

Mallikarjun B R, A. Tewari, H.-P. Seidel, M. Elgharib, and C. Theobalt, “Learning Complete 3D Morphable Face Models from Images and Videos,” 2020. [Online]. Available: https://arxiv.org/abs/2010.01679.

Abstract

Most 3D face reconstruction methods rely on 3D morphable models, which
disentangle the space of facial deformations into identity geometry,
expressions and skin reflectance. These models are typically learned from a
limited number of 3D scans and thus do not generalize well across different
identities and expressions. We present the first approach to learn complete 3D
models of face identity geometry, albedo and expression just from images and
videos. The virtually endless collection of such data, in combination with our
self-supervised learning-based approach allows for learning face models that
generalize beyond the span of existing approaches. Our network design and loss
functions ensure a disentangled parameterization of not only identity and
albedo, but also, for the first time, an expression basis. Our method also
allows for in-the-wild monocular reconstruction at test time. We show that our
learned models better generalize and lead to higher quality image-based
reconstructions than existing approaches.

BibTeX

@online{Mallikarjun_arXiv2010.01679,
TITLE = {Learning Complete {3D} Morphable Face Models from Images and Videos},
AUTHOR = {Mallikarjun B R and Tewari, Ayush and Seidel, Hans-Peter and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2010.01679},
EPRINT = {2010.01679},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Most 3D face reconstruction methods rely on 3D morphable models, which<br>disentangle the space of facial deformations into identity geometry,<br>expressions and skin reflectance. These models are typically learned from a<br>limited number of 3D scans and thus do not generalize well across different<br>identities and expressions. We present the first approach to learn complete 3D<br>models of face identity geometry, albedo and expression just from images and<br>videos. The virtually endless collection of such data, in combination with our<br>self-supervised learning-based approach allows for learning face models that<br>generalize beyond the span of existing approaches. Our network design and loss<br>functions ensure a disentangled parameterization of not only identity and<br>albedo, but also, for the first time, an expression basis. Our method also<br>allows for in-the-wild monocular reconstruction at test time. We show that our<br>learned models better generalize and lead to higher quality image-based<br>reconstructions than existing approaches.<br>},
}

Endnote

%0 Report
%A Mallikarjun B R, 
%A Tewari, Ayush
%A Seidel, Hans-Peter
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Learning Complete 3D Morphable Face Models from Images and Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B6FB-1
%U https://arxiv.org/abs/2010.01679
%D 2020
%X   Most 3D face reconstruction methods rely on 3D morphable models, which<br>disentangle the space of facial deformations into identity geometry,<br>expressions and skin reflectance. These models are typically learned from a<br>limited number of 3D scans and thus do not generalize well across different<br>identities and expressions. We present the first approach to learn complete 3D<br>models of face identity geometry, albedo and expression just from images and<br>videos. The virtually endless collection of such data, in combination with our<br>self-supervised learning-based approach allows for learning face models that<br>generalize beyond the span of existing approaches. Our network design and loss<br>functions ensure a disentangled parameterization of not only identity and<br>albedo, but also, for the first time, an expression basis. Our method also<br>allows for in-the-wild monocular reconstruction at test time. We show that our<br>learned models better generalize and lead to higher quality image-based<br>reconstructions than existing approaches.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG,Computer Science, Multimedia, cs.MM

Paper

Mallikarjun B R, A. Tewari, T.-H. Oh, T. Weyrich, B. Bickel, H.-P. Seidel, H. Pfister, W. Matusik, M. Elgharib, and C. Theobalt, “Monocular Reconstruction of Neural Face Reflectance Fields,” 2020. [Online]. Available: https://arxiv.org/abs/2008.10247.

Abstract

The reflectance field of a face describes the reflectance properties
responsible for complex lighting effects including diffuse, specular,
inter-reflection and self shadowing. Most existing methods for estimating the
face reflectance from a monocular image assume faces to be diffuse with very
few approaches adding a specular component. This still leaves out important
perceptual aspects of reflectance as higher-order global illumination effects
and self-shadowing are not modeled. We present a new neural representation for
face reflectance where we can estimate all components of the reflectance
responsible for the final appearance from a single monocular image. Instead of
modeling each component of the reflectance separately using parametric models,
our neural representation allows us to generate a basis set of faces in a
geometric deformation-invariant space, parameterized by the input light
direction, viewpoint and face geometry. We learn to reconstruct this
reflectance field of a face just from a monocular image, which can be used to
render the face from any viewpoint in any light condition. Our method is
trained on a light-stage training dataset, which captures 300 people
illuminated with 150 light conditions from 8 viewpoints. We show that our
method outperforms existing monocular reflectance reconstruction methods, in
terms of photorealism due to better capturing of physical premitives, such as
sub-surface scattering, specularities, self-shadows and other higher-order
effects.

BibTeX

@online{Mallikarjun_2008.10247,
TITLE = {Monocular Reconstruction of Neural Face Reflectance Fields},
AUTHOR = {Mallikarjun B R and Tewari, Ayush and Oh, Tae-Hyun and Weyrich, Tim and Bickel, Bernd and Seidel, Hans-Peter and Pfister, Hanspeter and Matusik, Wojciech and Elgharib, Mohamed and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2008.10247},
EPRINT = {2008.10247},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {The reflectance field of a face describes the reflectance properties<br>responsible for complex lighting effects including diffuse, specular,<br>inter-reflection and self shadowing. Most existing methods for estimating the<br>face reflectance from a monocular image assume faces to be diffuse with very<br>few approaches adding a specular component. This still leaves out important<br>perceptual aspects of reflectance as higher-order global illumination effects<br>and self-shadowing are not modeled. We present a new neural representation for<br>face reflectance where we can estimate all components of the reflectance<br>responsible for the final appearance from a single monocular image. Instead of<br>modeling each component of the reflectance separately using parametric models,<br>our neural representation allows us to generate a basis set of faces in a<br>geometric deformation-invariant space, parameterized by the input light<br>direction, viewpoint and face geometry. We learn to reconstruct this<br>reflectance field of a face just from a monocular image, which can be used to<br>render the face from any viewpoint in any light condition. Our method is<br>trained on a light-stage training dataset, which captures 300 people<br>illuminated with 150 light conditions from 8 viewpoints. We show that our<br>method outperforms existing monocular reflectance reconstruction methods, in<br>terms of photorealism due to better capturing of physical premitives, such as<br>sub-surface scattering, specularities, self-shadows and other higher-order<br>effects.<br>},
}

Endnote

%0 Report
%A Mallikarjun B R, 
%A Tewari, Ayush
%A Oh, Tae-Hyun
%A Weyrich, Tim
%A Bickel, Bernd
%A Seidel, Hans-Peter
%A Pfister, Hanspeter
%A Matusik, Wojciech
%A Elgharib, Mohamed
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Monocular Reconstruction of Neural Face Reflectance Fields : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B110-E
%U https://arxiv.org/abs/2008.10247
%D 2020
%X   The reflectance field of a face describes the reflectance properties<br>responsible for complex lighting effects including diffuse, specular,<br>inter-reflection and self shadowing. Most existing methods for estimating the<br>face reflectance from a monocular image assume faces to be diffuse with very<br>few approaches adding a specular component. This still leaves out important<br>perceptual aspects of reflectance as higher-order global illumination effects<br>and self-shadowing are not modeled. We present a new neural representation for<br>face reflectance where we can estimate all components of the reflectance<br>responsible for the final appearance from a single monocular image. Instead of<br>modeling each component of the reflectance separately using parametric models,<br>our neural representation allows us to generate a basis set of faces in a<br>geometric deformation-invariant space, parameterized by the input light<br>direction, viewpoint and face geometry. We learn to reconstruct this<br>reflectance field of a face just from a monocular image, which can be used to<br>render the face from any viewpoint in any light condition. Our method is<br>trained on a light-stage training dataset, which captures 300 people<br>illuminated with 150 light conditions from 8 viewpoints. We show that our<br>method outperforms existing monocular reflectance reconstruction methods, in<br>terms of photorealism due to better capturing of physical premitives, such as<br>sub-surface scattering, specularities, self-shadows and other higher-order<br>effects.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Article

D4D2

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.-P. Seidel, H. Rhodin, G. Pons-Moll, and C. Theobalt

“XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2020), vol. 39, no. 4, 2020.

@article{Mehta_TOG2020,
TITLE = {{XNect}: {R}eal-time Multi-person {3D} Human Pose Estimation with a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Elgharib, Mohamed and Fua, Pascal and Seidel, Hans-Peter and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3386569.3392410},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {39},
NUMBER = {4},
EID = {82},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2020},
}

Endnote

%0 Journal Article
%A Mehta, Dushyant
%A Sotnychenko, Oleksandr
%A Mueller, Franziska
%A Xu, Weipeng
%A Elgharib, Mohamed
%A Fua, Pascal
%A Seidel, Hans-Peter
%A Rhodin, Helge
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-832D-3
%R 10.1145/3386569.3392410
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 4
%Z sequence number: 82
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2020
%O ACM SIGGRAPH 2020 Virtual Conference ; 2020, 17-28 August

Article

D4D2

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.-P. Seidel, H. Rhodin, G. Pons-Moll, and C. Theobalt

“XNect: Real-time Multi-person 3D Motion Capture with a Single RGB Camera,” ACM Transactions on Graphics, vol. 39, no. 4, 2020.

@article{DBLP:journals/tog/MehtaS0XEFSRPT20,
TITLE = {{XNect}: Real-time Multi-person {3D} Motion Capture with a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Elgharib, Mohamed and Fua, Pascal and Seidel, Hans-Peter and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3386569.3392410},
PUBLISHER = {Association for Computing Machinery},
ADDRESS = {New York, NY},
YEAR = {2020},
DATE = {2020},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {39},
NUMBER = {4},
PAGES = {1--17},
EID = {82},
}

Endnote

%0 Journal Article
%A Mehta, Dushyant
%A Sotnychenko, Oleksandr
%A Mueller, Franziska
%A Xu, Weipeng
%A Elgharib, Mohamed
%A Fua, Pascal
%A Seidel, Hans-Peter
%A Rhodin, Helge
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T XNect: Real-time Multi-person 3D Motion Capture with a Single RGB
Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-796D-0
%R 10.1145/3386569.3392410
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 4
%& 1
%P 1 - 17
%Z sequence number: 82
%I Association for Computing Machinery
%C New York, NY
%@ false

Article

A. Meka, R. Pandey, C. Häne, S. Orts-Escolano, P. Barnum, P. David-Son, D. Erickson, Y. Zhang, J. Taylor, S. Bouaziz, C. Legendre, W.-C. Ma, R. Overbeck, T. Beeler, P. Debevec, S. Izadi, C. Theobalt, C. Rhemann, and S. Fanello

“Deep Relightable Textures Volumetric Performance Capture with Neural Rendering,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Meka_ToG2020,
TITLE = {Deep Relightable Textures Volumetric Performance Capture with Neural Rendering},
AUTHOR = {Meka, Abhimitra and Pandey, Rohit and H{\"a}ne, Christian and Orts-Escolano, Sergio and Barnum, Peter and David-Son, Philip and Erickson, Daniel and Zhang, Yinda and Taylor, Jonathan and Bouaziz, Sofien and Legendre, Chloe and Ma, Wan-Chun and Overbeck, Ryan and Beeler, Thabo and Debevec, Paul and Izadi, Shahram and Theobalt, Christian and Rhemann, Christoph and Fanello, Sean},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417814},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {259},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Meka, Abhimitra
%A Pandey, Rohit
%A H&#228;ne, Christian
%A Orts-Escolano, Sergio
%A Barnum, Peter
%A David-Son, Philip
%A Erickson, Daniel
%A Zhang, Yinda
%A Taylor, Jonathan
%A Bouaziz, Sofien
%A Legendre, Chloe
%A Ma, Wan-Chun
%A Overbeck, Ryan
%A Beeler, Thabo
%A Debevec, Paul
%A Izadi, Shahram
%A Theobalt, Christian
%A Rhemann, Christoph
%A Fanello, Sean
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Deep Relightable Textures Volumetric Performance Capture with Neural Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-A6FA-4
%R 10.1145/3414685.3417814
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 259
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020
%U https://dl.acm.org/doi/pdf/10.1145/3414685.3417814

Thesis

D4IMPR-CS

A. Meka

“Live inverse rendering,” Universität des Saarlandes, Saarbrücken, 2020.

@phdthesis{Meka_2019,
TITLE = {Live inverse rendering},
AUTHOR = {Meka, Abhimitra},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-302066},
DOI = {http://dx.doi.org/10.22028/D291-30206},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2020},
DATE = {2020},
}

Endnote

%0 Thesis
%A Meka, Abhimitra
%Y Theobalt, Christian
%A referee: Drettakis, George
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Live inverse rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-715A-5
%R http://dx.doi.org/10.22028/D291-30206
%U urn:nbn:de:bsz:291--ds-302066
%F OTHER: hdl:20.500.11880/28721
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2020
%P 189 p.
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/28721

Conference paper

X. Meng, Q. Zheng, A. Varshney, G. Singh, and M. Zwicker

“Real-time Monte Carlo Denoising with the Neural Bilateral Grid,” in Rendering 2020 - DL-only Track (Eurographics Symposium on Rendering 2020), London, UK, 2020.

@inproceedings{Meng_EGRendering20,
TITLE = {Real-time {Monte Carlo} Denoising with the Neural Bilateral Grid},
AUTHOR = {Meng, Xiaoxu and Zheng, Quan and Varshney, Amitabh and Singh, Gurprit and Zwicker, Matthias},
LANGUAGE = {eng},
ISBN = {978-3-03868-117-5},
URL = {https://diglib.eg.org:443/handle/10.2312/sr20201133},
DOI = {10.2312/sr.20201133},
PUBLISHER = {The Eurographics Association},
YEAR = {2020},
BOOKTITLE = {Rendering 2020 -- DL-only Track (Eurographics Symposium on Rendering 2020)},
EDITOR = {Dachsbacher, Carsten and Pharr, Matt},
PAGES = {1--12},
ADDRESS = {London, UK},
}

Endnote

%0 Conference Proceedings
%A Meng, Xiaoxu
%A Zheng, Quan
%A Varshney, Amitabh
%A Singh, Gurprit
%A Zwicker, Matthias
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Real-time Monte Carlo Denoising with the Neural Bilateral Grid : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CEC2-6
%R 10.2312/sr.20201133
%U https://diglib.eg.org:443/handle/10.2312/sr20201133
%D 2020
%B 31st Eurographics Symposium on Rendering 
%Z date of event: 2020-06-29 - 2020-07-02
%C London, UK
%B Rendering 2020 - DL-only Track
%E Dachsbacher, Carsten; Pharr, Matt
%P 1 - 12
%I The Eurographics Association
%@ 978-3-03868-117-5

Article

D. Mlakar, M. Winter, P. Stadlbauer, H.-P. Seidel, M. Steinberger, and R. Zayer

“Subdivision-Specialized Linear Algebra Kernels for Static and Dynamic Mesh Connectivity on the GPU,” Computer Graphics Forum (Proc. EUROGRAPHICS 2020), vol. 39, no. 2, 2020.

@article{Mlakar_EG2020,
TITLE = {Subdivision-Specialized Linear Algebra Kernels for Static and Dynamic Mesh Connectivity on the {GPU}},
AUTHOR = {Mlakar, Daniel and Winter, M. and Stadlbauer, Pascal and Seidel, Hans-Peter and Steinberger, Markus and Zayer, Rhaleb},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13934},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2020},
DATE = {2020},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {39},
NUMBER = {2},
PAGES = {335--349},
BOOKTITLE = {The European Association for Computer Graphics 41st Annual Conference (EUROGRAPHICS 2020)},
EDITOR = {Panozzo, Daniele and Assarsson, Ulf},
}

Endnote

%0 Journal Article
%A Mlakar, Daniel
%A Winter, M.
%A Stadlbauer, Pascal
%A Seidel, Hans-Peter
%A Steinberger, Markus
%A Zayer, Rhaleb
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Subdivision-Specialized Linear Algebra Kernels for Static and Dynamic Mesh Connectivity on the GPU : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-DB80-2
%R 10.1111/cgf.13934
%7 2020
%D 2020
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 39
%N 2
%& 335
%P 335 - 349
%I Blackwell-Wiley
%C Oxford
%@ false
%B The European Association for Computer Graphics 41st Annual Conference
%O EUROGRAPHICS 2020 EG 2020 The European Association for Computer Graphics 41st Annual Conference ; Norrk&#246;ping, Sweden, May 25 &#8211; 29, 2020

Thesis

D4IMPR-CS

F. Mueller

“Real-time 3D Hand Reconstruction in Challenging Scenes from a Single Color or Depth Camera,” Universität des Saarlandes, Saarbrücken, 2020.

@phdthesis{MuellerFDiss_2020,
TITLE = {Real-time 3{D} Hand Reconstruction in Challenging Scenes from a Single Color or Depth Camera},
AUTHOR = {Mueller, Franziska},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-328467},
DOI = {10.22028/D291-32846},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2020},
DATE = {2020},
}

Endnote

%0 Thesis
%A Mueller, Franziska
%Y Theobalt, Christian
%A referee: Seidel, Hans-Peter
%A referee: Izadi, Shahram
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Real-time 3D Hand Reconstruction in Challenging Scenes from a Single Color or Depth Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D8C7-5
%R 10.22028/D291-32846
%F OTHER: hdl:20.500.11880/30313
%U urn:nbn:de:bsz:291--ds-328467
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2020
%P 155 p.
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/30313

Article

M. Piovarči, M. Foshey, V. Babaei, S. Rusinkiewicz, W. Matusik, and P. Didyk

“Towards Spatially Varying Gloss Reproduction for 3D Printing,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Piovarci_ToG2020,
TITLE = {Towards Spatially Varying Gloss Reproduction for {3D} Printing},
AUTHOR = {Piovar{\v c}i, Michal and Foshey, Michael and Babaei, Vahid and Rusinkiewicz, Szymon and Matusik, Wojciech and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417850},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {206},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Piovar&#269;i, Michal
%A Foshey, Michael
%A Babaei, Vahid
%A Rusinkiewicz, Szymon
%A Matusik, Wojciech
%A Didyk, Piotr
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T Towards Spatially Varying Gloss Reproduction for 3D Printing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-A6FE-0
%R 10.1145/3414685.3417850
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 206
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Conference paper

N. Qian, J. Wang, F. Mueller, F. Bernard, V. Golyanik, and C. Theobalt

“HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Qian_ECCV20,
TITLE = {{HTML}: {A} Parametric Hand Texture Model for {3D} Hand Reconstruction and Personalization},
AUTHOR = {Qian, Neng and Wang, Jiayi and Mueller, Franziska and Bernard, Florian and Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-3-030-58621-8},
DOI = {10.1007/978-3-030-58621-8_4},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {54--71},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12356},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Qian, Neng
%A Wang, Jiayi
%A Mueller, Franziska
%A Bernard, Florian
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D062-F
%R 10.1007/978-3-030-58621-8_4
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 54 - 71
%I Springer
%@ 978-3-030-58621-8
%B Lecture Notes in Computer Science
%N 12356

Report

N. Qian, J. Wang, F. Mueller, F. Bernard, V. Golyanik, and C. Theobalt

“Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization,” Max-Planck-Institut für Informatik, Saarbrücken, MPI-I-2020-4-001, 2020.

Abstract

3D hand reconstruction from image data is a widely-studied problem in com-
puter vision and graphics, and has a particularly high relevance for virtual
and augmented reality. Although several 3D hand reconstruction approaches
leverage hand models as a strong prior to resolve ambiguities and achieve a
more robust reconstruction, most existing models account only for the hand
shape and poses and do not model the texture. To ﬁll this gap, in this work
we present the ﬁrst parametric texture model of human hands. Our model
spans several dimensions of hand appearance variability (e.g., related to gen-
der, ethnicity, or age) and only requires a commodity camera for data acqui-
sition. Experimentally, we demonstrate that our appearance model can be
used to tackle a range of challenging problems such as 3D hand reconstruc-
tion from a single monocular image. Furthermore, our appearance model
can be used to deﬁne a neural rendering layer that enables training with a
self-supervised photometric loss. We make our model publicly available.

BibTeX

@techreport{Qian_report2020,
TITLE = {Parametric Hand Texture Model for {3D} Hand Reconstruction and Personalization},
AUTHOR = {Qian, Neng and Wang, Jiayi and Mueller, Franziska and Bernard, Florian and Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0946-011X},
NUMBER = {MPI-I-2020-4-001},
INSTITUTION = {Max-Planck-Institut f{\"u}r Informatik},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2020},
ABSTRACT = {3D hand reconstruction from image data is a widely-studied problem in com-<br>puter vision and graphics, and has a particularly high relevance for virtual<br>and augmented reality. Although several 3D hand reconstruction approaches<br>leverage hand models as a strong prior to resolve ambiguities and achieve a<br>more robust reconstruction, most existing models account only for the hand<br>shape and poses and do not model the texture. To {fi}ll this gap, in this work<br>we present the {fi}rst parametric texture model of human hands. Our model<br>spans several dimensions of hand appearance variability (e.g., related to gen-<br>der, ethnicity, or age) and only requires a commodity camera for data acqui-<br>sition. Experimentally, we demonstrate that our appearance model can be<br>used to tackle a range of challenging problems such as 3D hand reconstruc-<br>tion from a single monocular image. Furthermore, our appearance model<br>can be used to de{fi}ne a neural rendering layer that enables training with a<br>self-supervised photometric loss. We make our model publicly available.},
TYPE = {Research Report},
}

Endnote

%0 Report
%A Qian, Neng
%A Wang, Jiayi
%A Mueller, Franziska
%A Bernard, Florian
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Parametric Hand Texture Model for 3D Hand Reconstruction and
Personalization : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-9128-9
%Y Max-Planck-Institut f&#252;r Informatik
%C Saarbr&#252;cken
%D 2020
%P 37 p.
%X 3D hand reconstruction from image data is a widely-studied problem in com-<br>puter vision and graphics, and has a particularly high relevance for virtual<br>and augmented reality. Although several 3D hand reconstruction approaches<br>leverage hand models as a strong prior to resolve ambiguities and achieve a<br>more robust reconstruction, most existing models account only for the hand<br>shape and poses and do not model the texture. To &#64257;ll this gap, in this work<br>we present the &#64257;rst parametric texture model of human hands. Our model<br>spans several dimensions of hand appearance variability (e.g., related to gen-<br>der, ethnicity, or age) and only requires a commodity camera for data acqui-<br>sition. Experimentally, we demonstrate that our appearance model can be<br>used to tackle a range of challenging problems such as 3D hand reconstruc-<br>tion from a single monocular image. Furthermore, our appearance model<br>can be used to de&#64257;ne a neural rendering layer that enables training with a<br>self-supervised photometric loss. We make our model publicly available.
%K hand texture model, appearance modeling, hand tracking, 3D hand recon-
struction
%B Research Report
%@ false

Paper

S. Rao, D. Stutz, and B. Schiele

“Adversarial Training against Location-Optimized Adversarial Patches,” 2020. [Online]. Available: https://arxiv.org/abs/2005.02313.

Abstract

Deep neural networks have been shown to be susceptible to adversarial
examples -- small, imperceptible changes constructed to cause
mis-classification in otherwise highly accurate image classifiers. As a
practical alternative, recent work proposed so-called adversarial patches:
clearly visible, but adversarially crafted rectangular patches in images. These
patches can easily be printed and applied in the physical world. While defenses
against imperceptible adversarial examples have been studied extensively,
robustness against adversarial patches is poorly understood. In this work, we
first devise a practical approach to obtain adversarial patches while actively
optimizing their location within the image. Then, we apply adversarial training
on these location-optimized adversarial patches and demonstrate significantly
improved robustness on CIFAR10 and GTSRB. Additionally, in contrast to
adversarial training on imperceptible adversarial examples, our adversarial
patch training does not reduce accuracy.

BibTeX

@online{Rao_arXiv2005.02313,
TITLE = {Adversarial Training against Location-Optimized Adversarial Patches},
AUTHOR = {Rao, Sukrut and Stutz, David and Schiele, Bernt},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2005.02313},
EPRINT = {2005.02313},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Deep neural networks have been shown to be susceptible to adversarial<br>examples -- small, imperceptible changes constructed to cause<br>mis-classification in otherwise highly accurate image classifiers. As a<br>practical alternative, recent work proposed so-called adversarial patches:<br>clearly visible, but adversarially crafted rectangular patches in images. These<br>patches can easily be printed and applied in the physical world. While defenses<br>against imperceptible adversarial examples have been studied extensively,<br>robustness against adversarial patches is poorly understood. In this work, we<br>first devise a practical approach to obtain adversarial patches while actively<br>optimizing their location within the image. Then, we apply adversarial training<br>on these location-optimized adversarial patches and demonstrate significantly<br>improved robustness on CIFAR10 and GTSRB. Additionally, in contrast to<br>adversarial training on imperceptible adversarial examples, our adversarial<br>patch training does not reduce accuracy.<br>},
}

Endnote

%0 Report
%A Rao, Sukrut
%A Stutz, David
%A Schiele, Bernt
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Adversarial Training against Location-Optimized Adversarial Patches : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-80D0-C
%U https://arxiv.org/abs/2005.02313
%D 2020
%X   Deep neural networks have been shown to be susceptible to adversarial<br>examples -- small, imperceptible changes constructed to cause<br>mis-classification in otherwise highly accurate image classifiers. As a<br>practical alternative, recent work proposed so-called adversarial patches:<br>clearly visible, but adversarially crafted rectangular patches in images. These<br>patches can easily be printed and applied in the physical world. While defenses<br>against imperceptible adversarial examples have been studied extensively,<br>robustness against adversarial patches is poorly understood. In this work, we<br>first devise a practical approach to obtain adversarial patches while actively<br>optimizing their location within the image. Then, we apply adversarial training<br>on these location-optimized adversarial patches and demonstrate significantly<br>improved robustness on CIFAR10 and GTSRB. Additionally, in contrast to<br>adversarial training on imperceptible adversarial examples, our adversarial<br>patch training does not reduce accuracy.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Cryptography and Security, cs.CR,Computer Science, Learning, cs.LG,Statistics, Machine Learning, stat.ML

Article

A. Saberpour, R. D. Hersch, J. Fang, R. Zayer, H.-P. Seidel, and V. Babaei

“Fabrication of Moiré on Curved Surfaces,” Optics Express, vol. 28, no. 13, 2020.

@article{Saberpour2020,
TITLE = {Fabrication of Moir{\'e} on Curved Surfaces},
AUTHOR = {Saberpour, Artin and Hersch, Roger D. and Fang, Jiajing and Zayer, Rhaleb and Seidel, Hans-Peter and Babaei, Vahid},
LANGUAGE = {eng},
ISSN = {1094-4087},
DOI = {10.1364/OE.393843},
PUBLISHER = {Optical Society of America},
ADDRESS = {Washington, DC},
YEAR = {2020},
DATE = {2020},
JOURNAL = {Optics Express},
VOLUME = {28},
NUMBER = {13},
PAGES = {19413--19427},
}

Endnote

%0 Journal Article
%A Saberpour, Artin
%A Hersch, Roger D.
%A Fang, Jiajing
%A Zayer, Rhaleb
%A Seidel, Hans-Peter
%A Babaei, Vahid
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Fabrication of Moir&#233; on Curved Surfaces  : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-D39D-B
%R 10.1364/OE.393843
%7 2020
%D 2020
%J Optics Express
%O Opt. Express
%V 28
%N 13
%& 19413
%P 19413 - 19427
%I Optical Society of America
%C Washington, DC
%@ false

Conference paper

K. Sarkar, D. Mehta, W. Xu, V. Golyanik, and C. Theobalt

“Neural Re-rendering of Humans from a Single Image,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Sarkar_ECCV20,
TITLE = {Neural Re-rendering of Humans from a Single Image},
AUTHOR = {Sarkar, Kripasindhu and Mehta, Dushyant and Xu, Weipeng and Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-3-030-58621-8},
DOI = {10.1007/978-3-030-58621-8_35},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {596--613},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12356},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Sarkar, Kripasindhu
%A Mehta, Dushyant
%A Xu, Weipeng
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Re-rendering of Humans from a Single Image : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D0A4-4
%R 10.1007/978-3-030-58621-8_35
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 596 - 613
%I Springer
%@ 978-3-030-58621-8
%B Lecture Notes in Computer Science
%N 12356

Conference paper

M. Seelbach Benkner, V. Golyanik, C. Theobalt, and M. Moeller

“Adiabatic Quantum Graph Matching with Permutation Matrix Constraints,” in International Conference on 3D Vision, Fukuoka, Japan (Virtual Event), 2020.

@inproceedings{SeelbachBenkner2020,
TITLE = {Adiabatic Quantum Graph Matching with Permutation Matrix Constraints},
AUTHOR = {Seelbach Benkner, Marcel and Golyanik, Vladislav and Theobalt, Christian and Moeller, Michael},
LANGUAGE = {eng},
ISBN = {978-1-7281-8128-8},
DOI = {10.1109/3DV50981.2020.00068},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {International Conference on 3D Vision},
PAGES = {583--592},
ADDRESS = {Fukuoka, Japan (Virtual Event)},
}

Endnote

%0 Conference Proceedings
%A Seelbach Benkner, Marcel
%A Golyanik, Vladislav
%A Theobalt, Christian
%A Moeller, Michael
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Adiabatic Quantum Graph Matching with Permutation Matrix 
Constraints : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-0362-6
%R 10.1109/3DV50981.2020.00068
%D 2020
%B International Conference on 3D Vision
%Z date of event: 2020-11-25 - 2020-11-28
%C Fukuoka, Japan (Virtual Event)
%B International Conference on 3D Vision
%P 583 - 592
%I IEEE
%@ 978-1-7281-8128-8

Article

A. Serrano, D. Martin, D. Gutierrez, K. Myszkowski, and B. Masia

“Imperceptible Manipulation of Lateral Camera Motion for Improved Virtual Reality Applications,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Serrano2020,
TITLE = {Imperceptible Manipulation of Lateral Camera Motion for Improved Virtual Reality Applications},
AUTHOR = {Serrano, Ana and Martin, Daniel and Gutierrez, Diego and Myszkowski, Karol and Masia, Belen},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417773},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {268},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Serrano, Ana
%A Martin, Daniel
%A Gutierrez, Diego
%A Myszkowski, Karol
%A Masia, Belen
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Imperceptible Manipulation of Lateral Camera Motion for Improved Virtual Reality Applications : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-FBE8-A
%R 10.1145/3414685.3417773
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 268
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Article

A. A. Shahmirzadi, V. Babaei, and H.-P. Seidel

“A Multispectral Dataset of Oil and Watercolor Paints,” Electronic Imaging, vol. 32, 2020.

@article{shahmirzadi2020multispectral,
TITLE = {A Multispectral Dataset of Oil and Watercolor Paints},
AUTHOR = {Shahmirzadi, Azadeh Asadi and Babaei, Vahid and Seidel, Hans-Peter},
LANGUAGE = {eng},
DOI = {10.2352/ISSN.2470-1173.2020.5.MAAP-107},
PUBLISHER = {IS\&T},
PUBLISHER = {Society for Imaging Science and Technology},
ADDRESS = {Springfield, VA},
ADDRESS = {Springfield},
YEAR = {2020},
JOURNAL = {Electronic Imaging},
VOLUME = {32},
PAGES = {1--4},
EID = {107},
BOOKTITLE = {Proceedings of the Material Appearance 2020},
EDITOR = {H{\'e}bert, Mathieu and Simonot, Lionel and Tastl, Ingeborg},
}

Endnote

%0 Journal Article
%A Shahmirzadi, Azadeh Asadi
%A Babaei, Vahid
%A Seidel, Hans-Peter
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T A Multispectral Dataset of Oil and Watercolor Paints : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-F064-9
%R 10.2352/ISSN.2470-1173.2020.5.MAAP-107
%7 2020
%D 2020
%J Electronic Imaging
%V 32
%& 1
%P 1 - 4
%Z sequence number: 107
%I IS&T
%C Springfield, VA
%B Proceedings of the Material Appearance 2020
%O Burlingame, CA, USA, January 26-30, 2020
%I Society for Imaging Science and Technology
%C Springfield

Article

S. Shimada, V. Golyanik, W. Xu, and C. Theobalt

“PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Shimada_ToG2020,
TITLE = {{PhysCap}: {P}hysically Plausible Monocular {3D} Motion Capture in Real Time},
AUTHOR = {Shimada, Soshi and Golyanik, Vladislav and Xu, Weipeng and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417877},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {235},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Shimada, Soshi
%A Golyanik, Vladislav
%A Xu, Weipeng
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-A709-3
%R 10.1145/3414685.3417877
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 235
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Paper

S. Shimada, V. Golyanik, W. Xu, and C. Theobalt

“PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time,” 2020. [Online]. Available: https://arxiv.org/abs/2008.08880.

Abstract

Marker-less 3D human motion capture from a single colour camera has seen
significant progress. However, it is a very challenging and severely ill-posed
problem. In consequence, even the most accurate state-of-the-art approaches
have significant limitations. Purely kinematic formulations on the basis of
individual joints or skeletons, and the frequent frame-wise reconstruction in
state-of-the-art methods greatly limit 3D accuracy and temporal stability
compared to multi-view or marker-based motion capture. Further, captured 3D
poses are often physically incorrect and biomechanically implausible, or
exhibit implausible environment interactions (floor penetration, foot skating,
unnatural body leaning and strong shifting in depth), which is problematic for
any use case in computer graphics. We, therefore, present PhysCap, the first
algorithm for physically plausible, real-time and marker-less human 3D motion
capture with a single colour camera at 25 fps. Our algorithm first captures 3D
human poses purely kinematically. To this end, a CNN infers 2D and 3D joint
positions, and subsequently, an inverse kinematics step finds space-time
coherent joint angles and global 3D pose. Next, these kinematic reconstructions
are used as constraints in a real-time physics-based pose optimiser that
accounts for environment constraints (e.g., collision handling and floor
placement), gravity, and biophysical plausibility of human postures. Our
approach employs a combination of ground reaction force and residual force for
plausible root control, and uses a trained neural network to detect foot
contact events in images. Our method captures physically plausible and
temporally stable global 3D human motion, without physically implausible
postures, floor penetrations or foot skating, from video in real time and in
general scenes. The video is available at
gvv.mpi-inf.mpg.de/projects/PhysCap

BibTeX

@online{Shimada_2008.08880,
TITLE = {{PhysCap}: {P}hysically Plausible Monocular {3D} Motion Capture in Real Time},
AUTHOR = {Shimada, Soshi and Golyanik, Vladislav and Xu, Weipeng and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2008.08880},
EPRINT = {2008.08880},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Marker-less 3D human motion capture from a single colour camera has seen<br>significant progress. However, it is a very challenging and severely ill-posed<br>problem. In consequence, even the most accurate state-of-the-art approaches<br>have significant limitations. Purely kinematic formulations on the basis of<br>individual joints or skeletons, and the frequent frame-wise reconstruction in<br>state-of-the-art methods greatly limit 3D accuracy and temporal stability<br>compared to multi-view or marker-based motion capture. Further, captured 3D<br>poses are often physically incorrect and biomechanically implausible, or<br>exhibit implausible environment interactions (floor penetration, foot skating,<br>unnatural body leaning and strong shifting in depth), which is problematic for<br>any use case in computer graphics. We, therefore, present PhysCap, the first<br>algorithm for physically plausible, real-time and marker-less human 3D motion<br>capture with a single colour camera at 25 fps. Our algorithm first captures 3D<br>human poses purely kinematically. To this end, a CNN infers 2D and 3D joint<br>positions, and subsequently, an inverse kinematics step finds space-time<br>coherent joint angles and global 3D pose. Next, these kinematic reconstructions<br>are used as constraints in a real-time physics-based pose optimiser that<br>accounts for environment constraints (e.g., collision handling and floor<br>placement), gravity, and biophysical plausibility of human postures. Our<br>approach employs a combination of ground reaction force and residual force for<br>plausible root control, and uses a trained neural network to detect foot<br>contact events in images. Our method captures physically plausible and<br>temporally stable global 3D human motion, without physically implausible<br>postures, floor penetrations or foot skating, from video in real time and in<br>general scenes. The video is available at<br>http://gvv.mpi-inf.mpg.de/projects/PhysCap<br>},
}

Endnote

%0 Report
%A Shimada, Soshi
%A Golyanik, Vladislav
%A Xu, Weipeng
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Time : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E8F3-1
%U https://arxiv.org/abs/2008.08880
%D 2020
%X   Marker-less 3D human motion capture from a single colour camera has seen<br>significant progress. However, it is a very challenging and severely ill-posed<br>problem. In consequence, even the most accurate state-of-the-art approaches<br>have significant limitations. Purely kinematic formulations on the basis of<br>individual joints or skeletons, and the frequent frame-wise reconstruction in<br>state-of-the-art methods greatly limit 3D accuracy and temporal stability<br>compared to multi-view or marker-based motion capture. Further, captured 3D<br>poses are often physically incorrect and biomechanically implausible, or<br>exhibit implausible environment interactions (floor penetration, foot skating,<br>unnatural body leaning and strong shifting in depth), which is problematic for<br>any use case in computer graphics. We, therefore, present PhysCap, the first<br>algorithm for physically plausible, real-time and marker-less human 3D motion<br>capture with a single colour camera at 25 fps. Our algorithm first captures 3D<br>human poses purely kinematically. To this end, a CNN infers 2D and 3D joint<br>positions, and subsequently, an inverse kinematics step finds space-time<br>coherent joint angles and global 3D pose. Next, these kinematic reconstructions<br>are used as constraints in a real-time physics-based pose optimiser that<br>accounts for environment constraints (e.g., collision handling and floor<br>placement), gravity, and biophysical plausibility of human postures. Our<br>approach employs a combination of ground reaction force and residual force for<br>plausible root control, and uses a trained neural network to detect foot<br>contact events in images. Our method captures physically plausible and<br>temporally stable global 3D human motion, without physically implausible<br>postures, floor penetrations or foot skating, from video in real time and in<br>general scenes. The video is available at<br>http://gvv.mpi-inf.mpg.de/projects/PhysCap<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Conference paper

V. Sidhu, E. Tretschk, V. Golyanik, A. Agudo, and C. Theobalt

“Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Sidhu_ECCV20,
TITLE = {Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints},
AUTHOR = {Sidhu, Vikramjit and Tretschk, Edgar and Golyanik, Vladislav and Agudo, Antonio and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-3-030-58516-7},
DOI = {10.1007/978-3-030-58517-4_13},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {204--222},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12361},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Sidhu, Vikramjit
%A Tretschk, Edgar
%A Golyanik, Vladislav
%A Agudo, Antonio
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Dense Non-Rigid Structure from Motion with Latent Space Constraints : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D080-C
%R 10.1007/978-3-030-58517-4_13
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 204 - 222
%I Springer
%@ 978-3-030-58516-7
%B Lecture Notes in Computer Science
%N 12361

Article

G. Singh, K. Subr, D. Coeurjolly, V. Ostromoukhov, and W. Jarosz

“Fourier Analysis of Correlated Monte Carlo Importance Sampling,” Computer Graphics Forum, vol. 39, no. 1, 2020.

@article{SinghCGF2020,
TITLE = {Fourier Analysis of Correlated {Monte Carlo} Importance Sampling},
AUTHOR = {Singh, Gurprit and Subr, Kartic and Coeurjolly, David and Ostromoukhov, Victor and Jarosz, Wojciech},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13613},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2020},
DATE = {2020},
JOURNAL = {Computer Graphics Forum},
VOLUME = {39},
NUMBER = {1},
PAGES = {7--19},
}

Endnote

%0 Journal Article
%A Singh, Gurprit
%A Subr, Kartic
%A Coeurjolly, David
%A Ostromoukhov, Victor
%A Jarosz, Wojciech
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
%T Fourier Analysis of Correlated Monte Carlo Importance Sampling : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-978D-1
%R 10.1111/cgf.13613
%7 2020
%D 2020
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 39
%N 1
%& 7
%P 7 - 19
%I Blackwell-Wiley
%C Oxford
%@ false

Article

P. Stadlbauer, D. Mlakar, H.-P. Seidel, M. Steinberger, and R. Zayer

“Interactive Modeling of Cellular Structures on Surfaces with Application to Additive Manufacturing,” Computer Graphics Forum (Proc. EUROGRAPHICS 2020), vol. 39, no. 2, 2020.

@article{Stadlbauer_EG2020,
TITLE = {Interactive Modeling of Cellular Structures on Surfaces with Application to Additive Manufacturing},
AUTHOR = {Stadlbauer, Pascal and Mlakar, Daniel and Seidel, Hans-Peter and Steinberger, Markus and Zayer, Rhaleb},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13929},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2020},
DATE = {2020},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {39},
NUMBER = {2},
PAGES = {277--289},
BOOKTITLE = {The European Association for Computer Graphics 41st Annual Conference (EUROGRAPHICS 2020)},
EDITOR = {Panozzo, Daniele and Assarsson, Ulf},
}

Endnote

%0 Journal Article
%A Stadlbauer, Pascal
%A Mlakar, Daniel
%A Seidel, Hans-Peter
%A Steinberger, Markus
%A Zayer, Rhaleb
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Interactive Modeling of Cellular Structures on Surfaces with Application to Additive Manufacturing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-DB8A-8
%R 10.1111/cgf.13929
%7 2020
%D 2020
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 39
%N 2
%& 277
%P 277 - 289
%I Blackwell-Wiley
%C Oxford
%@ false
%B The European Association for Computer Graphics 41st Annual Conference
%O EUROGRAPHICS 2020 EG 2020 The European Association for Computer Graphics 41st Annual Conference ; Norrk&#246;ping, Sweden, May 25 &#8211; 29, 2020

Article

A. S. Sultan, M. Elgharib, T. Tavares, M. Jessri, and J. R. Basile

“The Use of Artificial Intelligence, Machine Learning and Deep Learning in Oncologic Histopathology,” Journal of Oral Pathology & Medicine, vol. 49, no. 9, 2020.

@article{Sultan2020,
TITLE = {The Use of Artificial Intelligence, Machine Learning and Deep Learning in Oncologic Histopathology},
AUTHOR = {Sultan, Ahmed S. and Elgharib, Mohamed and Tavares, Tiffany and Jessri, Maryam and Basile, John R.},
LANGUAGE = {eng},
ISSN = {0904-2512},
DOI = {10.1111/jop.13042},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford},
YEAR = {2020},
JOURNAL = {Journal of Oral Pathology \& Medicine},
VOLUME = {49},
NUMBER = {9},
PAGES = {849--856},
}

Endnote

%0 Journal Article
%A Sultan, Ahmed S.
%A Elgharib, Mohamed
%A Tavares, Tiffany
%A Jessri, Maryam
%A Basile, John R.
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T The Use of Artificial Intelligence, Machine Learning and Deep Learning in Oncologic Histopathology : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-A2C9-0
%R 10.1111/jop.13042
%7 2020
%D 2020
%J Journal of Oral Pathology & Medicine
%V 49
%N 9
%& 849
%P 849 - 856
%I Wiley-Blackwell
%C Oxford
%@ false

Conference paper

A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H.-P. Seidel, P. Pérez, M. Zollhöfer, and C. Theobalt

“StyleRig: Rigging StyleGAN for 3D Control Over Portrait Images,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{Tewari_CVPR2020,
TITLE = {{StyleRig}: {R}igging {StyleGAN} for {3D} Control Over Portrait Images},
AUTHOR = {Tewari, Ayush and Elgharib, Mohamed and Bharaj, Gaurav and Bernard, Florian and Seidel, Hans-Peter and P{\'e}rez, Patrick and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00618},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {6141--6150},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Tewari, Ayush
%A Elgharib, Mohamed
%A Bharaj, Gaurav
%A Bernard, Florian
%A Seidel, Hans-Peter
%A P&#233;rez, Patrick
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T StyleRig: Rigging StyleGAN for 3D Control Over Portrait Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B0E7-D
%R 10.1109/CVPR42600.2020.00618
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 6141 - 6150
%I IEEE
%@ 978-1-7281-7168-5

Paper

A. Tewari, O. Fried, J. Thies, V. Sitzmann, S. Lombardi, K. Sunkavalli, R. Martin-Brualla, T. Simon, J. Saragih, M. Nießner, R. Pandey, S. Fanello, G. Wetzstein, J.-Y. Zhu, C. Theobalt, M. Agrawala, E. Shechtman, D. B. Goldman, and M. Zollhöfer

“State of the Art on Neural Rendering,” 2020. [Online]. Available: https://arxiv.org/abs/2004.03805.

Abstract

Efficient rendering of photo-realistic virtual worlds is a long standing
effort of computer graphics. Modern graphics techniques have succeeded in
synthesizing photo-realistic images from hand-crafted scene representations.
However, the automatic generation of shape, materials, lighting, and other
aspects of scenes remains a challenging problem that, if solved, would make
photo-realistic computer graphics more widely accessible. Concurrently,
progress in computer vision and machine learning have given rise to a new
approach to image synthesis and editing, namely deep generative models. Neural
rendering is a new and rapidly emerging field that combines generative machine
learning techniques with physical knowledge from computer graphics, e.g., by
the integration of differentiable rendering into network training. With a
plethora of applications in computer graphics and vision, neural rendering is
poised to become a new area in the graphics community, yet no survey of this
emerging field exists. This state-of-the-art report summarizes the recent
trends and applications of neural rendering. We focus on approaches that
combine classic computer graphics techniques with deep generative models to
obtain controllable and photo-realistic outputs. Starting with an overview of
the underlying computer graphics and machine learning concepts, we discuss
critical aspects of neural rendering approaches. This state-of-the-art report
is focused on the many important use cases for the described algorithms such as
novel view synthesis, semantic photo manipulation, facial and body reenactment,
relighting, free-viewpoint video, and the creation of photo-realistic avatars
for virtual and augmented reality telepresence. Finally, we conclude with a
discussion of the social implications of such technology and investigate open
research problems.

BibTeX

@online{Tewari2004.03805,
TITLE = {State of the Art on Neural Rendering},
AUTHOR = {Tewari, Ayush and Fried, Ohad and Thies, Justus and Sitzmann, Vincent and Lombardi, Stephen and Sunkavalli, Kalyan and Martin-Brualla, Ricardo and Simon, Tomas and Saragih, Jason and Nie{\ss}ner, Matthias and Pandey, Rohit and Fanello, Sean and Wetzstein, Gordon and Zhu, Jun-Yan and Theobalt, Christian and Agrawala, Maneesh and Shechtman, Eli and Goldman, Dan B and Zollh{\"o}fer, Michael},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2004.03805},
EPRINT = {2004.03805},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Efficient rendering of photo-realistic virtual worlds is a long standing<br>effort of computer graphics. Modern graphics techniques have succeeded in<br>synthesizing photo-realistic images from hand-crafted scene representations.<br>However, the automatic generation of shape, materials, lighting, and other<br>aspects of scenes remains a challenging problem that, if solved, would make<br>photo-realistic computer graphics more widely accessible. Concurrently,<br>progress in computer vision and machine learning have given rise to a new<br>approach to image synthesis and editing, namely deep generative models. Neural<br>rendering is a new and rapidly emerging field that combines generative machine<br>learning techniques with physical knowledge from computer graphics, e.g., by<br>the integration of differentiable rendering into network training. With a<br>plethora of applications in computer graphics and vision, neural rendering is<br>poised to become a new area in the graphics community, yet no survey of this<br>emerging field exists. This state-of-the-art report summarizes the recent<br>trends and applications of neural rendering. We focus on approaches that<br>combine classic computer graphics techniques with deep generative models to<br>obtain controllable and photo-realistic outputs. Starting with an overview of<br>the underlying computer graphics and machine learning concepts, we discuss<br>critical aspects of neural rendering approaches. This state-of-the-art report<br>is focused on the many important use cases for the described algorithms such as<br>novel view synthesis, semantic photo manipulation, facial and body reenactment,<br>relighting, free-viewpoint video, and the creation of photo-realistic avatars<br>for virtual and augmented reality telepresence. Finally, we conclude with a<br>discussion of the social implications of such technology and investigate open<br>research problems.<br>},
}

Endnote

%0 Report
%A Tewari, Ayush
%A Fried, Ohad
%A Thies, Justus
%A Sitzmann, Vincent
%A Lombardi, Stephen
%A Sunkavalli, Kalyan
%A Martin-Brualla, Ricardo
%A Simon, Tomas
%A Saragih, Jason
%A Nie&#223;ner, Matthias
%A Pandey, Rohit
%A Fanello, Sean
%A Wetzstein, Gordon
%A Zhu, Jun-Yan
%A Theobalt, Christian
%A Agrawala, Maneesh
%A Shechtman, Eli
%A Goldman, Dan B
%A Zollh&#246;fer, Michael
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
%T State of the Art on Neural Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E114-4
%U https://arxiv.org/abs/2004.03805
%D 2020
%X   Efficient rendering of photo-realistic virtual worlds is a long standing<br>effort of computer graphics. Modern graphics techniques have succeeded in<br>synthesizing photo-realistic images from hand-crafted scene representations.<br>However, the automatic generation of shape, materials, lighting, and other<br>aspects of scenes remains a challenging problem that, if solved, would make<br>photo-realistic computer graphics more widely accessible. Concurrently,<br>progress in computer vision and machine learning have given rise to a new<br>approach to image synthesis and editing, namely deep generative models. Neural<br>rendering is a new and rapidly emerging field that combines generative machine<br>learning techniques with physical knowledge from computer graphics, e.g., by<br>the integration of differentiable rendering into network training. With a<br>plethora of applications in computer graphics and vision, neural rendering is<br>poised to become a new area in the graphics community, yet no survey of this<br>emerging field exists. This state-of-the-art report summarizes the recent<br>trends and applications of neural rendering. We focus on approaches that<br>combine classic computer graphics techniques with deep generative models to<br>obtain controllable and photo-realistic outputs. Starting with an overview of<br>the underlying computer graphics and machine learning concepts, we discuss<br>critical aspects of neural rendering approaches. This state-of-the-art report<br>is focused on the many important use cases for the described algorithms such as<br>novel view synthesis, semantic photo manipulation, facial and body reenactment,<br>relighting, free-viewpoint video, and the creation of photo-realistic avatars<br>for virtual and augmented reality telepresence. Finally, we conclude with a<br>discussion of the social implications of such technology and investigate open<br>research problems.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Paper

A. Tewari, M. Elgharib, G. Bharaj, F. Bernard, H.-P. Seidel, P. Pérez, M. Zollhöfer, and C. Theobalt

“StyleRig: Rigging StyleGAN for 3D Control over Portrait Images,” 2020. [Online]. Available: https://arxiv.org/abs/2004.00121.

Abstract

StyleGAN generates photorealistic portrait images of faces with eyes, teeth,
hair and context (neck, shoulders, background), but lacks a rig-like control
over semantic face parameters that are interpretable in 3D, such as face pose,
expressions, and scene illumination. Three-dimensional morphable face models
(3DMMs) on the other hand offer control over the semantic parameters, but lack
photorealism when rendered and only model the face interior, not other parts of
a portrait image (hair, mouth interior, background). We present the first
method to provide a face rig-like control over a pretrained and fixed StyleGAN
via a 3DMM. A new rigging network, RigNet is trained between the 3DMM's
semantic parameters and StyleGAN's input. The network is trained in a
self-supervised manner, without the need for manual annotations. At test time,
our method generates portrait images with the photorealism of StyleGAN and
provides explicit control over the 3D semantic parameters of the face.

BibTeX

@online{Tewari_2004.00121,
TITLE = {{StyleRig}: Rigging {StyleGAN} for {3D} Control over Portrait Images},
AUTHOR = {Tewari, Ayush and Elgharib, Mohamed and Bharaj, Gaurav and Bernard, Florian and Seidel, Hans-Peter and P{\'e}rez, Patrick and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2004.00121},
EPRINT = {2004.00121},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {StyleGAN generates photorealistic portrait images of faces with eyes, teeth,<br>hair and context (neck, shoulders, background), but lacks a rig-like control<br>over semantic face parameters that are interpretable in 3D, such as face pose,<br>expressions, and scene illumination. Three-dimensional morphable face models<br>(3DMMs) on the other hand offer control over the semantic parameters, but lack<br>photorealism when rendered and only model the face interior, not other parts of<br>a portrait image (hair, mouth interior, background). We present the first<br>method to provide a face rig-like control over a pretrained and fixed StyleGAN<br>via a 3DMM. A new rigging network, RigNet is trained between the 3DMM's<br>semantic parameters and StyleGAN's input. The network is trained in a<br>self-supervised manner, without the need for manual annotations. At test time,<br>our method generates portrait images with the photorealism of StyleGAN and<br>provides explicit control over the 3D semantic parameters of the face.<br>},
}

Endnote

%0 Report
%A Tewari, Ayush
%A Elgharib, Mohamed
%A Bharaj, Gaurav
%A Bernard, Florian
%A Seidel, Hans-Peter
%A P&#233;rez, Patrick
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T StyleRig: Rigging StyleGAN for 3D Control over Portrait Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B0FC-6
%U https://arxiv.org/abs/2004.00121
%D 2020
%X   StyleGAN generates photorealistic portrait images of faces with eyes, teeth,<br>hair and context (neck, shoulders, background), but lacks a rig-like control<br>over semantic face parameters that are interpretable in 3D, such as face pose,<br>expressions, and scene illumination. Three-dimensional morphable face models<br>(3DMMs) on the other hand offer control over the semantic parameters, but lack<br>photorealism when rendered and only model the face interior, not other parts of<br>a portrait image (hair, mouth interior, background). We present the first<br>method to provide a face rig-like control over a pretrained and fixed StyleGAN<br>via a 3DMM. A new rigging network, RigNet is trained between the 3DMM's<br>semantic parameters and StyleGAN's input. The network is trained in a<br>self-supervised manner, without the need for manual annotations. At test time,<br>our method generates portrait images with the photorealism of StyleGAN and<br>provides explicit control over the 3D semantic parameters of the face.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Article

A. Tewari, M. Zollhöfer, F. Bernard, P. Garrido, H. Kim, P. Pérez, and C. Theobalt

“High-Fidelity Monocular Face Reconstruction based on an Unsupervised Model-based Face Autoencoder,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, 2020.

@article{8496850,
TITLE = {High-Fidelity Monocular Face Reconstruction based on an Unsupervised Model-based Face Autoencoder},
AUTHOR = {Tewari, Ayush and Zollh{\"o}fer, Michael and Bernard, Florian and Garrido, Pablo and Kim, Hyeongwoo and P{\'e}rez, Patrick and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0162-8828},
DOI = {10.1109/TPAMI.2018.2876842},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2020},
DATE = {2020},
JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
VOLUME = {42},
NUMBER = {2},
PAGES = {357--370},
}

Endnote

%0 Journal Article
%A Tewari, Ayush
%A Zollh&#246;fer, Michael
%A Bernard, Florian
%A Garrido, Pablo
%A Kim, Hyeongwoo
%A P&#233;rez, Patrick
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T High-Fidelity Monocular Face Reconstruction based on an Unsupervised Model-based Face Autoencoder : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-EF5B-C
%R 10.1109/TPAMI.2018.2876842
%7 2018
%D 2020
%J IEEE Transactions on Pattern Analysis and Machine Intelligence
%O IEEE Trans. Pattern Anal. Mach. Intell.
%V 42
%N 2
%& 357
%P 357 - 370
%I IEEE
%C Piscataway, NJ
%@ false

Article

“State of the Art on Neural Rendering,” Computer Graphics Forum (Proc. EUROGRAPHICS 2020), vol. 39, no. 2, 2020.

@article{Tewari_EG2020,
TITLE = {State of the Art on Neural Rendering},
AUTHOR = {Tewari, Ayush and Fried, Ohad and Thies, Justus and Sitzmann, Vincent and Lombardi, Stephen and Sunkavalli, Kalyan and Martin-Brualla, Ricardo and Simon, Tomas and Saragih, Jason and Nie{\ss}ner, Matthias and Pandey, Rohit and Fanello, Sean and Wetzstein, Gordon and Zhu, Jun-Yan and Theobalt, Christian and Agrawala, Maneesh and Shechtman, Eli and Goldman, Dan B. and Zollh{\"o}fer, Michael},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.14022},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2020},
DATE = {2020},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {39},
NUMBER = {2},
PAGES = {701--727},
BOOKTITLE = {The European Association for Computer Graphics 41st Annual Conference (EUROGRAPHICS 2020)},
EDITOR = {Panozzo, Daniele and Assarsson, Ulf},
}

Endnote

%0 Journal Article
%A Tewari, Ayush
%A Fried, Ohad
%A Thies, Justus
%A Sitzmann, Vincent
%A Lombardi, Stephen
%A Sunkavalli, Kalyan
%A Martin-Brualla, Ricardo
%A Simon, Tomas
%A Saragih, Jason
%A Nie&#223;ner, Matthias
%A Pandey, Rohit
%A Fanello, Sean
%A Wetzstein, Gordon
%A Zhu, Jun-Yan
%A Theobalt, Christian
%A Agrawala, Maneesh
%A Shechtman, Eli
%A Goldman, Dan B.
%A Zollh&#246;fer, Michael
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
%T State of the Art on Neural Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-DB93-D
%R 10.1111/cgf.14022
%7 2020
%D 2020
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 39
%N 2
%& 701
%P 701 - 727
%I Blackwell-Wiley
%C Oxford
%@ false
%B The European Association for Computer Graphics 41st Annual Conference
%O EUROGRAPHICS 2020 EG 2020 The European Association for Computer Graphics 41st Annual Conference ; Norrk&#246;ping, Sweden, May 25 &#8211; 29, 2020

Article

A. Tewari, M. Elgharib

Mallikarjun B R, F. Bernard, H.-P. Seidel, P. Pérez, M. Zollhöfer, and C. Theobalt, “PIE: Portrait Image Embedding for Semantic Control,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Tewari_ToG2020,
TITLE = {{PIE}: {P}ortrait Image Embedding for Semantic Control},
AUTHOR = {Tewari, Ayush and Elgharib, Mohamed and Mallikarjun B R and Bernard, Florian and Seidel, Hans-Peter and P{\'e}rez, Patrick and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417803},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {223},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Tewari, Ayush
%A Elgharib, Mohamed
%A Mallikarjun B R, 
%A Bernard, Florian
%A Seidel, Hans-Peter
%A P&#233;rez, Patrick
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T PIE: Portrait Image Embedding for Semantic Control : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-9B0C-E
%R 10.1145/3414685.3417803
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 223
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Paper

A. Tewari, M. Elgharib

Mallikarjun B R, F. Bernard, H.-P. Seidel, P. Pérez, M. Zollhöfer, and C. Theobalt, “PIE: Portrait Image Embedding for Semantic Control,” 2020. [Online]. Available: https://arxiv.org/abs/2009.09485.

Abstract

Editing of portrait images is a very popular and important research topic
with a large variety of applications. For ease of use, control should be
provided via a semantically meaningful parameterization that is akin to
computer animation controls. The vast majority of existing techniques do not
provide such intuitive and fine-grained control, or only enable coarse editing
of a single isolated control parameter. Very recently, high-quality
semantically controlled editing has been demonstrated, however only on
synthetically created StyleGAN images. We present the first approach for
embedding real portrait images in the latent space of StyleGAN, which allows
for intuitive editing of the head pose, facial expression, and scene
illumination in the image. Semantic editing in parameter space is achieved
based on StyleRig, a pretrained neural network that maps the control space of a
3D morphable face model to the latent space of the GAN. We design a novel
hierarchical non-linear optimization problem to obtain the embedding. An
identity preservation energy term allows spatially coherent edits while
maintaining facial integrity. Our approach runs at interactive frame rates and
thus allows the user to explore the space of possible edits. We evaluate our
approach on a wide set of portrait photos, compare it to the current state of
the art, and validate the effectiveness of its components in an ablation study.

BibTeX

@online{Tewari_2009.09485,
TITLE = {{PIE}: {P}ortrait Image Embedding for Semantic Control},
AUTHOR = {Tewari, Ayush and Elgharib, Mohamed and Mallikarjun B R and Bernard, Florian and Seidel, Hans-Peter and P{\'e}rez, Patrick and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2009.09485},
EPRINT = {2009.09485},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Editing of portrait images is a very popular and important research topic<br>with a large variety of applications. For ease of use, control should be<br>provided via a semantically meaningful parameterization that is akin to<br>computer animation controls. The vast majority of existing techniques do not<br>provide such intuitive and fine-grained control, or only enable coarse editing<br>of a single isolated control parameter. Very recently, high-quality<br>semantically controlled editing has been demonstrated, however only on<br>synthetically created StyleGAN images. We present the first approach for<br>embedding real portrait images in the latent space of StyleGAN, which allows<br>for intuitive editing of the head pose, facial expression, and scene<br>illumination in the image. Semantic editing in parameter space is achieved<br>based on StyleRig, a pretrained neural network that maps the control space of a<br>3D morphable face model to the latent space of the GAN. We design a novel<br>hierarchical non-linear optimization problem to obtain the embedding. An<br>identity preservation energy term allows spatially coherent edits while<br>maintaining facial integrity. Our approach runs at interactive frame rates and<br>thus allows the user to explore the space of possible edits. We evaluate our<br>approach on a wide set of portrait photos, compare it to the current state of<br>the art, and validate the effectiveness of its components in an ablation study.<br>},
}

Endnote

%0 Report
%A Tewari, Ayush
%A Elgharib, Mohamed
%A Mallikarjun B R, 
%A Bernard, Florian
%A Seidel, Hans-Peter
%A P&#233;rez, Patrick
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T PIE: Portrait Image Embedding for Semantic Control : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B117-7
%U https://arxiv.org/abs/2009.09485
%D 2020
%X   Editing of portrait images is a very popular and important research topic<br>with a large variety of applications. For ease of use, control should be<br>provided via a semantically meaningful parameterization that is akin to<br>computer animation controls. The vast majority of existing techniques do not<br>provide such intuitive and fine-grained control, or only enable coarse editing<br>of a single isolated control parameter. Very recently, high-quality<br>semantically controlled editing has been demonstrated, however only on<br>synthetically created StyleGAN images. We present the first approach for<br>embedding real portrait images in the latent space of StyleGAN, which allows<br>for intuitive editing of the head pose, facial expression, and scene<br>illumination in the image. Semantic editing in parameter space is achieved<br>based on StyleRig, a pretrained neural network that maps the control space of a<br>3D morphable face model to the latent space of the GAN. We design a novel<br>hierarchical non-linear optimization problem to obtain the embedding. An<br>identity preservation energy term allows spatially coherent edits while<br>maintaining facial integrity. Our approach runs at interactive frame rates and<br>thus allows the user to explore the space of possible edits. We evaluate our<br>approach on a wide set of portrait photos, compare it to the current state of<br>the art, and validate the effectiveness of its components in an ablation study.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Paper

J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner

“Face2Face: Real-time Face Capture and Reenactment of RGB Videos,” 2020. [Online]. Available: https://arxiv.org/abs/2007.14808.

Abstract

We present Face2Face, a novel approach for real-time facial reenactment of a
monocular target video sequence (e.g., Youtube video). The source sequence is
also a monocular video stream, captured live with a commodity webcam. Our goal
is to animate the facial expressions of the target video by a source actor and
re-render the manipulated output video in a photo-realistic fashion. To this
end, we first address the under-constrained problem of facial identity recovery
from monocular video by non-rigid model-based bundling. At run time, we track
facial expressions of both source and target video using a dense photometric
consistency measure. Reenactment is then achieved by fast and efficient
deformation transfer between source and target. The mouth interior that best
matches the re-targeted expression is retrieved from the target sequence and
warped to produce an accurate fit. Finally, we convincingly re-render the
synthesized target face on top of the corresponding video stream such that it
seamlessly blends with the real-world illumination. We demonstrate our method
in a live setup, where Youtube videos are reenacted in real time.

BibTeX

@online{Thies_2007.14808,
TITLE = {{Face2Face}: {R}eal-time Face Capture and Reenactment of {RGB} Videos},
AUTHOR = {Thies, Justus and Zollh{\"o}fer, Michael and Stamminger, Marc and Theobalt, Christian and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2007.14808},
EPRINT = {2007.14808},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present Face2Face, a novel approach for real-time facial reenactment of a<br>monocular target video sequence (e.g., Youtube video). The source sequence is<br>also a monocular video stream, captured live with a commodity webcam. Our goal<br>is to animate the facial expressions of the target video by a source actor and<br>re-render the manipulated output video in a photo-realistic fashion. To this<br>end, we first address the under-constrained problem of facial identity recovery<br>from monocular video by non-rigid model-based bundling. At run time, we track<br>facial expressions of both source and target video using a dense photometric<br>consistency measure. Reenactment is then achieved by fast and efficient<br>deformation transfer between source and target. The mouth interior that best<br>matches the re-targeted expression is retrieved from the target sequence and<br>warped to produce an accurate fit. Finally, we convincingly re-render the<br>synthesized target face on top of the corresponding video stream such that it<br>seamlessly blends with the real-world illumination. We demonstrate our method<br>in a live setup, where Youtube videos are reenacted in real time.<br>},
}

Endnote

%0 Report
%A Thies, Justus
%A Zollh&#246;fer, Michael
%A Stamminger, Marc
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Face2Face: Real-time Face Capture and Reenactment of RGB Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E8E9-D
%U https://arxiv.org/abs/2007.14808
%D 2020
%X   We present Face2Face, a novel approach for real-time facial reenactment of a<br>monocular target video sequence (e.g., Youtube video). The source sequence is<br>also a monocular video stream, captured live with a commodity webcam. Our goal<br>is to animate the facial expressions of the target video by a source actor and<br>re-render the manipulated output video in a photo-realistic fashion. To this<br>end, we first address the under-constrained problem of facial identity recovery<br>from monocular video by non-rigid model-based bundling. At run time, we track<br>facial expressions of both source and target video using a dense photometric<br>consistency measure. Reenactment is then achieved by fast and efficient<br>deformation transfer between source and target. The mouth interior that best<br>matches the re-targeted expression is retrieved from the target sequence and<br>warped to produce an accurate fit. Finally, we convincingly re-render the<br>synthesized target face on top of the corresponding video stream such that it<br>seamlessly blends with the real-world illumination. We demonstrate our method<br>in a live setup, where Youtube videos are reenacted in real time.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

J. Thies, M. Elgharib, A. Tewari, C. Theobalt, and M. Nießner

“Neural Voice Puppetry: Audio-Driven Facial Reenactment,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Thies_ECCV20,
TITLE = {Neural Voice Puppetry: {A}udio-Driven Facial Reenactment},
AUTHOR = {Thies, Justus and Elgharib, Mohamed and Tewari, Ayush and Theobalt, Christian and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
ISBN = {978-3-030-58516-7},
DOI = {10.1007/978-3-030-58517-4_42},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {716--731},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12361},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Thies, Justus
%A Elgharib, Mohamed
%A Tewari, Ayush
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Neural Voice Puppetry: Audio-Driven Facial Reenactment : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D42F-6
%R 10.1007/978-3-030-58517-4_42
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 716 - 731
%I Springer
%@ 978-3-030-58516-7
%B Lecture Notes in Computer Science
%N 12361

Conference paper

J. Thies, M. Zollhöfer, C. Theobalt, M. Stamminger, and M. Nießner

“Image-guided Neural Object Rendering,” in International Conference on Learning Representations (ICLR 2020), Addis Ababa, Ethopia, 2020.

@inproceedings{Thies_ICLR2020,
TITLE = {Image-guided Neural Object Rendering},
AUTHOR = {Thies, Justus and Zollh{\"o}fer, Michael and Theobalt, Christian and Stamminger, Marc and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
URL = {https://openreview.net/forum?id=Hyg9anEFPS; https://iclr.cc/Conferences/2020},
PUBLISHER = {OpenReview.net},
YEAR = {2020},
BOOKTITLE = {International Conference on Learning Representations (ICLR 2020)},
ADDRESS = {Addis Ababa, Ethopia},
}

Endnote

%0 Conference Proceedings
%A Thies, Justus
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%A Stamminger, Marc
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Image-guided Neural Object Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D431-2
%U https://openreview.net/forum?id=Hyg9anEFPS
%D 2020
%B 8th International Conference on Learning Representations
%Z date of event: 2020-04-26 - 2020-04-30
%C Addis Ababa, Ethopia
%B International Conference on Learning Representations
%I OpenReview.net


%U https://openreview.net/forum?id=Hyg9anEFPS

Article

X. Tong, K. Myszkowski, and J. Huang

“Foreword to the Special Section on the International Conference on Computer-Aided Design and Computer Graphics (CAD/Graphics) 2019,” Computers and Graphics, vol. 86, 2020.

@article{Tong_CAD19,
TITLE = {Foreword to the Special Section on the {International Conference on Computer-Aided Design and Computer Graphics (CAD/Graphics)} 2019},
AUTHOR = {Tong, Xin and Myszkowski, Karol and Huang, Jin},
LANGUAGE = {eng},
ISSN = {0097-8493},
DOI = {10.1016/j.cag.2019.12.002},
PUBLISHER = {Elsevier},
ADDRESS = {Amsterdam},
YEAR = {2020},
DATE = {2020},
JOURNAL = {Computers and Graphics},
VOLUME = {86},
PAGES = {A5--A6},
}

Endnote

%0 Journal Article
%A Tong, Xin
%A Myszkowski, Karol
%A Huang, Jin
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Foreword to the Special Section on the International Conference on Computer-Aided Design and Computer Graphics (CAD/Graphics) 2019 : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CEAF-D
%R 10.1016/j.cag.2019.12.002
%7 2019
%D 2020
%J Computers and Graphics
%V 86
%& A5
%P A5 - A6
%I Elsevier
%C Amsterdam
%@ false

Paper

E. Tretschk, A. Tewari, V. Golyanik, M. Zollhöfer, C. Stoll, and C. Theobalt

“PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations,” 2020. [Online]. Available: https://arxiv.org/abs/2008.01639.

Abstract

Implicit surface representations, such as signed-distance functions, combined
with deep learning have led to impressive models which can represent detailed
shapes of objects with arbitrary topology. Since a continuous function is
learned, the reconstructions can also be extracted at any arbitrary resolution.
However, large datasets such as ShapeNet are required to train such models. In
this paper, we present a new mid-level patch-based surface representation. At
the level of patches, objects across different categories share similarities,
which leads to more generalizable models. We then introduce a novel method to
learn this patch-based representation in a canonical space, such that it is as
object-agnostic as possible. We show that our representation trained on one
category of objects from ShapeNet can also well represent detailed shapes from
any other category. In addition, it can be trained using much fewer shapes,
compared to existing approaches. We show several applications of our new
representation, including shape interpolation and partial point cloud
completion. Due to explicit control over positions, orientations and scales of
patches, our representation is also more controllable compared to object-level
representations, which enables us to deform encoded shapes non-rigidly.

BibTeX

@online{Tretschk_2008.01639,
TITLE = {{PatchNets}: {P}atch-Based Generalizable Deep Implicit {3D} Shape Representations},
AUTHOR = {Tretschk, Edgar and Tewari, Ayush and Golyanik, Vladislav and Zollh{\"o}fer, Michael and Stoll, Carsten and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2008.01639},
EPRINT = {2008.01639},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Implicit surface representations, such as signed-distance functions, combined<br>with deep learning have led to impressive models which can represent detailed<br>shapes of objects with arbitrary topology. Since a continuous function is<br>learned, the reconstructions can also be extracted at any arbitrary resolution.<br>However, large datasets such as ShapeNet are required to train such models. In<br>this paper, we present a new mid-level patch-based surface representation. At<br>the level of patches, objects across different categories share similarities,<br>which leads to more generalizable models. We then introduce a novel method to<br>learn this patch-based representation in a canonical space, such that it is as<br>object-agnostic as possible. We show that our representation trained on one<br>category of objects from ShapeNet can also well represent detailed shapes from<br>any other category. In addition, it can be trained using much fewer shapes,<br>compared to existing approaches. We show several applications of our new<br>representation, including shape interpolation and partial point cloud<br>completion. Due to explicit control over positions, orientations and scales of<br>patches, our representation is also more controllable compared to object-level<br>representations, which enables us to deform encoded shapes non-rigidly.<br>},
}

Endnote

%0 Report
%A Tretschk, Edgar
%A Tewari, Ayush
%A Golyanik, Vladislav
%A Zollh&#246;fer, Michael
%A Stoll, Carsten
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape
  Representations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E8ED-9
%U https://arxiv.org/abs/2008.01639
%D 2020
%X   Implicit surface representations, such as signed-distance functions, combined<br>with deep learning have led to impressive models which can represent detailed<br>shapes of objects with arbitrary topology. Since a continuous function is<br>learned, the reconstructions can also be extracted at any arbitrary resolution.<br>However, large datasets such as ShapeNet are required to train such models. In<br>this paper, we present a new mid-level patch-based surface representation. At<br>the level of patches, objects across different categories share similarities,<br>which leads to more generalizable models. We then introduce a novel method to<br>learn this patch-based representation in a canonical space, such that it is as<br>object-agnostic as possible. We show that our representation trained on one<br>category of objects from ShapeNet can also well represent detailed shapes from<br>any other category. In addition, it can be trained using much fewer shapes,<br>compared to existing approaches. We show several applications of our new<br>representation, including shape interpolation and partial point cloud<br>completion. Due to explicit control over positions, orientations and scales of<br>patches, our representation is also more controllable compared to object-level<br>representations, which enables us to deform encoded shapes non-rigidly.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Conference paper

E. Tretschk, A. Tewari, V. Golyanik, M. Zollhöfer, C. Stoll, and C. Theobalt

“PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Tretschk_ECCV20a,
TITLE = {{PatchNets}: {P}atch-Based Generalizable Deep Implicit {3D} Shape Representations},
AUTHOR = {Tretschk, Edgar and Tewari, Ayush and Golyanik, Vladislav and Zollh{\"o}fer, Michael and Stoll, Carsten and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-3-030-58516-7},
DOI = {10.1007/978-3-030-58517-4_18},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {293--309},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12361},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Tretschk, Edgar
%A Tewari, Ayush
%A Golyanik, Vladislav
%A Zollh&#246;fer, Michael
%A Stoll, Carsten
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T PatchNets: Patch-Based Generalizable Deep Implicit 3D Shape Representations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D095-5
%R 10.1007/978-3-030-58517-4_18
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 293 - 309
%I Springer
%@ 978-3-030-58516-7
%B Lecture Notes in Computer Science
%N 12361

Paper

E. Tretschk, A. Tewari, V. Golyanik, M. Zollhöfer, C. Lassner, and C. Theobalt

“Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video,” 2020. [Online]. Available: https://arxiv.org/abs/2012.12247.

Abstract

In this tech report, we present the current state of our ongoing work on
reconstructing Neural Radiance Fields (NERF) of general non-rigid scenes via
ray bending. Non-rigid NeRF (NR-NeRF) takes RGB images of a deforming object
(e.g., from a monocular video) as input and then learns a geometry and
appearance representation that not only allows to reconstruct the input
sequence but also to re-render any time step into novel camera views with high
fidelity. In particular, we show that a consumer-grade camera is sufficient to
synthesize convincing bullet-time videos of short and simple scenes. In
addition, the resulting representation enables correspondence estimation across
views and time, and provides rigidity scores for each point in the scene. We
urge the reader to watch the supplemental videos for qualitative results. We
will release our code.

BibTeX

@online{Tretschk_2012.12247,
TITLE = {Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video},
AUTHOR = {Tretschk, Edgar and Tewari, Ayush and Golyanik, Vladislav and Zollh{\"o}fer, Michael and Lassner, Christoph and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2012.12247},
EPRINT = {2012.12247},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {In this tech report, we present the current state of our ongoing work on<br>reconstructing Neural Radiance Fields (NERF) of general non-rigid scenes via<br>ray bending. Non-rigid NeRF (NR-NeRF) takes RGB images of a deforming object<br>(e.g., from a monocular video) as input and then learns a geometry and<br>appearance representation that not only allows to reconstruct the input<br>sequence but also to re-render any time step into novel camera views with high<br>fidelity. In particular, we show that a consumer-grade camera is sufficient to<br>synthesize convincing bullet-time videos of short and simple scenes. In<br>addition, the resulting representation enables correspondence estimation across<br>views and time, and provides rigidity scores for each point in the scene. We<br>urge the reader to watch the supplemental videos for qualitative results. We<br>will release our code.<br>},
}

Endnote

%0 Report
%A Tretschk, Edgar
%A Tewari, Ayush
%A Golyanik, Vladislav
%A Zollh&#246;fer, Michael
%A Lassner, Christoph
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Non-Rigid Neural Radiance Fields: Reconstruction and Novel View
  Synthesis of a Deforming Scene from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-EA00-1
%U https://arxiv.org/abs/2012.12247
%D 2020
%X   In this tech report, we present the current state of our ongoing work on<br>reconstructing Neural Radiance Fields (NERF) of general non-rigid scenes via<br>ray bending. Non-rigid NeRF (NR-NeRF) takes RGB images of a deforming object<br>(e.g., from a monocular video) as input and then learns a geometry and<br>appearance representation that not only allows to reconstruct the input<br>sequence but also to re-render any time step into novel camera views with high<br>fidelity. In particular, we show that a consumer-grade camera is sufficient to<br>synthesize convincing bullet-time videos of short and simple scenes. In<br>addition, the resulting representation enables correspondence estimation across<br>views and time, and provides rigidity scores for each point in the scene. We<br>urge the reader to watch the supplemental videos for qualitative results. We<br>will release our code.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Conference paper

E. Tretschk, A. Tewari, M. Zollhöfer, V. Golyanik, and C. Theobalt

“DEMEA: Deep Mesh Autoencoders for Non-rigidly Deforming Objects,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Tretschk_ECCV20,
TITLE = {{DEMEA}: {D}eep Mesh Autoencoders for Non-rigidly Deforming Objects},
AUTHOR = {Tretschk, Edgar and Tewari, Ayush and Zollh{\"o}fer, Michael and Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-3-030-58516-7},
DOI = {10.1007/978-3-030-58548-8_35},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {601--617},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12349},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Tretschk, Edgar
%A Tewari, Ayush
%A Zollh&#246;fer, Michael
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T DEMEA: Deep Mesh Autoencoders for Non-rigidly Deforming Objects : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-D425-0
%R 10.1007/978-3-030-58548-8_35
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 601 - 617
%I Springer
%@ 978-3-030-58516-7
%B Lecture Notes in Computer Science
%N 12349

Conference paper

J. Wang, F. Mueller, F. Bernard, and C. Theobalt

“Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation,” in 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), Buenos Aires, Argentina, 2020.

@inproceedings{Wang_FG2020,
TITLE = {Generative Model-Based Loss to the Rescue: {A} Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation},
AUTHOR = {Wang, Jiayi and Mueller, Franziska and Bernard, Florian and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3079-8},
DOI = {10.1109/FG47880.2020.00013.},
PUBLISHER = {IEEE},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020)},
EDITOR = {{\v S}truc, Vitomir and G{\'o}mez-Fern{\'a}ndez, Francisco},
PAGES = {101--108},
ADDRESS = {Buenos Aires, Argentina},
}

Endnote

%0 Conference Proceedings
%A Wang, Jiayi
%A Mueller, Franziska
%A Bernard, Florian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-1687-7
%R 10.1109/FG47880.2020.00013.
%D 2020
%B 15th IEEE International Conference on Automatic Face and Gesture Recognition
%Z date of event: 2020-11-16 - 2020-11-20
%C Buenos Aires, Argentina
%B 15th IEEE International Conference on Automatic Face and Gesture Recognition
%E &#352;truc, Vitomir; G&#243;mez-Fern&#225;ndez, Francisco
%P 101 - 108
%I IEEE
%@ 978-1-7281-3079-8

Article

J. Wang, F. Mueller, F. Bernard, S. Sorli, O. Sotnychenko, N. Qian, M. A. Otaduy, D. Casas, and C. Theobalt

“RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Wang_ToG2020,
TITLE = {{RGB2Hands}: {R}eal-Time Tracking of {3D} Hand Interactions from Monocular {RGB} Video},
AUTHOR = {Wang, Jiayi and Mueller, Franziska and Bernard, Florian and Sorli, Suzanne and Sotnychenko, Oleksandr and Qian, Neng and Otaduy, Miguel A. and Casas, Dan and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417852},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {218},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Wang, Jiayi
%A Mueller, Franziska
%A Bernard, Florian
%A Sorli, Suzanne
%A Sotnychenko, Oleksandr
%A Qian, Neng
%A Otaduy, Miguel A.
%A Casas, Dan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CF20-C
%R 10.1145/3414685.3417852
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 218
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2020

%O ACM SIGGRAPH Asia 2020 SA'20 SA 2020

Paper

J. Wang, F. Mueller, F. Bernard, and C. Theobalt

“Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation,” 2020. [Online]. Available: https://arxiv.org/abs/2007.03073.

Abstract

We propose to use a model-based generative loss for training hand pose
estimators on depth images based on a volumetric hand model. This additional
loss allows training of a hand pose estimator that accurately infers the entire
set of 21 hand keypoints while only using supervision for 6 easy-to-annotate
keypoints (fingertips and wrist). We show that our partially-supervised method
achieves results that are comparable to those of fully-supervised methods which
enforce articulation consistency. Moreover, for the first time we demonstrate
that such an approach can be used to train on datasets that have erroneous
annotations, i.e. "ground truth" with notable measurement errors, while
obtaining predictions that explain the depth images better than the given
"ground truth".

BibTeX

@online{Wang_2007.03073,
TITLE = {Generative Model-Based Loss to the Rescue: {A} Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation},
AUTHOR = {Wang, Jiayi and Mueller, Franziska and Bernard, Florian and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2007.03073},
EPRINT = {2007.03073},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We propose to use a model-based generative loss for training hand pose<br>estimators on depth images based on a volumetric hand model. This additional<br>loss allows training of a hand pose estimator that accurately infers the entire<br>set of 21 hand keypoints while only using supervision for 6 easy-to-annotate<br>keypoints (fingertips and wrist). We show that our partially-supervised method<br>achieves results that are comparable to those of fully-supervised methods which<br>enforce articulation consistency. Moreover, for the first time we demonstrate<br>that such an approach can be used to train on datasets that have erroneous<br>annotations, i.e. "ground truth" with notable measurement errors, while<br>obtaining predictions that explain the depth images better than the given<br>"ground truth".<br>},
}

Endnote

%0 Report
%A Wang, Jiayi
%A Mueller, Franziska
%A Bernard, Florian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Generative Model-Based Loss to the Rescue: A Method to Overcome Annotation Errors for Depth-Based Hand Pose Estimation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E89A-6
%U https://arxiv.org/abs/2007.03073
%D 2020
%X   We propose to use a model-based generative loss for training hand pose<br>estimators on depth images based on a volumetric hand model. This additional<br>loss allows training of a hand pose estimator that accurately infers the entire<br>set of 21 hand keypoints while only using supervision for 6 easy-to-annotate<br>keypoints (fingertips and wrist). We show that our partially-supervised method<br>achieves results that are comparable to those of fully-supervised methods which<br>enforce articulation consistency. Moreover, for the first time we demonstrate<br>that such an approach can be used to train on datasets that have erroneous<br>annotations, i.e. "ground truth" with notable measurement errors, while<br>obtaining predictions that explain the depth images better than the given<br>"ground truth".<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

P. Wang, L. Liu, N. Chen, H.-K. Chu, C. Theobalt, and W. Wang

“Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2020), vol. 39, no. 4, 2020.

@article{Wang_SIGGRAPH2020,
TITLE = {{Vid2Curve}: {S}imultaneous Camera Motion Estimation and Thin Structure Reconstruction from an {RGB} Video},
AUTHOR = {Wang, Peng and Liu, Lingjie and Chen, Nenglun and Chu, Hung-Kuo and Theobalt, Christian and Wang, Wenping},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3386569.3392476},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {39},
NUMBER = {4},
EID = {132},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2020},
}

Endnote

%0 Journal Article
%A Wang, Peng
%A Liu, Lingjie
%A Chen, Nenglun
%A Chu, Hung-Kuo
%A Theobalt, Christian
%A Wang, Wenping
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-9A74-9
%R 10.1145/3386569.3392476
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 4
%Z sequence number: 132
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2020
%O ACM SIGGRAPH 2020 Virtual Conference ; 2020, 17-28  August

Paper

P. Wang, L. Liu, N. Chen, H.-K. Chu, C. Theobalt, and W. Wang

“Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video,” 2020. [Online]. Available: https://arxiv.org/abs/2005.03372.

Abstract

Thin structures, such as wire-frame sculptures, fences, cables, power lines,
and tree branches, are common in the real world. It is extremely challenging to
acquire their 3D digital models using traditional image-based or depth-based
reconstruction methods because thin structures often lack distinct point
features and have severe self-occlusion. We propose the first approach that
simultaneously estimates camera motion and reconstructs the geometry of complex
3D thin structures in high quality from a color video captured by a handheld
camera. Specifically, we present a new curve-based approach to estimate
accurate camera poses by establishing correspondences between featureless thin
objects in the foreground in consecutive video frames, without requiring visual
texture in the background scene to lock on. Enabled by this effective
curve-based camera pose estimation strategy, we develop an iterative
optimization method with tailored measures on geometry, topology as well as
self-occlusion handling for reconstructing 3D thin structures. Extensive
validations on a variety of thin structures show that our method achieves
accurate camera pose estimation and faithful reconstruction of 3D thin
structures with complex shape and topology at a level that has not been
attained by other existing reconstruction methods.

BibTeX

@online{Wang2005.03372,
TITLE = {{Vid2Curve}: {S}imultaneous Camera Motion Estimation and Thin Structure Reconstruction from an {RGB} Video},
AUTHOR = {Wang, Peng and Liu, Lingjie and Chen, Nenglun and Chu, Hung-Kuo and Theobalt, Christian and Wang, Wenping},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2005.03372},
EPRINT = {2005.03372},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Thin structures, such as wire-frame sculptures, fences, cables, power lines,<br>and tree branches, are common in the real world. It is extremely challenging to<br>acquire their 3D digital models using traditional image-based or depth-based<br>reconstruction methods because thin structures often lack distinct point<br>features and have severe self-occlusion. We propose the first approach that<br>simultaneously estimates camera motion and reconstructs the geometry of complex<br>3D thin structures in high quality from a color video captured by a handheld<br>camera. Specifically, we present a new curve-based approach to estimate<br>accurate camera poses by establishing correspondences between featureless thin<br>objects in the foreground in consecutive video frames, without requiring visual<br>texture in the background scene to lock on. Enabled by this effective<br>curve-based camera pose estimation strategy, we develop an iterative<br>optimization method with tailored measures on geometry, topology as well as<br>self-occlusion handling for reconstructing 3D thin structures. Extensive<br>validations on a variety of thin structures show that our method achieves<br>accurate camera pose estimation and faithful reconstruction of 3D thin<br>structures with complex shape and topology at a level that has not been<br>attained by other existing reconstruction methods.<br>},
}

Endnote

%0 Report
%A Wang, Peng
%A Liu, Lingjie
%A Chen, Nenglun
%A Chu, Hung-Kuo
%A Theobalt, Christian
%A Wang, Wenping
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure
  Reconstruction from an RGB Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E122-4
%U https://arxiv.org/abs/2005.03372
%D 2020
%X   Thin structures, such as wire-frame sculptures, fences, cables, power lines,<br>and tree branches, are common in the real world. It is extremely challenging to<br>acquire their 3D digital models using traditional image-based or depth-based<br>reconstruction methods because thin structures often lack distinct point<br>features and have severe self-occlusion. We propose the first approach that<br>simultaneously estimates camera motion and reconstructs the geometry of complex<br>3D thin structures in high quality from a color video captured by a handheld<br>camera. Specifically, we present a new curve-based approach to estimate<br>accurate camera poses by establishing correspondences between featureless thin<br>objects in the foreground in consecutive video frames, without requiring visual<br>texture in the background scene to lock on. Enabled by this effective<br>curve-based camera pose estimation strategy, we develop an iterative<br>optimization method with tailored measures on geometry, topology as well as<br>self-occlusion handling for reconstructing 3D thin structures. Extensive<br>validations on a variety of thin structures show that our method achieves<br>accurate camera pose estimation and faithful reconstruction of 3D thin<br>structures with complex shape and topology at a level that has not been<br>attained by other existing reconstruction methods.<br>
%K Computer Science, Graphics, cs.GR,Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.IV

Conference paper

L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang, and C. Theobalt

“EventCap: Monocular 3D Capture of High-Speed Human Motions Using an Event Camera,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{Xu_CVPR2020,
TITLE = {{EventCap}: {M}onocular {3D} Capture of High-Speed Human Motions Using an Event Camera},
AUTHOR = {Xu, Lan and Xu, Weipeng and Golyanik, Vladislav and Habermann, Marc and Fang, Lu and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00502},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {4967--4977},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Xu, Lan
%A Xu, Weipeng
%A Golyanik, Vladislav
%A Habermann, Marc
%A Fang, Lu
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T EventCap: Monocular 3D Capture of High-Speed Human Motions Using an Event Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CF57-F
%R 10.1109/CVPR42600.2020.00502
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 4967 - 4977
%I IEEE
%@ 978-1-7281-7168-5

Conference paper

Y. Xu, T. Fan, Y. Yuan, and G. Singh

“Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{Xu_ECCV20,
TITLE = {Ladybird: {Quasi-Monte Carlo} Sampling for Deep Implicit Field Based {3D} Reconstruction with Symmetry},
AUTHOR = {Xu, Yifan and Fan, Tianqi and Yuan, Yi and Singh, Gurprit},
LANGUAGE = {eng},
ISBN = {978-3-030-58451-1},
DOI = {10.1007/978-3-030-58452-8_15},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {248--263},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12346},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Xu, Yifan
%A Fan, Tianqi
%A Yuan, Yi
%A Singh, Gurprit
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CEBE-C
%R 10.1007/978-3-030-58452-8_15
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 248 - 263
%I Springer
%@ 978-3-030-58451-1
%B Lecture Notes in Computer Science
%N 12346

Paper

Y. Xu, T. Fan, Y. Yuan, and G. Singh

“Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry,” 2020. [Online]. Available: https://arxiv.org/abs/2007.13393.

Abstract

Deep implicit field regression methods are effective for 3D reconstruction
from single-view images. However, the impact of different sampling patterns on
the reconstruction quality is not well-understood. In this work, we first study
the effect of point set discrepancy on the network training. Based on Farthest
Point Sampling algorithm, we propose a sampling scheme that theoretically
encourages better generalization performance, and results in fast convergence
for SGD-based optimization algorithms. Secondly, based on the reflective
symmetry of an object, we propose a feature fusion method that alleviates
issues due to self-occlusions which makes it difficult to utilize local image
features. Our proposed system Ladybird is able to create high quality 3D object
reconstructions from a single input image. We evaluate Ladybird on a large
scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms
of Chamfer distance, Earth Mover's distance and Intersection Over Union (IoU).

BibTeX

@online{Xu_arXiv2007.13393,
TITLE = {Ladybird: {Quasi-Monte Carlo} Sampling for Deep Implicit Field Based {3D} Reconstruction with Symmetry},
AUTHOR = {Xu, Yifan and Fan, Tianqi and Yuan, Yi and Singh, Gurprit},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2007.13393},
EPRINT = {2007.13393},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {Deep implicit field regression methods are effective for 3D reconstruction<br>from single-view images. However, the impact of different sampling patterns on<br>the reconstruction quality is not well-understood. In this work, we first study<br>the effect of point set discrepancy on the network training. Based on Farthest<br>Point Sampling algorithm, we propose a sampling scheme that theoretically<br>encourages better generalization performance, and results in fast convergence<br>for SGD-based optimization algorithms. Secondly, based on the reflective<br>symmetry of an object, we propose a feature fusion method that alleviates<br>issues due to self-occlusions which makes it difficult to utilize local image<br>features. Our proposed system Ladybird is able to create high quality 3D object<br>reconstructions from a single input image. We evaluate Ladybird on a large<br>scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms<br>of Chamfer distance, Earth Mover's distance and Intersection Over Union (IoU).<br>},
}

Endnote

%0 Report
%A Xu, Yifan
%A Fan, Tianqi
%A Yuan, Yi
%A Singh, Gurprit
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D
  Reconstruction with Symmetry : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-CECA-E
%U https://arxiv.org/abs/2007.13393
%D 2020
%X   Deep implicit field regression methods are effective for 3D reconstruction<br>from single-view images. However, the impact of different sampling patterns on<br>the reconstruction quality is not well-understood. In this work, we first study<br>the effect of point set discrepancy on the network training. Based on Farthest<br>Point Sampling algorithm, we propose a sampling scheme that theoretically<br>encourages better generalization performance, and results in fast convergence<br>for SGD-based optimization algorithms. Secondly, based on the reflective<br>symmetry of an object, we propose a feature fusion method that alleviates<br>issues due to self-occlusions which makes it difficult to utilize local image<br>features. Our proposed system Ladybird is able to create high quality 3D object<br>reconstructions from a single input image. We evaluate Ladybird on a large<br>scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms<br>of Chamfer distance, Earth Mover's distance and Intersection Over Union (IoU).<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Paper

T. Yenamandra, A. Tewari, F. Bernard, H.-P. Seidel, M. Elgharib, D. Cremers, and C. Theobalt

“i3DMM: Deep Implicit 3D Morphable Model of Human Heads,” 2020. [Online]. Available: https://arxiv.org/abs/2011.14143.

Abstract

We present the first deep implicit 3D morphable model (i3DMM) of full heads.
Unlike earlier morphable face models it not only captures identity-specific
geometry, texture, and expressions of the frontal face, but also models the
entire head, including hair. We collect a new dataset consisting of 64 people
with different expressions and hairstyles to train i3DMM. Our approach has the
following favorable properties: (i) It is the first full head morphable model
that includes hair. (ii) In contrast to mesh-based models it can be trained on
merely rigidly aligned scans, without requiring difficult non-rigid
registration. (iii) We design a novel architecture to decouple the shape model
into an implicit reference shape and a deformation of this reference shape.
With that, dense correspondences between shapes can be learned implicitly. (iv)
This architecture allows us to semantically disentangle the geometry and color
components, as color is learned in the reference space. Geometry is further
disentangled as identity, expressions, and hairstyle, while color is
disentangled as identity and hairstyle components. We show the merits of i3DMM
using ablation studies, comparisons to state-of-the-art models, and
applications such as semantic head editing and texture transfer. We will make
our model publicly available.

BibTeX

@online{Yenamandra_arXiv2011.14143,
TITLE = {i{3D}MM: Deep Implicit {3D} Morphable Model of Human Heads},
AUTHOR = {Yenamandra, Tarun and Tewari, Ayush and Bernard, Florian and Seidel, Hans-Peter and Elgharib, Mohamed and Cremers, Daniel and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2011.14143},
EPRINT = {2011.14143},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present the first deep implicit 3D morphable model (i3DMM) of full heads.<br>Unlike earlier morphable face models it not only captures identity-specific<br>geometry, texture, and expressions of the frontal face, but also models the<br>entire head, including hair. We collect a new dataset consisting of 64 people<br>with different expressions and hairstyles to train i3DMM. Our approach has the<br>following favorable properties: (i) It is the first full head morphable model<br>that includes hair. (ii) In contrast to mesh-based models it can be trained on<br>merely rigidly aligned scans, without requiring difficult non-rigid<br>registration. (iii) We design a novel architecture to decouple the shape model<br>into an implicit reference shape and a deformation of this reference shape.<br>With that, dense correspondences between shapes can be learned implicitly. (iv)<br>This architecture allows us to semantically disentangle the geometry and color<br>components, as color is learned in the reference space. Geometry is further<br>disentangled as identity, expressions, and hairstyle, while color is<br>disentangled as identity and hairstyle components. We show the merits of i3DMM<br>using ablation studies, comparisons to state-of-the-art models, and<br>applications such as semantic head editing and texture transfer. We will make<br>our model publicly available.<br>},
}

Endnote

%0 Report
%A Yenamandra, Tarun
%A Tewari, Ayush
%A Bernard, Florian
%A Seidel, Hans-Peter
%A Elgharib, Mohamed
%A Cremers, Daniel
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T i3DMM: Deep Implicit 3D Morphable Model of Human Heads : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B702-8
%U https://arxiv.org/abs/2011.14143
%D 2020
%X   We present the first deep implicit 3D morphable model (i3DMM) of full heads.<br>Unlike earlier morphable face models it not only captures identity-specific<br>geometry, texture, and expressions of the frontal face, but also models the<br>entire head, including hair. We collect a new dataset consisting of 64 people<br>with different expressions and hairstyles to train i3DMM. Our approach has the<br>following favorable properties: (i) It is the first full head morphable model<br>that includes hair. (ii) In contrast to mesh-based models it can be trained on<br>merely rigidly aligned scans, without requiring difficult non-rigid<br>registration. (iii) We design a novel architecture to decouple the shape model<br>into an implicit reference shape and a deformation of this reference shape.<br>With that, dense correspondences between shapes can be learned implicitly. (iv)<br>This architecture allows us to semantically disentangle the geometry and color<br>components, as color is learned in the reference space. Geometry is further<br>disentangled as identity, expressions, and hairstyle, while color is<br>disentangled as identity and hairstyle components. We show the merits of i3DMM<br>using ablation studies, comparisons to state-of-the-art models, and<br>applications such as semantic head editing and texture transfer. We will make<br>our model publicly available.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Paper

J. S. Yoon, L. Liu, V. Golyanik, K. Sarkar, H. S. Park, and C. Theobalt

“Pose-Guided Human Animation from a Single Image in the Wild,” 2020. [Online]. Available: https://arxiv.org/abs/2012.03796.

Abstract

We present a new pose transfer method for synthesizing a human animation from
a single image of a person controlled by a sequence of body poses. Existing
pose transfer methods exhibit significant visual artifacts when applying to a
novel scene, resulting in temporal inconsistency and failures in preserving the
identity and textures of the person. To address these limitations, we design a
compositional neural network that predicts the silhouette, garment labels, and
textures. Each modular network is explicitly dedicated to a subtask that can be
learned from the synthetic data. At the inference time, we utilize the trained
network to produce a unified representation of appearance and its labels in UV
coordinates, which remains constant across poses. The unified representation
provides an incomplete yet strong guidance to generating the appearance in
response to the pose change. We use the trained network to complete the
appearance and render it with the background. With these strategies, we are
able to synthesize human animations that can preserve the identity and
appearance of the person in a temporally coherent way without any fine-tuning
of the network on the testing scene. Experiments show that our method
outperforms the state-of-the-arts in terms of synthesis quality, temporal
coherence, and generalization ability.

BibTeX

@online{Yoon_2012.03796,
TITLE = {Pose-Guided Human Animation from a Single Image in the Wild},
AUTHOR = {Yoon, Jae Shin and Liu, Lingjie and Golyanik, Vladislav and Sarkar, Kripasindhu and Park, Hyun Soo and Theobalt, Christian},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2012.03796},
EPRINT = {2012.03796},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present a new pose transfer method for synthesizing a human animation from<br>a single image of a person controlled by a sequence of body poses. Existing<br>pose transfer methods exhibit significant visual artifacts when applying to a<br>novel scene, resulting in temporal inconsistency and failures in preserving the<br>identity and textures of the person. To address these limitations, we design a<br>compositional neural network that predicts the silhouette, garment labels, and<br>textures. Each modular network is explicitly dedicated to a subtask that can be<br>learned from the synthetic data. At the inference time, we utilize the trained<br>network to produce a unified representation of appearance and its labels in UV<br>coordinates, which remains constant across poses. The unified representation<br>provides an incomplete yet strong guidance to generating the appearance in<br>response to the pose change. We use the trained network to complete the<br>appearance and render it with the background. With these strategies, we are<br>able to synthesize human animations that can preserve the identity and<br>appearance of the person in a temporally coherent way without any fine-tuning<br>of the network on the testing scene. Experiments show that our method<br>outperforms the state-of-the-arts in terms of synthesis quality, temporal<br>coherence, and generalization ability.<br>},
}

Endnote

%0 Report
%A Yoon, Jae Shin
%A Liu, Lingjie
%A Golyanik, Vladislav
%A Sarkar, Kripasindhu
%A Park, Hyun Soo
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Pose-Guided Human Animation from a Single Image in the Wild : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E9F3-0
%U https://arxiv.org/abs/2012.03796
%D 2020
%X   We present a new pose transfer method for synthesizing a human animation from<br>a single image of a person controlled by a sequence of body poses. Existing<br>pose transfer methods exhibit significant visual artifacts when applying to a<br>novel scene, resulting in temporal inconsistency and failures in preserving the<br>identity and textures of the person. To address these limitations, we design a<br>compositional neural network that predicts the silhouette, garment labels, and<br>textures. Each modular network is explicitly dedicated to a subtask that can be<br>learned from the synthetic data. At the inference time, we utilize the trained<br>network to produce a unified representation of appearance and its labels in UV<br>coordinates, which remains constant across poses. The unified representation<br>provides an incomplete yet strong guidance to generating the appearance in<br>response to the pose change. We use the trained network to complete the<br>appearance and render it with the background. With these strategies, we are<br>able to synthesize human animations that can preserve the identity and<br>appearance of the person in a temporally coherent way without any fine-tuning<br>of the network on the testing scene. Experiments show that our method<br>outperforms the state-of-the-arts in terms of synthesis quality, temporal<br>coherence, and generalization ability.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

Y. Yu, A. Meka, M. Elgharib, H.-P. Seidel, C. Theobalt, and W. A. P. Smith

“Self-supervised Outdoor Scene Relighting,” in Computer Vision -- ECCV 2020, Glasgow, UK, 2020.

@inproceedings{yu_ECCV20,
TITLE = {Self-supervised Outdoor Scene Relighting},
AUTHOR = {Yu, Ye and Meka, Abhimitra and Elgharib, Mohamed and Seidel, Hans-Peter and Theobalt, Christian and Smith, William A. P.},
LANGUAGE = {eng},
ISBN = {978-3-030-58541-9},
DOI = {10.1007/978-3-030-58542-6_6},
PUBLISHER = {Springer},
YEAR = {2020},
DATE = {2020},
BOOKTITLE = {Computer Vision -- ECCV 2020},
EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael},
PAGES = {84--101},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12367},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Yu, Ye
%A Meka, Abhimitra
%A Elgharib, Mohamed
%A Seidel, Hans-Peter
%A Theobalt, Christian
%A Smith, William A. P.
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Self-supervised Outdoor Scene Relighting : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-B0F6-C
%R 10.1007/978-3-030-58542-6_6
%D 2020
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV 2020
%E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael
%P 84 - 101
%I Springer
%@ 978-3-030-58541-9
%B Lecture Notes in Computer Science
%N 12367

Article

Q. Zheng, V. Babaei, G. Wetzstein, H.-P. Seidel, M. Zwicker, and G. Singh

“Neural Light Field 3D Printing,” ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2020), vol. 39, no. 6, 2020.

@article{Zheng_TOG2020,
TITLE = {Neural Light Field {3D} Printing},
AUTHOR = {Zheng, Quan and Babaei, Vahid and Wetzstein, Gordon and Seidel, Hans-Peter and Zwicker, Matthias and Singh, Gurprit},
ISSN = {0730-0301},
DOI = {10.1145/3414685.3417879},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2020},
JOURNAL = {ACM Transactions on Graphics (Proc. SIGGRAPH Asia)},
VOLUME = {39},
NUMBER = {6},
EID = {207},
BOOKTITLE = {Proceedings of the SIGGRAPH Asia 2020},
EDITOR = {Myszkowski, Karol},
}

Endnote

%0 Journal Article
%A Zheng, Quan
%A Babaei, Vahid
%A Wetzstein, Gordon
%A Seidel, Hans-Peter
%A Zwicker, Matthias
%A Singh, Gurprit
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Light Field 3D Printing : 
%U http://hdl.handle.net/21.11116/0000-0007-9AA8-E
%R 10.1145/3414685.3417879
%7 2020
%D 2020
%J ACM Transactions on Graphics
%V 39
%N 6
%Z sequence number: 207
%I ACM
%C New York, NY
%@ false
%B Proceedings of the SIGGRAPH Asia 2020
%O SIGGRAPH Asia 2020 SA'20 SA 2020

Paper

Y. Zhou, M. Habermann, I. Habibie, A. Tewari, C. Theobalt, and F. Xu

“Monocular Real-time Full Body Capture with Inter-part Correlations,” 2020. [Online]. Available: https://arxiv.org/abs/2012.06087.

Abstract

We present the first method for real-time full body capture that estimates
shape and motion of body and hands together with a dynamic 3D face model from a
single color image. Our approach uses a new neural network architecture that
exploits correlations between body and hands at high computational efficiency.
Unlike previous works, our approach is jointly trained on multiple datasets
focusing on hand, body or face separately, without requiring data where all the
parts are annotated at the same time, which is much more difficult to create at
sufficient variety. The possibility of such multi-dataset training enables
superior generalization ability. In contrast to earlier monocular full body
methods, our approach captures more expressive 3D face geometry and color by
estimating the shape, expression, albedo and illumination parameters of a
statistical face model. Our method achieves competitive accuracy on public
benchmarks, while being significantly faster and providing more complete face
reconstructions.

BibTeX

@online{Zhou_2012.06087,
TITLE = {Monocular Real-time Full Body Capture with Inter-part Correlations},
AUTHOR = {Zhou, Yuxiao and Habermann, Marc and Habibie, Ikhsanul and Tewari, Ayush and Theobalt, Christian and Xu, Feng},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2012.06087},
EPRINT = {2012.06087},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present the first method for real-time full body capture that estimates<br>shape and motion of body and hands together with a dynamic 3D face model from a<br>single color image. Our approach uses a new neural network architecture that<br>exploits correlations between body and hands at high computational efficiency.<br>Unlike previous works, our approach is jointly trained on multiple datasets<br>focusing on hand, body or face separately, without requiring data where all the<br>parts are annotated at the same time, which is much more difficult to create at<br>sufficient variety. The possibility of such multi-dataset training enables<br>superior generalization ability. In contrast to earlier monocular full body<br>methods, our approach captures more expressive 3D face geometry and color by<br>estimating the shape, expression, albedo and illumination parameters of a<br>statistical face model. Our method achieves competitive accuracy on public<br>benchmarks, while being significantly faster and providing more complete face<br>reconstructions.<br>},
}

Endnote

%0 Report
%A Zhou, Yuxiao
%A Habermann, Marc
%A Habibie, Ikhsanul
%A Tewari, Ayush
%A Theobalt, Christian
%A Xu, Feng
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Monocular Real-time Full Body Capture with Inter-part Correlations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E9FB-8
%U https://arxiv.org/abs/2012.06087
%D 2020
%X   We present the first method for real-time full body capture that estimates<br>shape and motion of body and hands together with a dynamic 3D face model from a<br>single color image. Our approach uses a new neural network architecture that<br>exploits correlations between body and hands at high computational efficiency.<br>Unlike previous works, our approach is jointly trained on multiple datasets<br>focusing on hand, body or face separately, without requiring data where all the<br>parts are annotated at the same time, which is much more difficult to create at<br>sufficient variety. The possibility of such multi-dataset training enables<br>superior generalization ability. In contrast to earlier monocular full body<br>methods, our approach captures more expressive 3D face geometry and color by<br>estimating the shape, expression, albedo and illumination parameters of a<br>statistical face model. Our method achieves competitive accuracy on public<br>benchmarks, while being significantly faster and providing more complete face<br>reconstructions.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

Y. Zhou, M. Habermann, W. Xu, I. Habibie, C. Theobalt, and F. Xu

“Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), Seattle, WA, USA (Virtual), 2020.

@inproceedings{zhou2019monocular,
TITLE = {Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data},
AUTHOR = {Zhou, Yuxiao and Habermann, Marc and Xu, Weipeng and Habibie, Ikhsanul and Theobalt, Christian and Xu, Feng},
LANGUAGE = {eng},
ISBN = {978-1-7281-7168-5},
DOI = {10.1109/CVPR42600.2020.00539},
PUBLISHER = {IEEE},
YEAR = {2020},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)},
PAGES = {5345--5354},
ADDRESS = {Seattle, WA, USA (Virtual)},
}

Endnote

%0 Conference Proceedings
%A Zhou, Yuxiao
%A Habermann, Marc
%A Xu, Weipeng
%A Habibie, Ikhsanul
%A Theobalt, Christian
%A Xu, Feng
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-A89E-B
%R 10.1109/CVPR42600.2020.00539
%D 2020
%B 33rd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2020-06-14 - 2020-06-19
%C Seattle, WA, USA (Virtual)
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 5345 - 5354
%I IEEE
%@ 978-1-7281-7168-5

Paper

Y. Zhou, M. Habermann, W. Xu, I. Habibie, C. Theobalt, and F. Xu

“Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data,” 2020. [Online]. Available: https://arxiv.org/abs/2003.09572.

Abstract

We present a novel method for monocular hand shape and pose estimation at
unprecedented runtime performance of 100fps and at state-of-the-art accuracy.
This is enabled by a new learning based architecture designed such that it can
make use of all the sources of available hand training data: image data with
either 2D or 3D annotations, as well as stand-alone 3D animations without
corresponding image data. It features a 3D hand joint detection module and an
inverse kinematics module which regresses not only 3D joint positions but also
maps them to joint rotations in a single feed-forward pass. This output makes
the method more directly usable for applications in computer vision and
graphics compared to only regressing 3D joint positions. We demonstrate that
our architectural design leads to a significant quantitative and qualitative
improvement over the state of the art on several challenging benchmarks. Our
model is publicly available for future research.

BibTeX

@online{Zhou2003.09572,
TITLE = {Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data},
AUTHOR = {Zhou, Yuxiao and Habermann, Marc and Xu, Weipeng and Habibie, Ikhsanul and Theobalt, Christian and Xu, Feng},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2003.09572},
EPRINT = {2003.09572},
EPRINTTYPE = {arXiv},
YEAR = {2020},
ABSTRACT = {We present a novel method for monocular hand shape and pose estimation at<br>unprecedented runtime performance of 100fps and at state-of-the-art accuracy.<br>This is enabled by a new learning based architecture designed such that it can<br>make use of all the sources of available hand training data: image data with<br>either 2D or 3D annotations, as well as stand-alone 3D animations without<br>corresponding image data. It features a 3D hand joint detection module and an<br>inverse kinematics module which regresses not only 3D joint positions but also<br>maps them to joint rotations in a single feed-forward pass. This output makes<br>the method more directly usable for applications in computer vision and<br>graphics compared to only regressing 3D joint positions. We demonstrate that<br>our architectural design leads to a significant quantitative and qualitative<br>improvement over the state of the art on several challenging benchmarks. Our<br>model is publicly available for future research.<br>},
}

Endnote

%0 Report
%A Zhou, Yuxiao
%A Habermann, Marc
%A Xu, Weipeng
%A Habibie, Ikhsanul
%A Theobalt, Christian
%A Xu, Feng
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-E0D3-D
%U https://arxiv.org/abs/2003.09572
%D 2020
%X   We present a novel method for monocular hand shape and pose estimation at<br>unprecedented runtime performance of 100fps and at state-of-the-art accuracy.<br>This is enabled by a new learning based architecture designed such that it can<br>make use of all the sources of available hand training data: image data with<br>either 2D or 3D annotations, as well as stand-alone 3D animations without<br>corresponding image data. It features a 3D hand joint detection module and an<br>inverse kinematics module which regresses not only 3D joint positions but also<br>maps them to joint rotations in a single feed-forward pass. This output makes<br>the method more directly usable for applications in computer vision and<br>graphics compared to only regressing 3D joint positions. We demonstrate that<br>our architectural design leads to a significant quantitative and qualitative<br>improvement over the state of the art on several challenging benchmarks. Our<br>model is publicly available for future research.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

2019

Article

I. Aharon, R. Chen, D. Zorin, and O. Weber

“Bounded Distortion Tetrahedral Metric Interpolation,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2019), vol. 38, no. 6, 2019.

@article{Aharon2019,
TITLE = {Bounded Distortion Tetrahedral Metric Interpolation},
AUTHOR = {Aharon, Ido and Chen, Renjie and Zorin, Denis and Weber, Ofir},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3355089.3356569},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {38},
NUMBER = {6},
EID = {182},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2019},
}

Endnote

%0 Journal Article
%A Aharon, Ido
%A Chen, Renjie
%A Zorin, Denis
%A Weber, Ofir
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Bounded Distortion Tetrahedral Metric Interpolation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-04BA-2
%R 10.1145/3355089.3356569
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 6
%Z sequence number: 182
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2019

%O ACM SIGGRAPH Asia 2019 Brisbane, Australia, 17 - 20 November 2019  SA'19 SA 2019

Conference paper

D2D4

T. Alldieck, G. Pons-Moll, C. Theobalt, and M. A. Magnor

“Tex2Shape: Detailed Full Human Body Geometry from a Single Image,” in International Conference on Computer Vision (ICCV 2019), Seoul, Korea, 2019.

Abstract

We present a simple yet effective method to infer detailed full human body
shape from only a single photograph. Our model can infer full-body shape
including face, hair, and clothing including wrinkles at interactive
frame-rates. Results feature details even on parts that are occluded in the
input image. Our main idea is to turn shape regression into an aligned
image-to-image translation problem. The input to our method is a partial
texture map of the visible region obtained from off-the-shelf methods. From a
partial texture, we estimate detailed normal and vector displacement maps,
which can be applied to a low-resolution smooth body model to add detail and
clothing. Despite being trained purely with synthetic data, our model
generalizes well to real-world photographs. Numerous results demonstrate the
versatility and robustness of our method.

BibTeX

@inproceedings{Alldieck_ICCV2019,
TITLE = {{Tex2Shape}: Detailed Full Human Body Geometry from a Single Image},
AUTHOR = {Alldieck, Thiemo and Pons-Moll, Gerard and Theobalt, Christian and Magnor, Marcus A.},
LANGUAGE = {eng},
ISBN = {978-1-7281-4803-8},
DOI = {10.1109/ICCV.2019.00238},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
ABSTRACT = {We present a simple yet effective method to infer detailed full human body<br>shape from only a single photograph. Our model can infer full-body shape<br>including face, hair, and clothing including wrinkles at interactive<br>frame-rates. Results feature details even on parts that are occluded in the<br>input image. Our main idea is to turn shape regression into an aligned<br>image-to-image translation problem. The input to our method is a partial<br>texture map of the visible region obtained from off-the-shelf methods. From a<br>partial texture, we estimate detailed normal and vector displacement maps,<br>which can be applied to a low-resolution smooth body model to add detail and<br>clothing. Despite being trained purely with synthetic data, our model<br>generalizes well to real-world photographs. Numerous results demonstrate the<br>versatility and robustness of our method.<br>},
BOOKTITLE = {International Conference on Computer Vision (ICCV 2019)},
PAGES = {2293--2303},
ADDRESS = {Seoul, Korea},
}

Endnote

%0 Conference Proceedings
%A Alldieck, Thiemo
%A Pons-Moll, Gerard
%A Theobalt, Christian
%A Magnor, Marcus A.
%+ External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Tex2Shape: Detailed Full Human Body Geometry from a Single Image : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-ECBE-E
%R 10.1109/ICCV.2019.00238
%D 2019
%B International Conference on Computer Vision
%Z date of event: 2019-10-27 - 2019-11-02
%C Seoul, Korea
%X   We present a simple yet effective method to infer detailed full human body<br>shape from only a single photograph. Our model can infer full-body shape<br>including face, hair, and clothing including wrinkles at interactive<br>frame-rates. Results feature details even on parts that are occluded in the<br>input image. Our main idea is to turn shape regression into an aligned<br>image-to-image translation problem. The input to our method is a partial<br>texture map of the visible region obtained from off-the-shelf methods. From a<br>partial texture, we estimate detailed normal and vector displacement maps,<br>which can be applied to a low-resolution smooth body model to add detail and<br>clothing. Despite being trained purely with synthetic data, our model<br>generalizes well to real-world photographs. Numerous results demonstrate the<br>versatility and robustness of our method.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%B International Conference on Computer Vision
%P 2293 - 2303
%I IEEE
%@ 978-1-7281-4803-8

Paper

T. Alldieck, G. Pons-Moll, C. Theobalt, and M. A. Magnor

“Tex2Shape: Detailed Full Human Body Geometry From a Single Image,” 2019. [Online]. Available: http://arxiv.org/abs/1904.08645.

Abstract

BibTeX

@online{Alldieck_arXiv1904.08645,
TITLE = {{Tex2Shape}: Detailed Full Human Body Geometry From a Single Image},
AUTHOR = {Alldieck, Thiemo and Pons-Moll, Gerard and Theobalt, Christian and Magnor, Marcus A.},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1904.08645},
EPRINT = {1904.08645},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We present a simple yet effective method to infer detailed full human body<br>shape from only a single photograph. Our model can infer full-body shape<br>including face, hair, and clothing including wrinkles at interactive<br>frame-rates. Results feature details even on parts that are occluded in the<br>input image. Our main idea is to turn shape regression into an aligned<br>image-to-image translation problem. The input to our method is a partial<br>texture map of the visible region obtained from off-the-shelf methods. From a<br>partial texture, we estimate detailed normal and vector displacement maps,<br>which can be applied to a low-resolution smooth body model to add detail and<br>clothing. Despite being trained purely with synthetic data, our model<br>generalizes well to real-world photographs. Numerous results demonstrate the<br>versatility and robustness of our method.<br>},
}

Endnote

%0 Report
%A Alldieck, Thiemo
%A Pons-Moll, Gerard
%A Theobalt, Christian
%A Magnor, Marcus A.
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Tex2Shape: Detailed Full Human Body Geometry From a Single Image : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7CF6-B
%U http://arxiv.org/abs/1904.08645
%D 2019
%X   We present a simple yet effective method to infer detailed full human body<br>shape from only a single photograph. Our model can infer full-body shape<br>including face, hair, and clothing including wrinkles at interactive<br>frame-rates. Results feature details even on parts that are occluded in the<br>input image. Our main idea is to turn shape regression into an aligned<br>image-to-image translation problem. The input to our method is a partial<br>texture map of the visible region obtained from off-the-shelf methods. From a<br>partial texture, we estimate detailed normal and vector displacement maps,<br>which can be applied to a low-resolution smooth body model to add detail and<br>clothing. Despite being trained purely with synthetic data, our model<br>generalizes well to real-world photographs. Numerous results demonstrate the<br>versatility and robustness of our method.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

D2D4

T. Alldieck, M. A. Magnor, B. L. Bhatnagar, C. Theobalt, and G. Pons-Moll

“Learning to Reconstruct People in Clothing from a Single RGB Camera,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, 2019.

@inproceedings{alldieck19cvpr,
TITLE = {Learning to Reconstruct People in Clothing from a Single {RGB} Camera},
AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Bhatnagar, Bharat Lal and Theobalt, Christian and Pons-Moll, Gerard},
ISBN = {978-1-7281-3293-8},
DOI = {10.1109/CVPR.2019.00127},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
PAGES = {1175--1186},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Alldieck, Thiemo
%A Magnor, Marcus A.
%A Bhatnagar, Bharat Lal
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Learning to Reconstruct People in Clothing from a Single RGB Camera : 
%U http://hdl.handle.net/21.11116/0000-0003-5F97-9
%R 10.1109/CVPR.2019.00127
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2019-06-15 - 2019-06-20
%C Long Beach, CA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 1175 - 1186
%I IEEE
%@ 978-1-7281-3293-8

Paper

T. Alldieck, M. A. Magnor, B. L. Bhatnagar, C. Theobalt, and G. Pons-Moll

“Learning to Reconstruct People in Clothing from a Single RGB Camera,” 2019. [Online]. Available: http://arxiv.org/abs/1903.05885.

Abstract

We present a learning-based model to infer the personalized 3D shape of
people from a few frames (1-8) of a monocular video in which the person is
moving, in less than 10 seconds with a reconstruction accuracy of 5mm. Our
model learns to predict the parameters of a statistical body model and instance
displacements that add clothing and hair to the shape. The model achieves fast
and accurate predictions based on two key design choices. First, by predicting
shape in a canonical T-pose space, the network learns to encode the images of
the person into pose-invariant latent codes, where the information is fused.
Second, based on the observation that feed-forward predictions are fast but do
not always align with the input images, we predict using both, bottom-up and
top-down streams (one per view) allowing information to flow in both
directions. Learning relies only on synthetic 3D data. Once learned, the model
can take a variable number of frames as input, and is able to reconstruct
shapes even from a single image with an accuracy of 6mm. Results on 3 different
datasets demonstrate the efficacy and accuracy of our approach.

BibTeX

@online{Alldieck_arXiv1903.05885,
TITLE = {Learning to Reconstruct People in Clothing from a Single {RGB} Camera},
AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Bhatnagar, Bharat Lal and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1903.05885},
EPRINT = {1903.05885},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We present a learning-based model to infer the personalized 3D shape of<br>people from a few frames (1-8) of a monocular video in which the person is<br>moving, in less than 10 seconds with a reconstruction accuracy of 5mm. Our<br>model learns to predict the parameters of a statistical body model and instance<br>displacements that add clothing and hair to the shape. The model achieves fast<br>and accurate predictions based on two key design choices. First, by predicting<br>shape in a canonical T-pose space, the network learns to encode the images of<br>the person into pose-invariant latent codes, where the information is fused.<br>Second, based on the observation that feed-forward predictions are fast but do<br>not always align with the input images, we predict using both, bottom-up and<br>top-down streams (one per view) allowing information to flow in both<br>directions. Learning relies only on synthetic 3D data. Once learned, the model<br>can take a variable number of frames as input, and is able to reconstruct<br>shapes even from a single image with an accuracy of 6mm. Results on 3 different<br>datasets demonstrate the efficacy and accuracy of our approach.<br>},
}

Endnote

%0 Report
%A Alldieck, Thiemo
%A Magnor, Marcus A.
%A Bhatnagar, Bharat Lal
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Learning to Reconstruct People in Clothing from a Single RGB Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-FE01-E
%U http://arxiv.org/abs/1903.05885
%D 2019
%X   We present a learning-based model to infer the personalized 3D shape of<br>people from a few frames (1-8) of a monocular video in which the person is<br>moving, in less than 10 seconds with a reconstruction accuracy of 5mm. Our<br>model learns to predict the parameters of a statistical body model and instance<br>displacements that add clothing and hair to the shape. The model achieves fast<br>and accurate predictions based on two key design choices. First, by predicting<br>shape in a canonical T-pose space, the network learns to encode the images of<br>the person into pose-invariant latent codes, where the information is fused.<br>Second, based on the observation that feed-forward predictions are fast but do<br>not always align with the input images, we predict using both, bottom-up and<br>top-down streams (one per view) allowing information to flow in both<br>directions. Learning relies only on synthetic 3D data. Once learned, the model<br>can take a variable number of frames as input, and is able to reconstruct<br>shapes even from a single image with an accuracy of 6mm. Results on 3 different<br>datasets demonstrate the efficacy and accuracy of our approach.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Paper

M. Bemana, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“Neural View-Interpolation for Sparse Light Field Video,” 2019. [Online]. Available: http://arxiv.org/abs/1910.13921.

Abstract

We suggest representing light field (LF) videos as "one-off" neural networks
(NN), i.e., a learned mapping from view-plus-time coordinates to
high-resolution color values, trained on sparse views. Initially, this sounds
like a bad idea for three main reasons: First, a NN LF will likely have less
quality than a same-sized pixel basis representation. Second, only few training
data, e.g., 9 exemplars per frame are available for sparse LF videos. Third,
there is no generalization across LFs, but across view and time instead.
Consequently, a network needs to be trained for each LF video. Surprisingly,
these problems can turn into substantial advantages: Other than the linear
pixel basis, a NN has to come up with a compact, non-linear i.e., more
intelligent, explanation of color, conditioned on the sparse view and time
coordinates. As observed for many NN however, this representation now is
interpolatable: if the image output for sparse view coordinates is plausible,
it is for all intermediate, continuous coordinates as well. Our specific
network architecture involves a differentiable occlusion-aware warping step,
which leads to a compact set of trainable parameters and consequently fast
learning and fast execution.

BibTeX

@online{Bemana_arXiv1910.13921,
TITLE = {Neural View-Interpolation for Sparse Light Field Video},
AUTHOR = {Bemana, Mojtaba and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1910.13921},
EPRINT = {1910.13921},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We suggest representing light field (LF) videos as "one-off" neural networks<br>(NN), i.e., a learned mapping from view-plus-time coordinates to<br>high-resolution color values, trained on sparse views. Initially, this sounds<br>like a bad idea for three main reasons: First, a NN LF will likely have less<br>quality than a same-sized pixel basis representation. Second, only few training<br>data, e.g., 9 exemplars per frame are available for sparse LF videos. Third,<br>there is no generalization across LFs, but across view and time instead.<br>Consequently, a network needs to be trained for each LF video. Surprisingly,<br>these problems can turn into substantial advantages: Other than the linear<br>pixel basis, a NN has to come up with a compact, non-linear i.e., more<br>intelligent, explanation of color, conditioned on the sparse view and time<br>coordinates. As observed for many NN however, this representation now is<br>interpolatable: if the image output for sparse view coordinates is plausible,<br>it is for all intermediate, continuous coordinates as well. Our specific<br>network architecture involves a differentiable occlusion-aware warping step,<br>which leads to a compact set of trainable parameters and consequently fast<br>learning and fast execution.<br>},
}

Endnote

%0 Report
%A Bemana, Mojtaba
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Neural View-Interpolation for Sparse Light Field Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7B16-9
%U http://arxiv.org/abs/1910.13921
%D 2019
%X   We suggest representing light field (LF) videos as "one-off" neural networks<br>(NN), i.e., a learned mapping from view-plus-time coordinates to<br>high-resolution color values, trained on sparse views. Initially, this sounds<br>like a bad idea for three main reasons: First, a NN LF will likely have less<br>quality than a same-sized pixel basis representation. Second, only few training<br>data, e.g., 9 exemplars per frame are available for sparse LF videos. Third,<br>there is no generalization across LFs, but across view and time instead.<br>Consequently, a network needs to be trained for each LF video. Surprisingly,<br>these problems can turn into substantial advantages: Other than the linear<br>pixel basis, a NN has to come up with a compact, non-linear i.e., more<br>intelligent, explanation of color, conditioned on the sparse view and time<br>coordinates. As observed for many NN however, this representation now is<br>interpolatable: if the image output for sparse view coordinates is plausible,<br>it is for all intermediate, continuous coordinates as well. Our specific<br>network architecture involves a differentiable occlusion-aware warping step,<br>which leads to a compact set of trainable parameters and consequently fast<br>learning and fast execution.<br>
%K Computer Science, Graphics, cs.GR,Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Learning, cs.LG,eess.IV

Article

M. Bemana, J. Keinert, K. Myszkowski, M. Bätz, M. Ziegler, H.-P. Seidel, and T. Ritschel

“Learning to Predict Image-based Rendering Artifacts with Respect to a Hidden Reference Image,” Computer Graphics Forum (Proc. Pacific Graphics 2019), vol. 38, no. 7, 2019.

@article{Bemana_PG2019,
TITLE = {Learning to Predict Image-based Rendering Artifacts with Respect to a Hidden Reference Image},
AUTHOR = {Bemana, Mojtaba and Keinert, Joachim and Myszkowski, Karol and B{\"a}tz, Michel and Ziegler, Matthias and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {1467-8659},
DOI = {10.1111/cgf.13862},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford, UK},
YEAR = {2019},
DATE = {2019},
JOURNAL = {Computer Graphics Forum (Proc. Pacific Graphics)},
VOLUME = {38},
NUMBER = {7},
PAGES = {579--589},
BOOKTITLE = {27th Annual International Conference on Computer Graphics and Applications (Pacific Graphics 2019)},
}

Endnote

%0 Journal Article
%A Bemana, Mojtaba
%A Keinert, Joachim
%A Myszkowski, Karol
%A B&#228;tz, Michel
%A Ziegler, Matthias
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Learning to Predict Image-based Rendering Artifacts with Respect to a Hidden Reference Image : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-9BC5-F
%R 10.1111/cgf.13862
%7 2019
%D 2019
%J Computer Graphics Forum
%V 38
%N 7
%& 579
%P 579 - 589
%I Wiley-Blackwell
%C Oxford, UK
%@ false
%B 27th Annual International Conference on Computer Graphics and Applications
%O Pacific Graphics 2019 PG 2019 Seoul, October 14-17, 2019

Article

F. Bernard, J. Thunberg, J. Goncalves, and C. Theobalt

“Synchronisation of Partial Multi-Matchings via Non-negative Factorisations,” Pattern Recognition, vol. 92, 2019.

@article{Bernard2019,
TITLE = {Synchronisation of Partial Multi-Matchings via Non-negative Factorisations},
AUTHOR = {Bernard, Florian and Thunberg, Johan and Goncalves, Jorge and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0031-3203},
DOI = {10.1016/j.patcog.2019.03.021},
PUBLISHER = {Pergamon},
ADDRESS = {Oxford},
YEAR = {2019},
DATE = {2019},
JOURNAL = {Pattern Recognition},
VOLUME = {92},
PAGES = {146--155},
}

Endnote

%0 Journal Article
%A Bernard, Florian
%A Thunberg, Johan
%A Goncalves, Jorge
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Synchronisation of Partial Multi-Matchings via Non-negative
  Factorisations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-B2EC-A
%R 10.1016/j.patcog.2019.03.021
%7 2019-03-23
%D 2019
%J Pattern Recognition
%O Pattern Recognit.
%V 92
%& 146
%P 146 - 155
%I Pergamon
%C Oxford
%@ false

Conference paper

D4D2

F. Bernard, J. Thunberg, P. Swoboda, and C. Theobalt

“HiPPI: Higher-Order Projected Power Iterations for Scalable Multi-Matching,” in International Conference on Computer Vision (ICCV 2019), Seoul, Korea, 2019.

@inproceedings{Bernard_ICCV2019,
TITLE = {{HiPPI}: {H}igher-Order Projected Power Iterations for Scalable Multi-Matching},
AUTHOR = {Bernard, Florian and Thunberg, Johan and Swoboda, Paul and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-4803-8},
DOI = {10.1109/ICCV.2019.01038},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {International Conference on Computer Vision (ICCV 2019)},
PAGES = {10283--10292},
ADDRESS = {Seoul, Korea},
}

Endnote

%0 Conference Proceedings
%A Bernard, Florian
%A Thunberg, Johan
%A Swoboda, Paul
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T HiPPI: Higher-Order Projected Power Iterations for Scalable Multi-Matching : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0006-DC81-0
%R 10.1109/ICCV.2019.01038
%D 2019
%B International Conference  on Computer Vision
%Z date of event: 2019-10-27 - 2019-11-02
%C Seoul, Korea
%B International Conference  on Computer Vision
%P 10283 - 10292
%I IEEE
%@ 978-1-7281-4803-8

Paper

B. L. Bhatnagar, G. Tiwari, C. Theobalt, and G. Pons-Moll

“Multi-Garment Net: Learning to Dress 3D People from Images,” 2019. [Online]. Available: http://arxiv.org/abs/1908.06903.

Abstract

We present Multi-Garment Network (MGN), a method to predict body shape and
clothing, layered on top of the SMPL model from a few frames (1-8) of a video.
Several experiments demonstrate that this representation allows higher level of
control when compared to single mesh or voxel representations of shape. Our
model allows to predict garment geometry, relate it to the body shape, and
transfer it to new body shapes and poses. To train MGN, we leverage a digital
wardrobe containing 712 digital garments in correspondence, obtained with a
novel method to register a set of clothing templates to a dataset of real 3D
scans of people in different clothing and poses. Garments from the digital
wardrobe, or predicted by MGN, can be used to dress any body shape in arbitrary
poses. We will make publicly available the digital wardrobe, the MGN model, and
code to dress SMPL with the garments.

BibTeX

@online{Bhatnagar_arXiv1908.06903,
TITLE = {Multi-Garment Net: {L}earning to Dress {3D} People from Images},
AUTHOR = {Bhatnagar, Bharat Lal and Tiwari, Garvita and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1908.06903},
EPRINT = {1908.06903},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We present Multi-Garment Network (MGN), a method to predict body shape and<br>clothing, layered on top of the SMPL model from a few frames (1-8) of a video.<br>Several experiments demonstrate that this representation allows higher level of<br>control when compared to single mesh or voxel representations of shape. Our<br>model allows to predict garment geometry, relate it to the body shape, and<br>transfer it to new body shapes and poses. To train MGN, we leverage a digital<br>wardrobe containing 712 digital garments in correspondence, obtained with a<br>novel method to register a set of clothing templates to a dataset of real 3D<br>scans of people in different clothing and poses. Garments from the digital<br>wardrobe, or predicted by MGN, can be used to dress any body shape in arbitrary<br>poses. We will make publicly available the digital wardrobe, the MGN model, and<br>code to dress SMPL with the garments.<br>},
}

Endnote

%0 Report
%A Bhatnagar, Bharat Lal
%A Tiwari, Garvita
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Multi-Garment Net: Learning to Dress 3D People from Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7D67-C
%U http://arxiv.org/abs/1908.06903
%D 2019
%X   We present Multi-Garment Network (MGN), a method to predict body shape and<br>clothing, layered on top of the SMPL model from a few frames (1-8) of a video.<br>Several experiments demonstrate that this representation allows higher level of<br>control when compared to single mesh or voxel representations of shape. Our<br>model allows to predict garment geometry, relate it to the body shape, and<br>transfer it to new body shapes and poses. To train MGN, we leverage a digital<br>wardrobe containing 712 digital garments in correspondence, obtained with a<br>novel method to register a set of clothing templates to a dataset of real 3D<br>scans of people in different clothing and poses. Garments from the digital<br>wardrobe, or predicted by MGN, can be used to dress any body shape in arbitrary<br>poses. We will make publicly available the digital wardrobe, the MGN model, and<br>code to dress SMPL with the garments.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

D2D4

B. L. Bhatnagar, G. Tiwari, C. Theobalt, and G. Pons-Moll

“Multi-Garment Net: Learning to Dress 3D People from Images,” in International Conference on Computer Vision (ICCV 2019), Seoul, Korea, 2019.

@inproceedings{bhatnagar_ICCV2019,
TITLE = {Multi-Garment {N}et: {L}earning to Dress {3D} People from Images},
AUTHOR = {Bhatnagar, Bharat Lal and Tiwari, Garvita and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
ISBN = {978-1-7281-4803-8},
DOI = {10.1109/ICCV.2019.00552},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {International Conference on Computer Vision (ICCV 2019)},
PAGES = {5419--5429},
ADDRESS = {Seoul, Korea},
}

Endnote

%0 Conference Proceedings
%A Bhatnagar, Bharat Lal
%A Tiwari, Garvita
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Multi-Garment Net: Learning to Dress 3D People from Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-89E8-C
%R 10.1109/ICCV.2019.00552
%D 2019
%B International Conference on Computer Vision
%Z date of event: 2019-10-27 - 2019-11-02
%C Seoul, Korea
%B International Conference on Computer Vision
%P 5419 - 5429
%I IEEE
%@ 978-1-7281-4803-8

Conference paper

A. K. Bojja, F. Mueller, S. R. Malireddi, M. Oberweger, V. Lepetit, C. Theobalt, K. M. Yi, and A. Tagliasacchi

“HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images,” in 16th Conference on Computer and Robot Vision (CRV 2019), Kingston, Canada, 2019.

@inproceedings{Malireddi_CRV2019,
TITLE = {{HandSeg}: {An Automatically Labeled Dataset for Hand Segmentation from Depth Images}},
AUTHOR = {Bojja, Abhishake Kumar and Mueller, Franziska and Malireddi, Sri Raghu and Oberweger, Markus and Lepetit, Vincent and Theobalt, Christian and Yi, Kwang Moo and Tagliasacchi, Andrea},
LANGUAGE = {eng},
ISBN = {978-1-7281-1838-3},
DOI = {10.1109/CRV.2019.00028},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {16th Conference on Computer and Robot Vision (CRV 2019)},
PAGES = {151--158},
ADDRESS = {Kingston, Canada},
}

Endnote

%0 Conference Proceedings
%A Bojja, Abhishake Kumar
%A Mueller, Franziska
%A Malireddi, Sri Raghu
%A Oberweger, Markus
%A Lepetit, Vincent
%A Theobalt, Christian
%A Yi, Kwang Moo
%A Tagliasacchi, Andrea
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-6BC4-6
%R 10.1109/CRV.2019.00028
%D 2019
%B 16th Conference on Computer and Robot Vision
%Z date of event: 2019-05-29 - 2019-05-31
%C Kingston, Canada
%B 16th Conference on Computer and Robot Vision
%P 151 - 158
%I IEEE
%@ 978-1-7281-1838-3

Paper

A. Božič, M. Zollhöfer, C. Theobalt, and M. Nießner

“DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data,” 2019. [Online]. Available: http://arxiv.org/abs/1912.04302.

Abstract

Applying data-driven approaches to non-rigid 3D reconstruction has been
difficult, which we believe can be attributed to the lack of a large-scale
training corpus. One recent approach proposes self-supervision based on
non-rigid reconstruction. Unfortunately, this method fails for important cases
such as highly non-rigid deformations. We first address this problem of lack of
data by introducing a novel semi-supervised strategy to obtain dense
inter-frame correspondences from a sparse set of annotations. This way, we
obtain a large dataset of 400 scenes, over 390,000 RGB-D frames, and 2,537
densely aligned frame pairs; in addition, we provide a test set along with
several metrics for evaluation. Based on this corpus, we introduce a
data-driven non-rigid feature matching approach, which we integrate into an
optimization-based reconstruction pipeline. Here, we propose a new neural
network that operates on RGB-D frames, while maintaining robustness under large
non-rigid deformations and producing accurate predictions. Our approach
significantly outperforms both existing non-rigid reconstruction methods that
do not use learned data terms, as well as learning-based approaches that only
use self-supervision.

BibTeX

@online{Bozic_arXiv1912.04302,
TITLE = {{DeepDeform}: Learning Non-rigid {RGB}-D Reconstruction with Semi-supervised Data},
AUTHOR = {Bo{\v z}i{\v c}, Alja{\v z} and Zollh{\"o}fer, Michael and Theobalt, Christian and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1912.04302},
EPRINT = {1912.04302},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {Applying data-driven approaches to non-rigid 3D reconstruction has been<br>difficult, which we believe can be attributed to the lack of a large-scale<br>training corpus. One recent approach proposes self-supervision based on<br>non-rigid reconstruction. Unfortunately, this method fails for important cases<br>such as highly non-rigid deformations. We first address this problem of lack of<br>data by introducing a novel semi-supervised strategy to obtain dense<br>inter-frame correspondences from a sparse set of annotations. This way, we<br>obtain a large dataset of 400 scenes, over 390,000 RGB-D frames, and 2,537<br>densely aligned frame pairs; in addition, we provide a test set along with<br>several metrics for evaluation. Based on this corpus, we introduce a<br>data-driven non-rigid feature matching approach, which we integrate into an<br>optimization-based reconstruction pipeline. Here, we propose a new neural<br>network that operates on RGB-D frames, while maintaining robustness under large<br>non-rigid deformations and producing accurate predictions. Our approach<br>significantly outperforms both existing non-rigid reconstruction methods that<br>do not use learned data terms, as well as learning-based approaches that only<br>use self-supervision.<br>},
}

Endnote

%0 Report
%A Bo&#382;i&#269;, Alja&#382;
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised
  Data : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7DDE-6
%U http://arxiv.org/abs/1912.04302
%D 2019
%X   Applying data-driven approaches to non-rigid 3D reconstruction has been<br>difficult, which we believe can be attributed to the lack of a large-scale<br>training corpus. One recent approach proposes self-supervision based on<br>non-rigid reconstruction. Unfortunately, this method fails for important cases<br>such as highly non-rigid deformations. We first address this problem of lack of<br>data by introducing a novel semi-supervised strategy to obtain dense<br>inter-frame correspondences from a sparse set of annotations. This way, we<br>obtain a large dataset of 400 scenes, over 390,000 RGB-D frames, and 2,537<br>densely aligned frame pairs; in addition, we provide a test set along with<br>several metrics for evaluation. Based on this corpus, we introduce a<br>data-driven non-rigid feature matching approach, which we integrate into an<br>optimization-based reconstruction pipeline. Here, we propose a new neural<br>network that operates on RGB-D frames, while maintaining robustness under large<br>non-rigid deformations and producing accurate predictions. Our approach<br>significantly outperforms both existing non-rigid reconstruction methods that<br>do not use learned data terms, as well as learning-based approaches that only<br>use self-supervision.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Conference paper

L. Castelli Aleardi, S. Salihoglu, G. Singh, and M. Ovsjanikov

“Spectral Measures of Distortion for Change Detection in Dynamic Graphs,” in Complex Networks and Their Applications VII, Cambridge, UK, 2019.

@inproceedings{Castelli_COMPLEX2018,
TITLE = {Spectral Measures of Distortion for Change Detection in Dynamic Graphs},
AUTHOR = {Castelli Aleardi, Luca and Salihoglu, Semih and Singh, Gurprit and Ovsjanikov, Maks},
LANGUAGE = {eng},
ISBN = {978-3-030-05413-7; 978-3-030-05414-4},
DOI = {10.1007/978-3-030-05414-4_5},
PUBLISHER = {Springer},
YEAR = {2018},
DATE = {2019},
BOOKTITLE = {Complex Networks and Their Applications VII},
EDITOR = {Aiello, Luca Maria and Cherifi, Chantal and Cherifi, Hocine and Lambiotte, Renaud and Li{\'o}, Pietro and Rocha, Luis M.},
PAGES = {54--66},
SERIES = {Studies in Computational Intelligence},
VOLUME = {813},
ADDRESS = {Cambridge, UK},
}

Endnote

%0 Conference Proceedings
%A Castelli Aleardi, Luca
%A Salihoglu, Semih
%A Singh, Gurprit
%A Ovsjanikov, Maks
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Spectral Measures of Distortion for Change Detection in Dynamic Graphs : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-F1F9-4
%R 10.1007/978-3-030-05414-4_5
%D 2019
%B 7th International Conference on Complex Networks and Their Applications 
%Z date of event: 2018-12-11 - 2018-12-13
%C Cambridge, UK
%B Complex Networks and Their Applications VII
%E Aiello, Luca Maria; Cherifi, Chantal; Cherifi, Hocine; Lambiotte, Renaud; Li&#243;, Pietro; Rocha, Luis M.
%P 54 - 66
%I Springer
%@ 978-3-030-05413-7 978-3-030-05414-4
%B Studies in Computational Intelligence
%N 813

Conference paper

J. M. Díaz Barros, V. Golyanik, K. Varanasi, and D. Stricker

“Face It!: A Pipeline for Real-Time Performance-Driven Facial Animation,” in IEEE International Conference on Image Processing (ICIP 2019), Taipei, Taiwan, 2019.

@inproceedings{DiazBarros_ICIP2019,
TITLE = {Face It!: {A} Pipeline for Real-Time Performance-Driven Facial Animation},
AUTHOR = {D{\'i}az Barros, Jilliam Mar{\'i}a and Golyanik, Vladislav and Varanasi, Kiran and Stricker, Didier},
LANGUAGE = {eng},
ISBN = {978-1-5386-6249-6},
DOI = {10.1109/ICIP.2019.8803330},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE International Conference on Image Processing (ICIP 2019)},
PAGES = {2209--2213},
ADDRESS = {Taipei, Taiwan},
}

Endnote

%0 Conference Proceedings
%A D&#237;az Barros, Jilliam Mar&#237;a
%A Golyanik, Vladislav
%A Varanasi, Kiran
%A Stricker, Didier
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Face It!: A Pipeline for Real-Time Performance-Driven Facial Animation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-982B-0
%R 10.1109/ICIP.2019.8803330
%D 2019
%B IEEE International Conference on Image Processing
%Z date of event: 2019-09-22 - 2019-09-25
%C Taipei, Taiwan
%B IEEE International Conference on Image Processing
%P 2209 - 2213
%I IEEE
%@ 978-1-5386-6249-6

Article

M. Dokter, J. Hladký, M. Parger, D. Schmalstieg, H.-P. Seidel, and M. Steinberger

“Hierarchical Rasterization of Curved Primitives for Vector Graphics Rendering on the GPU,” Computer Graphics Forum (Proc. EUROGRAPHICS 2019), vol. 38, no. 2, 2019.

@article{Dokter_EG2019,
TITLE = {Hierarchical Rasterization of Curved Primitives for Vector Graphics Rendering on the {GPU}},
AUTHOR = {Dokter, Mark and Hladk{\'y}, Jozef and Parger, Mathias and Schmalstieg, Dieter and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13622},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford},
YEAR = {2019},
DATE = {2019},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {38},
NUMBER = {2},
PAGES = {93--103},
BOOKTITLE = {EUROGRAPHICS 2019 STAR -- State of The Art Reports},
}

Endnote

%0 Journal Article
%A Dokter, Mark
%A Hladk&#253;, Jozef
%A Parger, Mathias
%A Schmalstieg, Dieter
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Hierarchical Rasterization of Curved Primitives for Vector Graphics Rendering on the GPU : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-FC80-1
%R 10.1111/cgf.13622
%7 2019
%D 2019
%J Computer Graphics Forum
%V 38
%N 2
%& 93
%P 93 - 103
%I Wiley-Blackwell
%C Oxford
%@ false
%B EUROGRAPHICS 2019 STAR &#8211; State of The Art Reports
%O EUROGRAPHICS 2019 The 40th Annual Conference of the European Association for Computer Graphics ; Genova, Italy, May 6-10, 2019 EG 2019

Paper

B. Egger, W. A. P. Smith, A. Tewari, S. Wuhrer, M. Zollhöfer, T. Beeler, F. Bernard, T. Bolkart, A. Kortylewski, S. Romdhani, C. Theobalt, V. Blanz, and T. Vetter

“3D Morphable Face Models -- Past, Present and Future,” 2019. [Online]. Available: http://arxiv.org/abs/1909.01815.

Abstract

In this paper, we provide a detailed survey of 3D Morphable Face Models over
the 20 years since they were first proposed. The challenges in building and
applying these models, namely capture, modeling, image formation, and image
analysis, are still active research topics, and we review the state-of-the-art
in each of these areas. We also look ahead, identifying unsolved challenges,
proposing directions for future research and highlighting the broad range of
current and future applications.

BibTeX

@online{Egger_arXIv1909.01815,
TITLE = {{3D} Morphable Face Models -- Past, Present and Future},
AUTHOR = {Egger, Bernhard and Smith, William A. P. and Tewari, Ayush and Wuhrer, Stefanie and Zollh{\"o}fer, Michael and Beeler, Thabo and Bernard, Florian and Bolkart, Timo and Kortylewski, Adam and Romdhani, Sami and Theobalt, Christian and Blanz, Volker and Vetter, Thomas},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1909.01815},
EPRINT = {1909.01815},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {In this paper, we provide a detailed survey of 3D Morphable Face Models over<br>the 20 years since they were first proposed. The challenges in building and<br>applying these models, namely capture, modeling, image formation, and image<br>analysis, are still active research topics, and we review the state-of-the-art<br>in each of these areas. We also look ahead, identifying unsolved challenges,<br>proposing directions for future research and highlighting the broad range of<br>current and future applications.<br>},
}

Endnote

%0 Report
%A Egger, Bernhard
%A Smith, William A. P.
%A Tewari, Ayush
%A Wuhrer, Stefanie
%A Zollh&#246;fer, Michael
%A Beeler, Thabo
%A Bernard, Florian
%A Bolkart, Timo
%A Kortylewski, Adam
%A Romdhani, Sami
%A Theobalt, Christian
%A Blanz, Volker
%A Vetter, Thomas
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T 3D Morphable Face Models -- Past, Present and Future : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7D8E-0
%U http://arxiv.org/abs/1909.01815
%D 2019
%X   In this paper, we provide a detailed survey of 3D Morphable Face Models over<br>the 20 years since they were first proposed. The challenges in building and<br>applying these models, namely capture, modeling, image formation, and image<br>analysis, are still active research topics, and we review the state-of-the-art<br>in each of these areas. We also look ahead, identifying unsolved challenges,<br>proposing directions for future research and highlighting the broad range of<br>current and future applications.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Proceedings

B. Egger, W. Smith, C. Theobalt, and T. Vetter

Eds., 3D Morphable Models, no. 3. Schloss Dagstuhl, 2019.

@proceedings{Egger_2019,
TITLE = {3D Morphable Models},
EDITOR = {Egger, Bernhard and Smith, William and Theobalt, Christian and Vetter, Thomas},
LANGUAGE = {eng},
URL = {urn:nbn:de:0030-drops-112894},
DOI = {10.4230/DagRep.9.3.16},
PUBLISHER = {Schloss Dagstuhl},
YEAR = {2019},
SERIES = {Dagstuhl Reports},
VOLUME = {9},
ISSUE = {3},
PAGES = {16--38},
ADDRESS = {Dagstuhl, Germany},
}

Endnote

%0 Conference Proceedings
%E Egger, Bernhard
%E Smith, William
%E Theobalt, Christian
%E Vetter, Thomas
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T 3D Morphable Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7DF9-7
%U urn:nbn:de:0030-drops-112894
%R  10.4230/DagRep.9.3.16
%I Schloss Dagstuhl
%D 2019
%B Dagstuhl Seminar 17201 "3D Morphable Models"
%Z date of event:  - 
%C Dagstuhl, Germany
%S Dagstuhl Reports
%V 9
%P 16 - 38
%U http://drops.dagstuhl.de/opus/volltexte/2019/11289/

Paper

M. Elgharib

Mallikarjun B R, A. Tewari, H. Kim, W. Liu, H.-P. Seidel, and C. Theobalt, “EgoFace: Egocentric Face Performance Capture and Videorealistic Reenactment,” 2019. [Online]. Available: http://arxiv.org/abs/1905.10822.

Abstract

Face performance capture and reenactment techniques use multiple cameras and
sensors, positioned at a distance from the face or mounted on heavy wearable
devices. This limits their applications in mobile and outdoor environments. We
present EgoFace, a radically new lightweight setup for face performance capture
and front-view videorealistic reenactment using a single egocentric RGB camera.
Our lightweight setup allows operations in uncontrolled environments, and lends
itself to telepresence applications such as video-conferencing from dynamic
environments. The input image is projected into a low dimensional latent space
of the facial expression parameters. Through careful adversarial training of
the parameter-space synthetic rendering, a videorealistic animation is
produced. Our problem is challenging as the human visual system is sensitive to
the smallest face irregularities that could occur in the final results. This
sensitivity is even stronger for video results. Our solution is trained in a
pre-processing stage, through a supervised manner without manual annotations.
EgoFace captures a wide variety of facial expressions, including mouth
movements and asymmetrical expressions. It works under varying illuminations,
background, movements, handles people from different ethnicities and can
operate in real time.

BibTeX

@online{Elgharib_arXiv1905.10822,
TITLE = {{EgoFace}: Egocentric Face Performance Capture and Videorealistic Reenactment},
AUTHOR = {Elgharib, Mohamed and Mallikarjun B R and Tewari, Ayush and Kim, Hyeongwoo and Liu, Wentao and Seidel, Hans-Peter and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1905.10822},
EPRINT = {1905.10822},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {Face performance capture and reenactment techniques use multiple cameras and<br>sensors, positioned at a distance from the face or mounted on heavy wearable<br>devices. This limits their applications in mobile and outdoor environments. We<br>present EgoFace, a radically new lightweight setup for face performance capture<br>and front-view videorealistic reenactment using a single egocentric RGB camera.<br>Our lightweight setup allows operations in uncontrolled environments, and lends<br>itself to telepresence applications such as video-conferencing from dynamic<br>environments. The input image is projected into a low dimensional latent space<br>of the facial expression parameters. Through careful adversarial training of<br>the parameter-space synthetic rendering, a videorealistic animation is<br>produced. Our problem is challenging as the human visual system is sensitive to<br>the smallest face irregularities that could occur in the final results. This<br>sensitivity is even stronger for video results. Our solution is trained in a<br>pre-processing stage, through a supervised manner without manual annotations.<br>EgoFace captures a wide variety of facial expressions, including mouth<br>movements and asymmetrical expressions. It works under varying illuminations,<br>background, movements, handles people from different ethnicities and can<br>operate in real time.<br>},
}

Endnote

%0 Report
%A Elgharib, Mohamed
%A Mallikarjun B R, 
%A Tewari, Ayush
%A Kim, Hyeongwoo
%A Liu, Wentao
%A Seidel, Hans-Peter
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T EgoFace: Egocentric Face Performance Capture and Videorealistic
  Reenactment : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-F1E6-9
%U http://arxiv.org/abs/1905.10822
%D 2019
%X   Face performance capture and reenactment techniques use multiple cameras and<br>sensors, positioned at a distance from the face or mounted on heavy wearable<br>devices. This limits their applications in mobile and outdoor environments. We<br>present EgoFace, a radically new lightweight setup for face performance capture<br>and front-view videorealistic reenactment using a single egocentric RGB camera.<br>Our lightweight setup allows operations in uncontrolled environments, and lends<br>itself to telepresence applications such as video-conferencing from dynamic<br>environments. The input image is projected into a low dimensional latent space<br>of the facial expression parameters. Through careful adversarial training of<br>the parameter-space synthetic rendering, a videorealistic animation is<br>produced. Our problem is challenging as the human visual system is sensitive to<br>the smallest face irregularities that could occur in the final results. This<br>sensitivity is even stronger for video results. Our solution is trained in a<br>pre-processing stage, through a supervised manner without manual annotations.<br>EgoFace captures a wide variety of facial expressions, including mouth<br>movements and asymmetrical expressions. It works under varying illuminations,<br>background, movements, handles people from different ethnicities and can<br>operate in real time.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR
%U http://gvv.mpi-inf.mpg.de/projects/EgoFace/

Paper

O. Fried, A. Tewari, M. Zollhöfer, A. Finkelstein, E. Shechtman, D. B. Goldman, K. Genova, Z. Jin, C. Theobalt, and M. Agrawala

“Text-based Editing of Talking-head Video,” 2019. [Online]. Available: http://arxiv.org/abs/1906.01524.

Abstract

Editing talking-head video to change the speech content or to remove filler
words is challenging. We propose a novel method to edit talking-head video
based on its transcript to produce a realistic output video in which the
dialogue of the speaker has been modified, while maintaining a seamless
audio-visual flow (i.e. no jump cuts). Our method automatically annotates an
input talking-head video with phonemes, visemes, 3D face pose and geometry,
reflectance, expression and scene illumination per frame. To edit a video, the
user has to only edit the transcript, and an optimization strategy then chooses
segments of the input corpus as base material. The annotated parameters
corresponding to the selected segments are seamlessly stitched together and
used to produce an intermediate video representation in which the lower half of
the face is rendered with a parametric face model. Finally, a recurrent video
generation network transforms this representation to a photorealistic video
that matches the edited transcript. We demonstrate a large variety of edits,
such as the addition, removal, and alteration of words, as well as convincing
language translation and full sentence synthesis.

BibTeX

@online{Fried_arXiv1906.01524,
TITLE = {Text-based Editing of Talking-head Video},
AUTHOR = {Fried, Ohad and Tewari, Ayush and Zollh{\"o}fer, Michael and Finkelstein, Adam and Shechtman, Eli and Goldman, Dan B. and Genova, Kyle and Jin, Zeyu and Theobalt, Christian and Agrawala, Maneesh},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1906.01524},
EPRINT = {1906.01524},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {Editing talking-head video to change the speech content or to remove filler<br>words is challenging. We propose a novel method to edit talking-head video<br>based on its transcript to produce a realistic output video in which the<br>dialogue of the speaker has been modified, while maintaining a seamless<br>audio-visual flow (i.e. no jump cuts). Our method automatically annotates an<br>input talking-head video with phonemes, visemes, 3D face pose and geometry,<br>reflectance, expression and scene illumination per frame. To edit a video, the<br>user has to only edit the transcript, and an optimization strategy then chooses<br>segments of the input corpus as base material. The annotated parameters<br>corresponding to the selected segments are seamlessly stitched together and<br>used to produce an intermediate video representation in which the lower half of<br>the face is rendered with a parametric face model. Finally, a recurrent video<br>generation network transforms this representation to a photorealistic video<br>that matches the edited transcript. We demonstrate a large variety of edits,<br>such as the addition, removal, and alteration of words, as well as convincing<br>language translation and full sentence synthesis.<br>},
}

Endnote

%0 Report
%A Fried, Ohad
%A Tewari, Ayush
%A Zollh&#246;fer, Michael
%A Finkelstein, Adam
%A Shechtman, Eli
%A Goldman, Dan B.
%A Genova, Kyle
%A Jin, Zeyu
%A Theobalt, Christian
%A Agrawala, Maneesh
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Text-based Editing of Talking-head Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-FE15-8
%U http://arxiv.org/abs/1906.01524
%D 2019
%X   Editing talking-head video to change the speech content or to remove filler<br>words is challenging. We propose a novel method to edit talking-head video<br>based on its transcript to produce a realistic output video in which the<br>dialogue of the speaker has been modified, while maintaining a seamless<br>audio-visual flow (i.e. no jump cuts). Our method automatically annotates an<br>input talking-head video with phonemes, visemes, 3D face pose and geometry,<br>reflectance, expression and scene illumination per frame. To edit a video, the<br>user has to only edit the transcript, and an optimization strategy then chooses<br>segments of the input corpus as base material. The annotated parameters<br>corresponding to the selected segments are seamlessly stitched together and<br>used to produce an intermediate video representation in which the lower half of<br>the face is rendered with a parametric face model. Finally, a recurrent video<br>generation network transforms this representation to a photorealistic video<br>that matches the edited transcript. We demonstrate a large variety of edits,<br>such as the addition, removal, and alteration of words, as well as convincing<br>language translation and full sentence synthesis.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR,Computer Science, Learning, cs.LG

Article

O. Fried, A. Tewari, M. Zollhöfer, A. Finkelstein, E. Shechtman, D. B. Goldman, K. Genova, Z. Jin, C. Theobalt, and M. Agrawala

“Text-based Editing of Talking-head Video,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2019), vol. 38, no. 4, 2019.

@article{Fried_SIGGRAPH2019,
TITLE = {Text-based Editing of Talking-head Video},
AUTHOR = {Fried, Ohad and Tewari, Ayush and Zollh{\"o}fer, Michael and Finkelstein, Adam and Shechtman, Eli and Goldman, Dan B. and Genova, Kyle and Jin, Zeyu and Theobalt, Christian and Agrawala, Maneesh},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3306346.3323028},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {38},
NUMBER = {4},
EID = {68},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2019},
}

Endnote

%0 Journal Article
%A Fried, Ohad
%A Tewari, Ayush
%A Zollh&#246;fer, Michael
%A Finkelstein, Adam
%A Shechtman, Eli
%A Goldman, Dan B.
%A Genova, Kyle
%A Jin, Zeyu
%A Theobalt, Christian
%A Agrawala, Maneesh
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Text-based Editing of Talking-head Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-8458-4
%R 10.1145/3306346.3323028
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 4
%Z sequence number: 68
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2019
%O ACM SIGGRAPH 2019 Los Angeles, CA, USA, 28 July - 1 August

Paper

V. Golyanik and C. Theobalt

“A Quantum Computational Approach to Correspondence Problems on Point Sets,” 2019. [Online]. Available: http://arxiv.org/abs/1912.12296.

Abstract

Modern adiabatic quantum computers (AQC) are already used to solve difficult
combinatorial optimisation problems in various domains of science. Currently,
only a few applications of AQC in computer vision have been demonstrated. We
review modern AQC and derive the first algorithm for transformation estimation
and point set alignment suitable for AQC. Our algorithm has a subquadratic
computational complexity of state preparation. We perform a systematic
experimental analysis of the proposed approach and show several examples of
successful point set alignment by simulated sampling. With this paper, we hope
to boost the research on AQC for computer vision.

BibTeX

@online{Golyanik_arXiv1912.12296,
TITLE = {A Quantum Computational Approach to Correspondence Problems on Point Sets},
AUTHOR = {Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1912.12296},
EPRINT = {1912.12296},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {Modern adiabatic quantum computers (AQC) are already used to solve difficult<br>combinatorial optimisation problems in various domains of science. Currently,<br>only a few applications of AQC in computer vision have been demonstrated. We<br>review modern AQC and derive the first algorithm for transformation estimation<br>and point set alignment suitable for AQC. Our algorithm has a subquadratic<br>computational complexity of state preparation. We perform a systematic<br>experimental analysis of the proposed approach and show several examples of<br>successful point set alignment by simulated sampling. With this paper, we hope<br>to boost the research on AQC for computer vision.<br>},
}

Endnote

%0 Report
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T A Quantum Computational Approach to Correspondence Problems on Point Sets : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7DF0-0
%U http://arxiv.org/abs/1912.12296
%D 2019
%X   Modern adiabatic quantum computers (AQC) are already used to solve difficult<br>combinatorial optimisation problems in various domains of science. Currently,<br>only a few applications of AQC in computer vision have been demonstrated. We<br>review modern AQC and derive the first algorithm for transformation estimation<br>and point set alignment suitable for AQC. Our algorithm has a subquadratic<br>computational complexity of state preparation. We perform a systematic<br>experimental analysis of the proposed approach and show several examples of<br>successful point set alignment by simulated sampling. With this paper, we hope<br>to boost the research on AQC for computer vision.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,cs.ET,Quantum Physics, quant-ph

Conference paper

V. Golyanik and C. Theobalt

“Optimising for Scale in Globally Multiply-Linked Gravitational Point Set Registration Leads to Singularities,” in International Conference on 3D Vision, Québec City, Canada, 2019.

@inproceedings{Golyanik_3DV2019,
TITLE = {Optimising for Scale in Globally Multiply-Linked Gravitational Point Set Registration Leads to Singularities},
AUTHOR = {Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3131-3},
DOI = {10.1109/3DV.2019.00027},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {International Conference on 3D Vision},
PAGES = {164--172},
ADDRESS = {Qu{\'e}bec City, Canada},
}

Endnote

%0 Conference Proceedings
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Optimising for Scale in Globally Multiply-Linked Gravitational Point Set Registration Leads to Singularities : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7B4E-B
%R 10.1109/3DV.2019.00027
%D 2019
%B International Conference on 3D Vision
%Z date of event: 2019-09-16 - 2019-09-19
%C Qu&#233;bec City, Canada
%B International Conference on 3D Vision
%P 164 - 172
%I IEEE
%@ 978-1-7281-3131-3

Paper

V. Golyanik, A. Jonas, D. Stricker, and C. Theobalt

“Intrinsic Dynamic Shape Prior for Fast, Sequential and Dense Non-Rigid Structure from Motion with Detection of Temporally-Disjoint Rigidity,” 2019. [Online]. Available: http://arxiv.org/abs/1909.02468.

Abstract

While dense non-rigid structure from motion (NRSfM) has been extensively
studied from the perspective of the reconstructability problem over the recent
years, almost no attempts have been undertaken to bring it into the practical
realm. The reasons for the slow dissemination are the severe ill-posedness,
high sensitivity to motion and deformation cues and the difficulty to obtain
reliable point tracks in the vast majority of practical scenarios. To fill this
gap, we propose a hybrid approach that extracts prior shape knowledge from an
input sequence with NRSfM and uses it as a dynamic shape prior for sequential
surface recovery in scenarios with recurrence. Our Dynamic Shape Prior
Reconstruction (DSPR) method can be combined with existing dense NRSfM
techniques while its energy functional is optimised with stochastic gradient
descent at real-time rates for new incoming point tracks. The proposed
versatile framework with a new core NRSfM approach outperforms several other
methods in the ability to handle inaccurate and noisy point tracks, provided we
have access to a representative (in terms of the deformation variety) image
sequence. Comprehensive experiments highlight convergence properties and the
accuracy of DSPR under different disturbing effects. We also perform a joint
study of tracking and reconstruction and show applications to shape compression
and heart reconstruction under occlusions. We achieve state-of-the-art metrics
(accuracy and compression ratios) in different scenarios.

BibTeX

@online{Golyanik_arXiv1909.02468,
TITLE = {Intrinsic Dynamic Shape Prior for Fast, Sequential and Dense Non-Rigid Structure from Motion with Detection of Temporally-Disjoint Rigidity},
AUTHOR = {Golyanik, Vladislav and Jonas, Andr{\'e} and Stricker, Didier and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1909.02468},
EPRINT = {1909.02468},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {While dense non-rigid structure from motion (NRSfM) has been extensively<br>studied from the perspective of the reconstructability problem over the recent<br>years, almost no attempts have been undertaken to bring it into the practical<br>realm. The reasons for the slow dissemination are the severe ill-posedness,<br>high sensitivity to motion and deformation cues and the difficulty to obtain<br>reliable point tracks in the vast majority of practical scenarios. To fill this<br>gap, we propose a hybrid approach that extracts prior shape knowledge from an<br>input sequence with NRSfM and uses it as a dynamic shape prior for sequential<br>surface recovery in scenarios with recurrence. Our Dynamic Shape Prior<br>Reconstruction (DSPR) method can be combined with existing dense NRSfM<br>techniques while its energy functional is optimised with stochastic gradient<br>descent at real-time rates for new incoming point tracks. The proposed<br>versatile framework with a new core NRSfM approach outperforms several other<br>methods in the ability to handle inaccurate and noisy point tracks, provided we<br>have access to a representative (in terms of the deformation variety) image<br>sequence. Comprehensive experiments highlight convergence properties and the<br>accuracy of DSPR under different disturbing effects. We also perform a joint<br>study of tracking and reconstruction and show applications to shape compression<br>and heart reconstruction under occlusions. We achieve state-of-the-art metrics<br>(accuracy and compression ratios) in different scenarios.<br>},
}

Endnote

%0 Report
%A Golyanik, Vladislav
%A Jonas, Andr&#233;
%A Stricker, Didier
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Intrinsic Dynamic Shape Prior for Fast, Sequential and Dense Non-Rigid
  Structure from Motion with Detection of Temporally-Disjoint Rigidity : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7D9A-2
%U http://arxiv.org/abs/1909.02468
%D 2019
%X   While dense non-rigid structure from motion (NRSfM) has been extensively<br>studied from the perspective of the reconstructability problem over the recent<br>years, almost no attempts have been undertaken to bring it into the practical<br>realm. The reasons for the slow dissemination are the severe ill-posedness,<br>high sensitivity to motion and deformation cues and the difficulty to obtain<br>reliable point tracks in the vast majority of practical scenarios. To fill this<br>gap, we propose a hybrid approach that extracts prior shape knowledge from an<br>input sequence with NRSfM and uses it as a dynamic shape prior for sequential<br>surface recovery in scenarios with recurrence. Our Dynamic Shape Prior<br>Reconstruction (DSPR) method can be combined with existing dense NRSfM<br>techniques while its energy functional is optimised with stochastic gradient<br>descent at real-time rates for new incoming point tracks. The proposed<br>versatile framework with a new core NRSfM approach outperforms several other<br>methods in the ability to handle inaccurate and noisy point tracks, provided we<br>have access to a representative (in terms of the deformation variety) image<br>sequence. Comprehensive experiments highlight convergence properties and the<br>accuracy of DSPR under different disturbing effects. We also perform a joint<br>study of tracking and reconstruction and show applications to shape compression<br>and heart reconstruction under occlusions. We achieve state-of-the-art metrics<br>(accuracy and compression ratios) in different scenarios.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

V. Golyanik, C. Theobalt, and D. Stricker

“Accelerated Gravitational Point Set Alignment with Altered Physical Laws,” in International Conference on Computer Vision (ICCV 2019), Seoul, Korea, 2019.

@inproceedings{BHRGA2019,
TITLE = {Accelerated Gravitational Point Set Alignment with Altered Physical Laws},
AUTHOR = {Golyanik, Vladislav and Theobalt, Christian and Stricker, Didier},
LANGUAGE = {eng},
ISBN = {978-1-7281-4803-8},
DOI = {10.1109/ICCV.2019.00217},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {International Conference on Computer Vision (ICCV 2019)},
PAGES = {2080--2089},
ADDRESS = {Seoul, Korea},
}

Endnote

%0 Conference Proceedings
%A Golyanik, Vladislav
%A Theobalt, Christian
%A Stricker, Didier
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Accelerated Gravitational Point Set Alignment with Altered Physical Laws : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-9473-2
%R 10.1109/ICCV.2019.00217
%D 2019
%B International Conference on Computer Vision
%Z date of event: 2019-10-27 - 2019-11-02
%C Seoul, Korea
%B International Conference on Computer Vision
%P 2080 - 2089
%I IEEE
%@  978-1-7281-4803-8

Conference paper

V. Golyanik, A. Jonas, and D. Stricker

“Consolidating Segmentwise Non-Rigid Structure from Motion,” in Proceedings of the Sixteenth International Conference on Machine Vision Applications (MVA 2019), Tokyo, Japan, 2019.

@inproceedings{Golyanik_MVA2019,
TITLE = {Consolidating Segmentwise Non-Rigid Structure from Motion},
AUTHOR = {Golyanik, Vladislav and Jonas, Andr{\'e} and Stricker, Didier},
LANGUAGE = {eng},
ISBN = {978-4-901122-18-4},
DOI = {10.23919/MVA.2019.8757909},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {Proceedings of the Sixteenth International Conference on Machine Vision Applications (MVA 2019)},
PAGES = {1--6},
ADDRESS = {Tokyo, Japan},
}

Endnote

%0 Conference Proceedings
%A Golyanik, Vladislav
%A Jonas, Andr&#233;
%A Stricker, Didier
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Consolidating Segmentwise Non-Rigid Structure from Motion : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-9823-8
%R 10.23919/MVA.2019.8757909
%D 2019
%B Sixteenth International Conference on Machine Vision Applications
%Z date of event: 2019-05-27 - 2019-05-31
%C Tokyo, Japan
%B Proceedings of the Sixteenth International Conference on Machine Vision Applications
%P 1 - 6
%I IEEE
%@ 978-4-901122-18-4

Article

D4D2

M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll, and C. Theobalt

“LiveCap: Real-time Human Performance Capture from Monocular Video,” ACM Transactions on Graphics, vol. 38, no. 2, 2019.

@article{Habermann_TOG19,
TITLE = {{LiveCap}: {R}eal-time Human Performance Capture from Monocular Video},
AUTHOR = {Habermann, Marc and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3311970},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {38},
NUMBER = {2},
EID = {14},
}

Endnote

%0 Journal Article
%A Habermann, Marc
%A Xu, Weipeng
%A Zollh&#246;fer, Michael
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T LiveCap: Real-time Human Performance Capture from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-B947-E
%R 10.1145/3311970
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 2
%Z sequence number: 14
%I ACM
%C New York, NY
%@ false

Conference paper

D4D2

M. Habermann, W. Xu, H. Rohdin, M. Zollhöfer, G. Pons-Moll, and C. Theobalt

“NRST: Non-rigid Surface Tracking from Monocular Video,” in Pattern Recognition (GCPR 2018), Stuttgart, Germany, 2019.

@inproceedings{Habermann_GVPR18,
TITLE = {{NRST}: {N}on-rigid Surface Tracking from Monocular Video},
AUTHOR = {Habermann, Marc and Xu, Weipeng and Rohdin, Helge and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-3-030-12938-5},
DOI = {10.1007/978-3-030-12939-2_23},
PUBLISHER = {Springer},
YEAR = {2018},
DATE = {2019},
BOOKTITLE = {Pattern Recognition (GCPR 2018)},
EDITOR = {Brox, Thomas and Bruhn, Andr{\'e}s and Fritz, Mario},
PAGES = {335--348},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {11269},
ADDRESS = {Stuttgart, Germany},
}

Endnote

%0 Conference Proceedings
%A Habermann, Marc
%A Xu, Weipeng
%A Rohdin, Helge
%A Zollh&#246;fer, Michael
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T NRST: Non-rigid Surface Tracking from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-B94C-9
%R 10.1007/978-3-030-12939-2_23
%D 2019
%B 40th German Conference on Pattern Recognition
%Z date of event: 2018-10-09 - 2018-10-12
%C Stuttgart, Germany
%B Pattern Recognition
%E Brox, Thomas; Bruhn, Andr&#233;s; Fritz, Mario
%P 335 - 348
%I Springer
%@ 978-3-030-12938-5
%B Lecture Notes in Computer Science
%N 11269

Conference paper

D4D2

I. Habibie, W. Xu, D. Mehta, G. Pons-Moll, and C. Theobalt

“In the Wild Human Pose Estimation using Explicit 2D Features and Intermediate 3D Representations,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, 2019.

@inproceedings{habibieCVPR19,
TITLE = {In the Wild Human Pose Estimation using Explicit {2D} Features and Intermediate {3D} Representations},
AUTHOR = {Habibie, Ikhsanul and Xu, Weipeng and Mehta, Dushyant and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3293-8},
DOI = {10.1109/CVPR.2019.01116},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
PAGES = {10897--10906},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Habibie, Ikhsanul
%A Xu, Weipeng
%A Mehta, Dushyant
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T In the Wild Human Pose Estimation using Explicit 2D Features and Intermediate 3D Representations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-6520-7
%R 10.1109/CVPR.2019.01116
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2019-06-15 - 2019-06-20
%C Long Beach, CA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 10897 - 10906
%I IEEE
%@ 978-1-7281-3293-8

Paper

I. Habibie, W. Xu, D. Mehta, G. Pons-Moll, and C. Theobalt

“In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representations,” 2019. [Online]. Available: http://arxiv.org/abs/1904.03289.

Abstract

Convolutional Neural Network based approaches for monocular 3D human pose
estimation usually require a large amount of training images with 3D pose
annotations. While it is feasible to provide 2D joint annotations for large
corpora of in-the-wild images with humans, providing accurate 3D annotations to
such in-the-wild corpora is hardly feasible in practice. Most existing 3D
labelled data sets are either synthetically created or feature in-studio
images. 3D pose estimation algorithms trained on such data often have limited
ability to generalize to real world scene diversity. We therefore propose a new
deep learning based method for monocular 3D human pose estimation that shows
high accuracy and generalizes better to in-the-wild scenes. It has a network
architecture that comprises a new disentangled hidden space encoding of
explicit 2D and 3D features, and uses supervision by a new learned projection
model from predicted 3D pose. Our algorithm can be jointly trained on image
data with 3D labels and image data with only 2D labels. It achieves
state-of-the-art accuracy on challenging in-the-wild data.

BibTeX

@online{Habibie_arXiv1904.03289,
TITLE = {In the Wild Human Pose Estimation Using Explicit {2D} Features and Intermediate {3D} Representations},
AUTHOR = {Habibie, Ikhsanul and Xu, Weipeng and Mehta, Dushyant and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1904.03289},
EPRINT = {1904.03289},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {Convolutional Neural Network based approaches for monocular 3D human pose<br>estimation usually require a large amount of training images with 3D pose<br>annotations. While it is feasible to provide 2D joint annotations for large<br>corpora of in-the-wild images with humans, providing accurate 3D annotations to<br>such in-the-wild corpora is hardly feasible in practice. Most existing 3D<br>labelled data sets are either synthetically created or feature in-studio<br>images. 3D pose estimation algorithms trained on such data often have limited<br>ability to generalize to real world scene diversity. We therefore propose a new<br>deep learning based method for monocular 3D human pose estimation that shows<br>high accuracy and generalizes better to in-the-wild scenes. It has a network<br>architecture that comprises a new disentangled hidden space encoding of<br>explicit 2D and 3D features, and uses supervision by a new learned projection<br>model from predicted 3D pose. Our algorithm can be jointly trained on image<br>data with 3D labels and image data with only 2D labels. It achieves<br>state-of-the-art accuracy on challenging in-the-wild data.<br>},
}

Endnote

%0 Report
%A Habibie, Ikhsanul
%A Xu, Weipeng
%A Mehta, Dushyant
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T In the Wild Human Pose Estimation Using Explicit 2D Features and
  Intermediate 3D Representations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-F76E-C
%U http://arxiv.org/abs/1904.03289
%D 2019
%X   Convolutional Neural Network based approaches for monocular 3D human pose<br>estimation usually require a large amount of training images with 3D pose<br>annotations. While it is feasible to provide 2D joint annotations for large<br>corpora of in-the-wild images with humans, providing accurate 3D annotations to<br>such in-the-wild corpora is hardly feasible in practice. Most existing 3D<br>labelled data sets are either synthetically created or feature in-studio<br>images. 3D pose estimation algorithms trained on such data often have limited<br>ability to generalize to real world scene diversity. We therefore propose a new<br>deep learning based method for monocular 3D human pose estimation that shows<br>high accuracy and generalizes better to in-the-wild scenes. It has a network<br>architecture that comprises a new disentangled hidden space encoding of<br>explicit 2D and 3D features, and uses supervision by a new learned projection<br>model from predicted 3D pose. Our algorithm can be jointly trained on image<br>data with 3D labels and image data with only 2D labels. It achieves<br>state-of-the-art accuracy on challenging in-the-wild data.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

J. Hladký, H.-P. Seidel, and M. Steinberger

“The Camera Offset Space: Real-time Potentially Visible Set Computations for Streaming Rendering,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2019), vol. 38, no. 6, 2019.

@article{Hladky_SA2019,
TITLE = {The Camera Offset Space: Real-time Potentially Visible Set Computations for Streaming Rendering},
AUTHOR = {Hladk{\'y}, Jozef and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0730-0301},
ISBN = {978-1-4503-6008-1},
DOI = {10.1145/3355089.3356530},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {38},
NUMBER = {6},
EID = {231},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2019},
}

Endnote

%0 Journal Article
%A Hladk&#253;, Jozef
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T The Camera Offset Space: Real-time Potentially Visible Set Computations for Streaming Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-4E4F-D
%R 10.1145/3355089.3356530
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 6
%Z sequence number: 231
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2019

%O ACM SIGGRAPH Asia 2019 Brisbane, Australia, 17 - 20 November 2019  SA'19 SA 2019
%@ 978-1-4503-6008-1

Article

J. Hladký, H.-P. Seidel, and M. Steinberger

“Tessellated Shading Streaming,” Computer Graphics Forum (Proc. Eurographics Symposium on Rendering 2019), vol. 38, no. 4, 2019.

@article{Hladky_EGSR2019,
TITLE = {Tessellated Shading Streaming},
AUTHOR = {Hladk{\'y}, Jozef and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0167-7055},
URL = {https://diglib.eg.org/handle/10.1111/cgf13780},
DOI = {10.1111/cgf.13780},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford},
YEAR = {2019},
DATE = {2019},
JOURNAL = {Computer Graphics Forum (Proc. Eurographics Symposium on Rendering)},
VOLUME = {38},
NUMBER = {4},
PAGES = {171--182},
BOOKTITLE = {Eurographics Symposium on Rendering 2019},
EDITOR = {Boubekeur, Tamy and Sen, Pradeep},
}

Endnote

%0 Journal Article
%A Hladk&#253;, Jozef
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Tessellated Shading Streaming : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-4897-1
%R 10.1111/cgf.13780
%U https://diglib.eg.org/handle/10.1111/cgf13780
%7 2019
%D 2019
%J Computer Graphics Forum
%V 38
%N 4
%& 171
%P 171 - 182
%I Wiley-Blackwell
%C Oxford
%@ false
%B Eurographics Symposium on Rendering 2019
%O Eurographics Symposium on Rendering 2019 EGSR 2019 Strasbourg, France, July 10 - 12, 2109

Paper

C. Jiang, C. Tang, H.-P. Seidel, R. Chen, and P. Wonka

“Computational Design of Lightweight Trusses,” 2019. [Online]. Available: http://arxiv.org/abs/1901.05637.

Abstract

Trusses are load-carrying light-weight structures consisting of bars
connected at joints ubiquitously applied in a variety of engineering scenarios.
Designing optimal trusses that satisfy functional specifications with a minimal
amount of material has interested both theoreticians and practitioners for more
than a century. In this paper, we introduce two main ideas to improve upon the
state of the art. First, we formulate an alternating linear programming problem
for geometry optimization. Second, we introduce two sets of complementary
topological operations, including a novel subdivision scheme for global
topology refinement inspired by Michell's famed theoretical study. Based on
these two ideas, we build an efficient computational framework for the design
of lightweight trusses. \AD{We illustrate our framework with a variety of
functional specifications and extensions. We show that our method achieves
trusses with smaller volumes and is over two orders of magnitude faster
compared with recent state-of-the-art approaches.

BibTeX

@online{Jiang_arXIv1901.05637,
TITLE = {Computational Design of Lightweight Trusses},
AUTHOR = {Jiang, Caigui and Tang, Chengcheng and Seidel, Hans-Peter and Chen, Renjie and Wonka, Peter},
URL = {http://arxiv.org/abs/1901.05637},
EPRINT = {1901.05637},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {Trusses are load-carrying light-weight structures consisting of bars<br>connected at joints ubiquitously applied in a variety of engineering scenarios.<br>Designing optimal trusses that satisfy functional specifications with a minimal<br>amount of material has interested both theoreticians and practitioners for more<br>than a century. In this paper, we introduce two main ideas to improve upon the<br>state of the art. First, we formulate an alternating linear programming problem<br>for geometry optimization. Second, we introduce two sets of complementary<br>topological operations, including a novel subdivision scheme for global<br>topology refinement inspired by Michell's famed theoretical study. Based on<br>these two ideas, we build an efficient computational framework for the design<br>of lightweight trusses. \AD{We illustrate our framework with a variety of<br>functional specifications and extensions. We show that our method achieves<br>trusses with smaller volumes and is over two orders of magnitude faster<br>compared with recent state-of-the-art approaches.<br>},
}

Endnote

%0 Report
%A Jiang, Caigui
%A Tang, Chengcheng
%A Seidel, Hans-Peter
%A Chen, Renjie
%A Wonka, Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Computational Design of Lightweight Trusses : 
%U http://hdl.handle.net/21.11116/0000-0003-A7E9-A
%U http://arxiv.org/abs/1901.05637
%D 2019
%X   Trusses are load-carrying light-weight structures consisting of bars<br>connected at joints ubiquitously applied in a variety of engineering scenarios.<br>Designing optimal trusses that satisfy functional specifications with a minimal<br>amount of material has interested both theoreticians and practitioners for more<br>than a century. In this paper, we introduce two main ideas to improve upon the<br>state of the art. First, we formulate an alternating linear programming problem<br>for geometry optimization. Second, we introduce two sets of complementary<br>topological operations, including a novel subdivision scheme for global<br>topology refinement inspired by Michell's famed theoretical study. Based on<br>these two ideas, we build an efficient computational framework for the design<br>of lightweight trusses. \AD{We illustrate our framework with a variety of<br>functional specifications and extensions. We show that our method achieves<br>trusses with smaller volumes and is over two orders of magnitude faster<br>compared with recent state-of-the-art approaches.<br>
%K Computer Science, Graphics, cs.GR

Thesis

D4IMPR-CS

H. Kim

“Learning-based Face Reconstruction and Editing,” Universität des Saarlandes, Saarbrücken, 2019.

@phdthesis{Kim_2019,
TITLE = {Learning-based Face Reconstruction and Editing},
AUTHOR = {Kim, Hyeongwoo},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-323942},
DOI = {10.22028/D291-32394},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2019},
DATE = {2019},
}

Endnote

%0 Thesis
%A Kim, Hyeongwoo
%Y Theobalt, Christian
%A referee: Izadi, Shahram
%A referee: Richardt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Learning-based Face Reconstruction and Editing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-6FFA-4
%R 10.22028/D291-32394
%U urn:nbn:de:bsz:291--ds-323942
%F OTHER: hdl:20.500.11880/29769
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2019
%P 124 p.
%V phd
%9 phd
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/29769

Article

H. Kim, M. Elgharib, M. Zollhöfer, H.-P. Seidel, T. Beeler, C. Richardt, and C. Theobalt

“Neural Style-preserving Visual Dubbing,” ACM Transactions on Graphics, vol. 38, no. 6, 2019.

@article{Kim2019,
TITLE = {Neural Style-preserving Visual Dubbing},
AUTHOR = {Kim, Hyeongwoo and Elgharib, Mohamed and Zollh{\"o}fer, Michael and Seidel, Hans-Peter and Beeler, Thabo and Richardt, Christian and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3355089.3356500},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {38},
NUMBER = {6},
EID = {178},
}

Endnote

%0 Journal Article
%A Kim, Hyeongwoo
%A Elgharib, Mohamed
%A Zollh&#246;fer, Michael
%A Seidel, Hans-Peter
%A Beeler, Thabo
%A Richardt, Christian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Style-preserving Visual Dubbing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-6AC0-B
%R 10.1145/3355089.3356500
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 6
%Z sequence number: 178
%I ACM
%C New York, NY
%@ false

Article

O. Kovalenko, V. Golyanik, J. Malik, A. Elhayek, and D. Stricker

“Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data,” Sensors, vol. 19, no. 20, 2019.

@article{Kovalenko2019,
TITLE = {Structure from Articulated Motion: {A}ccurate and Stable Monocular {3D} Reconstruction without Training Data},
AUTHOR = {Kovalenko, Onorina and Golyanik, Vladislav and Malik, Jameel and Elhayek, Ahmed and Stricker, Didier},
LANGUAGE = {eng},
ISSN = {1424-8220},
DOI = {10.3390/s19204603},
PUBLISHER = {MDPI},
YEAR = {2019},
JOURNAL = {Sensors},
VOLUME = {19},
NUMBER = {20},
EID = {4603},
}

Endnote

%0 Journal Article
%A Kovalenko, Onorina
%A Golyanik, Vladislav
%A Malik, Jameel
%A Elhayek, Ahmed
%A Stricker, Didier
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-5CB5-8
%R 10.3390/s19204603
%7 2019
%D 2019
%J Sensors
%V 19
%N 20
%Z sequence number: 4603
%I  MDPI
%@ false

Article

T. Leimkühler, G. Singh, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“Deep Point Correlation Design,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2019), vol. 38, no. 6, 2019.

@article{Leimkuehler_SA2019,
TITLE = {Deep Point Correlation Design},
AUTHOR = {Leimk{\"u}hler, Thomas and Singh, Gurprit and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3355089.3356562},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {38},
NUMBER = {6},
EID = {226},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2019},
}

Endnote

%0 Journal Article
%A Leimk&#252;hler, Thomas
%A Singh, Gurprit
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Deep Point Correlation Design : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-9BF3-B
%R 10.1145/3355089.3356562
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 6
%Z sequence number: 226
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2019

%O ACM SIGGRAPH Asia 2019 Brisbane, Australia, 17 - 20 November 2019  SA'19 SA 2019

Thesis

D4IMPR-CS

T. Leimkühler

“Artificial Intelligence for Efficient Image-based View Synthesis,” Universität des Saarlandes, Saarbrücken, 2019.

Abstract

Synthesizing novel views from image data is a widely investigated topic in both computer graphics and computer vision, and has many applications like stereo or multi-view rendering for virtual reality, light field reconstruction, and image post-processing. While image-based approaches have the advantage of reduced computational load compared to classical model-based rendering, efficiency is still a major concern. This thesis demonstrates how concepts and tools from artificial intelligence can be used to increase the efficiency of image-based view synthesis algorithms. In particular it is shown how machine learning can help to generate point patterns useful for a variety of computer graphics tasks, how path planning can guide image warping, how sparsity-enforcing optimization can lead to significant speedups in interactive distribution effect rendering, and how probabilistic inference can be used to perform real-time 2D-to-3D conversion.

BibTeX

@phdthesis{Leimphd2019,
TITLE = {Artificial Intelligence for Efficient Image-based View Synthesis},
AUTHOR = {Leimk{\"u}hler, Thomas},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-283793},
DOI = {10.22028/D291-28379},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2019},
DATE = {2019},
ABSTRACT = {Synthesizing novel views from image data is a widely investigated topic in both computer graphics and computer vision, and has many applications like stereo or multi-view rendering for virtual reality, light field reconstruction, and image post-processing. While image-based approaches have the advantage of reduced computational load compared to classical model-based rendering, efficiency is still a major concern. This thesis demonstrates how concepts and tools from artificial intelligence can be used to increase the efficiency of image-based view synthesis algorithms. In particular it is shown how machine learning can help to generate point patterns useful for a variety of computer graphics tasks, how path planning can guide image warping, how sparsity-enforcing optimization can lead to significant speedups in interactive distribution effect rendering, and how probabilistic inference can be used to perform real-time 2D-to-3D conversion.},
}

Endnote

%0 Thesis
%A Leimk&#252;hler, Thomas
%Y Seidel, Hans-Peter
%A referee: Ritschel, Tobias
%A referee: Lensch, Hendrik
%A referee: Drettakis, George
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Artificial Intelligence for Efficient Image-based View Synthesis : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-A589-7
%R 10.22028/D291-28379
%U urn:nbn:de:bsz:291--ds-283793
%F OTHER: hdl:20.500.11880/27664
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2019
%P 136 p.
%V phd
%9 phd
%X Synthesizing novel views from image data is a widely investigated topic in both computer graphics and computer vision, and has many applications like stereo or multi-view rendering for virtual reality, light field reconstruction, and image post-processing. While image-based approaches have the advantage of reduced computational load compared to classical model-based rendering, efficiency is still a major concern. This thesis demonstrates how concepts and tools from artificial intelligence can be used to increase the efficiency of image-based view synthesis algorithms. In particular it is shown how machine learning can help to generate point patterns useful for a variety of computer graphics tasks, how path planning can guide image warping, how sparsity-enforcing optimization can lead to significant speedups in interactive distribution effect rendering, and how probabilistic inference can be used to perform real-time 2D-to-3D conversion.
%U https://publikationen.sulb.uni-saarland.de/handle/20.500.11880/27664

Article

L. Liu, W. Xu, M. Zollhöfer, H. Kim, F. Bernard, M. Habermann, W. Wang, and C. Theobalt

“Neural Rendering and Reenactment of Human Actor Videos,” ACM Transactions on Graphics, vol. 38, no. 5, 2019.

@article{Liu_2019,
TITLE = {Neural Rendering and Reenactment of Human Actor Videos},
AUTHOR = {Liu, Lingjie and Xu, Weipeng and Zollh{\"o}fer, Michael and Kim, Hyeongwoo and Bernard, Florian and Habermann, Marc and Wang, Wenping and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3333002},
PUBLISHER = {Association for Computing Machinery},
ADDRESS = {New York, NY},
YEAR = {2019},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {38},
NUMBER = {5},
EID = {139},
}

Endnote

%0 Journal Article
%A Liu, Lingjie
%A Xu, Weipeng
%A Zollh&#246;fer, Michael
%A Kim, Hyeongwoo
%A Bernard, Florian
%A Habermann, Marc
%A Wang, Wenping
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Rendering and Reenactment of Human Actor Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7B28-5
%R 10.1145/3333002
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 5
%Z sequence number: 139
%I Association for Computing Machinery
%C New York, NY
%@ false

Paper

D. Mehta, K. I. Kim, and C. Theobalt

“Implicit Filter Sparsification In Convolutional Neural Networks,” 2019. [Online]. Available: http://arxiv.org/abs/1905.04967.

Abstract

We show implicit filter level sparsity manifests in convolutional neural
networks (CNNs) which employ Batch Normalization and ReLU activation, and are
trained with adaptive gradient descent techniques and L2 regularization or
weight decay. Through an extensive empirical study (Mehta et al., 2019) we
hypothesize the mechanism behind the sparsification process, and find
surprising links to certain filter sparsification heuristics proposed in
literature. Emergence of, and the subsequent pruning of selective features is
observed to be one of the contributing mechanisms, leading to feature sparsity
at par or better than certain explicit sparsification / pruning approaches. In
this workshop article we summarize our findings, and point out corollaries of
selective-featurepenalization which could also be employed as heuristics for
filter pruning

BibTeX

@online{Mehta_arXiv1905.04967,
TITLE = {Implicit Filter Sparsification In Convolutional Neural Networks},
AUTHOR = {Mehta, Dushyant and Kim, Kwang In and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1905.04967},
EPRINT = {1905.04967},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We show implicit filter level sparsity manifests in convolutional neural<br>networks (CNNs) which employ Batch Normalization and ReLU activation, and are<br>trained with adaptive gradient descent techniques and L2 regularization or<br>weight decay. Through an extensive empirical study (Mehta et al., 2019) we<br>hypothesize the mechanism behind the sparsification process, and find<br>surprising links to certain filter sparsification heuristics proposed in<br>literature. Emergence of, and the subsequent pruning of selective features is<br>observed to be one of the contributing mechanisms, leading to feature sparsity<br>at par or better than certain explicit sparsification / pruning approaches. In<br>this workshop article we summarize our findings, and point out corollaries of<br>selective-featurepenalization which could also be employed as heuristics for<br>filter pruning<br>},
}

Endnote

%0 Report
%A Mehta, Dushyant
%A Kim, Kwang In
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Implicit Filter Sparsification In Convolutional Neural Networks : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-FE07-8
%U http://arxiv.org/abs/1905.04967
%D 2019
%X   We show implicit filter level sparsity manifests in convolutional neural<br>networks (CNNs) which employ Batch Normalization and ReLU activation, and are<br>trained with adaptive gradient descent techniques and L2 regularization or<br>weight decay. Through an extensive empirical study (Mehta et al., 2019) we<br>hypothesize the mechanism behind the sparsification process, and find<br>surprising links to certain filter sparsification heuristics proposed in<br>literature. Emergence of, and the subsequent pruning of selective features is<br>observed to be one of the contributing mechanisms, leading to feature sparsity<br>at par or better than certain explicit sparsification / pruning approaches. In<br>this workshop article we summarize our findings, and point out corollaries of<br>selective-featurepenalization which could also be employed as heuristics for<br>filter pruning<br>
%K Computer Science, Learning, cs.LG,Computer Science, Computer Vision and Pattern Recognition, cs.CV,Statistics, Machine Learning, stat.ML

Conference paper

D4D2

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, H.-P. Seidel, P. Fua, M. Elgharib, H. Rhodin, G. Pons-Moll, and C. Theobalt

“XNect Demo (v2): Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera,” in CVPR 2019 Demonstrations, Long Beach, CA, USA, 2019.

@inproceedings{XNectDemoV2_CVPR2019,
TITLE = {Demo of {VNect} (v2): {R}eal-time {3D} Human Pose Estimation with a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Seidel, Hans-Peter and Fua, Pascal and Elgharib, Mohamed and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
YEAR = {2019},
BOOKTITLE = {CVPR 2019 Demonstrations},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Mehta, Dushyant
%A Sotnychenko, Oleksandr
%A Mueller, Franziska
%A Xu, Weipeng
%A Seidel, Hans-Peter
%A Fua, Pascal
%A Elgharib, Mohamed
%A Rhodin, Helge
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T XNect Demo (v2): Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-71DB-6
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition 
%Z date of event: 2019-06-16 - 2019-06-20
%C Long Beach, CA, USA
%B CVPR 2019 Demonstrations
%U http://gvv.mpi-inf.mpg.de/projects/XNectDemoV2/http://gvv.mpi-inf.mpg.de/projects/XNectDemoV2/

Conference paper

D. Mehta, K. I. Kim, and C. Theobalt

“On Implicit Filter Level Sparsity in Convolutional Neural Networks,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, 2019.

@inproceedings{Mehta_CVPR2019,
TITLE = {On Implicit Filter Level Sparsity in Convolutional Neural Networks},
AUTHOR = {Mehta, Dushyant and Kim, Kwang In and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3293-8},
DOI = {10.1109/CVPR.2019.00061},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
PAGES = {520--528},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Mehta, Dushyant
%A Kim, Kwang In
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T On Implicit Filter Level Sparsity in Convolutional Neural Networks : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7CB8-1
%R 10.1109/CVPR.2019.00061
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2019-06-16 - 2019-06-20
%C Long Beach, CA,  USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition 
%P 520 - 528
%I IEEE
%@ 978-1-7281-3293-8

Paper

D4D2

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, M. Elgharib, P. Fua, H.-P. Seidel, H. Rhodin, G. Pons-Moll, and C. Theobalt

“XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera,” 2019. [Online]. Available: http://arxiv.org/abs/1907.00837.

Abstract

We present a real-time approach for multi-person 3D motion capture at over 30
fps using a single RGB camera. It operates in generic scenes and is robust to
difficult occlusions both by other people and objects. Our method operates in
subsequent stages. The first stage is a convolutional neural network (CNN) that
estimates 2D and 3D pose features along with identity assignments for all
visible joints of all individuals. We contribute a new architecture for this
CNN, called SelecSLS Net, that uses novel selective long and short range skip
connections to improve the information flow allowing for a drastically faster
network without compromising accuracy. In the second stage, a fully-connected
neural network turns the possibly partial (on account of occlusion) 2D pose and
3D pose features for each subject into a complete 3D pose estimate per
individual. The third stage applies space-time skeletal model fitting to the
predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose,
and enforce temporal coherence. Our method returns the full skeletal pose in
joint angles for each subject. This is a further key distinction from previous
work that neither extracted global body positions nor joint angle results of a
coherent skeleton in real time for multi-person scenes. The proposed system
runs on consumer hardware at a previously unseen speed of more than 30 fps
given 512x320 images as input while achieving state-of-the-art accuracy, which
we will demonstrate on a range of challenging real-world scenes.

BibTeX

@online{Mehta_arXiv1907.00837,
TITLE = {{XNect}: Real-time Multi-person {3D} Human Pose Estimation with a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Elgharib, Mohamed and Fua, Pascal and Seidel, Hans-Peter and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1907.00837},
EPRINT = {1907.00837},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We present a real-time approach for multi-person 3D motion capture at over 30<br>fps using a single RGB camera. It operates in generic scenes and is robust to<br>difficult occlusions both by other people and objects. Our method operates in<br>subsequent stages. The first stage is a convolutional neural network (CNN) that<br>estimates 2D and 3D pose features along with identity assignments for all<br>visible joints of all individuals. We contribute a new architecture for this<br>CNN, called SelecSLS Net, that uses novel selective long and short range skip<br>connections to improve the information flow allowing for a drastically faster<br>network without compromising accuracy. In the second stage, a fully-connected<br>neural network turns the possibly partial (on account of occlusion) 2D pose and<br>3D pose features for each subject into a complete 3D pose estimate per<br>individual. The third stage applies space-time skeletal model fitting to the<br>predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose,<br>and enforce temporal coherence. Our method returns the full skeletal pose in<br>joint angles for each subject. This is a further key distinction from previous<br>work that neither extracted global body positions nor joint angle results of a<br>coherent skeleton in real time for multi-person scenes. The proposed system<br>runs on consumer hardware at a previously unseen speed of more than 30 fps<br>given 512x320 images as input while achieving state-of-the-art accuracy, which<br>we will demonstrate on a range of challenging real-world scenes.<br>},
}

Endnote

%0 Report
%A Mehta, Dushyant
%A Sotnychenko, Oleksandr
%A Mueller, Franziska
%A Xu, Weipeng
%A Elgharib, Mohamed
%A Fua, Pascal
%A Seidel, Hans-Peter
%A Rhodin, Helge
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-FE21-A
%U http://arxiv.org/abs/1907.00837
%D 2019
%X   We present a real-time approach for multi-person 3D motion capture at over 30<br>fps using a single RGB camera. It operates in generic scenes and is robust to<br>difficult occlusions both by other people and objects. Our method operates in<br>subsequent stages. The first stage is a convolutional neural network (CNN) that<br>estimates 2D and 3D pose features along with identity assignments for all<br>visible joints of all individuals. We contribute a new architecture for this<br>CNN, called SelecSLS Net, that uses novel selective long and short range skip<br>connections to improve the information flow allowing for a drastically faster<br>network without compromising accuracy. In the second stage, a fully-connected<br>neural network turns the possibly partial (on account of occlusion) 2D pose and<br>3D pose features for each subject into a complete 3D pose estimate per<br>individual. The third stage applies space-time skeletal model fitting to the<br>predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose,<br>and enforce temporal coherence. Our method returns the full skeletal pose in<br>joint angles for each subject. This is a further key distinction from previous<br>work that neither extracted global body positions nor joint angle results of a<br>coherent skeleton in real time for multi-person scenes. The proposed system<br>runs on consumer hardware at a previously unseen speed of more than 30 fps<br>given 512x320 images as input while achieving state-of-the-art accuracy, which<br>we will demonstrate on a range of challenging real-world scenes.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Article

A. Meka, C. Hane, R. Pandey, M. Zollhöfer, S. Fanello, G. Fyffe, A. Kowdle, X. Yu, J. Busch, J. Dour-Garian, P. Denny, S. Bouaziz, P. Lincoln, M. Whalen, G. Harvey, J. Taylor, S. Izadi, A. Tagliasacchi, P. Debevec, C. Theobalt, J. Valentin, and C. Rhemann

“Deep Reflectance Fields High-Quality Facial Reflectance Field Inference from Color Gradient Illumination,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2019), vol. 38, no. 4, 2019.

@article{Meka_SIGGRAPH2019,
TITLE = {Deep Reflectance Fields High-Quality Facial Reflectance Field Inference from Color Gradient Illumination},
AUTHOR = {Meka, Abhimitra and Hane, Christian and Pandey, Rohit and Zollh{\"o}fer, Michael and Fanello, Sean and Fyffe, Graham and Kowdle, Adarsh and Yu, Xueming and Busch, Jay and Dour-Garian, Jason and Denny, Peter and Bouaziz, Sofien and Lincoln, Peter and Whalen, Matt and Harvey, Geoff and Taylor, Jonathan and Izadi, Shahram and Tagliasacchi, Andrea and Debevec, Paul and Theobalt, Christian and Valentin, Julien and Rhemann, Christoph},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3306346.3323027},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {38},
NUMBER = {4},
EID = {77},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2019},
}

Endnote

%0 Journal Article
%A Meka, Abhimitra
%A Hane, Christian
%A Pandey, Rohit
%A Zollh&#246;fer, Michael
%A Fanello, Sean
%A Fyffe, Graham
%A Kowdle, Adarsh
%A Yu, Xueming
%A Busch, Jay
%A Dour-Garian, Jason
%A Denny, Peter
%A Bouaziz, Sofien
%A Lincoln, Peter
%A Whalen, Matt
%A Harvey, Geoff
%A Taylor, Jonathan
%A Izadi, Shahram
%A Tagliasacchi, Andrea
%A Debevec, Paul
%A Theobalt, Christian
%A Valentin, Julien
%A Rhemann, Christoph
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Deep Reflectance Fields High-Quality Facial Reflectance Field Inference from Color Gradient Illumination : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-8453-9
%R 10.1145/3306346.3323027
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 4
%Z sequence number: 77
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2019
%O ACM SIGGRAPH 2019 Los Angeles, CA, USA, 28 July - 1 August

Article

F. Mueller, M. Davis, F. Bernard, O. Sotnychenko, M. Verschoor, M. A. Otaduy, D. Casas, and C. Theobalt

“Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2019), vol. 38, no. 4, 2019.

@article{MuellerTOG2019,
TITLE = {Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera},
AUTHOR = {Mueller, Franziska and Davis, Micah and Bernard, Florian and Sotnychenko, Oleksandr and Verschoor, Mickeal and Otaduy, Miguel A. and Casas, Dan and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3306346.3322958},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {38},
NUMBER = {4},
EID = {49},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2019},
}

Endnote

%0 Journal Article
%A Mueller, Franziska
%A Davis, Micah
%A Bernard, Florian
%A Sotnychenko, Oleksandr
%A Verschoor, Mickeal
%A Otaduy, Miguel A.
%A Casas, Dan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-844A-4
%R 10.1145/3306346.3322958
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 4
%Z sequence number: 49
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2019
%O ACM SIGGRAPH 2019 Los Angeles, CA, USA, 28 July - 1 August

Thesis

D4IMPR-CS

N. Robertini

“Model-based Human Performance Capture in Outdoor Scenes,” Universität des Saarlandes, Saarbrücken, 2019.

Abstract

Technologies for motion and performance capture of real actors have enabled the creation of realisticlooking virtual humans through detail and deformation transfer at the cost of extensive manual work and sophisticated in-studio marker-based systems. This thesis pushes the boundaries of performance capture by proposing automatic algorithms for robust 3D skeleton and detailed surface tracking in less constrained multi-view outdoor scenarios. Contributions include new multi-layered human body representations designed for effective model-based time-consistent reconstruction in complex dynamic environments with varying illumination, from a set of vision cameras. We design dense surface refinement approaches to enable smooth silhouette-free model-to-image alignment, as well as coarse-to-fine tracking techniques to enable joint estimation of skeleton motion and finescale surface deformations in complicated scenarios. High-quality results attained on challenging application scenarios confirm the contributions and show great potential for the automatic creation of personalized 3D virtual humans.

BibTeX

@phdthesis{Robertini_PhD2019,
TITLE = {Model-based Human Performance Capture in Outdoor Scenes},
AUTHOR = {Robertini, Nadia},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291--ds-285887},
DOI = {10.22028/D291-28588},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2019},
DATE = {2019},
ABSTRACT = {Technologies for motion and performance capture of real actors have enabled the creation of realisticlooking virtual humans through detail and deformation transfer at the cost of extensive manual work and sophisticated in-studio marker-based systems. This thesis pushes the boundaries of performance capture by proposing automatic algorithms for robust 3D skeleton and detailed surface tracking in less constrained multi-view outdoor scenarios. Contributions include new multi-layered human body representations designed for effective model-based time-consistent reconstruction in complex dynamic environments with varying illumination, from a set of vision cameras. We design dense surface refinement approaches to enable smooth silhouette-free model-to-image alignment, as well as coarse-to-fine tracking techniques to enable joint estimation of skeleton motion and finescale surface deformations in complicated scenarios. High-quality results attained on challenging application scenarios confirm the contributions and show great potential for the automatic creation of personalized 3D virtual humans.},
}

Endnote

%0 Thesis
%A Robertini, Nadia
%Y Theobalt, Christian
%A referee: Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Model-based Human Performance Capture in Outdoor Scenes : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-9B2E-B
%U urn:nbn:de:bsz:291--ds-285887
%F OTHER: hdl:20.500.11880/27667
%R 10.22028/D291-28588
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2019
%P XIX, 136, XI p.
%V phd
%9 phd
%X Technologies for motion and performance capture of real actors have enabled the creation of realisticlooking virtual humans through detail and deformation transfer at the cost of extensive manual work and sophisticated in-studio marker-based systems. This thesis pushes the boundaries of performance capture by proposing automatic algorithms for robust 3D skeleton and detailed surface tracking in less constrained multi-view outdoor scenarios. Contributions include new multi-layered human body representations designed for effective model-based time-consistent reconstruction in complex dynamic environments with varying illumination, from a set of vision cameras. We design dense surface refinement approaches to enable smooth silhouette-free model-to-image alignment, as well as coarse-to-fine tracking techniques to enable joint estimation of skeleton motion and finescale surface deformations in complicated scenarios. High-quality results attained on challenging application scenarios confirm the contributions and show great potential for the automatic creation of personalized 3D virtual humans.
%U https://scidok.sulb.uni-saarland.de/handle/20.500.11880/27667

Conference paper

S. Shekhar, A. Semmo, M. Trapp, O. T. Tursun, S. Pasewaldt, K. Myszkowski, and J. Döllner

“Consistent Filtering of Videos and Dense Light-Fields without Optic-Flow,” in Vision, Modeling and Visualization 2019 (VMV 2019), Rostock, Germany, 2019.

@inproceedings{Shekhar_VMV2019,
TITLE = {Consistent Filtering of Videos and Dense Light-Fields without Optic-Flow},
AUTHOR = {Shekhar, Sumit and Semmo, Amir and Trapp, Matthias and Tursun, Okan Tarhan and Pasewaldt, Sebastian and Myszkowski, Karol and D{\"o}llner, J{\"u}rgen},
LANGUAGE = {eng},
ISBN = {978-3-03868-098-7},
DOI = {10.2312/vmv.20191326},
PUBLISHER = {Eurographics Association},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {Vision, Modeling and Visualization 2019 (VMV 2019)},
PAGES = {125--134},
ADDRESS = {Rostock, Germany},
}

Endnote

%0 Conference Proceedings
%A Shekhar, Sumit
%A Semmo, Amir
%A Trapp, Matthias
%A Tursun, Okan Tarhan
%A Pasewaldt, Sebastian
%A Myszkowski, Karol
%A D&#246;llner, J&#252;rgen
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Consistent Filtering of Videos and Dense Light-Fields without 
Optic-Flow : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-9C10-A
%R 10.2312/vmv.20191326
%D 2019
%B 24th International Symposium on Vision, Modeling, and Visualization
%Z date of event: 2019-09-30 - 2019-10-02
%C Rostock, Germany
%B Vision, Modeling and Visualization 2019
%P 125 - 134
%I Eurographics Association
%@ 978-3-03868-098-7

Conference paper

S. Shimada, V. Golyanik, E. Tretschk, D. Stricker, and C. Theobalt

“DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies,” in International Conference on 3D Vision, Québec City, Canada, 2019.

@inproceedings{Shimada_3DV2019,
TITLE = {{DispVoxNets}: {N}on-Rigid Point Set Alignment with Supervised Learning Proxies},
AUTHOR = {Shimada, Soshi and Golyanik, Vladislav and Tretschk, Edgar and Stricker, Didier and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3131-3},
DOI = {10.1109/3DV.2019.00013},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {International Conference on 3D Vision},
PAGES = {27--36},
ADDRESS = {Qu{\'e}bec City, Canada},
}

Endnote

%0 Conference Proceedings
%A Shimada, Soshi
%A Golyanik, Vladislav
%A Tretschk, Edgar
%A Stricker, Didier
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7B48-1
%R 10.1109/3DV.2019.00013
%D 2019
%B International Conference on 3D Vision
%Z date of event: 2019-09-16 - 2019-09-19
%C Qu&#233;bec City, Canada
%B International Conference on 3D Vision
%P 27 - 36
%I IEEE
%@ 978-1-7281-3131-3

Paper

S. Shimada, V. Golyanik, C. Theobalt, and D. Stricker

“IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction,” 2019. [Online]. Available: http://arxiv.org/abs/1904.12144.

Abstract

The majority of the existing methods for non-rigid 3D surface regression from
monocular 2D images require an object template or point tracks over multiple
frames as an input, and are still far from real-time processing rates. In this
work, we present the Isometry-Aware Monocular Generative Adversarial Network
(IsMo-GAN) - an approach for direct 3D reconstruction from a single image,
trained for the deformation model in an adversarial manner on a light-weight
synthetic dataset. IsMo-GAN reconstructs surfaces from real images under
varying illumination, camera poses, textures and shading at over 250 Hz. In
multiple experiments, it consistently outperforms several approaches in the
reconstruction accuracy, runtime, generalisation to unknown surfaces and
robustness to occlusions. In comparison to the state-of-the-art, we reduce the
reconstruction error by 10-30% including the textureless case and our surfaces
evince fewer artefacts qualitatively.

BibTeX

@online{Shimada_arXiv1904.12144,
TITLE = {{IsMo}-{GAN}: Adversarial Learning for Monocular Non-Rigid {3D} Reconstruction},
AUTHOR = {Shimada, Soshi and Golyanik, Vladislav and Theobalt, Christian and Stricker, Didier},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1904.12144},
EPRINT = {1904.12144},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {The majority of the existing methods for non-rigid 3D surface regression from<br>monocular 2D images require an object template or point tracks over multiple<br>frames as an input, and are still far from real-time processing rates. In this<br>work, we present the Isometry-Aware Monocular Generative Adversarial Network<br>(IsMo-GAN) -- an approach for direct 3D reconstruction from a single image,<br>trained for the deformation model in an adversarial manner on a light-weight<br>synthetic dataset. IsMo-GAN reconstructs surfaces from real images under<br>varying illumination, camera poses, textures and shading at over 250 Hz. In<br>multiple experiments, it consistently outperforms several approaches in the<br>reconstruction accuracy, runtime, generalisation to unknown surfaces and<br>robustness to occlusions. In comparison to the state-of-the-art, we reduce the<br>reconstruction error by 10-30% including the textureless case and our surfaces<br>evince fewer artefacts qualitatively.<br>},
}

Endnote

%0 Report
%A Shimada, Soshi
%A Golyanik, Vladislav
%A Theobalt, Christian
%A Stricker, Didier
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-FE04-B
%U http://arxiv.org/abs/1904.12144
%D 2019
%X   The majority of the existing methods for non-rigid 3D surface regression from<br>monocular 2D images require an object template or point tracks over multiple<br>frames as an input, and are still far from real-time processing rates. In this<br>work, we present the Isometry-Aware Monocular Generative Adversarial Network<br>(IsMo-GAN) - an approach for direct 3D reconstruction from a single image,<br>trained for the deformation model in an adversarial manner on a light-weight<br>synthetic dataset. IsMo-GAN reconstructs surfaces from real images under<br>varying illumination, camera poses, textures and shading at over 250 Hz. In<br>multiple experiments, it consistently outperforms several approaches in the<br>reconstruction accuracy, runtime, generalisation to unknown surfaces and<br>robustness to occlusions. In comparison to the state-of-the-art, we reduce the<br>reconstruction error by 10-30% including the textureless case and our surfaces<br>evince fewer artefacts qualitatively.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Paper

S. Shimada, V. Golyanik, E. Tretschk, D. Stricker, and C. Theobalt

“DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning Proxies,” 2019. [Online]. Available: http://arxiv.org/abs/1907.10367.

Abstract

We introduce a supervised-learning framework for non-rigid point set
alignment of a new kind - Displacements on Voxels Networks (DispVoxNets) -
which abstracts away from the point set representation and regresses 3D
displacement fields on regularly sampled proxy 3D voxel grids. Thanks to
recently released collections of deformable objects with known intra-state
correspondences, DispVoxNets learn a deformation model and further priors
(e.g., weak point topology preservation) for different object categories such
as cloths, human bodies and faces. DispVoxNets cope with large deformations,
noise and clustered outliers more robustly than the state-of-the-art. At test
time, our approach runs orders of magnitude faster than previous techniques.
All properties of DispVoxNets are ascertained numerically and qualitatively in
extensive experiments and comparisons to several previous methods.

BibTeX

@online{Shimada_arXiv1907.10367,
TITLE = {{DispVoxNets}: {N}on-Rigid Point Set Alignment with Supervised Learning Proxies},
AUTHOR = {Shimada, Soshi and Golyanik, Vladislav and Tretschk, Edgar and Stricker, Didier and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1907.10367},
EPRINT = {1907.10367},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We introduce a supervised-learning framework for non-rigid point set<br>alignment of a new kind -- Displacements on Voxels Networks (DispVoxNets) -<br>which abstracts away from the point set representation and regresses 3D<br>displacement fields on regularly sampled proxy 3D voxel grids. Thanks to<br>recently released collections of deformable objects with known intra-state<br>correspondences, DispVoxNets learn a deformation model and further priors<br>(e.g., weak point topology preservation) for different object categories such<br>as cloths, human bodies and faces. DispVoxNets cope with large deformations,<br>noise and clustered outliers more robustly than the state-of-the-art. At test<br>time, our approach runs orders of magnitude faster than previous techniques.<br>All properties of DispVoxNets are ascertained numerically and qualitatively in<br>extensive experiments and comparisons to several previous methods.<br>},
}

Endnote

%0 Report
%A Shimada, Soshi
%A Golyanik, Vladislav
%A Tretschk, Edgar
%A Stricker, Didier
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T DispVoxNets: Non-Rigid Point Set Alignment with Supervised Learning
  Proxies : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7D04-B
%U http://arxiv.org/abs/1907.10367
%D 2019
%X   We introduce a supervised-learning framework for non-rigid point set<br>alignment of a new kind - Displacements on Voxels Networks (DispVoxNets) -<br>which abstracts away from the point set representation and regresses 3D<br>displacement fields on regularly sampled proxy 3D voxel grids. Thanks to<br>recently released collections of deformable objects with known intra-state<br>correspondences, DispVoxNets learn a deformation model and further priors<br>(e.g., weak point topology preservation) for different object categories such<br>as cloths, human bodies and faces. DispVoxNets cope with large deformations,<br>noise and clustered outliers more robustly than the state-of-the-art. At test<br>time, our approach runs orders of magnitude faster than previous techniques.<br>All properties of DispVoxNets are ascertained numerically and qualitatively in<br>extensive experiments and comparisons to several previous methods.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Computational Geometry, cs.CG

Conference paper

S. Shimada, V. Golyanik, C. Theobalt, and D. Stricker

“IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2019), Long Beach, CA, USA, 2019.

@inproceedings{Shimada_2019,
TITLE = {{IsMo-GAN}: {A}dversarial Learning for Monocular Non-Rigid {3D} Reconstruction},
AUTHOR = {Shimada, Soshi and Golyanik, Vladislav and Theobalt, Christian and Stricker, Didier},
LANGUAGE = {eng},
PUBLISHER = {Computer Vision Foundation},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW 2019)},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Shimada, Soshi
%A Golyanik, Vladislav
%A Theobalt, Christian
%A Stricker, Didier
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D 
Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-9410-1
%D 2019
%B Photogrammetric Computer Vision Workshop
%Z date of event: 2019-06-17 - 2019-06-17
%C Long Beach, CA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
%I Computer Vision Foundation

Article

G. Singh, C. Öztireli, A. G. M. Ahmed, D. Coeurjolly, K. Subr, V. Ostromoukhov, O. Deussen, R. Ramamoorthi, and W. Jarosz

“Analysis of Sample Correlations for Monte Carlo Rendering,” Computer Graphics Forum (Proc. EUROGRAPHICS 2019), vol. 38, no. 2, 2019.

@article{Singh_EG2019STAR,
TITLE = {Analysis of Sample Correlations for {Monte Carlo} Rendering},
AUTHOR = {Singh, Gurprit and {\"O}ztireli, Cengiz and Ahmed, Abdalla G.M. and Coeurjolly, David and Subr, Kartic and Ostromoukhov, Victor and Deussen, Oliver and Ramamoorthi, Ravi and Jarosz, Wojciech},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13653},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2019},
DATE = {2019},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {38},
NUMBER = {2},
PAGES = {473--491},
BOOKTITLE = {EUROGRAPHICS 2019 STAR -- State of The Art Reports},
}

Endnote

%0 Journal Article
%A Singh, Gurprit
%A &#214;ztireli, Cengiz
%A Ahmed, Abdalla G.M.
%A Coeurjolly, David
%A Subr, Kartic
%A Ostromoukhov, Victor
%A Deussen, Oliver
%A Ramamoorthi, Ravi
%A Jarosz, Wojciech
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Analysis of Sample Correlations for Monte Carlo Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F487-2
%R 10.1111/cgf.13653
%7 2019
%D 2019
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 38
%N 2
%& 473
%P 473 - 491
%I Blackwell-Wiley
%C Oxford
%@ false
%B EUROGRAPHICS 2019 STAR &#8211; State of The Art Reports
%O EUROGRAPHICS 2019 EG 2019 The 40th Annual Conference of the European Association for Computer Graphics ; Genova, Italy, May 6-10

Article

D. Sumin, T. Rittig, V. Babaei, T. Nindel, A. Wilkie, P. Didyk, B. Bickel, J. Křivánek, K. Myszkowski, and T. Weyrich

“Geometry-Aware Scattering Compensation for 3D Printing,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2019), vol. 38, no. 4, 2019.

@article{SuminRittig2019,
TITLE = {Geometry-Aware Scattering Compensation for {3D} Printing},
AUTHOR = {Sumin, Denis and Rittig, Tobias and Babaei, Vahid and Nindel, Thomas and Wilkie, Alexander and Didyk, Piotr and Bickel, Bernd and K{\v r}iv{\'a}nek, Jaroslav and Myszkowski, Karol and Weyrich, Tim},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3306346.3322992},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {38},
NUMBER = {4},
EID = {111},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2019},
}

Endnote

%0 Journal Article
%A Sumin, Denis
%A Rittig, Tobias
%A Babaei, Vahid
%A Nindel, Thomas
%A Wilkie, Alexander
%A Didyk, Piotr
%A Bickel, Bernd
%A K&#345;iv&#225;nek, Jaroslav
%A Myszkowski, Karol
%A Weyrich, Tim
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Geometry-Aware Scattering Compensation for 3D Printing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-7D65-0
%R 10.1145/3306346.3322992
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 4
%Z sequence number: 111
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2019
%O ACM SIGGRAPH 2019 Los Angeles, CA, USA, 28 July - 1 August

Conference paper

Y. Su, V. Golyanik, N. Minaskan, S. A. Ali, and D. Stricker

“A Shape Completion Component for Monocular Non-Rigid SLAM,” in Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct 2019), Beijing, China, 2019.

@inproceedings{Su_ISMAR2019,
TITLE = {A Shape Completion Component for Monocular Non-Rigid {SLAM}},
AUTHOR = {Su, Yongzhi and Golyanik, Vladislav and Minaskan, Nareg and Ali, Sk Aziz and Stricker, Didier},
LANGUAGE = {eng},
ISBN = {978-1-7281-4765-9},
DOI = {10.1109/ISMAR-Adjunct.2019.00-18},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct 2019)},
PAGES = {332--337},
ADDRESS = {Beijing, China},
}

Endnote

%0 Conference Proceedings
%A Su, Yongzhi
%A Golyanik, Vladislav
%A Minaskan, Nareg
%A Ali, Sk Aziz
%A Stricker, Didier
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T A Shape Completion Component for Monocular Non-Rigid SLAM : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-9832-7
%R 10.1109/ISMAR-Adjunct.2019.00-18
%D 2019
%B IEEE International Symposium on Mixed and Augmented Reality
%Z date of event: 2019-10-14 - 2019-10-18
%C Beijing, China
%B Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality
%P 332 - 337
%I IEEE
%@ 978-1-7281-4765-9

Conference paper

D2D4

P. Swoboda, D. Kainmüller, A. Mokarian, C. Theobalt, and F. Bernard

“A Convex Relaxation for Multi-Graph Matching,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, 2019.

@inproceedings{SwobodaCVPR2019a,
TITLE = {A Convex Relaxation for Multi-Graph Matching},
AUTHOR = {Swoboda, Paul and Kainm{\"u}ller, Dagmar and Mokarian, Ashkan and Theobalt, Christian and Bernard, Florian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3293-8},
DOI = {10.1109/CVPR.2019.01141},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
PAGES = {11156--11165},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Swoboda, Paul
%A Kainm&#252;ller, Dagmar
%A Mokarian, Ashkan
%A Theobalt, Christian
%A Bernard, Florian
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T A Convex Relaxation for Multi-Graph Matching : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-74B8-9
%R 10.1109/CVPR.2019.01141
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2019-06-15 - 2019-06-20
%C Long Beach, CA,  USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 11156 - 11165
%I IEEE
%@ 978-1-7281-3293-8

Conference paper

A. Tewari, F. Bernard, P. Garrido, G. Bharaj, M. Elgharib, H.-P. Seidel, P. Pérez, M. Zollhöfer, and C. Theobalt

“FML: Face Model Learning From Videos,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, 2019.

@inproceedings{TewariCVPR2019,
TITLE = {{FML}: {F}ace Model Learning From Videos},
AUTHOR = {Tewari, Ayush and Bernard, Florian and Garrido, Pablo and Bharaj, Gaurav and Elgharib, Mohamed and Seidel, Hans-Peter and P{\'e}rez, Patrick and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3293-8},
DOI = {10.1109/CVPR.2019.01107},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
PAGES = {10812--10822},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Tewari, Ayush
%A Bernard, Florian
%A Garrido, Pablo
%A Bharaj, Gaurav
%A Elgharib, Mohamed
%A Seidel, Hans-Peter
%A P&#233;rez, Patrick
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T FML: Face Model Learning From Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7B0C-5
%R 10.1109/CVPR.2019.01107
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2019-06-16 - 2019-06-20
%C Long Beach, CA,  USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition 
%P 10812 - 10822
%I IEEE
%@ 978-1-7281-3293-8

Paper

J. Thies, M. Elgharib, A. Tewari, C. Theobalt, and M. Nießner

“Neural Voice Puppetry: Audio-driven Facial Reenactment,” 2019. [Online]. Available: http://arxiv.org/abs/1912.05566.

Abstract

We present Neural Voice Puppetry, a novel approach for audio-driven facial
video synthesis. Given an audio sequence of a source person or digital
assistant, we generate a photo-realistic output video of a target person that
is in sync with the audio of the source input. This audio-driven facial
reenactment is driven by a deep neural network that employs a latent 3D face
model space. Through the underlying 3D representation, the model inherently
learns temporal stability while we leverage neural rendering to generate
photo-realistic output frames. Our approach generalizes across different
people, allowing us to synthesize videos of a target actor with the voice of
any unknown source actor or even synthetic voices that can be generated
utilizing standard text-to-speech approaches. Neural Voice Puppetry has a
variety of use-cases, including audio-driven video avatars, video dubbing, and
text-driven video synthesis of a talking head. We demonstrate the capabilities
of our method in a series of audio- and text-based puppetry examples. Our
method is not only more general than existing works since we are generic to the
input person, but we also show superior visual and lip sync quality compared to
photo-realistic audio- and video-driven reenactment techniques.

BibTeX

@online{Thies_arXiv1912.05566,
TITLE = {Neural Voice Puppetry: Audio-driven Facial Reenactment},
AUTHOR = {Thies, Justus and Elgharib, Mohamed and Tewari, Ayush and Theobalt, Christian and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1912.05566},
EPRINT = {1912.05566},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We present Neural Voice Puppetry, a novel approach for audio-driven facial<br>video synthesis. Given an audio sequence of a source person or digital<br>assistant, we generate a photo-realistic output video of a target person that<br>is in sync with the audio of the source input. This audio-driven facial<br>reenactment is driven by a deep neural network that employs a latent 3D face<br>model space. Through the underlying 3D representation, the model inherently<br>learns temporal stability while we leverage neural rendering to generate<br>photo-realistic output frames. Our approach generalizes across different<br>people, allowing us to synthesize videos of a target actor with the voice of<br>any unknown source actor or even synthetic voices that can be generated<br>utilizing standard text-to-speech approaches. Neural Voice Puppetry has a<br>variety of use-cases, including audio-driven video avatars, video dubbing, and<br>text-driven video synthesis of a talking head. We demonstrate the capabilities<br>of our method in a series of audio- and text-based puppetry examples. Our<br>method is not only more general than existing works since we are generic to the<br>input person, but we also show superior visual and lip sync quality compared to<br>photo-realistic audio- and video-driven reenactment techniques.<br>},
}

Endnote

%0 Report
%A Thies, Justus
%A Elgharib, Mohamed
%A Tewari, Ayush
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Neural Voice Puppetry: Audio-driven Facial Reenactment : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7DE3-F
%U http://arxiv.org/abs/1912.05566
%D 2019
%X   We present Neural Voice Puppetry, a novel approach for audio-driven facial<br>video synthesis. Given an audio sequence of a source person or digital<br>assistant, we generate a photo-realistic output video of a target person that<br>is in sync with the audio of the source input. This audio-driven facial<br>reenactment is driven by a deep neural network that employs a latent 3D face<br>model space. Through the underlying 3D representation, the model inherently<br>learns temporal stability while we leverage neural rendering to generate<br>photo-realistic output frames. Our approach generalizes across different<br>people, allowing us to synthesize videos of a target actor with the voice of<br>any unknown source actor or even synthetic voices that can be generated<br>utilizing standard text-to-speech approaches. Neural Voice Puppetry has a<br>variety of use-cases, including audio-driven video avatars, video dubbing, and<br>text-driven video synthesis of a talking head. We demonstrate the capabilities<br>of our method in a series of audio- and text-based puppetry examples. Our<br>method is not only more general than existing works since we are generic to the<br>input person, but we also show superior visual and lip sync quality compared to<br>photo-realistic audio- and video-driven reenactment techniques.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Article

J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner

“Face2Face: Real-Time Face Capture and Reenactment of RGB Videos,” Communications of the ACM, vol. 62, no. 1, 2019.

@article{thies2019face,
TITLE = {{Face2Face}: {R}eal-Time Face Capture and Reenactment of {RGB} Videos},
AUTHOR = {Thies, Justus and Zollh{\"o}fer, Michael and Stamminger, Marc and Theobalt, Christian and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
ISSN = {0001-0782},
DOI = {10.1145/3292039},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {Communications of the ACM},
VOLUME = {62},
NUMBER = {1},
PAGES = {96--104},
}

Endnote

%0 Journal Article
%A Thies, Justus
%A Zollh&#246;fer, Michael
%A Stamminger, Marc
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Face2Face: Real-Time Face Capture and Reenactment of RGB Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-C0A7-8
%R 10.1145/3292039
%7 2019
%D 2019
%J Communications of the ACM
%V 62
%N 1
%& 96
%P 96 - 104
%I ACM
%C New York, NY
%@ false

Paper

E. Tretschk, A. Tewari, M. Zollhöfer, V. Golyanik, and C. Theobalt

“DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects,” 2019. [Online]. Available: http://arxiv.org/abs/1905.10290.

Abstract

Mesh autoencoders are commonly used for dimensionality reduction, sampling
and mesh modeling. We propose a general-purpose DEep MEsh Autoencoder (DEMEA)
which adds a novel embedded deformation layer to a graph-convolutional mesh
autoencoder. The embedded deformation layer (EDL) is a differentiable
deformable geometric proxy which explicitly models point displacements of
non-rigid deformations in a lower dimensional space and serves as a local
rigidity regularizer. DEMEA decouples the parameterization of the deformation
from the final mesh resolution since the deformation is defined over a lower
dimensional embedded deformation graph. We perform a large-scale study on four
different datasets of deformable objects. Reasoning about the local rigidity of
meshes using EDL allows us to achieve higher-quality results for highly
deformable objects, compared to directly regressing vertex positions. We
demonstrate multiple applications of DEMEA, including non-rigid 3D
reconstruction from depth and shading cues, non-rigid surface tracking, as well
as the transfer of deformations over different meshes.

BibTeX

@online{Tretschk_arXIv1905.10290,
TITLE = {{DEMEA}: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects},
AUTHOR = {Tretschk, Edgar and Tewari, Ayush and Zollh{\"o}fer, Michael and Golyanik, Vladislav and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1905.10290},
EPRINT = {1905.10290},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {Mesh autoencoders are commonly used for dimensionality reduction, sampling<br>and mesh modeling. We propose a general-purpose DEep MEsh Autoencoder (DEMEA)<br>which adds a novel embedded deformation layer to a graph-convolutional mesh<br>autoencoder. The embedded deformation layer (EDL) is a differentiable<br>deformable geometric proxy which explicitly models point displacements of<br>non-rigid deformations in a lower dimensional space and serves as a local<br>rigidity regularizer. DEMEA decouples the parameterization of the deformation<br>from the final mesh resolution since the deformation is defined over a lower<br>dimensional embedded deformation graph. We perform a large-scale study on four<br>different datasets of deformable objects. Reasoning about the local rigidity of<br>meshes using EDL allows us to achieve higher-quality results for highly<br>deformable objects, compared to directly regressing vertex positions. We<br>demonstrate multiple applications of DEMEA, including non-rigid 3D<br>reconstruction from depth and shading cues, non-rigid surface tracking, as well<br>as the transfer of deformations over different meshes.<br>},
}

Endnote

%0 Report
%A Tretschk, Edgar
%A Tewari, Ayush
%A Zollh&#246;fer, Michael
%A Golyanik, Vladislav
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-FE0C-3
%U http://arxiv.org/abs/1905.10290
%D 2019
%X   Mesh autoencoders are commonly used for dimensionality reduction, sampling<br>and mesh modeling. We propose a general-purpose DEep MEsh Autoencoder (DEMEA)<br>which adds a novel embedded deformation layer to a graph-convolutional mesh<br>autoencoder. The embedded deformation layer (EDL) is a differentiable<br>deformable geometric proxy which explicitly models point displacements of<br>non-rigid deformations in a lower dimensional space and serves as a local<br>rigidity regularizer. DEMEA decouples the parameterization of the deformation<br>from the final mesh resolution since the deformation is defined over a lower<br>dimensional embedded deformation graph. We perform a large-scale study on four<br>different datasets of deformable objects. Reasoning about the local rigidity of<br>meshes using EDL allows us to achieve higher-quality results for highly<br>deformable objects, compared to directly regressing vertex positions. We<br>demonstrate multiple applications of DEMEA, including non-rigid 3D<br>reconstruction from depth and shading cues, non-rigid surface tracking, as well<br>as the transfer of deformations over different meshes.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Article

O. T. Tursun, E. Arabadzhiyska, M. Wernikowski, R. Mantiuk, H.-P. Seidel, K. Myszkowski, and P. Didyk

“Luminance-Contrast-Aware Foveated Rendering,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2019), vol. 38, no. 4, 2019.

@article{Tursun2019Luminance,
TITLE = {Luminance-Contrast-Aware Foveated Rendering},
AUTHOR = {Tursun, Okan Tarhan and Arabadzhiyska, Elena and Wernikowski, Marek and Mantiuk, Rados{\l}aw and Seidel, Hans-Peter and Myszkowski, Karol and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3306346.3322985},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {38},
NUMBER = {4},
EID = {98},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2019},
}

Endnote

%0 Journal Article
%A Tursun, Okan Tarhan
%A Arabadzhiyska, Elena
%A Wernikowski, Marek
%A Mantiuk, Rados&#322;aw
%A Seidel, Hans-Peter
%A Myszkowski, Karol
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Luminance-Contrast-Aware Foveated Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-75D5-9
%R 10.1145/3306346.3322985
%7 2019
%D 2019
%J ACM Transactions on Graphics
%V 38
%N 4
%Z sequence number: 98
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2019
%O ACM SIGGRAPH 2019 Los Angeles, CA, USA, 28 July - 1 August

Conference paper

M. Winter, D. Mlakar, R. Zayer, H.-P. Seidel, and M. Steinberger

“Adaptive Sparse Matrix-Matrix Multiplication on the GPU,” in PPoPP’19, 24th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Washington, DC, USA, 2019.

@inproceedings{PPOPP:2019:ASPMM,
TITLE = {Adaptive Sparse Matrix-Matrix Multiplication on the {GPU}},
AUTHOR = {Winter, Martin and Mlakar, Daniel and Zayer, Rhaleb and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISBN = {978-1-4503-6225-2},
DOI = {10.1145/3293883.3295701},
PUBLISHER = {ACM},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {PPoPP'19, 24th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming},
PAGES = {68--81},
ADDRESS = {Washington, DC, USA},
}

Endnote

%0 Conference Proceedings
%A Winter, Martin
%A Mlakar, Daniel
%A Zayer, Rhaleb
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Adaptive Sparse Matrix-Matrix Multiplication on the GPU : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-EFE9-B
%R 10.1145/3293883.3295701
%D 2019
%B 24th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
%Z date of event: 2019-02-16 - 2019-02-20
%C Washington, DC, USA
%B PPoPP'19
%P 68 - 81
%I ACM
%@ 978-1-4503-6225-2
%U https://doi.org/10.5281/zenodo.14885935

Article

K. Wolski, D. Giunchi, S. Kinuwaki, P. Didyk, K. Myszkowski, A. Steed, and R. K. Mantiuk

“Selecting Texture Resolution Using a Task-specific Visibility Metric,” Computer Graphics Forum (Proc. Pacific Graphics 2019), vol. 38, no. 7, 2019.

@article{Wolski_PG2019,
TITLE = {Selecting Texture Resolution Using a Task-specific Visibility Metric},
AUTHOR = {Wolski, Krzysztof and Giunchi,, Daniele and Kinuwaki, Shinichi and Didyk, Piotr and Myszkowski, Karol and Steed, Anthony and Mantiuk, Rafa{\l} K.},
LANGUAGE = {eng},
ISSN = {1467-8659},
DOI = {10.1111/cgf.13871},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford, UK},
YEAR = {2019},
DATE = {2019},
JOURNAL = {Computer Graphics Forum (Proc. Pacific Graphics)},
VOLUME = {38},
NUMBER = {7},
PAGES = {685--696},
BOOKTITLE = {27th Annual International Conference on Computer Graphics and Applications (Pacific Graphics 2019)},
}

Endnote

%0 Journal Article
%A Wolski, Krzysztof
%A Giunchi,, Daniele
%A Kinuwaki, Shinichi
%A Didyk, Piotr
%A Myszkowski, Karol
%A Steed, Anthony
%A Mantiuk, Rafa&#322; K.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Selecting Texture Resolution Using a Task-specific Visibility Metric : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0004-9BB3-3
%R 10.1111/cgf.13871
%7 2019
%D 2019
%J Computer Graphics Forum
%V 38
%N 7
%& 685
%P 685 - 696
%I Wiley-Blackwell
%C Oxford, UK
%@ false
%B 27th Annual International Conference on Computer Graphics and Applications
%O Pacific Graphics 2019 PG 2019 Seoul, October 14-17, 2019

Paper

L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang, and C. Theobalt

“EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera,” 2019. [Online]. Available: http://arxiv.org/abs/1908.11505.

Abstract

The high frame rate is a critical requirement for capturing fast human
motions. In this setting, existing markerless image-based methods are
constrained by the lighting requirement, the high data bandwidth and the
consequent high computation overhead. In this paper, we propose EventCap ---
the first approach for 3D capturing of high-speed human motions using a single
event camera. Our method combines model-based optimization and CNN-based human
pose detection to capture high-frequency motion details and to reduce the
drifting in the tracking. As a result, we can capture fast motions at
millisecond resolution with significantly higher data efficiency than using
high frame rate videos. Experiments on our new event-based fast human motion
dataset demonstrate the effectiveness and accuracy of our method, as well as
its robustness to challenging lighting conditions.

BibTeX

@online{Xu_arXiv1908.11505,
TITLE = {{EventCap}: Monocular {3D} Capture of High-Speed Human Motions using an Event Camera},
AUTHOR = {Xu, Lan and Xu, Weipeng and Golyanik, Vladislav and Habermann, Marc and Fang, Lu and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1908.11505},
EPRINT = {1908.11505},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {The high frame rate is a critical requirement for capturing fast human<br>motions. In this setting, existing markerless image-based methods are<br>constrained by the lighting requirement, the high data bandwidth and the<br>consequent high computation overhead. In this paper, we propose EventCap ---<br>the first approach for 3D capturing of high-speed human motions using a single<br>event camera. Our method combines model-based optimization and CNN-based human<br>pose detection to capture high-frequency motion details and to reduce the<br>drifting in the tracking. As a result, we can capture fast motions at<br>millisecond resolution with significantly higher data efficiency than using<br>high frame rate videos. Experiments on our new event-based fast human motion<br>dataset demonstrate the effectiveness and accuracy of our method, as well as<br>its robustness to challenging lighting conditions.<br>},
}

Endnote

%0 Report
%A Xu, Lan
%A Xu, Weipeng
%A Golyanik, Vladislav
%A Habermann, Marc
%A Fang, Lu
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T EventCap: Monocular 3D Capture of High-Speed Human Motions using an
  Event Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7D7B-6
%U http://arxiv.org/abs/1908.11505
%D 2019
%X   The high frame rate is a critical requirement for capturing fast human<br>motions. In this setting, existing markerless image-based methods are<br>constrained by the lighting requirement, the high data bandwidth and the<br>consequent high computation overhead. In this paper, we propose EventCap ---<br>the first approach for 3D capturing of high-speed human motions using a single<br>event camera. Our method combines model-based optimization and CNN-based human<br>pose detection to capture high-frequency motion details and to reduce the<br>drifting in the tracking. As a result, we can capture fast motions at<br>millisecond resolution with significantly higher data efficiency than using<br>high frame rate videos. Experiments on our new event-based fast human motion<br>dataset demonstrate the effectiveness and accuracy of our method, as well as<br>its robustness to challenging lighting conditions.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Article

W. Xu, A. Chatterjee, M. Zollhöfer, H. Rhodin, P. Fua, H.-P. Seidel, and C. Theobalt

“Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera,” IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2019), vol. 25, no. 5, 2019.

@article{Xu2019Mo2Cap2,
TITLE = {{Mo2Cap2}: Real-time Mobile {3D} Motion Capture with a Cap-mounted Fisheye Camera},
AUTHOR = {Xu, Weipeng and Chatterjee, Avishek and Zollh{\"o}fer, Michael and Rhodin, Helge and Fua, Pascal and Seidel, Hans-Peter and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2019.2898650},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2019},
DATE = {2019},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR)},
VOLUME = {25},
NUMBER = {5},
PAGES = {2093--2101},
BOOKTITLE = {Selected Proceedings IEEE Virtual Reality 2019 (IEEE VR 2019)},
}

Endnote

%0 Journal Article
%A Xu, Weipeng
%A Chatterjee, Avishek
%A Zollh&#246;fer, Michael
%A Rhodin, Helge
%A Fua, Pascal
%A Seidel, Hans-Peter
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F1DB-7
%R 10.1109/TVCG.2019.2898650
%7 2019
%D 2019
%J IEEE Transactions on Visualization and Computer Graphics
%V 25
%N 5
%& 2093
%P 2093 - 2101
%I IEEE
%C Piscataway, NJ
%@ false
%B Selected Proceedings IEEE Virtual Reality 2019 
%O IEEE VR 2019 Osaka, Japan, March 23rd - 27th

Paper

T. Yenamandra, F. Bernard, J. Wang, F. Mueller, and C. Theobalt

“Convex Optimisation for Inverse Kinematics,” 2019. [Online]. Available: http://arxiv.org/abs/1910.11016.

Abstract

We consider the problem of inverse kinematics (IK), where one wants to find
the parameters of a given kinematic skeleton that best explain a set of
observed 3D joint locations. The kinematic skeleton has a tree structure, where
each node is a joint that has an associated geometric transformation that is
propagated to all its child nodes. The IK problem has various applications in
vision and graphics, for example for tracking or reconstructing articulated
objects, such as human hands or bodies. Most commonly, the IK problem is
tackled using local optimisation methods. A major downside of these approaches
is that, due to the non-convex nature of the problem, such methods are prone to
converge to unwanted local optima and therefore require a good initialisation.
In this paper we propose a convex optimisation approach for the IK problem
based on semidefinite programming, which admits a polynomial-time algorithm
that globally solves (a relaxation of) the IK problem. Experimentally, we
demonstrate that the proposed method significantly outperforms local
optimisation methods using different real-world skeletons.

BibTeX

@online{Yenamandra_arXiv1910.11016,
TITLE = {Convex Optimisation for Inverse Kinematics},
AUTHOR = {Yenamandra, Tarum and Bernard, Florian and Wang, Jiayi and Mueller, Franziska and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1910.11016},
EPRINT = {1910.11016},
EPRINTTYPE = {arXiv},
YEAR = {2019},
ABSTRACT = {We consider the problem of inverse kinematics (IK), where one wants to find<br>the parameters of a given kinematic skeleton that best explain a set of<br>observed 3D joint locations. The kinematic skeleton has a tree structure, where<br>each node is a joint that has an associated geometric transformation that is<br>propagated to all its child nodes. The IK problem has various applications in<br>vision and graphics, for example for tracking or reconstructing articulated<br>objects, such as human hands or bodies. Most commonly, the IK problem is<br>tackled using local optimisation methods. A major downside of these approaches<br>is that, due to the non-convex nature of the problem, such methods are prone to<br>converge to unwanted local optima and therefore require a good initialisation.<br>In this paper we propose a convex optimisation approach for the IK problem<br>based on semidefinite programming, which admits a polynomial-time algorithm<br>that globally solves (a relaxation of) the IK problem. Experimentally, we<br>demonstrate that the proposed method significantly outperforms local<br>optimisation methods using different real-world skeletons.<br>},
}

Endnote

%0 Report
%A Yenamandra, Tarum
%A Bernard, Florian
%A Wang, Jiayi
%A Mueller, Franziska
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Convex Optimisation for Inverse Kinematics : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7DA8-2
%U http://arxiv.org/abs/1910.11016
%D 2019
%X   We consider the problem of inverse kinematics (IK), where one wants to find<br>the parameters of a given kinematic skeleton that best explain a set of<br>observed 3D joint locations. The kinematic skeleton has a tree structure, where<br>each node is a joint that has an associated geometric transformation that is<br>propagated to all its child nodes. The IK problem has various applications in<br>vision and graphics, for example for tracking or reconstructing articulated<br>objects, such as human hands or bodies. Most commonly, the IK problem is<br>tackled using local optimisation methods. A major downside of these approaches<br>is that, due to the non-convex nature of the problem, such methods are prone to<br>converge to unwanted local optima and therefore require a good initialisation.<br>In this paper we propose a convex optimisation approach for the IK problem<br>based on semidefinite programming, which admits a polynomial-time algorithm<br>that globally solves (a relaxation of) the IK problem. Experimentally, we<br>demonstrate that the proposed method significantly outperforms local<br>optimisation methods using different real-world skeletons.<br>
%K Computer Science, Learning, cs.LG,Computer Science, Computer Vision and Pattern Recognition, cs.CV,Statistics, Machine Learning, stat.ML

Conference paper

T. Yenamandra, F. Bernard, J. Wang, F. Mueller, and C. Theobalt

“Convex Optimisation for Inverse Kinematics,” in International Conference on 3D Vision, Québec City, Canada, 2019.

@inproceedings{Yenamandra_3DV2019,
TITLE = {Convex Optimisation for Inverse Kinematics},
AUTHOR = {Yenamandra, Tarum and Bernard, Florian and Wang, Jiayi and Mueller, Franziska and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-7281-3131-3},
DOI = {10.1109/3DV.2019.00043},
PUBLISHER = {IEEE},
YEAR = {2019},
DATE = {2019},
BOOKTITLE = {International Conference on 3D Vision},
PAGES = {318--327},
ADDRESS = {Qu{\'e}bec City, Canada},
}

Endnote

%0 Conference Proceedings
%A Yenamandra, Tarum
%A Bernard, Florian
%A Wang, Jiayi
%A Mueller, Franziska
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Convex Optimisation for Inverse Kinematics : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0005-7B63-2
%R 10.1109/3DV.2019.00043
%D 2019
%B International Conference on 3D Vision
%Z date of event: 2019-09-16 - 2019-09-19
%C Qu&#233;bec City, Canada
%B International Conference on 3D Vision
%P 318 - 327
%I IEEE
%@ 978-1-7281-3131-3

Conference paper

N. Ye, K. Wolski, and R. K. Mantiuk

“Predicting Visible Image Differences under Varying Display Brightness and Viewing Distance,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, 2019.

@inproceedings{Ye19,
TITLE = {Predicting Visible Image Differences under Varying Display Brightness and Viewing Distance},
AUTHOR = {Ye, Nanyang and Wolski, Krzysztof and Mantiuk, Rafa{\l} K.},
LANGUAGE = {eng},
ISBN = {978-1-7281-3293-8},
DOI = {10.1109/CVPR.2019.00558},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
PAGES = {5429--5437},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Ye, Nanyang
%A Wolski, Krzysztof
%A Mantiuk, Rafa&#322; K.
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Predicting Visible Image Differences under Varying Display 
Brightness and Viewing Distance : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-2748-1
%R 10.1109/CVPR.2019.00558
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2019-06-16 - 2019-06-20
%C Long Beach, CA,  USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition 
%P 5429 - 5437
%I IEEE
%@ 978-1-7281-3293-8

Article

H. Yu, M. Bemana, M. Wernikowski, M. Chwesiuk, O. T. Tursun, G. Singh, K. Myszkowski, R. Mantiuk, H.-P. Seidel, and P. Didyk

“A Perception-driven Hybrid Decomposition for Multi-layer Accommodative Displays,” IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2019), vol. 25, no. 5, 2019.

@article{Yu_VR2019,
TITLE = {A Perception-driven Hybrid Decomposition for Multi-layer Accommodative Displays},
AUTHOR = {Yu, Hyeonseung and Bemana, Mojtaba and Wernikowski, Marek and Chwesiuk, Micha{\l} and Tursun, Okan Tarhan and Singh, Gurprit and Myszkowski, Karol and Mantiuk, Rados{\l}aw and Seidel, Hans-Peter and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2019.2898821},
PUBLISHER = {IEEE Computer Society},
ADDRESS = {New York, NY},
YEAR = {2019},
DATE = {2019},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR)},
VOLUME = {25},
NUMBER = {5},
PAGES = {1940--1950},
BOOKTITLE = {Selected Proceedings IEEE Virtual Reality 2019 (IEEE VR 2019)},
EDITOR = {Thomas, Bruce and Welch, Greg and Kuhlen, Torsten and Johnson, Kyle},
}

Endnote

%0 Journal Article
%A Yu, Hyeonseung
%A Bemana, Mojtaba
%A Wernikowski, Marek
%A Chwesiuk, Micha&#322;
%A Tursun, Okan Tarhan
%A Singh, Gurprit
%A Myszkowski, Karol
%A Mantiuk, Rados&#322;aw
%A Seidel, Hans-Peter
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T A Perception-driven Hybrid Decomposition for Multi-layer Accommodative Displays : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-DCB5-A
%R 10.1109/TVCG.2019.2898821
%7 2019
%D 2019
%J IEEE Transactions on Visualization and Computer Graphics
%V 25
%N 5
%& 1940
%P 1940 - 1950
%I IEEE Computer Society
%C New York, NY
%@ false
%B Selected Proceedings IEEE Virtual Reality 2019
%O IEEE VR 2019 Osaka, Japan, 23rd - 27th March

Conference paper

D2D4

T. Yu, Z. Zheng, Y. Zhong, J. Zhao, D. Quionhai, G. Pons-Moll, and Y. Liu

“SimulCap : Single-View Human Performance Capture with Cloth Simulation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA, 2019.

@inproceedings{SimulCap19,
TITLE = {{SimulCap} : {S}ingle-View Human Performance Capture with Cloth Simulation},
AUTHOR = {Yu, Tao and Zheng, Zerong and Zhong, Yuan and Zhao, Jianhui and Quionhai, Dai and Pons-Moll, Gerard and Liu, Yebin},
LANGUAGE = {eng},
ISBN = {978-1-7281-3293-8},
DOI = {10.1109/CVPR.2019.00565},
PUBLISHER = {IEEE},
YEAR = {2019},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)},
PAGES = {5499--5509},
ADDRESS = {Long Beach, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Yu, Tao
%A Zheng, Zerong
%A Zhong, Yuan
%A Zhao, Jianhui
%A Quionhai, Dai
%A Pons-Moll, Gerard
%A Liu, Yebin
%+ External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T SimulCap : Single-View Human Performance Capture with Cloth Simulation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-651E-B
%R 10.1109/CVPR.2019.00565
%D 2019
%B 32nd IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2019-06-16 - 2019-06-20
%C Long Beach, CA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 5499 - 5509
%I IEEE
%@ 978-1-7281-3293-8

Conference paper

M. Ziegler, M. Bemana, J. Keinert, and K. Myszkowski

“Near Real-time Light Field Reconstruction and Rendering for On-set Capture Quality Evaluation,” in European Light Field Imaging Workshop (ELFI 2019), Borovets, Bulgaria, 2019.

@inproceedings{bemana2019near,
TITLE = {Near Real-time Light Field Reconstruction and Rendering for On-set Capture Quality Evaluation},
AUTHOR = {Ziegler, Matthias and Bemana, Mojtaba and Keinert, Joachim and Myszkowski, Karol},
LANGUAGE = {eng},
URL = {https://www.eurasip.org/Proceedings/Ext/ELFI_2019/Proceedings.html},
PUBLISHER = {EURASIP},
YEAR = {2019},
BOOKTITLE = {European Light Field Imaging Workshop (ELFI 2019)},
ADDRESS = {Borovets, Bulgaria},
}

Endnote

%0 Conference Proceedings
%A Ziegler, Matthias
%A Bemana, Mojtaba
%A Keinert, Joachim
%A Myszkowski, Karol
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Near Real-time Light Field Reconstruction and Rendering for On-set Capture Quality Evaluation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0007-DDC0-7
%D 2019
%B European Light Field Imaging Workshop
%Z date of event: 2019-06-04 - 2019-06-06
%C Borovets, Bulgaria
%B European Light Field Imaging Workshop
%I EURASIP
%U https://www.eurasip.org/Proceedings/Ext/ELFI_2019/Proceedings.html

2018

Conference paper

D4D2

T. Alldieck, M. A. Magnor, W. Xu, C. Theobalt, and G. Pons-Moll

“Video Based Reconstruction of 3D People Models,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA, 2018.

@inproceedings{alldieck2018video,
TITLE = {Video Based Reconstruction of {3D} People Models},
AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Xu, Weipeng and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
ISBN = {978-1-5386-6420-9},
DOI = {10.1109/CVPR.2018.00875},
PUBLISHER = {IEEE},
YEAR = {2018},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)},
PAGES = {8387--8397},
ADDRESS = {Salt Lake City, UT, USA},
}

Endnote

%0 Conference Proceedings
%A Alldieck, Thiemo
%A Magnor, Marcus A.
%A Xu, Weipeng
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
%T Video Based Reconstruction of 3D People Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-1E24-6
%R 10.1109/CVPR.2018.00875
%D 2018
%B 31st IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2018-06-18 - 2018-06-22
%C Salt Lake City, UT, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition 
%P 8387 - 8397
%I IEEE
%@ 978-1-5386-6420-9

Paper

T. Alldieck, M. A. Magnor, W. Xu, C. Theobalt, and G. Pons-Moll

“Detailed Human Avatars from Monocular Video,” 2018. [Online]. Available: http://arxiv.org/abs/1808.01338.

Abstract

We present a novel method for high detail-preserving human avatar creation
from monocular video. A parameterized body model is refined and optimized to
maximally resemble subjects from a video showing them from all sides. Our
avatars feature a natural face, hairstyle, clothes with garment wrinkles, and
high-resolution texture. Our paper contributes facial landmark and
shading-based human body shape refinement, a semantic texture prior, and a
novel texture stitching strategy, resulting in the most sophisticated-looking
human avatars obtained from a single video to date. Numerous results show the
robustness and versatility of our method. A user study illustrates its
superiority over the state-of-the-art in terms of identity preservation, level
of detail, realism, and overall user preference.

BibTeX

@online{Alldieck_arXiv1808.01338,
TITLE = {Detailed Human Avatars from Monocular Video},
AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Xu, Weipeng and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1808.01338},
EPRINT = {1808.01338},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We present a novel method for high detail-preserving human avatar creation<br>from monocular video. A parameterized body model is refined and optimized to<br>maximally resemble subjects from a video showing them from all sides. Our<br>avatars feature a natural face, hairstyle, clothes with garment wrinkles, and<br>high-resolution texture. Our paper contributes facial landmark and<br>shading-based human body shape refinement, a semantic texture prior, and a<br>novel texture stitching strategy, resulting in the most sophisticated-looking<br>human avatars obtained from a single video to date. Numerous results show the<br>robustness and versatility of our method. A user study illustrates its<br>superiority over the state-of-the-art in terms of identity preservation, level<br>of detail, realism, and overall user preference.<br>},
}

Endnote

%0 Report
%A Alldieck, Thiemo
%A Magnor, Marcus A.
%A Xu, Weipeng
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
%T Detailed Human Avatars from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5E0F-6
%U http://arxiv.org/abs/1808.01338
%D 2018
%X   We present a novel method for high detail-preserving human avatar creation<br>from monocular video. A parameterized body model is refined and optimized to<br>maximally resemble subjects from a video showing them from all sides. Our<br>avatars feature a natural face, hairstyle, clothes with garment wrinkles, and<br>high-resolution texture. Our paper contributes facial landmark and<br>shading-based human body shape refinement, a semantic texture prior, and a<br>novel texture stitching strategy, resulting in the most sophisticated-looking<br>human avatars obtained from a single video to date. Numerous results show the<br>robustness and versatility of our method. A user study illustrates its<br>superiority over the state-of-the-art in terms of identity preservation, level<br>of detail, realism, and overall user preference.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

D2D4

T. Alldieck, M. A. Magnor, W. Xu, C. Theobalt, and G. Pons-Moll

“Detailed Human Avatars from Monocular Video,” in 3DV 2018 , International Conference on 3D Vision, Verona, Italy, 2018.

@inproceedings{Alldieck_3DV2018,
TITLE = {Detailed Human Avatars from Monocular Video},
AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Xu, Weipeng and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
ISBN = {978-1-5386-8425-2 ; 978-1-5386-8426-9},
DOI = {10.1109/3DV.2018.00022},
PUBLISHER = {IEEE},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {3DV 2018 , International Conference on 3D Vision},
PAGES = {98--109},
ADDRESS = {Verona, Italy},
}

Endnote

%0 Conference Proceedings
%A Alldieck, Thiemo
%A Magnor, Marcus A.
%A Xu, Weipeng
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
%T Detailed Human Avatars from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5C40-F
%R 10.1109/3DV.2018.00022
%D 2018
%B International Conference on 3D Vision
%Z date of event: 2018-09-05 - 2018-09-08
%C Verona, Italy
%B 3DV 2018 
%P 98 - 109
%I IEEE
%@ 978-1-5386-8425-2  978-1-5386-8426-9

Paper

T. Alldieck, M. A. Magnor, W. Xu, C. Theobalt, and G. Pons-Moll

“Video Based Reconstruction of 3D People Models,” 2018. [Online]. Available: http://arxiv.org/abs/1803.04758.

Abstract

This paper describes how to obtain accurate 3D body models and texture of

arbitrary people from a single, monocular video in which a person is moving.

Based on a parametric body model, we present a robust processing pipeline

achieving 3D model fits with 5mm accuracy also for clothed people. Our main

contribution is a method to nonrigidly deform the silhouette cones

corresponding to the dynamic human silhouettes, resulting in a visual hull in a

common reference frame that enables surface reconstruction. This enables

efficient estimation of a consensus 3D shape, texture and implanted animation

skeleton based on a large number of frames. We present evaluation results for a

number of test subjects and analyze overall performance. Requiring only a

smartphone or webcam, our method enables everyone to create their own fully

animatable digital double, e.g., for social VR applications or virtual try-on

for online fashion shopping.

BibTeX

@online{Alldieck_arXiv1803.04758,
TITLE = {Video Based Reconstruction of {3D} People Models},
AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Xu, Weipeng and Theobalt, Christian and Pons-Moll, Gerard},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1803.04758},
EPRINT = {1803.04758},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {This paper describes how to obtain accurate 3D body models and texture of arbitrary people from a single, monocular video in which a person is moving. Based on a parametric body model, we present a robust processing pipeline achieving 3D model fits with 5mm accuracy also for clothed people. Our main contribution is a method to nonrigidly deform the silhouette cones corresponding to the dynamic human silhouettes, resulting in a visual hull in a common reference frame that enables surface reconstruction. This enables efficient estimation of a consensus 3D shape, texture and implanted animation skeleton based on a large number of frames. We present evaluation results for a number of test subjects and analyze overall performance. Requiring only a smartphone or webcam, our method enables everyone to create their own fully animatable digital double, e.g., for social VR applications or virtual try-on for online fashion shopping.},
}

Endnote

%0 Report
%A Alldieck, Thiemo
%A Magnor, Marcus A.
%A Xu, Weipeng
%A Theobalt, Christian
%A Pons-Moll, Gerard
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
D2 Extern
%T Video Based Reconstruction of 3D People Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-40CD-0
%U http://arxiv.org/abs/1803.04758
%D 2018
%X   This paper describes how to obtain accurate 3D body models and texture of
arbitrary people from a single, monocular video in which a person is moving.
Based on a parametric body model, we present a robust processing pipeline
achieving 3D model fits with 5mm accuracy also for clothed people. Our main
contribution is a method to nonrigidly deform the silhouette cones
corresponding to the dynamic human silhouettes, resulting in a visual hull in a
common reference frame that enables surface reconstruction. This enables
efficient estimation of a consensus 3D shape, texture and implanted animation
skeleton based on a large number of frames. We present evaluation results for a
number of test subjects and analyze overall performance. Requiring only a
smartphone or webcam, our method enables everyone to create their own fully
animatable digital double, e.g., for social VR applications or virtual try-on
for online fashion shopping.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

S. Beigpour, S. Shekhar, M. Mansouryar, K. Myszkowski, and H.-P. Seidel

“Light-Field Appearance Editing Based on Intrinsic Decomposition,” Journal of Perceptual Imaging, vol. 1, no. 1, 2018.

@article{Beigpour2018,
TITLE = {Light-Field Appearance Editing Based on Intrinsic Decomposition},
AUTHOR = {Beigpour, Shida and Shekhar, Sumit and Mansouryar, Mohsen and Myszkowski, Karol and Seidel, Hans-Peter},
LANGUAGE = {eng},
DOI = {10.2352/J.Percept.Imaging.2018.1.1.010502},
YEAR = {2018},
JOURNAL = {Journal of Perceptual Imaging},
VOLUME = {1},
NUMBER = {1},
PAGES = {1--15},
EID = {10502},
}

Endnote

%0 Journal Article
%A Beigpour, Shida
%A Shekhar, Sumit
%A Mansouryar, Mohsen
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Light-Field Appearance Editing Based on Intrinsic Decomposition : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-5F88-C
%R 10.2352/J.Percept.Imaging.2018.1.1.010502
%7 2018
%D 2018
%J Journal of Perceptual Imaging
%O JPI
%V 1
%N 1
%& 1
%P 1 - 15
%Z sequence number: 10502

Paper

D4D2

F. Bernard, J. Thunberg, P. Swoboda, and C. Theobalt

“Higher-order Projected Power Iterations for Scalable Multi-Matching,” 2018. [Online]. Available: http://arxiv.org/abs/1811.10541.

Abstract

The matching of multiple objects (e.g. shapes or images) is a fundamental
problem in vision and graphics. In order to robustly handle ambiguities, noise
and repetitive patterns in challenging real-world settings, it is essential to
take geometric consistency between points into account. Computationally, the
multi-matching problem is difficult. It can be phrased as simultaneously
solving multiple (NP-hard) quadratic assignment problems (QAPs) that are
coupled via cycle-consistency constraints. The main limitations of existing
multi-matching methods are that they either ignore geometric consistency and
thus have limited robustness, or they are restricted to small-scale problems
due to their (relatively) high computational cost. We address these
shortcomings by introducing a Higher-order Projected Power Iteration method,
which is (i) efficient and scales to tens of thousands of points, (ii)
straightforward to implement, (iii) able to incorporate geometric consistency,
and (iv) guarantees cycle-consistent multi-matchings. Experimentally we show
that our approach is superior to existing methods.

BibTeX

@online{Bernard_arXIv1811.10541,
TITLE = {Higher-order Projected Power Iterations for Scalable Multi-Matching},
AUTHOR = {Bernard, Florian and Thunberg, Johan and Swoboda, Paul and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1811.10541},
EPRINT = {1811.10541},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {The matching of multiple objects (e.g. shapes or images) is a fundamental<br>problem in vision and graphics. In order to robustly handle ambiguities, noise<br>and repetitive patterns in challenging real-world settings, it is essential to<br>take geometric consistency between points into account. Computationally, the<br>multi-matching problem is difficult. It can be phrased as simultaneously<br>solving multiple (NP-hard) quadratic assignment problems (QAPs) that are<br>coupled via cycle-consistency constraints. The main limitations of existing<br>multi-matching methods are that they either ignore geometric consistency and<br>thus have limited robustness, or they are restricted to small-scale problems<br>due to their (relatively) high computational cost. We address these<br>shortcomings by introducing a Higher-order Projected Power Iteration method,<br>which is (i) efficient and scales to tens of thousands of points, (ii)<br>straightforward to implement, (iii) able to incorporate geometric consistency,<br>and (iv) guarantees cycle-consistent multi-matchings. Experimentally we show<br>that our approach is superior to existing methods.<br>},
}

Endnote

%0 Report
%A Bernard, Florian
%A Thunberg, Johan
%A Swoboda, Paul
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Higher-order Projected Power Iterations for Scalable Multi-Matching : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-A8D0-5
%U http://arxiv.org/abs/1811.10541
%D 2018
%X   The matching of multiple objects (e.g. shapes or images) is a fundamental<br>problem in vision and graphics. In order to robustly handle ambiguities, noise<br>and repetitive patterns in challenging real-world settings, it is essential to<br>take geometric consistency between points into account. Computationally, the<br>multi-matching problem is difficult. It can be phrased as simultaneously<br>solving multiple (NP-hard) quadratic assignment problems (QAPs) that are<br>coupled via cycle-consistency constraints. The main limitations of existing<br>multi-matching methods are that they either ignore geometric consistency and<br>thus have limited robustness, or they are restricted to small-scale problems<br>due to their (relatively) high computational cost. We address these<br>shortcomings by introducing a Higher-order Projected Power Iteration method,<br>which is (i) efficient and scales to tens of thousands of points, (ii)<br>straightforward to implement, (iii) able to incorporate geometric consistency,<br>and (iv) guarantees cycle-consistent multi-matchings. Experimentally we show<br>that our approach is superior to existing methods.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Statistics, Machine Learning, stat.ML

Paper

F. Bernard, J. Thunberg, J. Goncalves, and C. Theobalt

“Synchronisation of Partial Multi-Matchings via Non-negative Factorisations,” 2018. [Online]. Available: http://arxiv.org/abs/1803.06320.

Abstract

In this work we study permutation synchronisation for the challenging case of

partial permutations, which plays an important role for the problem of matching

multiple objects (e.g. images or shapes). The term synchronisation refers to

the property that the set of pairwise matchings is cycle-consistent, i.e. in

the full matching case all compositions of pairwise matchings over cycles must

be equal to the identity. Motivated by clustering and matrix factorisation

perspectives of cycle-consistency, we derive an algorithm to tackle the

permutation synchronisation problem based on non-negative factorisations. In

order to deal with the inherent non-convexity of the permutation

synchronisation problem, we use an initialisation procedure based on a novel

rotation scheme applied to the solution of the spectral relaxation. Moreover,

this rotation scheme facilitates a convenient Euclidean projection to obtain a

binary solution after solving our relaxed problem. In contrast to

state-of-the-art methods, our approach is guaranteed to produce

cycle-consistent results. We experimentally demonstrate the efficacy of our

method and show that it achieves better results compared to existing methods.

BibTeX

@online{Bernard_arXiv1803.06320,
TITLE = {Synchronisation of Partial Multi-Matchings via Non-negative Factorisations},
AUTHOR = {Bernard, Florian and Thunberg, Johan and Goncalves, Jorge and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1803.06320},
EPRINT = {1803.06320},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {In this work we study permutation synchronisation for the challenging case of partial permutations, which plays an important role for the problem of matching multiple objects (e.g. images or shapes). The term synchronisation refers to the property that the set of pairwise matchings is cycle-consistent, i.e. in the full matching case all compositions of pairwise matchings over cycles must be equal to the identity. Motivated by clustering and matrix factorisation perspectives of cycle-consistency, we derive an algorithm to tackle the permutation synchronisation problem based on non-negative factorisations. In order to deal with the inherent non-convexity of the permutation synchronisation problem, we use an initialisation procedure based on a novel rotation scheme applied to the solution of the spectral relaxation. Moreover, this rotation scheme facilitates a convenient Euclidean projection to obtain a binary solution after solving our relaxed problem. In contrast to state-of-the-art methods, our approach is guaranteed to produce cycle-consistent results. We experimentally demonstrate the efficacy of our method and show that it achieves better results compared to existing methods.},
}

Endnote

%0 Report
%A Bernard, Florian
%A Thunberg, Johan
%A Goncalves, Jorge
%A Theobalt, Christian
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Synchronisation of Partial Multi-Matchings via Non-negative
Factorisations :
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-40BC-3
%U http://arxiv.org/abs/1803.06320
%D 2018
%X In this work we study permutation synchronisation for the challenging case of
partial permutations, which plays an important role for the problem of matching
multiple objects (e.g. images or shapes). The term synchronisation refers to
the property that the set of pairwise matchings is cycle-consistent, i.e. in
the full matching case all compositions of pairwise matchings over cycles must
be equal to the identity. Motivated by clustering and matrix factorisation
perspectives of cycle-consistency, we derive an algorithm to tackle the
permutation synchronisation problem based on non-negative factorisations. In
order to deal with the inherent non-convexity of the permutation
synchronisation problem, we use an initialisation procedure based on a novel
rotation scheme applied to the solution of the spectral relaxation. Moreover,
this rotation scheme facilitates a convenient Euclidean projection to obtain a
binary solution after solving our relaxed problem. In contrast to
state-of-the-art methods, our approach is guaranteed to produce
cycle-consistent results. We experimentally demonstrate the efficacy of our
method and show that it achieves better results compared to existing methods.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Mathematics, Optimization and Control, math.OC,Statistics, Machine Learning, stat.ML

Conference paper

F. Bernard, C. Theobalt, and M. Moeller

“DS*: Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA, 2018.

@inproceedings{BernardCVPR2018,
TITLE = {DS*: {T}ighter Lifting-Free Convex Relaxations for Quadratic Matching Problems},
AUTHOR = {Bernard, Florian and Theobalt, Christian and Moeller, Michael},
LANGUAGE = {eng},
ISBN = {978-1-5386-6420-9},
DOI = {10.1109/CVPR.2018.00453},
PUBLISHER = {IEEE},
YEAR = {2018},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)},
PAGES = {4310--4319},
ADDRESS = {Salt Lake City, UT, USA},
}

Endnote

%0 Conference Proceedings
%A Bernard, Florian
%A Theobalt, Christian
%A Moeller, Michael
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T DS*: Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-E92F-4
%R 10.1109/CVPR.2018.00453
%D 2018
%B 31st IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2018-06-18 - 2018-06-22
%C Salt Lake City, UT, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 4310 - 4319
%I IEEE
%@ 978-1-5386-6420-9

Paper

L. Castelli Aleardi, S. Salihoglu, G. Singh, and M. Ovsjanikov

“Spectral Measures of Distortion for Change Detection in Dynamic Graphs.” HAL, 2018.

@misc{Castelli_hal-01864079v2,
TITLE = {Spectral Measures of Distortion for Change Detection in Dynamic Graphs},
AUTHOR = {Castelli Aleardi, Luca and Salihoglu, Semih and Singh, Gurprit and Ovsjanikov, Maks},
LANGUAGE = {eng},
URL = {https://hal.archives-ouvertes.fr/hal-01864079v2},
PUBLISHER = {HAL},
YEAR = {2018},
}

Endnote

%0 Report
%A Castelli Aleardi, Luca
%A Salihoglu, Semih
%A Singh, Gurprit
%A Ovsjanikov, Maks
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Spectral Measures of Distortion for Change Detection in Dynamic Graphs : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-9F5D-4
%F OTHER: hal-01864079v2
%U https://hal.archives-ouvertes.fr/hal-01864079v2
%I HAL
%D 2018
%U https://hal.archives-ouvertes.fr/hal-01864079/

Article

R. Chen, C. Gotsman, and K. Hormann

“Path Planning with Divergence-Based Distance Functions,” Computer Aided Geometric Design, vol. 66, 2018.

@article{Chen_CAGD2018,
TITLE = {Path Planning with Divergence-Based Distance Functions},
AUTHOR = {Chen, Renjie and Gotsman, Craig and Hormann, Kai},
LANGUAGE = {eng},
ISSN = {0167-8396},
DOI = {10.1016/j.cagd.2018.09.002},
PUBLISHER = {North-Holland},
ADDRESS = {Amsterdam},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Computer Aided Geometric Design},
VOLUME = {66},
PAGES = {52--74},
}

Endnote

%0 Journal Article
%A Chen, Renjie
%A Gotsman, Craig
%A Hormann, Kai
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Path Planning with Divergence-Based Distance Functions : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-72C1-3
%R 10.1016/j.cagd.2018.09.002
%7 2018
%D 2018
%J Computer Aided Geometric Design
%V 66
%& 52
%P 52 - 74
%I North-Holland
%C Amsterdam
%@ false

Article

R. Chen, C. Gotsman, and K. Hormann

“Efficient Path Generation with Reduced Coordinates,” Computer Graphics Forum (Proc. Eurographics Symposium on Geometric Processing 2018), vol. 37, no. 5, 2018.

@article{ChenSGP2018,
TITLE = {Efficient Path Generation with Reduced Coordinates},
AUTHOR = {Chen, Renjie and Gotsman, Craig and Hormann, Kai},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13489},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Chichester},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Computer Graphics Forum (Proc. Eurographics Symposium on Geometric Processing)},
VOLUME = {37},
NUMBER = {5},
PAGES = {37--48},
BOOKTITLE = {Symposium on Geometry Processing 2018 (Eurographics Symposium on Geometric Processing 2018)},
EDITOR = {Ju, Tao and Vaxman, Amir},
}

Endnote

%0 Journal Article
%A Chen, Renjie
%A Gotsman, Craig
%A Hormann, Kai
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Efficient Path Generation with Reduced Coordinates : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-E6D6-A
%R 10.1111/cgf.13489
%7 2018
%D 2018
%J Computer Graphics Forum
%V 37
%N 5
%& 37
%P 37 - 48
%I Wiley-Blackwell
%C Chichester
%@ false
%B Symposium on Geometry Processing 2018
%O Paris, France, July 7 &#8211; 11, 2018 SGP 2018 Eurographics Symposium on Geometric Processing 2018

Article

X. Du, X. Liu, D.-M. Yan, C. Jiang, J. Ye, and H. Zhang

“Field-Aligned Isotropic Surface Remeshing,” Computer Graphics Forum, vol. 37, no. 6, 2018.

@article{DuCGF2018,
TITLE = {Field-Aligned Isotropic Surface Remeshing},
AUTHOR = {Du, Xingyi and Liu, Xiaohan and Yan, Dong-Ming and Jiang, Caigui and Ye, Juntao and Zhang, Hui},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13329},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Computer Graphics Forum},
VOLUME = {37},
NUMBER = {6},
PAGES = {343--357},
}

Endnote

%0 Journal Article
%A Du, Xingyi
%A Liu, Xiaohan
%A Yan, Dong-Ming
%A Jiang, Caigui
%A Ye, Juntao
%A Zhang, Hui
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Field-Aligned Isotropic Surface Remeshing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-E209-6
%R 10.1111/cgf.13329
%7 2018
%D 2018
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 37
%N 6
%& 343
%P 343 - 357
%I Blackwell-Wiley
%C Oxford
%@ false

Article

B. Golla, H.-P. Seidel, and R. Chen

“Piecewise Linear Mapping Optimization Based on the Complex View,” Computer Graphics Forum (Proc. Pacific Graphics 2018), vol. 37, no. 7, 2018.

@article{Golla_PG2018,
TITLE = {Piecewise Linear Mapping Optimization Based on the Complex View},
AUTHOR = {Golla, Bj{\"o}rn and Seidel, Hans-Peter and Chen, Renjie},
LANGUAGE = {eng},
ISSN = {1467-8659},
DOI = {10.1111/cgf.13563},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford, UK},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Computer Graphics Forum (Proc. Pacific Graphics)},
VOLUME = {37},
NUMBER = {7},
PAGES = {233--243},
BOOKTITLE = {The 26th Pacific Conference on Computer Graphics and Applications (Pacific Graphics 2018)},
}

Endnote

%0 Journal Article
%A Golla, Bj&#246;rn
%A Seidel, Hans-Peter
%A Chen, Renjie
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Piecewise Linear Mapping Optimization Based on the Complex View : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-72CD-7
%R 10.1111/cgf.13563
%7 2018
%D 2018
%J Computer Graphics Forum
%V 37
%N 7
%& 233
%P 233 - 243
%I Wiley-Blackwell
%C Oxford, UK
%@ false
%B The 26th Pacific Conference on Computer Graphics and Applications
%O Pacific Graphics 2018 PG 2018 Hong Kong, 8-11 October 2018

Thesis

H. Hajipour

“Weakly-supervised Surface Reconstruction Using Floating Radial Basis Functions,” Universität des Saarlandes, Saarbrücken, 2018.

@mastersthesis{HajipourMSc2018,
TITLE = {Weakly-supervised Surface Reconstruction Using Floating Radial Basis Functions},
AUTHOR = {Hajipour, Hossein},
LANGUAGE = {eng},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2018},
DATE = {2018},
}

Endnote

%0 Thesis
%A Hajipour, Hossein
%Y Theobalt, Christian
%A referee: Tewari, Ayush
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Weakly-supervised Surface Reconstruction Using Floating Radial Basis Functions : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0003-E013-A
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2018
%V master
%9 master

Article

J. Hladký and R. Ďurikovič

“Fire Simulation in 3D Computer Animation with Turbulence Dynamics including Fire Separation and Profile Modeling,” International Journal of Networking and Computing, vol. 8, no. 2, 2018.

@article{DBLP:journals/ijnc/HladkyD18,
TITLE = {Fire Simulation in {3D} Computer Animation with Turbulence Dynamics including Fire Separation and Profile Modeling},
AUTHOR = {Hladk{\'y}, Jozef and {\v D}urikovi{\v c}, Roman},
LANGUAGE = {eng},
ISSN = {2185-2847},
URL = {http://www.ijnc.org/index.php/ijnc/article/view/180},
YEAR = {2018},
JOURNAL = {International Journal of Networking and Computing},
VOLUME = {8},
NUMBER = {2},
PAGES = {186--204},
}

Endnote

%0 Journal Article
%A Hladk&#253;, Jozef
%A &#270;urikovi&#269;, Roman
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Fire Simulation in 3D Computer Animation with Turbulence Dynamics including Fire Separation and Profile Modeling : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F652-C
%U http://www.ijnc.org/index.php/ijnc/article/view/180
%7 2018
%D 2018
%J International Journal of Networking and Computing
%V 8
%N 2
%& 186
%P 186 - 204
%@ false

Article

M. Kenzel, B. Kerbl, D. Schmalstieg, and M. Steinberger

“A High-Performance Software Graphics Pipeline Architecture for the GPU,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2018), vol. 37, no. 4, 2018.

@article{Kenzel_SIGGRAPH2018,
TITLE = {A High-Performance Software Graphics Pipeline Architecture for the {GPU}},
AUTHOR = {Kenzel, Michael and Kerbl, Bernhard and Schmalstieg, Dieter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3197517.3201374},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {37},
NUMBER = {4},
PAGES = {1--15},
EID = {140},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2018},
}

Endnote

%0 Journal Article
%A Kenzel, Michael
%A Kerbl, Bernhard
%A Schmalstieg, Dieter
%A Steinberger, Markus
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T A High-Performance Software Graphics Pipeline Architecture for the GPU : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-72E1-F
%R 10.1145/3197517.3201374
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 4
%& 1
%P 1 - 15
%Z sequence number: 140
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2018
%O ACM SIGGRAPH 2018 Vancouver, Canada , 12 - 16 August

Paper

H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies, M. Nießner, P. Pérez, C. Richardt, M. Zollhöfer, and C. Theobalt

“Deep Video Portraits,” 2018. [Online]. Available: http://arxiv.org/abs/1805.11714.

Abstract

We present a novel approach that enables photo-realistic re-animation of
portrait videos using only an input video. In contrast to existing approaches
that are restricted to manipulations of facial expressions only, we are the
first to transfer the full 3D head position, head rotation, face expression,
eye gaze, and eye blinking from a source actor to a portrait video of a target
actor. The core of our approach is a generative neural network with a novel
space-time architecture. The network takes as input synthetic renderings of a
parametric face model, based on which it predicts photo-realistic video frames
for a given target actor. The realism in this rendering-to-video transfer is
achieved by careful adversarial training, and as a result, we can create
modified target videos that mimic the behavior of the synthetically-created
input. In order to enable source-to-target video re-animation, we render a
synthetic target video with the reconstructed head animation parameters from a
source video, and feed it into the trained network -- thus taking full control
of the target. With the ability to freely recombine source and target
parameters, we are able to demonstrate a large variety of video rewrite
applications without explicitly modeling hair, body or background. For
instance, we can reenact the full head using interactive user-controlled
editing, and realize high-fidelity visual dubbing. To demonstrate the high
quality of our output, we conduct an extensive series of experiments and
evaluations, where for instance a user study shows that our video edits are
hard to detect.

BibTeX

@online{Kim_arXiv1805.11714,
TITLE = {Deep Video Portraits},
AUTHOR = {Kim, Hyeongwoo and Garrido, Pablo and Tewari, Ayush and Xu, Weipeng and Thies, Justus and Nie{\ss}ner, Matthias and P{\'e}rez, Patrick and Richardt, Christian and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1805.11714},
EPRINT = {1805.11714},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We present a novel approach that enables photo-realistic re-animation of<br>portrait videos using only an input video. In contrast to existing approaches<br>that are restricted to manipulations of facial expressions only, we are the<br>first to transfer the full 3D head position, head rotation, face expression,<br>eye gaze, and eye blinking from a source actor to a portrait video of a target<br>actor. The core of our approach is a generative neural network with a novel<br>space-time architecture. The network takes as input synthetic renderings of a<br>parametric face model, based on which it predicts photo-realistic video frames<br>for a given target actor. The realism in this rendering-to-video transfer is<br>achieved by careful adversarial training, and as a result, we can create<br>modified target videos that mimic the behavior of the synthetically-created<br>input. In order to enable source-to-target video re-animation, we render a<br>synthetic target video with the reconstructed head animation parameters from a<br>source video, and feed it into the trained network -- thus taking full control<br>of the target. With the ability to freely recombine source and target<br>parameters, we are able to demonstrate a large variety of video rewrite<br>applications without explicitly modeling hair, body or background. For<br>instance, we can reenact the full head using interactive user-controlled<br>editing, and realize high-fidelity visual dubbing. To demonstrate the high<br>quality of our output, we conduct an extensive series of experiments and<br>evaluations, where for instance a user study shows that our video edits are<br>hard to detect.<br>},
}

Endnote

%0 Report
%A Kim, Hyeongwoo
%A Garrido, Pablo
%A Tewari, Ayush
%A Xu, Weipeng
%A Thies, Justus
%A Nie&#223;ner, Matthias
%A P&#233;rez, Patrick
%A Richardt, Christian
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Deep Video Portraits : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5E1E-5
%U http://arxiv.org/abs/1805.11714
%D 2018
%X   We present a novel approach that enables photo-realistic re-animation of<br>portrait videos using only an input video. In contrast to existing approaches<br>that are restricted to manipulations of facial expressions only, we are the<br>first to transfer the full 3D head position, head rotation, face expression,<br>eye gaze, and eye blinking from a source actor to a portrait video of a target<br>actor. The core of our approach is a generative neural network with a novel<br>space-time architecture. The network takes as input synthetic renderings of a<br>parametric face model, based on which it predicts photo-realistic video frames<br>for a given target actor. The realism in this rendering-to-video transfer is<br>achieved by careful adversarial training, and as a result, we can create<br>modified target videos that mimic the behavior of the synthetically-created<br>input. In order to enable source-to-target video re-animation, we render a<br>synthetic target video with the reconstructed head animation parameters from a<br>source video, and feed it into the trained network -- thus taking full control<br>of the target. With the ability to freely recombine source and target<br>parameters, we are able to demonstrate a large variety of video rewrite<br>applications without explicitly modeling hair, body or background. For<br>instance, we can reenact the full head using interactive user-controlled<br>editing, and realize high-fidelity visual dubbing. To demonstrate the high<br>quality of our output, we conduct an extensive series of experiments and<br>evaluations, where for instance a user study shows that our video edits are<br>hard to detect.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Graphics, cs.GR

Article

H. Kim, P. Garrido, A. Tewari, W. Xu, J. Thies, M. Nießner, P. Pérez, C. Richardt, M. Zollhöfer, and C. Theobalt

“Deep Video Portraits,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2018), vol. 37, no. 4, 2018.

@article{Kim_SIGGRAPH2018,
TITLE = {Deep Video Portraits},
AUTHOR = {Kim, Hyeongwoo and Garrido, Pablo and Tewari, Ayush and Xu, Weipeng and Thies, Justus and Nie{\ss}ner, Matthias and P{\'e}rez, Patrick and Richardt, Christian and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3197517.3201283},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {37},
NUMBER = {4},
PAGES = {1--14},
EID = {163},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2018},
}

Endnote

%0 Journal Article
%A Kim, Hyeongwoo
%A Garrido, Pablo
%A Tewari, Ayush
%A Xu, Weipeng
%A Thies, Justus
%A Nie&#223;ner, Matthias
%A P&#233;rez, Patrick
%A Richardt, Christian
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Deep Video Portraits : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5D8D-8
%R 10.1145/3197517.3201283
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 4
%& 1
%P 1 - 14
%Z sequence number: 163
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2018
%O ACM SIGGRAPH 2018 Vancouver, Canada , 12 - 16 August

Conference paper

H. Kim, M. Zollhöfer, A. Tewari, J. Thies, C. Richardt, and C. Theobalt

“InverseFaceNet: Deep Monocular Inverse Face Rendering,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA, 2018.

@inproceedings{kim2018inverse,
TITLE = {{InverseFaceNet}: {D}eep Monocular Inverse Face Rendering},
AUTHOR = {Kim, Hyeongwoo and Zollh{\"o}fer, Michael and Tewari, Ayush and Thies, Justus and Richardt, Christian and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-5386-6420-9},
DOI = {10.1109/CVPR.2018.00486},
PUBLISHER = {IEEE},
YEAR = {2018},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)},
PAGES = {4625--4634},
ADDRESS = {Salt Lake City, UT, USA},
}

Endnote

%0 Conference Proceedings
%A Kim, Hyeongwoo
%A Zollh&#246;fer, Michael
%A Tewari, Ayush
%A Thies, Justus
%A Richardt, Christian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T InverseFaceNet: Deep Monocular Inverse Face Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F476-6
%R 10.1109/CVPR.2018.00486
%D 2018
%B 31st IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2018-06-18 - 2018-06-22
%C Salt Lake City, UT, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 4625 - 4634
%I IEEE
%@ 978-1-5386-6420-9

Article

H. Lee and P. Didyk

“Real-time Apparent Resolution Enhancement for Head-mounted Displays,” Proceedings of the ACM on Computer Graphics and Interactive Techniques, vol. 1, no. 1, 2018.

@article{Lee:2018:RAR:3242771.3203202,
TITLE = {Real-time Apparent Resolution Enhancement for Head-mounted Displays},
AUTHOR = {Lee, Haebom and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {2577-6193},
DOI = {10.1145/3203202},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
JOURNAL = {Proceedings of the ACM on Computer Graphics and Interactive Techniques},
VOLUME = {1},
NUMBER = {1},
PAGES = {1--15},
EID = {19},
}

Endnote

%0 Journal Article
%A Lee, Haebom
%A Didyk, Piotr
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Real-time Apparent Resolution Enhancement for Head-mounted Displays : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-DB97-D
%R 10.1145/3203202
%7 2018
%D 2018
%J Proceedings of the ACM on Computer Graphics and Interactive Techniques
%V 1
%N 1
%& 1
%P 1 - 15
%Z sequence number: 19
%I ACM
%C New York, NY
%@ false

Paper

T. Leimkühler, G. Singh, K. Myszkowski, H.-P. Seidel, and T. Ritschel

“End-to-end Sampling Patterns,” 2018. [Online]. Available: http://arxiv.org/abs/1806.06710.

Abstract

Sample patterns have many uses in Computer Graphics, ranging from procedural

object placement over Monte Carlo image synthesis to non-photorealistic

depiction. Their properties such as discrepancy, spectra, anisotropy, or

progressiveness have been analyzed extensively. However, designing methods to

produce sampling patterns with certain properties can require substantial

hand-crafting effort, both in coding, mathematical derivation and compute time.

In particular, there is no systematic way to derive the best sampling algorithm

for a specific end-task.

Tackling this issue, we suggest another level of abstraction: a toolkit to

end-to-end optimize over all sampling methods to find the one producing

user-prescribed properties such as discrepancy or a spectrum that best fit the

end-task. A user simply implements the forward losses and the sampling method

is found automatically -- without coding or mathematical derivation -- by

making use of back-propagation abilities of modern deep learning frameworks.

While this optimization takes long, at deployment time the sampling method is

quick to execute as iterated unstructured non-linear filtering using radial

basis functions (RBFs) to represent high-dimensional kernels. Several important

previous methods are special cases of this approach, which we compare to

previous work and demonstrate its usefulness in several typical Computer

Graphics applications. Finally, we propose sampling patterns with properties

not shown before, such as high-dimensional blue noise with projective

properties.

BibTeX

@online{Leimkuehler_arXiv1806.06710,
TITLE = {End-to-end Sampling Patterns},
AUTHOR = {Leimk{\"u}hler, Thomas and Singh, Gurprit and Myszkowski, Karol and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1806.06710},
EPRINT = {1806.06710},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {Sample patterns have many uses in Computer Graphics, ranging from procedural object placement over Monte Carlo image synthesis to non-photorealistic depiction. Their properties such as discrepancy, spectra, anisotropy, or progressiveness have been analyzed extensively. However, designing methods to produce sampling patterns with certain properties can require substantial hand-crafting effort, both in coding, mathematical derivation and compute time. In particular, there is no systematic way to derive the best sampling algorithm for a specific end-task. Tackling this issue, we suggest another level of abstraction: a toolkit to end-to-end optimize over all sampling methods to find the one producing user-prescribed properties such as discrepancy or a spectrum that best fit the end-task. A user simply implements the forward losses and the sampling method is found automatically -- without coding or mathematical derivation -- by making use of back-propagation abilities of modern deep learning frameworks. While this optimization takes long, at deployment time the sampling method is quick to execute as iterated unstructured non-linear filtering using radial basis functions (RBFs) to represent high-dimensional kernels. Several important previous methods are special cases of this approach, which we compare to previous work and demonstrate its usefulness in several typical Computer Graphics applications. Finally, we propose sampling patterns with properties not shown before, such as high-dimensional blue noise with projective properties.},
}

Endnote

%0 Report
%A Leimk&#252;hler, Thomas
%A Singh, Gurprit
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T End-to-end Sampling Patterns : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-1376-4
%U http://arxiv.org/abs/1806.06710
%D 2018
%X   Sample patterns have many uses in Computer Graphics, ranging from procedural
object placement over Monte Carlo image synthesis to non-photorealistic
depiction. Their properties such as discrepancy, spectra, anisotropy, or
progressiveness have been analyzed extensively. However, designing methods to
produce sampling patterns with certain properties can require substantial
hand-crafting effort, both in coding, mathematical derivation and compute time.
In particular, there is no systematic way to derive the best sampling algorithm
for a specific end-task.
  Tackling this issue, we suggest another level of abstraction: a toolkit to
end-to-end optimize over all sampling methods to find the one producing
user-prescribed properties such as discrepancy or a spectrum that best fit the
end-task. A user simply implements the forward losses and the sampling method
is found automatically -- without coding or mathematical derivation -- by
making use of back-propagation abilities of modern deep learning frameworks.
While this optimization takes long, at deployment time the sampling method is
quick to execute as iterated unstructured non-linear filtering using radial
basis functions (RBFs) to represent high-dimensional kernels. Several important
previous methods are special cases of this approach, which we compare to
previous work and demonstrate its usefulness in several typical Computer
Graphics applications. Finally, we propose sampling patterns with properties
not shown before, such as high-dimensional blue noise with projective
properties.

%K Computer Science, Graphics, cs.GR

Article

T. Leimkühler, P. Kellnhofer, T. Ritschel, K. Myszkowski, and H.-P. Seidel

“Perceptual Real-Time 2D-to-3D Conversion Using Cue Fusion,” IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 6, 2018.

@article{Leimkuehler2018,
TITLE = {Perceptual real-time {2D}-to-{3D} conversion using cue fusion},
AUTHOR = {Leimk{\"u}hler, Thomas and Kellnhofer, Petr and Ritschel, Tobias and Myszkowski, Karol and Seidel, Hans-Peter},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2017.2703612},
PUBLISHER = {IEEE Computer Society},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics},
VOLUME = {24},
NUMBER = {6},
PAGES = {2037--2050},
}

Endnote

%0 Journal Article
%A Leimk&#252;hler, Thomas
%A Kellnhofer, Petr
%A Ritschel, Tobias
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perceptual Real-Time 2D-to-3D Conversion Using Cue Fusion : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-409A-9
%R 10.1109/TVCG.2017.2703612
%7 2018
%D 2018
%J IEEE Transactions on Visualization and Computer Graphics
%V 24
%N 6
%& 2037
%P 2037 - 2050
%I IEEE Computer Society
%C New York, NY
%@ false

Article

T. Leimkühler, H.-P. Seidel, and T. Ritschel

“Laplacian Kernel Splatting for Efficient Depth-of-field and Motion Blur Synthesis or Reconstruction,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2018), vol. 37, no. 4, 2018.

@article{LeimkuehlerSIGGRAPH2018,
TITLE = {Laplacian Kernel Splatting for Efficient Depth-of-field and Motion Blur Synthesis or Reconstruction},
AUTHOR = {Leimk{\"u}hler, Thomas and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3197517.3201379},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {37},
NUMBER = {4},
PAGES = {1--11},
EID = {55},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2018},
}

Endnote

%0 Journal Article
%A Leimk&#252;hler, Thomas
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Laplacian Kernel Splatting for Efficient Depth-of-field and Motion Blur 
Synthesis or Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-0630-1
%R 10.1145/3197517.3201379
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 4
%& 1
%P 1 - 11
%Z sequence number: 55
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2018
%O ACM SIGGRAPH 2018 Vancouver, Canada , 12 - 16 August

Paper

D4D2

K. Z. Lin, W. Xu, Q. Sun, C. Theobalt, and T.-S. Chua

“Learning a Disentangled Embedding for Monocular 3D Shape Retrieval and Pose Estimation,” 2018. [Online]. Available: http://arxiv.org/abs/1812.09899.

Abstract

We propose a novel approach to jointly perform 3D object retrieval and pose
estimation from monocular images.In order to make the method robust to real
world scene variations in the images, e.g. texture, lighting and background,we
learn an embedding space from 3D data that only includes the relevant
information, namely the shape and pose.Our method can then be trained for
robustness under real world scene variations without having to render a large
training set simulating these variations. Our learned embedding explicitly
disentangles a shape vector and a pose vector, which alleviates both pose bias
for 3D shape retrieval and categorical bias for pose estimation. Having the
learned disentangled embedding, we train a CNN to map the images to the
embedding space, and then retrieve the closest 3D shape from the database and
estimate the 6D pose of the object using the embedding vectors. Our method
achieves 10.8 median error for pose estimation and 0.514 top-1-accuracy for
category agnostic 3D object retrieval on the Pascal3D+ dataset. It therefore
outperforms the previous state-of-the-art methods on both tasks.

BibTeX

@online{Kyaw_arXiv1812.09899,
TITLE = {{Learning a Disentangled Embedding for Monocular 3D Shape Retrieval and Pose Estimation}},
AUTHOR = {Lin, Kyaw Zaw and Xu, Weipeng and Sun, Qianru and Theobalt, Christian and Chua, Tat-Seng},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1812.09899},
EPRINT = {1812.09899},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We propose a novel approach to jointly perform 3D object retrieval and pose<br>estimation from monocular images.In order to make the method robust to real<br>world scene variations in the images, e.g. texture, lighting and background,we<br>learn an embedding space from 3D data that only includes the relevant<br>information, namely the shape and pose.Our method can then be trained for<br>robustness under real world scene variations without having to render a large<br>training set simulating these variations. Our learned embedding explicitly<br>disentangles a shape vector and a pose vector, which alleviates both pose bias<br>for 3D shape retrieval and categorical bias for pose estimation. Having the<br>learned disentangled embedding, we train a CNN to map the images to the<br>embedding space, and then retrieve the closest 3D shape from the database and<br>estimate the 6D pose of the object using the embedding vectors. Our method<br>achieves 10.8 median error for pose estimation and 0.514 top-1-accuracy for<br>category agnostic 3D object retrieval on the Pascal3D+ dataset. It therefore<br>outperforms the previous state-of-the-art methods on both tasks.<br>},
}

Endnote

%0 Report
%A Lin, Kyaw Zaw
%A Xu, Weipeng
%A Sun, Qianru
%A Theobalt, Christian
%A Chua, Tat-Seng
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Learning a Disentangled Embedding for Monocular 3D Shape Retrieval and Pose Estimation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-D519-2
%U http://arxiv.org/abs/1812.09899
%D 2018
%X   We propose a novel approach to jointly perform 3D object retrieval and pose<br>estimation from monocular images.In order to make the method robust to real<br>world scene variations in the images, e.g. texture, lighting and background,we<br>learn an embedding space from 3D data that only includes the relevant<br>information, namely the shape and pose.Our method can then be trained for<br>robustness under real world scene variations without having to render a large<br>training set simulating these variations. Our learned embedding explicitly<br>disentangles a shape vector and a pose vector, which alleviates both pose bias<br>for 3D shape retrieval and categorical bias for pose estimation. Having the<br>learned disentangled embedding, we train a CNN to map the images to the<br>embedding space, and then retrieve the closest 3D shape from the database and<br>estimate the 6D pose of the object using the embedding vectors. Our method<br>achieves 10.8 median error for pose estimation and 0.514 top-1-accuracy for<br>category agnostic 3D object retrieval on the Pascal3D+ dataset. It therefore<br>outperforms the previous state-of-the-art methods on both tasks.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

L. Liu, N. Chen, D. Ceylan, C. Theobalt, W. Wang, and N. J. Mitra

“CurveFusion: Reconstructing thin Structures from RGBD Sequences,” ACM Transactions on Graphics, vol. 37, no. 6, 2018.

@article{Liu:2018:CvF,
TITLE = {{CurveFusion}: reconstructing thin structures from {RGBD} sequences},
AUTHOR = {Liu, Lingjie and Chen, Nenglun and Ceylan, Duygu and Theobalt, Christian and Wang, Wenping and Mitra, Niloy J.},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3272127.3275097},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {37},
NUMBER = {6},
PAGES = {1--12},
EID = {2018},
}

Endnote

%0 Journal Article
%A Liu, Lingjie
%A Chen, Nenglun
%A Ceylan, Duygu
%A Theobalt, Christian
%A Wang, Wenping
%A Mitra, Niloy J.
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T CurveFusion: Reconstructing thin Structures from RGBD Sequences : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-E6F2-9
%R 10.1145/3272127.3275097
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 6
%& 1
%P 1 - 12
%Z sequence number: 2018
%I ACM
%C New York, NY
%@ false

Paper

L. Liu, W. Xu, M. Zollhöfer, H. Kim, F. Bernard, M. Habermann, W. Wang, and C. Theobalt

“Neural Animation and Reenactment of Human Actor Videos,” 2018. [Online]. Available: http://arxiv.org/abs/1809.03658.

Abstract

We propose a method for generating (near) video-realistic animations of real
humans under user control. In contrast to conventional human character
rendering, we do not require the availability of a production-quality
photo-realistic 3D model of the human, but instead rely on a video sequence in
conjunction with a (medium-quality) controllable 3D template model of the
person. With that, our approach significantly reduces production cost compared
to conventional rendering approaches based on production-quality 3D models, and
can also be used to realistically edit existing videos. Technically, this is
achieved by training a neural network that translates simple synthetic images
of a human character into realistic imagery. For training our networks, we
first track the 3D motion of the person in the video using the template model,
and subsequently generate a synthetically rendered version of the video. These
images are then used to train a conditional generative adversarial network that
translates synthetic images of the 3D model into realistic imagery of the
human. We evaluate our method for the reenactment of another person that is
tracked in order to obtain the motion data, and show video results generated
from artist-designed skeleton motion. Our results outperform the
state-of-the-art in learning-based human image synthesis. Project page:
gvv.mpi-inf.mpg.de/projects/wxu/HumanReenactment/

BibTeX

@online{Liu_arXiv1809.03658,
TITLE = {Neural Animation and Reenactment of Human Actor Videos},
AUTHOR = {Liu, Lingjie and Xu, Weipeng and Zollh{\"o}fer, Michael and Kim, Hyeongwoo and Bernard, Florian and Habermann, Marc and Wang, Wenping and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1809.03658},
EPRINT = {1809.03658},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We propose a method for generating (near) video-realistic animations of real<br>humans under user control. In contrast to conventional human character<br>rendering, we do not require the availability of a production-quality<br>photo-realistic 3D model of the human, but instead rely on a video sequence in<br>conjunction with a (medium-quality) controllable 3D template model of the<br>person. With that, our approach significantly reduces production cost compared<br>to conventional rendering approaches based on production-quality 3D models, and<br>can also be used to realistically edit existing videos. Technically, this is<br>achieved by training a neural network that translates simple synthetic images<br>of a human character into realistic imagery. For training our networks, we<br>first track the 3D motion of the person in the video using the template model,<br>and subsequently generate a synthetically rendered version of the video. These<br>images are then used to train a conditional generative adversarial network that<br>translates synthetic images of the 3D model into realistic imagery of the<br>human. We evaluate our method for the reenactment of another person that is<br>tracked in order to obtain the motion data, and show video results generated<br>from artist-designed skeleton motion. Our results outperform the<br>state-of-the-art in learning-based human image synthesis. Project page:<br>http://gvv.mpi-inf.mpg.de/projects/wxu/HumanReenactment/<br>},
}

Endnote

%0 Report
%A Liu, Lingjie
%A Xu, Weipeng
%A Zollh&#246;fer, Michael
%A Kim, Hyeongwoo
%A Bernard, Florian
%A Habermann, Marc
%A Wang, Wenping
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Neural Animation and Reenactment of Human Actor Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5E06-F
%U http://arxiv.org/abs/1809.03658
%D 2018
%X   We propose a method for generating (near) video-realistic animations of real<br>humans under user control. In contrast to conventional human character<br>rendering, we do not require the availability of a production-quality<br>photo-realistic 3D model of the human, but instead rely on a video sequence in<br>conjunction with a (medium-quality) controllable 3D template model of the<br>person. With that, our approach significantly reduces production cost compared<br>to conventional rendering approaches based on production-quality 3D models, and<br>can also be used to realistically edit existing videos. Technically, this is<br>achieved by training a neural network that translates simple synthetic images<br>of a human character into realistic imagery. For training our networks, we<br>first track the 3D motion of the person in the video using the template model,<br>and subsequently generate a synthetically rendered version of the video. These<br>images are then used to train a conditional generative adversarial network that<br>translates synthetic images of the 3D model into realistic imagery of the<br>human. We evaluate our method for the reenactment of another person that is<br>tracked in order to obtain the motion data, and show video results generated<br>from artist-designed skeleton motion. Our results outperform the<br>state-of-the-art in learning-based human image synthesis. Project page:<br>http://gvv.mpi-inf.mpg.de/projects/wxu/HumanReenactment/<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%U http://gvv.mpi-inf.mpg.de/projects/wxu/HumanReenactment/

Conference paper

D4D2

D. Mehta, O. Sotnychenko, F. Mueller, H. Rhodin, W. Xu, G. Pons-Moll, and C. Theobalt

“Demo of XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera,” in ECCV 2018 Demo Sessions, Munich, Germany, 2018.

@inproceedings{XNectDemo_ECCV2018,
TITLE = {Demo of {XNect}: Real-time Multi-person {3D} Human Pose Estimation with a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Rhodin, Helge and Xu, Weipeng and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://gvv.mpi-inf.mpg.de/projects/XNectDemo/},
YEAR = {2018},
BOOKTITLE = {ECCV 2018 Demo Sessions},
ADDRESS = {Munich, Germany},
}

Endnote

%0 Conference Proceedings
%A Mehta, Dushyant
%A Sotnychenko, Oleksandr
%A Mueller, Franziska
%A Rhodin, Helge
%A Xu, Weipeng
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Demo of XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F4DC-3
%U http://gvv.mpi-inf.mpg.de/projects/XNectDemo/
%D 2018
%B European Conference on Computer Vision
%Z date of event: 2018-09-08 - 2018-09-14
%C Munich, Germany
%B ECCV 2018 Demo Sessions
%U http://gvv.mpi-inf.mpg.de/projects/XNectDemo/

Conference paper

D4D2

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, S. Sridhar, G. Pons-Moll, and C. Theobalt

“Single-Shot Multi-person 3D Pose Estimation from Monocular RGB,” in 3DV 2018 , International Conference on 3D Vision, Verona, Italy, 2018.

@inproceedings{Mehta_3DV2018,
TITLE = {Single-Shot Multi-person {3D} Pose Estimation from Monocular {RGB}},
AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Sridhar, Srinath and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-5386-8425-2 ; 978-1-5386-8426-9},
DOI = {10.1109/3DV.2018.00024},
PUBLISHER = {IEEE},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {3DV 2018 , International Conference on 3D Vision},
PAGES = {120--130},
ADDRESS = {Verona, Italy},
}

Endnote

%0 Conference Proceedings
%A Mehta, Dushyant
%A Sotnychenko, Oleksandr
%A Mueller, Franziska
%A Xu, Weipeng
%A Sridhar, Srinath
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Single-Shot Multi-person 3D Pose Estimation from Monocular RGB : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5C46-9
%R 10.1109/3DV.2018.00024
%D 2018
%B International Conference on 3D Vision
%Z date of event: 2018-09-05 - 2018-09-08
%C Verona, Italy
%B 3DV 2018 
%P 120 - 130
%I IEEE
%@ 978-1-5386-8425-2  978-1-5386-8426-9

Paper

D. Mehta, K. I. Kim, and C. Theobalt

“On Implicit Filter Level Sparsity in Convolutional Neural Networks,” 2018. [Online]. Available: http://arxiv.org/abs/1811.12495.

Abstract

We investigate filter level sparsity that emerges in convolutional neural
networks (CNNs) which employ Batch Normalization and ReLU activation, and are
trained with adaptive gradient descent techniques and L2 regularization (or
weight decay). We conduct an extensive experimental study casting these initial
findings into hypotheses and conclusions about the mechanisms underlying the
emergent filter level sparsity. This study allows new insight into the
performance gap obeserved between adapative and non-adaptive gradient descent
methods in practice. Further, analysis of the effect of training strategies and
hyperparameters on the sparsity leads to practical suggestions in designing CNN
training strategies enabling us to explore the tradeoffs between feature
selectivity, network capacity, and generalization performance. Lastly, we show
that the implicit sparsity can be harnessed for neural network speedup at par
or better than explicit sparsification / pruning approaches, without needing
any modifications to the typical training pipeline.

BibTeX

@online{Mehta_arXIv1811.12495,
TITLE = {On Implicit Filter Level Sparsity in Convolutional Neural Networks},
AUTHOR = {Mehta, Dushyant and Kim, Kwang In and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1811.12495},
EPRINT = {1811.12495},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We investigate filter level sparsity that emerges in convolutional neural<br>networks (CNNs) which employ Batch Normalization and ReLU activation, and are<br>trained with adaptive gradient descent techniques and L2 regularization (or<br>weight decay). We conduct an extensive experimental study casting these initial<br>findings into hypotheses and conclusions about the mechanisms underlying the<br>emergent filter level sparsity. This study allows new insight into the<br>performance gap obeserved between adapative and non-adaptive gradient descent<br>methods in practice. Further, analysis of the effect of training strategies and<br>hyperparameters on the sparsity leads to practical suggestions in designing CNN<br>training strategies enabling us to explore the tradeoffs between feature<br>selectivity, network capacity, and generalization performance. Lastly, we show<br>that the implicit sparsity can be harnessed for neural network speedup at par<br>or better than explicit sparsification / pruning approaches, without needing<br>any modifications to the typical training pipeline.<br>},
}

Endnote

%0 Report
%A Mehta, Dushyant
%A Kim, Kwang In
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T On Implicit Filter Level Sparsity in Convolutional Neural Networks : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-E920-3
%U http://arxiv.org/abs/1811.12495
%D 2018
%X   We investigate filter level sparsity that emerges in convolutional neural<br>networks (CNNs) which employ Batch Normalization and ReLU activation, and are<br>trained with adaptive gradient descent techniques and L2 regularization (or<br>weight decay). We conduct an extensive experimental study casting these initial<br>findings into hypotheses and conclusions about the mechanisms underlying the<br>emergent filter level sparsity. This study allows new insight into the<br>performance gap obeserved between adapative and non-adaptive gradient descent<br>methods in practice. Further, analysis of the effect of training strategies and<br>hyperparameters on the sparsity leads to practical suggestions in designing CNN<br>training strategies enabling us to explore the tradeoffs between feature<br>selectivity, network capacity, and generalization performance. Lastly, we show<br>that the implicit sparsity can be harnessed for neural network speedup at par<br>or better than explicit sparsification / pruning approaches, without needing<br>any modifications to the typical training pipeline.<br>
%K Computer Science, Learning, cs.LG,Computer Science, Computer Vision and Pattern Recognition, cs.CV,eess.SP,Statistics, Machine Learning, stat.ML

Paper

A. Meka, M. Maximov, M. Zollhöfer, A. Chatterjee, H.-P. Seidel, C. Richardt, and C. Theobalt

“LIME: Live Intrinsic Material Estimation,” 2018. [Online]. Available: http://arxiv.org/abs/1801.01075.

Abstract

We present the first end to end approach for real time material estimation

for general object shapes with uniform material that only requires a single

color image as input. In addition to Lambertian surface properties, our

approach fully automatically computes the specular albedo, material shininess,

and a foreground segmentation. We tackle this challenging and ill posed inverse

rendering problem using recent advances in image to image translation

techniques based on deep convolutional encoder decoder architectures. The

underlying core representations of our approach are specular shading, diffuse

shading and mirror images, which allow to learn the effective and accurate

separation of diffuse and specular albedo. In addition, we propose a novel

highly efficient perceptual rendering loss that mimics real world image

formation and obtains intermediate results even during run time. The estimation

of material parameters at real time frame rates enables exciting mixed reality

applications, such as seamless illumination consistent integration of virtual

objects into real world scenes, and virtual material cloning. We demonstrate

our approach in a live setup, compare it to the state of the art, and

demonstrate its effectiveness through quantitative and qualitative evaluation.

BibTeX

@online{Meka_arXiv1801.01075,
TITLE = {LIME: {L}ive Intrinsic Material Estimation},
AUTHOR = {Meka, Abhimitra and Maximov, Maxim and Zollh{\"o}fer, Michael and Chatterjee, Avishek and Seidel, Hans-Peter and Richardt, Christian and Theobalt, Christian},
URL = {http://arxiv.org/abs/1801.01075},
EPRINT = {1801.01075},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We present the first end to end approach for real time material estimation for general object shapes with uniform material that only requires a single color image as input. In addition to Lambertian surface properties, our approach fully automatically computes the specular albedo, material shininess, and a foreground segmentation. We tackle this challenging and ill posed inverse rendering problem using recent advances in image to image translation techniques based on deep convolutional encoder decoder architectures. The underlying core representations of our approach are specular shading, diffuse shading and mirror images, which allow to learn the effective and accurate separation of diffuse and specular albedo. In addition, we propose a novel highly efficient perceptual rendering loss that mimics real world image formation and obtains intermediate results even during run time. The estimation of material parameters at real time frame rates enables exciting mixed reality applications, such as seamless illumination consistent integration of virtual objects into real world scenes, and virtual material cloning. We demonstrate our approach in a live setup, compare it to the state of the art, and demonstrate its effectiveness through quantitative and qualitative evaluation.},
}

Endnote

%0 Report
%A Meka, Abhimitra
%A Maximov, Maxim
%A Zollh&#246;fer, Michael
%A Chatterjee, Avishek
%A Seidel, Hans-Peter
%A Richardt, Christian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
D2 External
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T LIME: Live Intrinsic Material Estimation : 
%U http://hdl.handle.net/21.11116/0000-0001-40D9-2
%U http://arxiv.org/abs/1801.01075
%D 2018
%X   We present the first end to end approach for real time material estimation
for general object shapes with uniform material that only requires a single
color image as input. In addition to Lambertian surface properties, our
approach fully automatically computes the specular albedo, material shininess,
and a foreground segmentation. We tackle this challenging and ill posed inverse
rendering problem using recent advances in image to image translation
techniques based on deep convolutional encoder decoder architectures. The
underlying core representations of our approach are specular shading, diffuse
shading and mirror images, which allow to learn the effective and accurate
separation of diffuse and specular albedo. In addition, we propose a novel
highly efficient perceptual rendering loss that mimics real world image
formation and obtains intermediate results even during run time. The estimation
of material parameters at real time frame rates enables exciting mixed reality
applications, such as seamless illumination consistent integration of virtual
objects into real world scenes, and virtual material cloning. We demonstrate
our approach in a live setup, compare it to the state of the art, and
demonstrate its effectiveness through quantitative and qualitative evaluation.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

A. Meka, M. Maximov, M. Zollhöfer, A. Chatterjee, H.-P. Seidel, C. Richardt, and C. Theobalt

“LIME: Live Intrinsic Material Estimation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA, 2018.

@inproceedings{Meka:2018,
TITLE = {{LIME}: {L}ive Intrinsic Material Estimation},
AUTHOR = {Meka, Abhimitra and Maximov, Maxim and Zollh{\"o}fer, Michael and Chatterjee, Avishek and Seidel, Hans-Peter and Richardt, Christian and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-5386-6420-9},
DOI = {10.1109/CVPR.2018.00661},
PUBLISHER = {IEEE},
YEAR = {2018},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)},
PAGES = {6315--6324},
ADDRESS = {Salt Lake City, UT, USA},
}

Endnote

%0 Conference Proceedings
%A Meka, Abhimitra
%A Maximov, Maxim
%A Zollh&#246;fer, Michael
%A Chatterjee, Avishek
%A Seidel, Hans-Peter
%A Richardt, Christian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T LIME: Live Intrinsic Material Estimation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F391-7
%R 10.1109/CVPR.2018.00661
%D 2018
%B 31st IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2018-06-18 - 2018-06-22
%C Salt Lake City, UT, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 6315 - 6324
%I IEEE
%@ 978-1-5386-6420-9
%U http://gvv.mpi-inf.mpg.de/projects/LIME/

Paper

D. Mlakar, M. Winter, H.-P. Seidel, M. Steinberger, and R. Zayer

“AlSub: Fully Parallel and Modular Subdivision,” 2018. [Online]. Available: http://arxiv.org/abs/1809.06047.

Abstract

In recent years, mesh subdivision---the process of forging smooth free-form
surfaces from coarse polygonal meshes---has become an indispensable production
instrument. Although subdivision performance is crucial during simulation,
animation and rendering, state-of-the-art approaches still rely on serial
implementations for complex parts of the subdivision process. Therefore, they
often fail to harness the power of modern parallel devices, like the graphics
processing unit (GPU), for large parts of the algorithm and must resort to
time-consuming serial preprocessing. In this paper, we show that a complete
parallelization of the subdivision process for modern architectures is
possible. Building on sparse matrix linear algebra, we show how to structure
the complete subdivision process into a sequence of algebra operations. By
restructuring and grouping these operations, we adapt the process for different
use cases, such as regular subdivision of dynamic meshes, uniform subdivision
for immutable topology, and feature-adaptive subdivision for efficient
rendering of animated models. As the same machinery is used for all use cases,
identical subdivision results are achieved in all parts of the production
pipeline. As a second contribution, we show how these linear algebra
formulations can effectively be translated into efficient GPU kernels. Applying
our strategies to $\sqrt{3}$, Loop and Catmull-Clark subdivision shows
significant speedups of our approach compared to state-of-the-art solutions,
while we completely avoid serial preprocessing.

BibTeX

@online{Mlakar_arXiv1809.06047,
TITLE = {{AlSub}: {Fully Parallel and Modular Subdivision}},
AUTHOR = {Mlakar, Daniel and Winter, Martin and Seidel, Hans-Peter and Steinberger, Markus and Zayer, Rhaleb},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1809.06047},
EPRINT = {1809.06047},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {In recent years, mesh subdivision---the process of forging smooth free-form<br>surfaces from coarse polygonal meshes---has become an indispensable production<br>instrument. Although subdivision performance is crucial during simulation,<br>animation and rendering, state-of-the-art approaches still rely on serial<br>implementations for complex parts of the subdivision process. Therefore, they<br>often fail to harness the power of modern parallel devices, like the graphics<br>processing unit (GPU), for large parts of the algorithm and must resort to<br>time-consuming serial preprocessing. In this paper, we show that a complete<br>parallelization of the subdivision process for modern architectures is<br>possible. Building on sparse matrix linear algebra, we show how to structure<br>the complete subdivision process into a sequence of algebra operations. By<br>restructuring and grouping these operations, we adapt the process for different<br>use cases, such as regular subdivision of dynamic meshes, uniform subdivision<br>for immutable topology, and feature-adaptive subdivision for efficient<br>rendering of animated models. As the same machinery is used for all use cases,<br>identical subdivision results are achieved in all parts of the production<br>pipeline. As a second contribution, we show how these linear algebra<br>formulations can effectively be translated into efficient GPU kernels. Applying<br>our strategies to $\sqrt{3}$, Loop and Catmull-Clark subdivision shows<br>significant speedups of our approach compared to state-of-the-art solutions,<br>while we completely avoid serial preprocessing.<br>},
}

Endnote

%0 Report
%A Mlakar, Daniel
%A Winter, Martin
%A Seidel, Hans-Peter
%A Steinberger, Markus
%A Zayer, Rhaleb
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T AlSub: Fully Parallel and Modular Subdivision : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-E5E2-C
%U http://arxiv.org/abs/1809.06047
%D 2018
%X   In recent years, mesh subdivision---the process of forging smooth free-form<br>surfaces from coarse polygonal meshes---has become an indispensable production<br>instrument. Although subdivision performance is crucial during simulation,<br>animation and rendering, state-of-the-art approaches still rely on serial<br>implementations for complex parts of the subdivision process. Therefore, they<br>often fail to harness the power of modern parallel devices, like the graphics<br>processing unit (GPU), for large parts of the algorithm and must resort to<br>time-consuming serial preprocessing. In this paper, we show that a complete<br>parallelization of the subdivision process for modern architectures is<br>possible. Building on sparse matrix linear algebra, we show how to structure<br>the complete subdivision process into a sequence of algebra operations. By<br>restructuring and grouping these operations, we adapt the process for different<br>use cases, such as regular subdivision of dynamic meshes, uniform subdivision<br>for immutable topology, and feature-adaptive subdivision for efficient<br>rendering of animated models. As the same machinery is used for all use cases,<br>identical subdivision results are achieved in all parts of the production<br>pipeline. As a second contribution, we show how these linear algebra<br>formulations can effectively be translated into efficient GPU kernels. Applying<br>our strategies to $\sqrt{3}$, Loop and Catmull-Clark subdivision shows<br>significant speedups of our approach compared to state-of-the-art solutions,<br>while we completely avoid serial preprocessing.<br>
%K Computer Science, Graphics, cs.GR

Conference paper

F. Mueller, F. Bernard, O. Sotnychenko, D. Mehta, S. Sridhar, D. Casas, and C. Theobalt

“GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA, 2018.

@inproceedings{Mueller_CVPR2018,
TITLE = {{GANerated} Hands for Real-Time {3D} Hand Tracking from Monocular {RGB}},
AUTHOR = {Mueller, Franziska and Bernard, Florian and Sotnychenko, Oleksandr and Mehta, Dushyant and Sridhar, Srinath and Casas, Dan and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-5386-6420-9},
DOI = {10.1109/CVPR.2018.00013},
PUBLISHER = {IEEE},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)},
PAGES = {49--59},
ADDRESS = {Salt Lake City, UT, USA},
}

Endnote

%0 Conference Proceedings
%A Mueller, Franziska
%A Bernard, Florian
%A Sotnychenko, Oleksandr
%A Mehta, Dushyant
%A Sridhar, Srinath
%A Casas, Dan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB  : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-EFFA-8
%R 10.1109/CVPR.2018.00013
%D 2018
%B 31st IEEE Conference on Computer Vision and Pattern Recognition 
%Z date of event: 2018-06-18 - 2018-06-22
%C Salt Lake City, UT, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 49 - 59
%I IEEE
%@ 978-1-5386-6420-9

Conference paper

K. Myszkowski, O. T. Tursun, P. Kellnhofer, K. Templin, E. Arabadzhiyska, P. Didyk, and H.-P. Seidel

“Perceptual Display: Apparent Enhancement of Scene Detail and Depth,” in Electronic Imaging (Proc. HVEI 2018), San Francisco, CA, USA, 2018, vol. 2018.

Abstract

Predicting human visual perception of image differences has several applications such as compression, rendering, editing and retargeting. Current approaches however, ignore the fact that the human visual system compensates for geometric transformations, e.g. we see that an image and a rotated copy are identical. Instead, they will report a large, false-positive difference. At the same time, if the transformations become too strong or too spatially incoherent, comparing two images indeed gets increasingly difficult. Between these two extremes, we propose a system to quantify the effect of transformations, not only on the perception of image differences, but also on saliency and motion parallax. To this end, we first fit local homographies to a given optical flow field and then convert this field into a field of elementary transformations such as translation, rotation, scaling, and perspective. We conduct a perceptual experiment quantifying the increase of difficulty when compensating for elementary transformations. Transformation entropy is proposed as a novel measure of complexity in a flow field. This representation is then used for applications, such as comparison of non-aligned images, where transformations cause threshold elevation, and detection of salient transformations.

BibTeX

@inproceedings{Myszkowski2018Perceptual,
TITLE = {Perceptual Display: Apparent Enhancement of Scene Detail and Depth},
AUTHOR = {Myszkowski, Karol and Tursun, Okan Tarhan and Kellnhofer, Petr and Templin, Krzysztof and Arabadzhiyska, Elena and Didyk, Piotr and Seidel, Hans-Peter},
LANGUAGE = {eng},
ISSN = {2470-1173},
DOI = {10.2352/ISSN.2470-1173.2018.14.HVEI-501},
PUBLISHER = {SPIE/IS\&T},
YEAR = {2018},
ABSTRACT = {Predicting human visual perception of image differences has several applications such as compression, rendering, editing and retargeting. Current approaches however, ignore the fact that the human visual system compensates for geometric transformations, e.g. we see that an image and a rotated copy are identical. Instead, they will report a large, false-positive difference. At the same time, if the transformations become too strong or too spatially incoherent, comparing two images indeed gets increasingly difficult. Between these two extremes, we propose a system to quantify the effect of transformations, not only on the perception of image differences, but also on saliency and motion parallax. To this end, we first fit local homographies to a given optical flow field and then convert this field into a field of elementary transformations such as translation, rotation, scaling, and perspective. We conduct a perceptual experiment quantifying the increase of difficulty when compensating for elementary transformations. Transformation entropy is proposed as a novel measure of complexity in a flow field. This representation is then used for applications, such as comparison of non-aligned images, where transformations cause threshold elevation, and detection of salient transformations.},
BOOKTITLE = {Human Vision and Electronic Imaging (HVEI 2018)},
PAGES = {1--10},
EID = {501},
JOURNAL = {Electronic Imaging (Proc. HVEI)},
VOLUME = {2018},
ADDRESS = {San Francisco, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Myszkowski, Karol
%A Tursun, Okan Tarhan
%A Kellnhofer, Petr
%A Templin, Krzysztof
%A Arabadzhiyska, Elena
%A Didyk, Piotr
%A Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perceptual Display: Apparent Enhancement of Scene Detail 
and Depth : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-5F64-5
%R 10.2352/ISSN.2470-1173.2018.14.HVEI-501
%D 2018
%B Human Vision and Electronic Imaging 
%Z date of event: 2018-01-28 - 2018-02-02
%C San Francisco, CA, USA
%X Predicting human visual perception of image differences has several applications such as compression, rendering, editing and retargeting. Current approaches however, ignore the fact that the human visual system compensates for geometric transformations, e.g. we see that an image and a rotated copy are identical. Instead, they will report a large, false-positive difference. At the same time, if the transformations become too strong or too spatially incoherent, comparing two images indeed gets increasingly difficult. Between these two extremes, we propose a system to quantify the effect of transformations, not only on the perception of image differences, but also on saliency and motion parallax. To this end, we first fit local homographies to a given optical flow field and then convert this field into a field of elementary transformations such as translation, rotation, scaling, and perspective. We conduct a perceptual experiment quantifying the increase of difficulty when compensating for elementary transformations. Transformation entropy is proposed as a novel measure of complexity in a flow field. This representation is then used for applications, such as comparison of non-aligned images, where transformations cause threshold elevation, and detection of salient transformations.
%B Human Vision and Electronic Imaging
%P 1 - 10
%Z sequence number: 501
%I SPIE/IS&T
%J Electronic Imaging
%V 2018
%@ false

Conference paper

A. C. Öztireli and G. Singh

“Sampling Analysis Using Correlations for Monte Carlo Rendering,” in SIGGRAPH Asia 2018 Courses (ACM SIGGRAPH Asia 2018), Tokyo, Japan, 2018.

@inproceedings{Oeztireli_SIGGRAPHASIA18,
TITLE = {Sampling Analysis Using Correlations for {M}onte {C}arlo Rendering},
AUTHOR = {{\"O}ztireli, A. Cengiz and Singh, Gurprit},
LANGUAGE = {eng},
DOI = {10.1145/3277644.3277783},
PUBLISHER = {ACM},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {SIGGRAPH Asia 2018 Courses (ACM SIGGRAPH Asia 2018)},
PAGES = {1--48},
EID = {16},
ADDRESS = {Tokyo, Japan},
}

Endnote

%0 Conference Proceedings
%A &#214;ztireli, A. Cengiz
%A Singh, Gurprit
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Sampling Analysis Using Correlations for Monte Carlo Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-9F4D-6
%R 10.1145/3277644.3277783
%D 2018
%B 11th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia
%Z date of event: 2018-12-04 - 2018-12-07
%C Tokyo, Japan
%B SIGGRAPH Asia 2018 Courses
%P 1 - 48
%Z sequence number: 16
%I ACM

Article

M. Piovarči, D. I. W. Levin, D. M. Kaufman, and P. Didyk

“Perception-Aware Modeling and Fabrication of Digital Drawing Tools,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2018), vol. 37, no. 4, 2018.

@article{Piovarci_SIGGRAPH2018,
TITLE = {Perception-Aware Modeling and Fabrication of Digital Drawing Tools},
AUTHOR = {Piovar{\v c}i, Michal and Levin, David I. W. and Kaufman, Danny M. and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3197517.3201322},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {37},
NUMBER = {4},
PAGES = {1--15},
EID = {123},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2018},
}

Endnote

%0 Journal Article
%A Piovar&#269;i, Michal
%A Levin, David I. W.
%A Kaufman, Danny M.
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Perception-Aware Modeling and Fabrication of Digital Drawing Tools : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5D7D-B
%R 10.1145/3197517.3201322
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 4
%& 1
%P 1 - 15
%Z sequence number: 123
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2018
%O ACM SIGGRAPH 2018 Vancouver, Canada , 12 - 16 August

Article

M. Piovarči, M. Wessely, M. Jagielski, M. Alexa, W. Matusik, and P. Didyk

“Design and Analysis of Directional Front Projection Screens,” Computers and Graphics, vol. 74, 2018.

@article{Piovarci_2018,
TITLE = {Design and Analysis of Directional Front Projection Screens},
AUTHOR = {Piovar{\v c}i, Michal and Wessely, Michael and Jagielski, Michal and Alexa, Marc and Matusik, Wojciech and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {0097-8493},
DOI = {10.1016/j.cag.2018.05.010},
PUBLISHER = {Pergamon},
ADDRESS = {New York},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Computers and Graphics},
VOLUME = {74},
PAGES = {213--224},
}

Endnote

%0 Journal Article
%A Piovar&#269;i, Michal
%A Wessely, Michael
%A Jagielski, Michal
%A Alexa, Marc
%A Matusik, Wojciech
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Design and Analysis of Directional Front Projection Screens : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-16D9-1
%R 10.1016/j.cag.2018.05.010
%7 2018
%D 2018
%J Computers and Graphics
%V 74
%& 213
%P 213 - 224
%I Pergamon
%C New York
%@ false

Conference paper

N. Robertini, F. Bernard, W. Xu, and C. Theobalt

“Illumination-Invariant Robust Multiview 3D Human Motion Capture,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV 2018), Lake Tahoe, NV, USA, 2018.

@inproceedings{Robertini_WACV2018,
TITLE = {Illumination-Invariant Robust Multiview {3D} Human Motion Capture},
AUTHOR = {Robertini, Nadia and Bernard, Florian and Xu, Weipeng and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-5386-4886-5},
DOI = {10.1109/WACV.2018.00185},
PUBLISHER = {IEEE},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {2018 IEEE Winter Conference on Applications of Computer Vision (WACV 2018)},
PAGES = {1661--1670},
ADDRESS = {Lake Tahoe, NV, USA},
}

Endnote

%0 Conference Proceedings
%A Robertini, Nadia
%A Bernard, Florian
%A Xu, Weipeng
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Illumination-Invariant Robust Multiview 3D Human Motion Capture : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-A474-3
%R 10.1109/WACV.2018.00185
%D 2018
%B IEEE Winter Conference on Applications of Computer Vision
%Z date of event: 2018-03-12 - 2018-03-15
%C Lake Tahoe, NV, USA
%B 2018 IEEE Winter Conference on Applications of Computer Vision 
%P 1661 - 1670
%I IEEE
%@ 978-1-5386-4886-5

Conference paper

K. Sarkar, F. Bernard, K. Varanasi, C. Theobalt, and D. Stricker

“Structured Low-Rank Matrix Factorization for Point-Cloud Denoising,” in 3DV 2018 , International Conference on 3D Vision, Verona, Italy, 2018.

@inproceedings{Sarkar_3DV2018,
TITLE = {Structured Low-Rank Matrix Factorization for Point-Cloud Denoising},
AUTHOR = {Sarkar, Kripasindhu and Bernard, Florian and Varanasi, Kiran and Theobalt, Christian and Stricker, Didier},
LANGUAGE = {eng},
ISBN = {978-1-5386-8425-2 ; 978-1-5386-8426-9},
DOI = {10.1109/3DV.2018.00058},
PUBLISHER = {IEEE},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {3DV 2018 , International Conference on 3D Vision},
PAGES = {444--453},
ADDRESS = {Verona, Italy},
}

Endnote

%0 Conference Proceedings
%A Sarkar, Kripasindhu
%A Bernard, Florian
%A Varanasi, Kiran
%A Theobalt, Christian
%A Stricker, Didier
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Structured Low-Rank Matrix Factorization for Point-Cloud Denoising : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-D62C-C
%R 10.1109/3DV.2018.00058
%D 2018
%B International Conference on 3D Vision
%Z date of event: 2018-09-05 - 2018-09-08
%C Verona, Italy
%B 3DV 2018 
%P 444 - 453
%I IEEE
%@ 978-1-5386-8425-2  978-1-5386-8426-9

Paper

A. Serrano, D. Gutierrez, K. Myszkowski, H.-P. Seidel, and B. Masia

“An Intuitive Control Space for Material Appearance,” 2018. [Online]. Available: http://arxiv.org/abs/1806.04950.

Abstract

Many different techniques for measuring material appearance have been
proposed in the last few years. These have produced large public datasets,
which have been used for accurate, data-driven appearance modeling. However,
although these datasets have allowed us to reach an unprecedented level of
realism in visual appearance, editing the captured data remains a challenge. In
this paper, we present an intuitive control space for predictable editing of
captured BRDF data, which allows for artistic creation of plausible novel
material appearances, bypassing the difficulty of acquiring novel samples. We
first synthesize novel materials, extending the existing MERL dataset up to 400
mathematically valid BRDFs. We then design a large-scale experiment, gathering
56,000 subjective ratings on the high-level perceptual attributes that best
describe our extended dataset of materials. Using these ratings, we build and
train networks of radial basis functions to act as functionals mapping the
perceptual attributes to an underlying PCA-based representation of BRDFs. We
show that our functionals are excellent predictors of the perceived attributes
of appearance. Our control space enables many applications, including intuitive
material editing of a wide range of visual properties, guidance for gamut
mapping, analysis of the correlation between perceptual attributes, or novel
appearance similarity metrics. Moreover, our methodology can be used to derive
functionals applicable to classic analytic BRDF representations. We release our
code and dataset publicly, in order to support and encourage further research
in this direction.

BibTeX

@online{Serrano_arXiv1806.04950,
TITLE = {An Intuitive Control Space for Material Appearance},
AUTHOR = {Serrano, Ana and Gutierrez, Diego and Myszkowski, Karol and Seidel, Hans-Peter and Masia, Belen},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1806.04950},
EPRINT = {1806.04950},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {Many different techniques for measuring material appearance have been<br>proposed in the last few years. These have produced large public datasets,<br>which have been used for accurate, data-driven appearance modeling. However,<br>although these datasets have allowed us to reach an unprecedented level of<br>realism in visual appearance, editing the captured data remains a challenge. In<br>this paper, we present an intuitive control space for predictable editing of<br>captured BRDF data, which allows for artistic creation of plausible novel<br>material appearances, bypassing the difficulty of acquiring novel samples. We<br>first synthesize novel materials, extending the existing MERL dataset up to 400<br>mathematically valid BRDFs. We then design a large-scale experiment, gathering<br>56,000 subjective ratings on the high-level perceptual attributes that best<br>describe our extended dataset of materials. Using these ratings, we build and<br>train networks of radial basis functions to act as functionals mapping the<br>perceptual attributes to an underlying PCA-based representation of BRDFs. We<br>show that our functionals are excellent predictors of the perceived attributes<br>of appearance. Our control space enables many applications, including intuitive<br>material editing of a wide range of visual properties, guidance for gamut<br>mapping, analysis of the correlation between perceptual attributes, or novel<br>appearance similarity metrics. Moreover, our methodology can be used to derive<br>functionals applicable to classic analytic BRDF representations. We release our<br>code and dataset publicly, in order to support and encourage further research<br>in this direction.<br>},
}

Endnote

%0 Report
%A Serrano, Ana
%A Gutierrez, Diego
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Masia, Belen
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T An Intuitive Control Space for Material Appearance : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-151E-6
%U http://arxiv.org/abs/1806.04950
%D 2018
%X   Many different techniques for measuring material appearance have been<br>proposed in the last few years. These have produced large public datasets,<br>which have been used for accurate, data-driven appearance modeling. However,<br>although these datasets have allowed us to reach an unprecedented level of<br>realism in visual appearance, editing the captured data remains a challenge. In<br>this paper, we present an intuitive control space for predictable editing of<br>captured BRDF data, which allows for artistic creation of plausible novel<br>material appearances, bypassing the difficulty of acquiring novel samples. We<br>first synthesize novel materials, extending the existing MERL dataset up to 400<br>mathematically valid BRDFs. We then design a large-scale experiment, gathering<br>56,000 subjective ratings on the high-level perceptual attributes that best<br>describe our extended dataset of materials. Using these ratings, we build and<br>train networks of radial basis functions to act as functionals mapping the<br>perceptual attributes to an underlying PCA-based representation of BRDFs. We<br>show that our functionals are excellent predictors of the perceived attributes<br>of appearance. Our control space enables many applications, including intuitive<br>material editing of a wide range of visual properties, guidance for gamut<br>mapping, analysis of the correlation between perceptual attributes, or novel<br>appearance similarity metrics. Moreover, our methodology can be used to derive<br>functionals applicable to classic analytic BRDF representations. We release our<br>code and dataset publicly, in order to support and encourage further research<br>in this direction.<br>
%K Computer Science, Graphics, cs.GR

Conference paper

S. Shekhar, S. Kunz Beigpour, M. Ziegler, M. Chwesiuk, D. Paleń, K. Myszkowski, J. Keinert, R. Mantiuk, and P. Didyk

“Light-Field Intrinsic Dataset,” in Proceedings of the British Machine Vision Conference 2018 (BMVC), Newcastle, UK, 2018.

@inproceedings{Shekhar_BMVC2018,
TITLE = {Light-Field Intrinsic Dataset},
AUTHOR = {Shekhar, Sumit and Kunz Beigpour, Shida and Ziegler, Matthias and Chwesiuk, Micha{\l} and Pale{\'n}, Dawid and Myszkowski, Karol and Keinert, Joachim and Mantiuk, Rados{\l}aw and Didyk, Piotr},
LANGUAGE = {eng},
URL = {http://bmvc2018.org/programme/BMVC2018.zip},
PUBLISHER = {British Machine Vision Association},
YEAR = {2018},
BOOKTITLE = {Proceedings of the British Machine Vision Conference 2018 (BMVC)},
EID = {0120},
ADDRESS = {Newcastle, UK},
}

Endnote

%0 Conference Proceedings
%A Shekhar, Sumit
%A Kunz Beigpour, Shida
%A Ziegler, Matthias
%A Chwesiuk, Micha&#322;
%A Pale&#324;, Dawid
%A Myszkowski, Karol
%A Keinert, Joachim
%A Mantiuk, Rados&#322;aw
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Light-Field Intrinsic Dataset : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-0E38-1
%D 2018
%B British Machine Vision Conference 2018 (BMVC)
%Z date of event: 2018-09-03 - 2018-09-06
%C Newcastle, UK
%K Forschungsgruppe Geiger
%B Proceedings of the British Machine Vision Conference 2018 (BMVC)
%Z sequence number: 0120
%I British Machine Vision Association
%U http://bmvc2018.org/programme/BMVC2018.zip

Article

L. Shi, V. Babaei, C. Kim, M. Foshey, Y. Hu, P. Sitthi-Amorn, S. Rusinkiewicz, and W. Matusik

“Deep Multispectral Painting Reproduction via Multi-Layer, Custom-Ink Printing,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2018), vol. 37, no. 6, 2018.

@article{Shi2018,
TITLE = {Deep Multispectral Painting Reproduction via Multi-Layer, Custom-Ink Printing},
AUTHOR = {Shi, Liang and Babaei, Vahid and Kim, Chagil and Foshey, Michael and Hu, Yuanming and Sitthi-Amorn, Pitchaya and Rusinkiewicz, Szymon and Matusik, Wojciech},
LANGUAGE = {eng},
ISSN = {0730-0301},
ISBN = {978-1-4503-6008-1},
DOI = {10.1145/3272127.3275057},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {37},
NUMBER = {6},
EID = {271},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2018},
}

Endnote

%0 Journal Article
%A Shi, Liang
%A Babaei, Vahid
%A Kim, Chagil
%A Foshey, Michael
%A Hu, Yuanming
%A Sitthi-Amorn, Pitchaya
%A Rusinkiewicz, Szymon
%A Matusik, Wojciech
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T Deep Multispectral Painting Reproduction via Multi-Layer, Custom-Ink
Printing : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5C93-1
%R 10.1145/3272127.3275057
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 6
%Z sequence number: 271
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2018

%O ACM SIGGRAPH Asia 2018 Tokyo, Japan,  December 04 - 07, 2018  SA'18 SA 2018
%@ 978-1-4503-6008-1

Article

G. Singh, K. Subr, D. Coeurjolly, V. Ostromoukhov, and W. Jarosz

“Analyzing and Improving Correlated Monte Carlo Importance Sampling of Discontinuous functions,” Computer Graphics Forum.

@article{SinghCGF2018,
TITLE = {Analyzing and Improving Correlated {Monte Carlo} Importance Sampling of Discontinuous functions},
AUTHOR = {Singh, Gurprit and Subr, Kartic and Coeurjolly, David and Ostromoukhov, Victor and Jarosz, Wojciech},
LANGUAGE = {eng},
ISSN = {0167-7055},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2018},
PUBLREMARK = {Accepted},
JOURNAL = {Computer Graphics Forum},
}

Endnote

%0 Journal Article
%A Singh, Gurprit
%A Subr, Kartic
%A Coeurjolly, David
%A Ostromoukhov, Victor
%A Jarosz, Wojciech
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
%T Analyzing and Improving Correlated Monte Carlo Importance Sampling of Discontinuous functions : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-9F87-3
%D 2018
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%I Blackwell-Wiley
%C Oxford
%@ false

Conference paper

M. Soliman, F. Mueller, L. Hegemann, J. S. Roo, C. Theobalt, and J. Steimle

“FingerInput: Capturing Expressive Single-Hand Thumb-to-Finger Microgestures,” in ISS’18, 13th ACM International Conference on Interactive Surfaces and Spaces, Tokyo, Japan, 2018.

@inproceedings{Soliman_ISS2018,
TITLE = {{FingerInput}: {C}apturing Expressive Single-Hand Thumb-to-Finger Microgestures},
AUTHOR = {Soliman, Mohamed and Mueller, Franziska and Hegemann, Lena and Roo, Joan Sol and Theobalt, Christian and Steimle, J{\"u}rgen},
LANGUAGE = {eng},
ISBN = {978-1-4503-5694-7},
DOI = {10.1145/3279778.3279799},
PUBLISHER = {ACM},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {ISS'18, 13th ACM International Conference on Interactive Surfaces and Spaces},
PAGES = {177--187},
ADDRESS = {Tokyo, Japan},
}

Endnote

%0 Conference Proceedings
%A Soliman, Mohamed
%A Mueller, Franziska
%A Hegemann, Lena
%A Roo, Joan Sol
%A Theobalt, Christian
%A Steimle, J&#252;rgen
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T FingerInput: Capturing Expressive Single-Hand Thumb-to-Finger Microgestures : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-A36D-C
%R 10.1145/3279778.3279799
%D 2018
%B 13th ACM International Conference on Interactive Surfaces and Spaces
%Z date of event: 2018-11-25 - 2018-11-28
%C Tokyo, Japan
%B ISS'18
%P 177 - 187
%I ACM
%@ 978-1-4503-5694-7

Paper

Q. Sun, A. Tewari, W. Xu, M. Fritz, C. Theobalt, and B. Schiele

“A Hybrid Model for Identity Obfuscation by Face Replacement,” 2018. [Online]. Available: http://arxiv.org/abs/1804.04779.

Abstract

As more and more personal photos are shared and tagged in social media,
avoiding privacy risks such as unintended recognition becomes increasingly
challenging. We propose a new hybrid approach to obfuscate identities in photos
by head replacement. Our approach combines state of the art parametric face
synthesis with latest advances in Generative Adversarial Networks (GAN) for
data-driven image synthesis. On the one hand, the parametric part of our method
gives us control over the facial parameters and allows for explicit
manipulation of the identity. On the other hand, the data-driven aspects allow
for adding fine details and overall realism as well as seamless blending into
the scene context. In our experiments, we show highly realistic output of our
system that improves over the previous state of the art in obfuscation rate
while preserving a higher similarity to the original image content.

BibTeX

@online{Sun_arXiv1804.04779,
TITLE = {A Hybrid Model for Identity Obfuscation by Face Replacement},
AUTHOR = {Sun, Qianru and Tewari, Ayush and Xu, Weipeng and Fritz, Mario and Theobalt, Christian and Schiele, Bernt},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1804.04779},
EPRINT = {1804.04779},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {As more and more personal photos are shared and tagged in social media,<br>avoiding privacy risks such as unintended recognition becomes increasingly<br>challenging. We propose a new hybrid approach to obfuscate identities in photos<br>by head replacement. Our approach combines state of the art parametric face<br>synthesis with latest advances in Generative Adversarial Networks (GAN) for<br>data-driven image synthesis. On the one hand, the parametric part of our method<br>gives us control over the facial parameters and allows for explicit<br>manipulation of the identity. On the other hand, the data-driven aspects allow<br>for adding fine details and overall realism as well as seamless blending into<br>the scene context. In our experiments, we show highly realistic output of our<br>system that improves over the previous state of the art in obfuscation rate<br>while preserving a higher similarity to the original image content.<br>},
}

Endnote

%0 Report
%A Sun, Qianru
%A Tewari, Ayush
%A Xu, Weipeng
%A Fritz, Mario
%A Theobalt, Christian
%A Schiele, Bernt
%+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
%T A Hybrid Model for Identity Obfuscation by Face Replacement : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5E25-C
%U http://arxiv.org/abs/1804.04779
%D 2018
%X   As more and more personal photos are shared and tagged in social media,<br>avoiding privacy risks such as unintended recognition becomes increasingly<br>challenging. We propose a new hybrid approach to obfuscate identities in photos<br>by head replacement. Our approach combines state of the art parametric face<br>synthesis with latest advances in Generative Adversarial Networks (GAN) for<br>data-driven image synthesis. On the one hand, the parametric part of our method<br>gives us control over the facial parameters and allows for explicit<br>manipulation of the identity. On the other hand, the data-driven aspects allow<br>for adding fine details and overall realism as well as seamless blending into<br>the scene context. In our experiments, we show highly realistic output of our<br>system that improves over the previous state of the art in obfuscation rate<br>while preserving a higher similarity to the original image content.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Cryptography and Security, cs.CR

Conference paper

D2D4

Q. Sun, A. Tewari, W. Xu, M. Fritz, C. Theobalt, and B. Schiele

“A Hybrid Model for Identity Obfuscation by Face Replacement,” in Computer Vision -- ECCV 2018, Munich, Germany, 2018.

@inproceedings{Sun_ECCV2018,
TITLE = {A Hybrid Model for Identity Obfuscation by Face Replacement},
AUTHOR = {Sun, Qianru and Tewari, Ayush and Xu, Weipeng and Fritz, Mario and Theobalt, Christian and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {978-3-030-01245-8},
DOI = {10.1007/978-3-030-01246-5_34},
PUBLISHER = {Springer},
YEAR = {2018},
DATE = {2018},
BOOKTITLE = {Computer Vision -- ECCV 2018},
PAGES = {570--586},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {11205},
ADDRESS = {Munich, Germany},
}

Endnote

%0 Conference Proceedings
%A Sun, Qianru
%A Tewari, Ayush
%A Xu, Weipeng
%A Fritz, Mario
%A Theobalt, Christian
%A Schiele, Bernt
%+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
%T A Hybrid Model for Identity Obfuscation by Face Replacement : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-3F9B-B
%R 10.1007/978-3-030-01246-5_34
%D 2018
%B 15th European Conference on Computer Vision
%Z date of event: 2018-09-08 - 2018-09-14
%C Munich, Germany
%B Computer Vision -- ECCV 2018
%P 570 - 586
%I Springer
%@ 978-3-030-01245-8
%B Lecture Notes in Computer Science
%N 11205

Paper

A. Tewari, F. Bernard, P. Garrido, G. Bharaj, M. Elgharib, H.-P. Seidel, P. Pérez, M. Zollhöfer, and C. Theobalt

“FML: Face Model Learning from Videos,” 2018. [Online]. Available: http://arxiv.org/abs/1812.07603.

Abstract

Monocular image-based 3D reconstruction of faces is a long-standing problem
in computer vision. Since image data is a 2D projection of a 3D face, the
resulting depth ambiguity makes the problem ill-posed. Most existing methods
rely on data-driven priors that are built from limited 3D face scans. In
contrast, we propose multi-frame video-based self-supervised training of a deep
network that (i) learns a face identity model both in shape and appearance
while (ii) jointly learning to reconstruct 3D faces. Our face model is learned
using only corpora of in-the-wild video clips collected from the Internet. This
virtually endless source of training data enables learning of a highly general
3D face model. In order to achieve this, we propose a novel multi-frame
consistency loss that ensures consistent shape and appearance across multiple
frames of a subject's face, thus minimizing depth ambiguity. At test time we
can use an arbitrary number of frames, so that we can perform both monocular as
well as multi-frame reconstruction.

BibTeX

@online{tewari2018fml,
TITLE = {{FML}: {Face Model Learning from Videos}},
AUTHOR = {Tewari, Ayush and Bernard, Florian and Garrido, Pablo and Bharaj, Gaurav and Elgharib, Mohamed and Seidel, Hans-Peter and P{\'e}rez, Patrick and Zollh{\"o}fer, Michael and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1812.07603},
EPRINT = {1812.07603},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {Monocular image-based 3D reconstruction of faces is a long-standing problem<br>in computer vision. Since image data is a 2D projection of a 3D face, the<br>resulting depth ambiguity makes the problem ill-posed. Most existing methods<br>rely on data-driven priors that are built from limited 3D face scans. In<br>contrast, we propose multi-frame video-based self-supervised training of a deep<br>network that (i) learns a face identity model both in shape and appearance<br>while (ii) jointly learning to reconstruct 3D faces. Our face model is learned<br>using only corpora of in-the-wild video clips collected from the Internet. This<br>virtually endless source of training data enables learning of a highly general<br>3D face model. In order to achieve this, we propose a novel multi-frame<br>consistency loss that ensures consistent shape and appearance across multiple<br>frames of a subject's face, thus minimizing depth ambiguity. At test time we<br>can use an arbitrary number of frames, so that we can perform both monocular as<br>well as multi-frame reconstruction.<br>},
}

Endnote

%0 Report
%A Tewari, Ayush
%A Bernard, Florian
%A Garrido, Pablo
%A Bharaj, Gaurav
%A Elgharib, Mohamed
%A Seidel, Hans-Peter
%A P&#233;rez, Patrick
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T FML: Face Model Learning from Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-EF79-A
%U http://arxiv.org/abs/1812.07603
%D 2018
%X   Monocular image-based 3D reconstruction of faces is a long-standing problem<br>in computer vision. Since image data is a 2D projection of a 3D face, the<br>resulting depth ambiguity makes the problem ill-posed. Most existing methods<br>rely on data-driven priors that are built from limited 3D face scans. In<br>contrast, we propose multi-frame video-based self-supervised training of a deep<br>network that (i) learns a face identity model both in shape and appearance<br>while (ii) jointly learning to reconstruct 3D faces. Our face model is learned<br>using only corpora of in-the-wild video clips collected from the Internet. This<br>virtually endless source of training data enables learning of a highly general<br>3D face model. In order to achieve this, we propose a novel multi-frame<br>consistency loss that ensures consistent shape and appearance across multiple<br>frames of a subject's face, thus minimizing depth ambiguity. At test time we<br>can use an arbitrary number of frames, so that we can perform both monocular as<br>well as multi-frame reconstruction.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%U https://www.youtube.com/watch?v=SG2BwxCw0lQ

Conference paper

A. Tewari, M. Zollhöfer, P. Garrido, F. Bernard, H. Kim, P. Pérez, and C. Theobalt

“Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), Salt Lake City, UT, USA, 2018.

@inproceedings{Teware_CVPR2018,
TITLE = {Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 {Hz}},
AUTHOR = {Tewari, Ayush and Zollh{\"o}fer, Michael and Garrido, Pablo and Bernard, Florian and Kim, Hyeongwoo and P{\'e}rez, Patrick and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-5386-6420-9},
DOI = {10.1109/CVPR.2018.00270},
PUBLISHER = {IEEE},
YEAR = {2018},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)},
PAGES = {2549--2559},
ADDRESS = {Salt Lake City, UT, USA},
}

Endnote

%0 Conference Proceedings
%A Tewari, Ayush
%A Zollh&#246;fer, Michael
%A Garrido, Pablo
%A Bernard, Florian
%A Kim, Hyeongwoo
%A P&#233;rez, Patrick
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Self-Supervised Multi-level Face Model Learning for Monocular Reconstruction at Over 250 Hz : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-EF56-1
%R 10.1109/CVPR.2018.00270
%D 2018
%B 31st IEEE Conference on Computer Vision and Pattern Recognition 
%Z date of event: 2018-06-18 - 2018-06-22
%C Salt Lake City, UT, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 2549 - 2559
%I IEEE
%@ 978-1-5386-6420-9

Paper

J. Thies, M. Zollhöfer, C. Theobalt, M. Stamminger, and M. Nießner

“HeadOn: Real-time Reenactment of Human Portrait Videos,” 2018. [Online]. Available: http://arxiv.org/abs/1805.11729.

Abstract

We propose HeadOn, the first real-time source-to-target reenactment approach
for complete human portrait videos that enables transfer of torso and head
motion, face expression, and eye gaze. Given a short RGB-D video of the target
actor, we automatically construct a personalized geometry proxy that embeds a
parametric head, eye, and kinematic torso model. A novel real-time reenactment
algorithm employs this proxy to photo-realistically map the captured motion
from the source actor to the target actor. On top of the coarse geometric
proxy, we propose a video-based rendering technique that composites the
modified target portrait video via view- and pose-dependent texturing, and
creates photo-realistic imagery of the target actor under novel torso and head
poses, facial expressions, and gaze directions. To this end, we propose a
robust tracking of the face and torso of the source actor. We extensively
evaluate our approach and show significant improvements in enabling much
greater flexibility in creating realistic reenacted output videos.

BibTeX

@online{Thies_arXiv1805.11729,
TITLE = {{HeadOn}: {R}eal-time Reenactment of Human Portrait Videos},
AUTHOR = {Thies, Justus and Zollh{\"o}fer, Michael and Theobalt, Christian and Stamminger, Marc and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1805.11729},
EPRINT = {1805.11729},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We propose HeadOn, the first real-time source-to-target reenactment approach<br>for complete human portrait videos that enables transfer of torso and head<br>motion, face expression, and eye gaze. Given a short RGB-D video of the target<br>actor, we automatically construct a personalized geometry proxy that embeds a<br>parametric head, eye, and kinematic torso model. A novel real-time reenactment<br>algorithm employs this proxy to photo-realistically map the captured motion<br>from the source actor to the target actor. On top of the coarse geometric<br>proxy, we propose a video-based rendering technique that composites the<br>modified target portrait video via view- and pose-dependent texturing, and<br>creates photo-realistic imagery of the target actor under novel torso and head<br>poses, facial expressions, and gaze directions. To this end, we propose a<br>robust tracking of the face and torso of the source actor. We extensively<br>evaluate our approach and show significant improvements in enabling much<br>greater flexibility in creating realistic reenacted output videos.<br>},
}

Endnote

%0 Report
%A Thies, Justus
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%A Stamminger, Marc
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T HeadOn: Real-time Reenactment of Human Portrait Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5E14-F
%U http://arxiv.org/abs/1805.11729
%D 2018
%X   We propose HeadOn, the first real-time source-to-target reenactment approach<br>for complete human portrait videos that enables transfer of torso and head<br>motion, face expression, and eye gaze. Given a short RGB-D video of the target<br>actor, we automatically construct a personalized geometry proxy that embeds a<br>parametric head, eye, and kinematic torso model. A novel real-time reenactment<br>algorithm employs this proxy to photo-realistically map the captured motion<br>from the source actor to the target actor. On top of the coarse geometric<br>proxy, we propose a video-based rendering technique that composites the<br>modified target portrait video via view- and pose-dependent texturing, and<br>creates photo-realistic imagery of the target actor under novel torso and head<br>poses, facial expressions, and gaze directions. To this end, we propose a<br>robust tracking of the face and torso of the source actor. We extensively<br>evaluate our approach and show significant improvements in enabling much<br>greater flexibility in creating realistic reenacted output videos.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Paper

J. Thies, M. Zollhöfer, C. Theobalt, M. Stamminger, and M. Nießner

“IGNOR: Image-guided Neural Object Rendering,” 2018. [Online]. Available: http://arxiv.org/abs/1811.10720.

Abstract

We propose a new learning-based novel view synthesis approach for scanned
objects that is trained based on a set of multi-view images. Instead of using
texture mapping or hand-designed image-based rendering, we directly train a
deep neural network to synthesize a view-dependent image of an object. First,
we employ a coverage-based nearest neighbour look-up to retrieve a set of
reference frames that are explicitly warped to a given target view using
cross-projection. Our network then learns to best composite the warped images.
This enables us to generate photo-realistic results, while not having to
allocate capacity on `remembering' object appearance. Instead, the multi-view
images can be reused. While this works well for diffuse objects,
cross-projection does not generalize to view-dependent effects. Therefore, we
propose a decomposition network that extracts view-dependent effects and that
is trained in a self-supervised manner. After decomposition, the diffuse
shading is cross-projected, while the view-dependent layer of the target view
is regressed. We show the effectiveness of our approach both qualitatively and
quantitatively on real as well as synthetic data.

BibTeX

@online{Thies2018IGNORIN,
TITLE = {{IGNOR}: {Image-guided Neural Object Rendering}},
AUTHOR = {Thies, Justus and Zollh{\"o}fer, Michael and Theobalt, Christian and Stamminger, Marc and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1811.10720},
EPRINT = {1811.10720},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We propose a new learning-based novel view synthesis approach for scanned<br>objects that is trained based on a set of multi-view images. Instead of using<br>texture mapping or hand-designed image-based rendering, we directly train a<br>deep neural network to synthesize a view-dependent image of an object. First,<br>we employ a coverage-based nearest neighbour look-up to retrieve a set of<br>reference frames that are explicitly warped to a given target view using<br>cross-projection. Our network then learns to best composite the warped images.<br>This enables us to generate photo-realistic results, while not having to<br>allocate capacity on `remembering' object appearance. Instead, the multi-view<br>images can be reused. While this works well for diffuse objects,<br>cross-projection does not generalize to view-dependent effects. Therefore, we<br>propose a decomposition network that extracts view-dependent effects and that<br>is trained in a self-supervised manner. After decomposition, the diffuse<br>shading is cross-projected, while the view-dependent layer of the target view<br>is regressed. We show the effectiveness of our approach both qualitatively and<br>quantitatively on real as well as synthetic data.<br>},
}

Endnote

%0 Report
%A Thies, Justus
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%A Stamminger, Marc
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T IGNOR: Image-guided Neural Object Rendering : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F7EB-F
%U http://arxiv.org/abs/1811.10720
%D 2018
%X   We propose a new learning-based novel view synthesis approach for scanned<br>objects that is trained based on a set of multi-view images. Instead of using<br>texture mapping or hand-designed image-based rendering, we directly train a<br>deep neural network to synthesize a view-dependent image of an object. First,<br>we employ a coverage-based nearest neighbour look-up to retrieve a set of<br>reference frames that are explicitly warped to a given target view using<br>cross-projection. Our network then learns to best composite the warped images.<br>This enables us to generate photo-realistic results, while not having to<br>allocate capacity on `remembering' object appearance. Instead, the multi-view<br>images can be reused. While this works well for diffuse objects,<br>cross-projection does not generalize to view-dependent effects. Therefore, we<br>propose a decomposition network that extracts view-dependent effects and that<br>is trained in a self-supervised manner. After decomposition, the diffuse<br>shading is cross-projected, while the view-dependent layer of the target view<br>is regressed. We show the effectiveness of our approach both qualitatively and<br>quantitatively on real as well as synthetic data.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%U https://youtu.be/s79HG9yn7QM

Article

J. Thies, M. Zollhöfer, C. Theobalt, M. Stamminger, and M. Nießner

“HeadOn: Real-time Reenactment of Human Portrait Videos,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2018), vol. 37, no. 4, 2018.

@article{Thies_SIGGRAPH2018,
TITLE = {{HeadOn}: {R}eal-time Reenactment of Human Portrait Videos},
AUTHOR = {Thies, Justus and Zollh{\"o}fer, Michael and Theobalt, Christian and Stamminger, Marc and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3197517.3201350},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {37},
NUMBER = {4},
PAGES = {1--13},
EID = {164},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2018},
}

Endnote

%0 Journal Article
%A Thies, Justus
%A Zollh&#246;fer, Michael
%A Theobalt, Christian
%A Stamminger, Marc
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T HeadOn: Real-time Reenactment of Human Portrait Videos : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5D91-2
%R 10.1145/3197517.3201350
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 4
%& 1
%P 1 - 13
%Z sequence number: 164
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2018
%O ACM SIGGRAPH 2018 Vancouver, Canada , 12 - 16 August

Article

J. Thies, M. Zollhöfer, M. Stamminger, C. Theobalt, and M. Nießner

“FaceVR: Real-Time Gaze-Aware Facial Reenactment in Virtual Reality,” ACM Transactions on Graphics, vol. 37, no. 2, 2018.

@article{thies18FaceVR,
TITLE = {{FaceVR}: Real-Time Gaze-Aware Facial Reenactment in Virtual Reality},
AUTHOR = {Thies, Justus and Zollh{\"o}fer, Michael and Stamminger, Marc and Theobalt, Christian and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3182644},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {37},
NUMBER = {2},
PAGES = {1--15},
EID = {25},
}

Endnote

%0 Journal Article
%A Thies, Justus
%A Zollh&#246;fer, Michael
%A Stamminger, Marc
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T FaceVR: Real-Time Gaze-Aware Facial Reenactment in Virtual Reality : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5D96-D
%R 10.1145/3182644
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 2
%& 1
%P 1 - 15
%Z sequence number: 25
%I ACM
%C New York, NY
%@ false

Article

J. Thunberg, J. Markdahl, F. Bernard, and J. Goncalves

“A Lifting Method for Analyzing Distributed Synchronization on the Unit Sphere,” Automatica, vol. 96, 2018.

@article{Thunberg_2018,
TITLE = {A Lifting Method for Analyzing Distributed Synchronization on the Unit Sphere},
AUTHOR = {Thunberg, Johan and Markdahl, Johan and Bernard, Florian and Goncalves, Jorge},
LANGUAGE = {eng},
ISSN = {0005-1098},
DOI = {10.1016/j.automatica.2018.07.007},
PUBLISHER = {Elsevier},
ADDRESS = {Amsterdam},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Automatica},
VOLUME = {96},
PAGES = {253--258},
}

Endnote

%0 Journal Article
%A Thunberg, Johan
%A Markdahl, Johan
%A Bernard, Florian
%A Goncalves, Jorge
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T A Lifting Method for Analyzing Distributed Synchronization on the Unit Sphere : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5758-A
%R 10.1016/j.automatica.2018.07.007
%7 2018
%D 2018
%J Automatica
%V 96
%& 253
%P 253 - 258
%I Elsevier
%C Amsterdam
%@ false

Conference paper

D4D2

E. Tretschk, S. J. Oh, and M. Fritz

“Sequential Attacks on Agents for Long-Term Adversarial Goals,” in 2. ACM Computer Science in Cars Symposium (CSCS 2018), Munich, Germany, 2018.

@inproceedings{Tretschk_CSCS2018,
TITLE = {Sequential Attacks on Agents for Long-Term Adversarial Goals},
AUTHOR = {Tretschk, Edgar and Oh, Seong Joon and Fritz, Mario},
LANGUAGE = {eng},
YEAR = {2018},
BOOKTITLE = {2. ACM Computer Science in Cars Symposium (CSCS 2018)},
ADDRESS = {Munich, Germany},
}

Endnote

%0 Conference Proceedings
%A Tretschk, Edgar
%A Oh, Seong Joon
%A Fritz, Mario
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
%T Sequential Attacks on Agents for Long-Term Adversarial Goals : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5B46-A
%D 2018
%B 2. ACM Computer Science in Cars Symposium
%Z date of event: 2018-09-13 - 2018-09-14
%C Munich, Germany
%B 2. ACM Computer Science in Cars Symposium

Conference paper

M. Winter, D. Mlakar, R. Zayer, H.-P. Seidel, and M. Steinberger

“faimGraph: High Performance Management of Fully-Dynamic Graphs Under Tight Memory Constraints on the GPU,” in The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2018), Dallas, TX, USA, 2018.

@inproceedings{Winter:2018:FHP,
TITLE = {{faimGraph}: {H}igh Performance Management of Fully-Dynamic Graphs Under Tight Memory Constraints on the {GPU}},
AUTHOR = {Winter, Martin and Mlakar, Daniel and Zayer, Rhaleb and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISBN = {978-1-5386-8384-2},
URL = {http://conferences.computer.org/sc/2018/#!/home},
PUBLISHER = {IEEE},
YEAR = {2018},
BOOKTITLE = {The International Conference for High Performance Computing, Networking, Storage, and Analysis (SC 2018)},
PAGES = {754--766},
ADDRESS = {Dallas, TX, USA},
}

Endnote

%0 Conference Proceedings
%A Winter, Martin
%A Mlakar, Daniel
%A Zayer, Rhaleb
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T faimGraph: High Performance Management of Fully-Dynamic Graphs Under Tight Memory Constraints on the GPU : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-E5E6-8
%D 2018
%B The International Conference for High Performance Computing, Networking, Storage, and Analysis
%Z date of event: 2018-11-11 - 2018-11-16
%C Dallas, TX, USA
%B The International Conference for High Performance Computing, Networking, Storage, and Analysis
%P 754 - 766
%I IEEE
%@ 978-1-5386-8384-2

Article

K. Wolski, D. Giunchi, N. Ye, P. Didyk, K. Myszkowski, R. Mantiuk, H.-P. Seidel, A. Steed, and R. K. Mantiuk

“Dataset and Metrics for Predicting Local Visible Differences,” ACM Transactions on Graphics, vol. 37, no. 5, 2018.

@article{wolski2018dataset,
TITLE = {Dataset and Metrics for Predicting Local Visible Differences},
AUTHOR = {Wolski, Krzysztof and Giunchi, Daniele and Ye, Nanyang and Didyk, Piotr and Myszkowski, Karol and Mantiuk, Rados{\textbackslash}l{\textbraceleft}{\textbraceright}aw and Seidel, Hans-Peter and Steed, Anthony and Mantiuk, Rafa{\l} K.},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3196493},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {37},
NUMBER = {5},
EID = {172},
}

Endnote

%0 Journal Article
%A Wolski, Krzysztof
%A Giunchi, Daniele
%A Ye, Nanyang
%A Didyk, Piotr
%A Myszkowski, Karol
%A Mantiuk, Rados\l{}aw
%A Seidel, Hans-Peter
%A Steed, Anthony
%A Mantiuk, Rafa&#322; K.
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Dataset and Metrics for Predicting Local Visible Differences : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-5F75-2
%R 10.1145/3196493
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 5
%Z sequence number: 172
%I ACM
%C New York, NY
%@ false

Article

W. Xu, A. Chatterjee, M. Zollhöfer, H. Rhodin, D. Mehta, H.-P. Seidel, and C. Theobalt

“MonoPerfCap: Human Performance Capture from Monocular Video,” ACM Transactions on Graphics, vol. 37, no. 2, 2018.

Abstract

We present the first marker-less approach for temporally coherent 3D

performance capture of a human with general clothing from monocular video. Our

approach reconstructs articulated human skeleton motion as well as medium-scale

non-rigid surface deformations in general scenes. Human performance capture is

a challenging problem due to the large range of articulation, potentially fast

motion, and considerable non-rigid deformations, even from multi-view data.

Reconstruction from monocular video alone is drastically more challenging,

since strong occlusions and the inherent depth ambiguity lead to a highly

ill-posed reconstruction problem. We tackle these challenges by a novel

approach that employs sparse 2D and 3D human pose detections from a

convolutional neural network using a batch-based pose estimation strategy.

Joint recovery of per-batch motion allows to resolve the ambiguities of the

monocular reconstruction problem based on a low dimensional trajectory

subspace. In addition, we propose refinement of the surface geometry based on

fully automatically extracted silhouettes to enable medium-scale non-rigid

alignment. We demonstrate state-of-the-art performance capture results that

enable exciting applications such as video editing and free viewpoint video,

previously infeasible from monocular video. Our qualitative and quantitative

evaluation demonstrates that our approach significantly outperforms previous

monocular methods in terms of accuracy, robustness and scene complexity that

can be handled.

BibTeX

@article{Xu_ToG2018,
TITLE = {{MonoPerfCap}: Human Performance Capture from Monocular Video},
AUTHOR = {Xu, Weipeng and Chatterjee, Avishek and Zollh{\"o}fer, Michael and Rhodin, Helge and Mehta, Dushyant and Seidel, Hans-Peter and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3181973},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
ABSTRACT = {We present the first marker-less approach for temporally coherent 3D performance capture of a human with general clothing from monocular video. Our approach reconstructs articulated human skeleton motion as well as medium-scale non-rigid surface deformations in general scenes. Human performance capture is a challenging problem due to the large range of articulation, potentially fast motion, and considerable non-rigid deformations, even from multi-view data. Reconstruction from monocular video alone is drastically more challenging, since strong occlusions and the inherent depth ambiguity lead to a highly ill-posed reconstruction problem. We tackle these challenges by a novel approach that employs sparse 2D and 3D human pose detections from a convolutional neural network using a batch-based pose estimation strategy. Joint recovery of per-batch motion allows to resolve the ambiguities of the monocular reconstruction problem based on a low dimensional trajectory subspace. In addition, we propose refinement of the surface geometry based on fully automatically extracted silhouettes to enable medium-scale non-rigid alignment. We demonstrate state-of-the-art performance capture results that enable exciting applications such as video editing and free viewpoint video, previously infeasible from monocular video. Our qualitative and quantitative evaluation demonstrates that our approach significantly outperforms previous monocular methods in terms of accuracy, robustness and scene complexity that can be handled.},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {37},
NUMBER = {2},
EID = {27},
}

Endnote

%0 Journal Article
%A Xu, Weipeng
%A Chatterjee, Avishek
%A Zollh&#246;fer, Michael
%A Rhodin, Helge
%A Mehta, Dushyant
%A Seidel, Hans-Peter
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T MonoPerfCap: Human Performance Capture from Monocular Video : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-E20E-1
%R 10.1145/3181973
%7 2017
%D 2018
%X   We present the first marker-less approach for temporally coherent 3D
performance capture of a human with general clothing from monocular video. Our
approach reconstructs articulated human skeleton motion as well as medium-scale
non-rigid surface deformations in general scenes. Human performance capture is
a challenging problem due to the large range of articulation, potentially fast
motion, and considerable non-rigid deformations, even from multi-view data.
Reconstruction from monocular video alone is drastically more challenging,
since strong occlusions and the inherent depth ambiguity lead to a highly
ill-posed reconstruction problem. We tackle these challenges by a novel
approach that employs sparse 2D and 3D human pose detections from a
convolutional neural network using a batch-based pose estimation strategy.
Joint recovery of per-batch motion allows to resolve the ambiguities of the
monocular reconstruction problem based on a low dimensional trajectory
subspace. In addition, we propose refinement of the surface geometry based on
fully automatically extracted silhouettes to enable medium-scale non-rigid
alignment. We demonstrate state-of-the-art performance capture results that
enable exciting applications such as video editing and free viewpoint video,
previously infeasible from monocular video. Our qualitative and quantitative
evaluation demonstrates that our approach significantly outperforms previous
monocular methods in terms of accuracy, robustness and scene complexity that
can be handled.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR
%J ACM Transactions on Graphics
%V 37
%N 2
%Z sequence number: 27
%I ACM
%C New York, NY
%@ false

Paper

W. Xu, A. Chatterjee, M. Zollhöfer, H. Rhodin, P. Fua, H.-P. Seidel, and C. Theobalt

“Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera,” 2018. [Online]. Available: http://arxiv.org/abs/1803.05959.

Abstract

We propose the first real-time approach for the egocentric estimation of 3D

human body pose in a wide range of unconstrained everyday activities. This

setting has a unique set of challenges, such as mobility of the hardware setup,

and robustness to long capture sessions with fast recovery from tracking

failures. We tackle these challenges based on a novel lightweight setup that

converts a standard baseball cap to a device for high-quality pose estimation

based on a single cap-mounted fisheye camera. From the captured egocentric live

stream, our CNN based 3D pose estimation approach runs at 60Hz on a

consumer-level GPU. In addition to the novel hardware setup, our other main

contributions are: 1) a large ground truth training corpus of top-down fisheye

images and 2) a novel disentangled 3D pose estimation approach that takes the

unique properties of the egocentric viewpoint into account. As shown by our

evaluation, we achieve lower 3D joint error as well as better 2D overlay than

the existing baselines.

BibTeX

@online{Xu_arXiv1803.05959,
TITLE = {{Mo2Cap2}: Real-time Mobile {3D} Motion Capture with a Cap-mounted Fisheye Camera},
AUTHOR = {Xu, Weipeng and Chatterjee, Avishek and Zollh{\"o}fer, Michael and Rhodin, Helge and Fua, Pascal and Seidel, Hans-Peter and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1803.05959},
EPRINT = {1803.05959},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {We propose the first real-time approach for the egocentric estimation of 3D human body pose in a wide range of unconstrained everyday activities. This setting has a unique set of challenges, such as mobility of the hardware setup, and robustness to long capture sessions with fast recovery from tracking failures. We tackle these challenges based on a novel lightweight setup that converts a standard baseball cap to a device for high-quality pose estimation based on a single cap-mounted fisheye camera. From the captured egocentric live stream, our CNN based 3D pose estimation approach runs at 60Hz on a consumer-level GPU. In addition to the novel hardware setup, our other main contributions are: 1) a large ground truth training corpus of top-down fisheye images and 2) a novel disentangled 3D pose estimation approach that takes the unique properties of the egocentric viewpoint into account. As shown by our evaluation, we achieve lower 3D joint error as well as better 2D overlay than the existing baselines.},
}

Endnote

%0 Report
%A Xu, Weipeng
%A Chatterjee, Avishek
%A Zollh&#246;fer, Michael
%A Rhodin, Helge
%A Fua, Pascal
%A Seidel, Hans-Peter
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Mo2Cap2: Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-3C65-B
%U http://arxiv.org/abs/1803.05959
%D 2018
%X   We propose the first real-time approach for the egocentric estimation of 3D
human body pose in a wide range of unconstrained everyday activities. This
setting has a unique set of challenges, such as mobility of the hardware setup,
and robustness to long capture sessions with fast recovery from tracking
failures. We tackle these challenges based on a novel lightweight setup that
converts a standard baseball cap to a device for high-quality pose estimation
based on a single cap-mounted fisheye camera. From the captured egocentric live
stream, our CNN based 3D pose estimation approach runs at 60Hz on a
consumer-level GPU. In addition to the novel hardware setup, our other main
contributions are: 1) a large ground truth training corpus of top-down fisheye
images and 2) a novel disentangled 3D pose estimation approach that takes the
unique properties of the egocentric viewpoint into account. As shown by our
evaluation, we achieve lower 3D joint error as well as better 2D overlay than
the existing baselines.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

R. Zayer, D. Mlakar, M. Steinberger, and H.-P. Seidel

“Layered Fields for Natural Tessellations on Surfaces,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2018), vol. 37, no. 6, 2018.

@article{Zayer:2018:LFN,
TITLE = {Layered Fields for Natural Tessellations on Surfaces},
AUTHOR = {Zayer, Rhaleb and Mlakar, Daniel and Steinberger, Markus and Seidel, Hans-Peter},
LANGUAGE = {eng},
ISSN = {0730-0301},
ISBN = {978-1-4503-6008-1},
DOI = {10.1145/3272127.3275072},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2018},
DATE = {2018},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {37},
NUMBER = {6},
EID = {264},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2018},
}

Endnote

%0 Journal Article
%A Zayer, Rhaleb
%A Mlakar, Daniel
%A Steinberger, Markus
%A Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Layered Fields for Natural Tessellations on Surfaces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-E5E0-E
%R 10.1145/3272127.3275072
%7 2018
%D 2018
%J ACM Transactions on Graphics
%V 37
%N 6
%Z sequence number: 264
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2018

%O ACM SIGGRAPH Asia 2018 Tokyo, Japan,  December 04 - 07, 2018  SA'18 SA 2018
%@ 978-1-4503-6008-1

Paper

R. Zayer, D. Mlakar, M. Steinberger, and H.-P. Seidel

“Layered Fields for Natural Tessellations on Surfaces,” 2018. [Online]. Available: http://arxiv.org/abs/1804.09152.

Abstract

Mimicking natural tessellation patterns is a fascinating multi-disciplinary

problem. Geometric methods aiming at reproducing such partitions on surface

meshes are commonly based on the Voronoi model and its variants, and are often

faced with challenging issues such as metric estimation, geometric, topological

complications, and most critically parallelization. In this paper, we introduce

an alternate model which may be of value for resolving these issues. We drop

the assumption that regions need to be separated by lines. Instead, we regard

region boundaries as narrow bands and we model the partition as a set of smooth

functions layered over the surface. Given an initial set of seeds or regions,

the partition emerges as the solution of a time dependent set of partial

differential equations describing concurrently evolving fronts on the surface.

Our solution does not require geodesic estimation, elaborate numerical solvers,

or complicated bookkeeping data structures. The cost per time-iteration is

dominated by the multiplication and addition of two sparse matrices. Extension

of our approach in a Lloyd's algorithm fashion can be easily achieved and the

extraction of the dual mesh can be conveniently preformed in parallel through

matrix algebra. As our approach relies mainly on basic linear algebra kernels,

it lends itself to efficient implementation on modern graphics hardware.

BibTeX

@online{Zayer_arXiv1804.09152,
TITLE = {Layered Fields for Natural Tessellations on Surfaces},
AUTHOR = {Zayer, Rhaleb and Mlakar, Daniel and Steinberger, Markus and Seidel, Hans-Peter},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1804.09152},
EPRINT = {1804.09152},
EPRINTTYPE = {arXiv},
YEAR = {2018},
ABSTRACT = {Mimicking natural tessellation patterns is a fascinating multi-disciplinary problem. Geometric methods aiming at reproducing such partitions on surface meshes are commonly based on the Voronoi model and its variants, and are often faced with challenging issues such as metric estimation, geometric, topological complications, and most critically parallelization. In this paper, we introduce an alternate model which may be of value for resolving these issues. We drop the assumption that regions need to be separated by lines. Instead, we regard region boundaries as narrow bands and we model the partition as a set of smooth functions layered over the surface. Given an initial set of seeds or regions, the partition emerges as the solution of a time dependent set of partial differential equations describing concurrently evolving fronts on the surface. Our solution does not require geodesic estimation, elaborate numerical solvers, or complicated bookkeeping data structures. The cost per time-iteration is dominated by the multiplication and addition of two sparse matrices. Extension of our approach in a Lloyd's algorithm fashion can be easily achieved and the extraction of the dual mesh can be conveniently preformed in parallel through matrix algebra. As our approach relies mainly on basic linear algebra kernels, it lends itself to efficient implementation on modern graphics hardware.},
}

Endnote

%0 Report
%A Zayer, Rhaleb
%A Mlakar, Daniel
%A Steinberger, Markus
%A Seidel, Hans-Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Layered Fields for Natural Tessellations on Surfaces :
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-152D-5
%U http://arxiv.org/abs/1804.09152
%D 2018
%X Mimicking natural tessellation patterns is a fascinating multi-disciplinary
problem. Geometric methods aiming at reproducing such partitions on surface
meshes are commonly based on the Voronoi model and its variants, and are often
faced with challenging issues such as metric estimation, geometric, topological
complications, and most critically parallelization. In this paper, we introduce
an alternate model which may be of value for resolving these issues. We drop
the assumption that regions need to be separated by lines. Instead, we regard
region boundaries as narrow bands and we model the partition as a set of smooth
functions layered over the surface. Given an initial set of seeds or regions,
the partition emerges as the solution of a time dependent set of partial
differential equations describing concurrently evolving fronts on the surface.
Our solution does not require geodesic estimation, elaborate numerical solvers,
or complicated bookkeeping data structures. The cost per time-iteration is
dominated by the multiplication and addition of two sparse matrices. Extension
of our approach in a Lloyd's algorithm fashion can be easily achieved and the
extraction of the dual mesh can be conveniently preformed in parallel through
matrix algebra. As our approach relies mainly on basic linear algebra kernels,
it lends itself to efficient implementation on modern graphics hardware.

%K Computer Science, Graphics, cs.GR,Computer Science, Distributed, Parallel, and Cluster Computing, cs.DC

Article

M. Zollhöfer, P. Stotko, A. Görlitz, C. Theobalt, M. Nießner, R. Klein, and A. Kolb

“State of the Art on 3D Reconstruction with RGB-D Cameras,” Computer Graphics Forum (Proc. EUROGRAPHICS 2018), vol. 37, no. 2, 2018.

@article{Zollhoefer_EG2018STAR,
TITLE = {State of the Art on {3D} Reconstruction with {RGB}-{D} Cameras},
AUTHOR = {Zollh{\"o}fer, Michael and Stotko, Patrick and G{\"o}rlitz, Andreas and Theobalt, Christian and Nie{\ss}ner, Matthias and Klein, Reinhard and Kolb, Andreas},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13386},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {37},
NUMBER = {2},
PAGES = {625--652},
BOOKTITLE = {EUROGRAPHICS 2018 STAR -- State of The Art Reports},
EDITOR = {Hildebrandt, Klaus and Theobalt, Christian},
}

Endnote

%0 Journal Article
%A Zollh&#246;fer, Michael
%A Stotko, Patrick
%A G&#246;rlitz, Andreas
%A Theobalt, Christian
%A Nie&#223;ner, Matthias
%A Klein, Reinhard
%A Kolb, Andreas
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T State of the Art on 3D Reconstruction with RGB-D Cameras : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-6917-0
%R 10.1111/cgf.13386
%7 2018
%D 2018
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 37
%N 2
%& 625
%P 625 - 652
%I Blackwell-Wiley
%C Oxford
%@ false
%B EUROGRAPHICS 2018 STAR &#8211; State of The Art Reports
%O EUROGRAPHICS 2018 EG 2018 The European Association for Computer Graphics 39th Annual Conference ; Delft, The Netherlands, April 16th - 20th, 2018

Article

M. Zollhöfer, J. Thies, P. Garrido, D. Bradley, T. Beeler, P. Pérez, M. Stamminger, M. Nießner, and C. Theobalt

“State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications,” Computer Graphics Forum (Proc. EUROGRAPHICS 2018), vol. 37, no. 2, 2018.

@article{Zollhoefer_EG2018STARb,
TITLE = {State of the Art on Monocular {3D} Face Reconstruction, Tracking, and Applications},
AUTHOR = {Zollh{\"o}fer, Michael and Thies, Justus and Garrido, Pablo and Bradley, D. and Beeler, T. and P{\'e}rez, P. and Stamminger, Marc and Nie{\ss}ner, Matthias and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13382},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2018},
DATE = {2018},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {37},
NUMBER = {2},
PAGES = {523--550},
BOOKTITLE = {EUROGRAPHICS 2018 STAR -- State of The Art Reports},
EDITOR = {Hildebrandt, Klaus and Theobalt, Christian},
}

Endnote

%0 Journal Article
%A Zollh&#246;fer, Michael
%A Thies, Justus
%A Garrido, Pablo
%A Bradley, D.
%A Beeler, T.
%A P&#233;rez, P.
%A Stamminger, Marc
%A Nie&#223;ner, Matthias
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T State of the Art on Monocular 3D Face Reconstruction, Tracking, and Applications : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0001-691A-D
%R 10.1111/cgf.13382
%7 2018
%D 2018
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 37
%N 2
%& 523
%P 523 - 550
%I Blackwell-Wiley
%C Oxford
%@ false
%B EUROGRAPHICS 2018 STAR &#8211; State of The Art Reports
%O EUROGRAPHICS 2018 EG 2018 The European Association for Computer Graphics 39th Annual Conference ; Delft, The Netherlands, April 16th - 20th, 2018

2017

Paper

V. K. Adhikarla, M. Vinkler, D. Sumin, R. K. Mantiuk, K. Myszkowski, H.-P. Seidel, and P. Didyk

“Towards a Quality Metric for Dense Light Fields,” 2017. [Online]. Available: http://arxiv.org/abs/1704.07576.

Abstract

Light fields become a popular representation of three dimensional scenes, and

there is interest in their processing, resampling, and compression. As those

operations often result in loss of quality, there is a need to quantify it. In

this work, we collect a new dataset of dense reference and distorted light

fields as well as the corresponding quality scores which are scaled in

perceptual units. The scores were acquired in a subjective experiment using an

interactive light-field viewing setup. The dataset contains typical artifacts

that occur in light-field processing chain due to light-field reconstruction,

multi-view compression, and limitations of automultiscopic displays. We test a

number of existing objective quality metrics to determine how well they can

predict the quality of light fields. We find that the existing image quality

metrics provide good measures of light-field quality, but require dense

reference light- fields for optimal performance. For more complex tasks of

comparing two distorted light fields, their performance drops significantly,

which reveals the need for new, light-field-specific metrics.

BibTeX

@online{AdhikarlaArXiv17,
TITLE = {Towards a Quality Metric for Dense Light Fields},
AUTHOR = {Adhikarla, Vamsi Kiran and Vinkler, Marek and Sumin, Denis and Mantiuk, Rafa{\l} K. and Myszkowski, Karol and Seidel, Hans-Peter and Didyk, Piotr},
URL = {http://arxiv.org/abs/1704.07576},
EPRINT = {1704.07576},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {Light fields become a popular representation of three dimensional scenes, and there is interest in their processing, resampling, and compression. As those operations often result in loss of quality, there is a need to quantify it. In this work, we collect a new dataset of dense reference and distorted light fields as well as the corresponding quality scores which are scaled in perceptual units. The scores were acquired in a subjective experiment using an interactive light-field viewing setup. The dataset contains typical artifacts that occur in light-field processing chain due to light-field reconstruction, multi-view compression, and limitations of automultiscopic displays. We test a number of existing objective quality metrics to determine how well they can predict the quality of light fields. We find that the existing image quality metrics provide good measures of light-field quality, but require dense reference light- fields for optimal performance. For more complex tasks of comparing two distorted light fields, their performance drops significantly, which reveals the need for new, light-field-specific metrics.},
}

Endnote

%0 Report
%A Adhikarla, Vamsi Kiran
%A Vinkler, Marek
%A Sumin, Denis
%A Mantiuk, Rafa&#322; K.
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Towards a Quality Metric for Dense Light Fields : 
%U http://hdl.handle.net/11858/00-001M-0000-002D-2C2C-1
%U http://arxiv.org/abs/1704.07576
%D 2017
%X   Light fields become a popular representation of three dimensional scenes, and
there is interest in their processing, resampling, and compression. As those
operations often result in loss of quality, there is a need to quantify it. In
this work, we collect a new dataset of dense reference and distorted light
fields as well as the corresponding quality scores which are scaled in
perceptual units. The scores were acquired in a subjective experiment using an
interactive light-field viewing setup. The dataset contains typical artifacts
that occur in light-field processing chain due to light-field reconstruction,
multi-view compression, and limitations of automultiscopic displays. We test a
number of existing objective quality metrics to determine how well they can
predict the quality of light fields. We find that the existing image quality
metrics provide good measures of light-field quality, but require dense
reference light- fields for optimal performance. For more complex tasks of
comparing two distorted light fields, their performance drops significantly,
which reveals the need for new, light-field-specific metrics.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Conference paper

V. K. Adhikarla, M. Vinkler, D. Sumin, R. Mantiuk, K. Myszkowski, H.-P. Seidel, and P. Didyk

“Towards a Quality Metric for Dense Light Fields,” in 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA, 2017.

@inproceedings{Vamsi2017,
TITLE = {Towards a Quality Metric for Dense Light Fields},
AUTHOR = {Adhikarla, Vamsi Kiran and Vinkler, Marek and Sumin, Denis and Mantiuk, Rafa{\l} and Myszkowski, Karol and Seidel, Hans-Peter and Didyk, Piotr},
LANGUAGE = {eng},
ISBN = {978-1-5386-0458-8},
DOI = {10.1109/CVPR.2017.396},
PUBLISHER = {IEEE Computer Society},
YEAR = {2017},
DATE = {2017},
BOOKTITLE = {30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017)},
PAGES = {3720--3729},
ADDRESS = {Honolulu, HI, USA},
}

Endnote

%0 Conference Proceedings
%A Adhikarla, Vamsi Kiran
%A Vinkler, Marek
%A Sumin, Denis
%A Mantiuk, Rafa&#322;
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Didyk, Piotr
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Towards a Quality Metric for Dense Light Fields : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-E476-3
%R 10.1109/CVPR.2017.396
%D 2017
%B 30th IEEE Conference on Computer Vision and Pattern Recognition
%Z date of event: 2017-07-21 - 2017-07-26
%C Honolulu, HI, USA
%B 30th IEEE Conference on Computer Vision and Pattern Recognition
%P 3720 - 3729
%I IEEE Computer Society
%@ 978-1-5386-0458-8

Book chapter / section

A. O. Akyüz, O. T. Tursun, J. Hasić-Telalović, and K. Karađuzović-Hadžiabdić

“Ghosting in HDR Video,” in High Dynamic Range Video, Amsterdam: Elsevier, 2017.

@incollection{hdrvideo2017,
TITLE = {Ghosting in {HDR} Video},
AUTHOR = {Aky{\"u}z, Ahmet O{\u g}uz and Tursun, Okan Tarhan and Hasi{\'c}-Telalovi{\'c}, Jasminka and Kara{\dj}uzovi{\'c}-Had{\v z}iabdi{\'c}, Kanita},
LANGUAGE = {eng},
ISBN = {978-0-12-809477-8},
DOI = {10.1016/B978-0-12-809477-8.00001-7},
PUBLISHER = {Elsevier},
ADDRESS = {Amsterdam},
YEAR = {2017},
DATE = {2017},
BOOKTITLE = {High Dynamic Range Video},
EDITOR = {Chalmers, Alan and Campisi, Patrizio and Shiley, Peter and Olaizola, Igor G.},
PAGES = {3--44},
}

Endnote

%0 Book Section
%A Aky&#252;z, Ahmet O&#287;uz
%A Tursun, Okan Tarhan
%A Hasi&#263;-Telalovi&#263;, Jasminka
%A Kara&#273;uzovi&#263;-Had&#382;iabdi&#263;, Kanita
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Ghosting in HDR Video : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002C-4BAD-A
%R 10.1016/B978-0-12-809477-8.00001-7
%D 2017
%B High Dynamic Range Video
%E Chalmers, Alan; Campisi, Patrizio; Shiley, Peter; Olaizola, Igor G.
%P 3 - 44
%I Elsevier
%C Amsterdam
%@ 978-0-12-809477-8

Article

E. Arabadzhiyska, O. T. Tursun, K. Myszkowski, H.-P. Seidel, and P. Didyk

“Saccade Landing Position Prediction for Gaze-Contingent Rendering,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2017), vol. 36, no. 4, 2017.

@article{ArabadzhiyskaSIGGRAPH2017,
TITLE = {Saccade Landing Position Prediction for Gaze-Contingent Rendering},
AUTHOR = {Arabadzhiyska, Elena and Tursun, Okan Tarhan and Myszkowski, Karol and Seidel, Hans-Peter and Didyk, Piotr},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3072959.3073642},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {36},
NUMBER = {4},
PAGES = {1--12},
EID = {50},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2017},
}

Endnote

%0 Journal Article
%A Arabadzhiyska, Elena
%A Tursun, Okan Tarhan
%A Myszkowski, Karol
%A Seidel, Hans-Peter
%A Didyk, Piotr
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Saccade Landing Position Prediction for Gaze-Contingent Rendering : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-7D82-9
%R 10.1145/3072959.3073642
%7 2017
%D 2017
%J ACM Transactions on Graphics
%V 36
%N 4
%& 1
%P 1 - 12
%Z sequence number: 50
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2017
%O ACM SIGGRAPH 2017 Los Angeles, California, 30 July - 3 August

Article

V. Babaei, K. Vidimče, M. Foshey, A. Kaspar, P. Didyk, and W. Matusik

“Color Contoning for 3D Printing,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2017), vol. 36, no. 4, 2017.

@article{BabaeiSIGGRAPH2017,
TITLE = {Color Contoning for {3D} Printing},
AUTHOR = {Babaei, Vahid and Vidim{\v c}e, Kiril and Foshey, Michael and Kaspar, Alexandre and Didyk, Piotr and Matusik, Wojciech},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3072959.3073605},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {36},
NUMBER = {4},
PAGES = {1--15},
EID = {124},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2017},
}

Endnote

%0 Journal Article
%A Babaei, Vahid
%A Vidim&#269;e, Kiril
%A Foshey, Michael
%A Kaspar, Alexandre
%A Didyk, Piotr
%A Matusik, Wojciech
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Color Contoning for 3D Printing : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-F88E-E
%R 10.1145/3072959.3073605
%7 2017
%D 2017
%J ACM Transactions on Graphics
%V 36
%N 4
%& 1
%P 1 - 15
%Z sequence number: 124
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2017
%O ACM SIGGRAPH 2017 Los Angeles, California, 30 July - 3 August

Conference paper

V. Babaei, K. Vidimče, M. Foshey, A. Kaspar, P. Didyk, and W. Matusik

“3D Color Contoning,” in Proceedings SCF 2017, Cambridge, MA, USA, 2017.

@inproceedings{BabaeiSFC2017,
TITLE = {{3D} Color Contoning},
AUTHOR = {Babaei, Vahid and Vidim{\v c}e, Kiril and Foshey, Michael and Kaspar, Alexandre and Didyk, Piotr and Matusik, Wojciech},
LANGUAGE = {eng},
ISBN = {978-1-4503-4999-4},
DOI = {10.1145/3083157.3092885},
PUBLISHER = {ACM},
YEAR = {2017},
DATE = {2017},
BOOKTITLE = {Proceedings SCF 2017},
EID = {8},
ADDRESS = {Cambridge, MA, USA},
}

Endnote

%0 Conference Proceedings
%A Babaei, Vahid
%A Vidim&#269;e, Kiril
%A Foshey, Michael
%A Kaspar, Alexandre
%A Didyk, Piotr
%A Matusik, Wojciech
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T 3D Color Contoning : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002E-05A8-7
%R 10.1145/3083157.3092885
%D 2017
%B ACM Symposium On Computational Fabrication
%Z date of event: 2017-06-12 - 2017-06-13
%C Cambridge, MA, USA
%B Proceedings SCF 2017
%Z sequence number: 8
%I ACM
%@ 978-1-4503-4999-4

Paper

F. Bernard, C. Theobalt, and M. Moeller

“Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems,” 2017. [Online]. Available: http://arxiv.org/abs/1711.10733.

Abstract

In this work we study convex relaxations of quadratic optimisation problems

over permutation matrices. While existing semidefinite programming approaches

can achieve remarkably tight relaxations, they have the strong disadvantage

that they lift the original $n {\times} n$-dimensional variable to an $n^2

{\times} n^2$-dimensional variable, which limits their practical applicability.

In contrast, here we present a lifting-free convex relaxation that is provably

at least as tight as existing (lifting-free) convex relaxations. We demonstrate

experimentally that our approach is superior to existing convex and non-convex

methods for various problems, including image arrangement and multi-graph

matching.

BibTeX

@online{Bernard2017,
TITLE = {Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems},
AUTHOR = {Bernard, Florian and Theobalt, Christian and Moeller, Michael},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1711.10733},
EPRINT = {1711.10733},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {In this work we study convex relaxations of quadratic optimisation problems over permutation matrices. While existing semidefinite programming approaches can achieve remarkably tight relaxations, they have the strong disadvantage that they lift the original $n {\times} n$-dimensional variable to an $n^2 {\times} n^2$-dimensional variable, which limits their practical applicability. In contrast, here we present a lifting-free convex relaxation that is provably at least as tight as existing (lifting-free) convex relaxations. We demonstrate experimentally that our approach is superior to existing convex and non-convex methods for various problems, including image arrangement and multi-graph matching.},
}

Endnote

%0 Report
%A Bernard, Florian
%A Theobalt, Christian
%A Moeller, Michael
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0000-6143-7
%U http://arxiv.org/abs/1711.10733
%D 2017
%X   In this work we study convex relaxations of quadratic optimisation problems
over permutation matrices. While existing semidefinite programming approaches
can achieve remarkably tight relaxations, they have the strong disadvantage
that they lift the original $n {\times} n$-dimensional variable to an $n^2
{\times} n^2$-dimensional variable, which limits their practical applicability.
In contrast, here we present a lifting-free convex relaxation that is provably
at least as tight as existing (lifting-free) convex relaxations. We demonstrate
experimentally that our approach is superior to existing convex and non-convex
methods for various problems, including image arrangement and multi-graph
matching.

%K Mathematics, Optimization and Control, math.OC,Computer Science, Computer Vision and Pattern Recognition, cs.CV,Statistics, Machine Learning, stat.ML

Paper

A. K. Bojja, F. Mueller, S. R. Malireddi, M. Oberweger, V. Lepetit, C. Theobalt, K. M. Yi, and A. Tagliasacchi

“HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images,” 2017. [Online]. Available: http://arxiv.org/abs/1711.05944.

Abstract

We introduce a large-scale RGBD hand segmentation dataset, with detailed and
automatically generated high-quality ground-truth annotations. Existing
real-world datasets are limited in quantity due to the difficulty in manually
annotating ground-truth labels. By leveraging a pair of brightly colored gloves
and an RGBD camera, we propose an acquisition pipeline that eases the task of
annotating very large datasets with minimal human intervention. We then
quantify the importance of a large annotated dataset in this domain, and
compare the performance of existing datasets in the training of deep-learning
architectures. Finally, we propose a novel architecture employing strided
convolution/deconvolutions in place of max-pooling and unpooling layers. Our
variant outperforms baseline architectures while remaining computationally
efficient at inference time. Source and datasets will be made publicly
available.

BibTeX

@online{Malireddi2017,
TITLE = {{HandSeg}: {An Automatically Labeled Dataset for Hand Segmentation from Depth Images}},
AUTHOR = {Bojja, Abhishake Kumar and Mueller, Franziska and Malireddi, Sri Raghu and Oberweger, Markus and Lepetit, Vincent and Theobalt, Christian and Yi, Kwang Moo and Tagliasacchi, Andrea},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1711.05944},
EPRINT = {1711.05944},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {We introduce a large-scale RGBD hand segmentation dataset, with detailed and<br>automatically generated high-quality ground-truth annotations. Existing<br>real-world datasets are limited in quantity due to the difficulty in manually<br>annotating ground-truth labels. By leveraging a pair of brightly colored gloves<br>and an RGBD camera, we propose an acquisition pipeline that eases the task of<br>annotating very large datasets with minimal human intervention. We then<br>quantify the importance of a large annotated dataset in this domain, and<br>compare the performance of existing datasets in the training of deep-learning<br>architectures. Finally, we propose a novel architecture employing strided<br>convolution/deconvolutions in place of max-pooling and unpooling layers. Our<br>variant outperforms baseline architectures while remaining computationally<br>efficient at inference time. Source and datasets will be made publicly<br>available.<br>},
}

Endnote

%0 Report
%A Bojja, Abhishake Kumar
%A Mueller, Franziska
%A Malireddi, Sri Raghu
%A Oberweger, Markus
%A Lepetit, Vincent
%A Theobalt, Christian
%A Yi, Kwang Moo
%A Tagliasacchi, Andrea
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0000-6132-A
%U http://arxiv.org/abs/1711.05944
%D 2017
%X   We introduce a large-scale RGBD hand segmentation dataset, with detailed and<br>automatically generated high-quality ground-truth annotations. Existing<br>real-world datasets are limited in quantity due to the difficulty in manually<br>annotating ground-truth labels. By leveraging a pair of brightly colored gloves<br>and an RGBD camera, we propose an acquisition pipeline that eases the task of<br>annotating very large datasets with minimal human intervention. We then<br>quantify the importance of a large annotated dataset in this domain, and<br>compare the performance of existing datasets in the training of deep-learning<br>architectures. Finally, we propose a novel architecture employing strided<br>convolution/deconvolutions in place of max-pooling and unpooling layers. Our<br>variant outperforms baseline architectures while remaining computationally<br>efficient at inference time. Source and datasets will be made publicly<br>available.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%U https://arxiv.org/abs/1711.05944

Article

R. Chen and O. Weber

“GPU-Accelerated Locally Injective Shape Deformation,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2017), vol. 36, no. 6, 2017.

@article{ChenSIGASIA2017,
TITLE = {{GPU}-Accelerated Locally Injective Shape Deformation},
AUTHOR = {Chen, Renjie and Weber, Ofir},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3130800.3130843},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {36},
NUMBER = {6},
EID = {214},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2017},
}

Endnote

%0 Journal Article
%A Chen, Renjie
%A Weber, Ofir
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T GPU-Accelerated Locally Injective Shape Deformation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0000-045E-3
%R 10.1145/3130800.3130843
%7 2017
%D 2017
%J ACM Transactions on Graphics
%O TOG
%V 36
%N 6
%Z sequence number: 214
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2017
%O ACM SIGGRAPH Asia 2017 Bangkok, Thailand, 27 - 30 November 2017

Article

R. Chen and C. Gotsman

“Approximating Planar Conformal Maps Using Regular Polygonal Meshes,” Computer Graphics Forum, vol. 36, no. 8, 2017.

@article{Chen2017c,
TITLE = {Approximating Planar Conformal Maps Using Regular Polygonal Meshes},
AUTHOR = {Chen, Renjie and Gotsman, Craig},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13157},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2017},
DATE = {2017},
JOURNAL = {Computer Graphics Forum},
VOLUME = {36},
NUMBER = {8},
PAGES = {629--642},
}

Endnote

%0 Journal Article
%A Chen, Renjie
%A Gotsman, Craig
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Approximating Planar Conformal Maps Using Regular Polygonal Meshes : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0000-0422-5
%R 10.1111/cgf.13157
%7 2017
%D 2017
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 36
%N 8
%& 629
%P 629 - 642
%I Blackwell-Wiley
%C Oxford
%@ false

Paper

R. Chen, C. Gotsman, and K. Hormann

“Practical Distance Functions for Path-Planning in Planar Domains,” 2017. [Online]. Available: http://arxiv.org/abs/1708.05855.

Abstract

Path planning is an important problem in robotics. One way to plan a path

between two points $x,y$ within a (not necessarily simply-connected) planar

domain $\Omega$, is to define a non-negative distance function $d(x,y)$ on

$\Omega\times\Omega$ such that following the (descending) gradient of this

distance function traces such a path. This presents two equally important

challenges: A mathematical challenge -- to define $d$ such that $d(x,y)$ has a

single minimum for any fixed $y$ (and this is when $x=y$), since a local

minimum is in effect a "dead end", A computational challenge -- to define $d$

such that it may be computed efficiently. In this paper, given a description of

$\Omega$, we show how to assign coordinates to each point of $\Omega$ and

define a family of distance functions between points using these coordinates,

such that both the mathematical and the computational challenges are met. This

is done using the concepts of \emph{harmonic measure} and

\emph{$f$-divergences}.

In practice, path planning is done on a discrete network defined on a finite

set of \emph{sites} sampled from $\Omega$, so any method that works well on the

continuous domain must be adapted so that it still works well on the discrete

domain. Given a set of sites sampled from $\Omega$, we show how to define a

network connecting these sites such that a \emph{greedy routing} algorithm

(which is the discrete equivalent of continuous gradient descent) based on the

distance function mentioned above is guaranteed to generate a path in the

network between any two such sites. In many cases, this network is close to a

(desirable) planar graph, especially if the set of sites is dense.

BibTeX

@online{ChenarXiv2017,
TITLE = {Practical Distance Functions for Path-Planning in Planar Domains},
AUTHOR = {Chen, Renjie and Gotsman, Craig and Hormann, Kai},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1708.05855},
EPRINT = {1708.05855},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {Path planning is an important problem in robotics. One way to plan a path between two points $x,y$ within a (not necessarily simply-connected) planar domain $\Omega$, is to define a non-negative distance function $d(x,y)$ on $\Omega\times\Omega$ such that following the (descending) gradient of this distance function traces such a path. This presents two equally important challenges: A mathematical challenge -- to define $d$ such that $d(x,y)$ has a single minimum for any fixed $y$ (and this is when $x=y$), since a local minimum is in effect a "dead end", A computational challenge -- to define $d$ such that it may be computed efficiently. In this paper, given a description of $\Omega$, we show how to assign coordinates to each point of $\Omega$ and define a family of distance functions between points using these coordinates, such that both the mathematical and the computational challenges are met. This is done using the concepts of \emph{harmonic measure} and \emph{$f$-divergences}. In practice, path planning is done on a discrete network defined on a finite set of \emph{sites} sampled from $\Omega$, so any method that works well on the continuous domain must be adapted so that it still works well on the discrete domain. Given a set of sites sampled from $\Omega$, we show how to define a network connecting these sites such that a \emph{greedy routing} algorithm (which is the discrete equivalent of continuous gradient descent) based on the distance function mentioned above is guaranteed to generate a path in the network between any two such sites. In many cases, this network is close to a (desirable) planar graph, especially if the set of sites is dense.},
}

Endnote

%0 Report
%A Chen, Renjie
%A Gotsman, Craig
%A Hormann, Kai
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Practical Distance Functions for Path-Planning in Planar Domains : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002E-0634-3
%U http://arxiv.org/abs/1708.05855
%D 2017
%X   Path planning is an important problem in robotics. One way to plan a path
between two points $x,y$ within a (not necessarily simply-connected) planar
domain $\Omega$, is to define a non-negative distance function $d(x,y)$ on
$\Omega\times\Omega$ such that following the (descending) gradient of this
distance function traces such a path. This presents two equally important
challenges: A mathematical challenge -- to define $d$ such that $d(x,y)$ has a
single minimum for any fixed $y$ (and this is when $x=y$), since a local
minimum is in effect a "dead end", A computational challenge -- to define $d$
such that it may be computed efficiently. In this paper, given a description of
$\Omega$, we show how to assign coordinates to each point of $\Omega$ and
define a family of distance functions between points using these coordinates,
such that both the mathematical and the computational challenges are met. This
is done using the concepts of \emph{harmonic measure} and
\emph{$f$-divergences}.
  In practice, path planning is done on a discrete network defined on a finite
set of \emph{sites} sampled from $\Omega$, so any method that works well on the
continuous domain must be adapted so that it still works well on the discrete
domain. Given a set of sites sampled from $\Omega$, we show how to define a
network connecting these sites such that a \emph{greedy routing} algorithm
(which is the discrete equivalent of continuous gradient descent) based on the
distance function mentioned above is guaranteed to generate a path in the
network between any two such sites. In many cases, this network is close to a
(desirable) planar graph, especially if the set of sites is dense.

%K Computer Science, Robotics, cs.RO,

Paper

R. Chen, C. Gotsman, and K. Hormann

“Path Planning with Divergence-Based Distance Functions,” 2017. [Online]. Available: http://arxiv.org/abs/1708.02845.

Abstract

Distance functions between points in a domain are sometimes used to

automatically plan a gradient-descent path towards a given target point in the

domain, avoiding obstacles that may be present. A key requirement from such

distance functions is the absence of spurious local minima, which may foil such

an approach, and this has led to the common use of harmonic potential

functions. Based on the planar Laplace operator, the potential function

guarantees the absence of spurious minima, but is well known to be slow to

numerically compute and prone to numerical precision issues. To alleviate the

first of these problems, we propose a family of novel divergence distances.

These are based on f-divergence of the Poisson kernel of the domain. We define

the divergence distances and compare them to the harmonic potential function

and other related distance functions.

Our first result is theoretical: We show that the family of divergence

distances are equivalent to the harmonic potential function on simply-connected

domains, namely generate paths which are identical to those generated by the

potential function. The proof is based on the concept of conformal invariance.

Our other results are more practical and relate to two special cases of

divergence distances, one based on the Kullback-Leibler divergence and one

based on the total variation divergence. We show that using divergence

distances instead of the potential function and other distances has a

significant computational advantage, as, following a pre-processing stage, they

may be computed up to an order of magnitude faster than the others when taking

advantage of certain sparsity properties of the Poisson kernel. Furthermore,

the computation is "embarrassingly parallel", so may be implemented on a GPU

with up to three orders of magnitude speedup.

BibTeX

@online{chen_arXiv1708.02845,
TITLE = {Path Planning with Divergence-Based Distance Functions},
AUTHOR = {Chen, Renjie and Gotsman, Craig and Hormann, Kai},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1708.02845},
EPRINT = {1708.02845},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {Distance functions between points in a domain are sometimes used to automatically plan a gradient-descent path towards a given target point in the domain, avoiding obstacles that may be present. A key requirement from such distance functions is the absence of spurious local minima, which may foil such an approach, and this has led to the common use of harmonic potential functions. Based on the planar Laplace operator, the potential function guarantees the absence of spurious minima, but is well known to be slow to numerically compute and prone to numerical precision issues. To alleviate the first of these problems, we propose a family of novel divergence distances. These are based on f-divergence of the Poisson kernel of the domain. We define the divergence distances and compare them to the harmonic potential function and other related distance functions. Our first result is theoretical: We show that the family of divergence distances are equivalent to the harmonic potential function on simply-connected domains, namely generate paths which are identical to those generated by the potential function. The proof is based on the concept of conformal invariance. Our other results are more practical and relate to two special cases of divergence distances, one based on the Kullback-Leibler divergence and one based on the total variation divergence. We show that using divergence distances instead of the potential function and other distances has a significant computational advantage, as, following a pre-processing stage, they may be computed up to an order of magnitude faster than the others when taking advantage of certain sparsity properties of the Poisson kernel. Furthermore, the computation is "embarrassingly parallel", so may be implemented on a GPU with up to three orders of magnitude speedup.},
}

Endnote

%0 Report
%A Chen, Renjie
%A Gotsman, Craig
%A Hormann, Kai
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Path Planning with Divergence-Based Distance Functions : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002E-063A-8
%U http://arxiv.org/abs/1708.02845
%D 2017
%X   Distance functions between points in a domain are sometimes used to
automatically plan a gradient-descent path towards a given target point in the
domain, avoiding obstacles that may be present. A key requirement from such
distance functions is the absence of spurious local minima, which may foil such
an approach, and this has led to the common use of harmonic potential
functions. Based on the planar Laplace operator, the potential function
guarantees the absence of spurious minima, but is well known to be slow to
numerically compute and prone to numerical precision issues. To alleviate the
first of these problems, we propose a family of novel divergence distances.
These are based on f-divergence of the Poisson kernel of the domain. We define
the divergence distances and compare them to the harmonic potential function
and other related distance functions.
  Our first result is theoretical: We show that the family of divergence
distances are equivalent to the harmonic potential function on simply-connected
domains, namely generate paths which are identical to those generated by the
potential function. The proof is based on the concept of conformal invariance.
  Our other results are more practical and relate to two special cases of
divergence distances, one based on the Kullback-Leibler divergence and one
based on the total variation divergence. We show that using divergence
distances instead of the potential function and other distances has a
significant computational advantage, as, following a pre-processing stage, they
may be computed up to an order of magnitude faster than the others when taking
advantage of certain sparsity properties of the Poisson kernel. Furthermore,
the computation is "embarrassingly parallel", so may be implemented on a GPU
with up to three orders of magnitude speedup.

%K Computer Science, Robotics, cs.RO

Article

A. Dai, M. Nießner, M. Zollhöfer, S. Izadi, and C. Theobalt

“BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration,” ACM Transactions on Graphics, vol. 36, no. 3, 2017.

@article{dai2016bundleTOG,
TITLE = {{BundleFusion}: {R}eal-time Globally Consistent {3D} Reconstruction using On-the-fly Surface Re-integration},
AUTHOR = {Dai, Angela and Nie{\ss}ner, Matthias and Zollh{\"o}fer, Michael and Izadi, Shahram and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3054739},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {36},
NUMBER = {3},
EID = {24},
}

Endnote

%0 Journal Article
%A Dai, Angela
%A Nie&#223;ner, Matthias
%A Zollh&#246;fer, Michael
%A Izadi, Shahram
%A Theobalt, Christian
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002C-6796-B
%R 10.1145/3054739
%7 2017
%D 2017
%J ACM Transactions on Graphics
%V 36
%N 3
%Z sequence number: 24
%I ACM
%C New York, NY
%@ false

Conference paper

A. Derler, R. Zayer, H.-P. Seidel, and M. Steinberger

“Dynamic Scheduling for Efficient Hierarchical Sparse Matrix Operations on the GPU,” in ICS 2017, International Conference on Supercomputing, Chicago, IL, USA, 2017.

@inproceedings{DerlerICS2017,
TITLE = {Dynamic Scheduling for Efficient Hierarchical Sparse Matrix Operations on the {GPU}},
AUTHOR = {Derler, Andreas and Zayer, Rhaleb and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISBN = {978-1-4503-5020-4},
DOI = {10.1145/3079079.3079085},
PUBLISHER = {ACM},
YEAR = {2017},
DATE = {2017},
BOOKTITLE = {ICS 2017, International Conference on Supercomputing},
EID = {7},
ADDRESS = {Chicago, IL, USA},
}

Endnote

%0 Conference Proceedings
%A Derler, Andreas
%A Zayer, Rhaleb
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Dynamic Scheduling for Efficient Hierarchical Sparse Matrix
Operations on the GPU : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-7D73-D
%R 10.1145/3079079.3079085
%D 2017
%B International Conference on Supercomputing
%Z date of event: 2017-06-13 - 2017-06-16
%C Chicago, IL, USA
%B ICS 2017
%Z sequence number: 7
%I ACM
%@ 978-1-4503-5020-4

Article

Z. DeVito, M. Mara, M. Zollhöfer, G. Bernstein, J. Ragan-Kelley, C. Theobalt, P. Hanrahan, M. Fisher, and M. Nießner

“Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging,” ACM Transactions on Graphics, vol. 36, no. 5, 2017.

@article{DeVitoTOG2017,
TITLE = {Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging},
AUTHOR = {DeVito, Zachary and Mara, Michael and Zollh{\"o}fer, Michael and Bernstein, Gilbert and Ragan-Kelley, Jonathan and Theobalt, Christian and Hanrahan, Pat and Fisher, Matthew and Nie{\ss}ner, Matthias},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3132188},
PUBLISHER = {Association for Computing Machinery},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics},
VOLUME = {36},
NUMBER = {5},
EID = {171},
}

Endnote

%0 Journal Article
%A DeVito, Zachary
%A Mara, Michael
%A Zollh&#246;fer, Michael
%A Bernstein, Gilbert
%A Ragan-Kelley, Jonathan
%A Theobalt, Christian
%A Hanrahan, Pat
%A Fisher, Matthew
%A Nie&#223;ner, Matthias
%+ External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T Opt: A Domain Specific Language for Non-linear Least Squares
  Optimization in Graphics and Imaging : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002E-3111-9
%R 10.1145/3132188
%7 2017
%D 2017
%J ACM Transactions on Graphics
%V 36
%N 5
%Z sequence number: 171
%I Association for Computing Machinery
%C New York, NY
%@ false

Article

D. Dunn, C. Tippets, K. Torell, P. Kellnhofer, K. Akşit, P. Didyk, K. Myszkowski, D. Luebke, and H. Fuchs

“Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors,” IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR 2017), vol. 23, no. 4, 2017.

@article{DunnVR2017,
TITLE = {Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors},
AUTHOR = {Dunn, David and Tippets, Cary and Torell, Kent and Kellnhofer, Petr and Ak{\c s}it, Kaan and Didyk, Piotr and Myszkowski, Karol and Luebke, David and Fuchs, Henry},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2017.2657058},
PUBLISHER = {IEEE Computer Society},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VR)},
VOLUME = {23},
NUMBER = {4},
PAGES = {1322--1331},
BOOKTITLE = {Selected Proceedings IEEE Virtual Reality 2017 (IEEE VR 2017)},
}

Endnote

%0 Journal Article
%A Dunn, David
%A Tippets, Cary
%A Torell, Kent
%A Kellnhofer, Petr
%A Ak&#351;it, Kaan
%A Didyk, Piotr
%A Myszkowski, Karol
%A Luebke, David
%A Fuchs, Henry
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Wide Field Of View Varifocal Near-Eye Display Using See-Through Deformable Membrane Mirrors : (Best Paper Award)
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002C-3095-4
%R 10.1109/TVCG.2017.2657058
%7 2017
%D 2017
%J IEEE Transactions on Visualization and Computer Graphics
%V 23
%N 4
%& 1322
%P 1322 - 1331
%I IEEE Computer Society
%C New York, NY
%@ false
%B Selected Proceedings IEEE Virtual Reality 2017
%O IEEE VR 2017 Los Angeles, California on March 18-22, 2017
%U http://telepresence.web.unc.edu/research/dynamic-focus-augmented-reality-display/

Conference paper

D. Dunn, C. Tippets, K. Torell, H. Fuchs, P. Kellnhofer, K. Myszkowski, P. Didyk, K. Akşit, and D. Luebke

“Membrane AR: Varifocal, Wide Field of View Augmented Reality Display from Deformable Membranes,” in ACM SIGGRAPH 2017 Emerging Technologies, Los Angeles, CA, USA, 2017.

@inproceedings{Dunn2017,
TITLE = {Membrane {AR}: {V}arifocal, Wide Field of View Augmented Reality Display from Deformable Membranes},
AUTHOR = {Dunn, David and Tippets, Cary and Torell, Kent and Fuchs, Henry and Kellnhofer, Petr and Myszkowski, Karol and Didyk, Piotr and Ak{\c s}it, Kaan and Luebke, David},
LANGUAGE = {eng},
ISBN = {978-1-4503-5012-9},
DOI = {10.1145/3084822.3084846},
PUBLISHER = {ACM},
YEAR = {2017},
DATE = {2017},
BOOKTITLE = {ACM SIGGRAPH 2017 Emerging Technologies},
PAGES = {1--2},
EID = {15},
ADDRESS = {Los Angeles, CA, USA},
}

Endnote

%0 Conference Proceedings
%A Dunn, David
%A Tippets, Cary
%A Torell, Kent
%A Fuchs, Henry
%A Kellnhofer, Petr
%A Myszkowski, Karol
%A Didyk, Piotr
%A Ak&#351;it, Kaan
%A Luebke, David
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T Membrane AR: Varifocal, Wide Field of View Augmented Reality
Display from Deformable Membranes : (Digital Content Association of Japan Award)
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-E47B-A
%R 10.1145/3084822.3084846
%D 2017
%B 44th SIGGRAPH Conference on Computer Graphics and Interactive Techniques
%Z date of event: 2017-07-30 - 2017-08-03
%C Los Angeles, CA, USA
%B ACM SIGGRAPH 2017 Emerging Technologies
%P 1 - 2
%Z sequence number: 15
%I ACM
%@ 978-1-4503-5012-9

Conference paper

R. Ďurikovič and J. Hladký

“Complex Dynamics, Evolution, Separation, Profile Modeling of Fire in 3D for Computer Animation,” in Fifth International Symposium on Computing and Networking (CANDAR 2017), Aomori, Japan, 2017.

@inproceedings{DBLP:conf/ic-nc/DurikovicH17,
TITLE = {Complex Dynamics, Evolution, Separation, Profile Modeling of Fire in {3D} for Computer Animation},
AUTHOR = {{\v D}urikovi{\v c}, Roman and Hladk{\'y}, Jozef},
LANGUAGE = {eng},
ISBN = {978-1-5386-2087-8},
DOI = {10.1109/CANDAR.2017.11},
PUBLISHER = {IEEE},
YEAR = {2017},
BOOKTITLE = {Fifth International Symposium on Computing and Networking (CANDAR 2017)},
PAGES = {80--86},
ADDRESS = {Aomori, Japan},
}

Endnote

%0 Conference Proceedings
%A &#270;urikovi&#269;, Roman
%A Hladk&#253;, Jozef
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T Complex Dynamics, Evolution, Separation, Profile Modeling of Fire in 3D for Computer Animation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F650-E
%R 10.1109/CANDAR.2017.11
%D 2017
%B Fifth International Symposium on Computing and Networking
%Z date of event: 2017-11-19 - 2017-11-22
%C Aomori, Japan
%B Fifth International Symposium  on Computing and Networking
%P 80 - 86
%I IEEE
%@ 978-1-5386-2087-8

Article

O. Elek, D. Sumin, R. Zhang, T. Weyrich, K. Myszkowski, B. Bickel, A. Wilkie, and J. Křivánek

“Scattering-aware Texture Reproduction for 3D Printing,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2017), vol. 36, no. 6, 2017.

@article{ElekSIGASIA2017,
TITLE = {Scattering-aware Texture Reproduction for {3D} Printing},
AUTHOR = {Elek, Oskar and Sumin, Denis and Zhang, Ran and Weyrich, Tim and Myszkowski, Karol and Bickel, Bernd and Wilkie, Alexander and K{\v r}iv{\'a}nek, Jaroslav},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3130800.3130890},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)},
VOLUME = {36},
NUMBER = {6},
EID = {241},
BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2017},
}

Endnote

%0 Journal Article
%A Elek, Oskar
%A Sumin, Denis
%A Zhang, Ran
%A Weyrich, Tim
%A Myszkowski, Karol
%A Bickel, Bernd
%A Wilkie, Alexander
%A K&#345;iv&#225;nek, Jaroslav
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T Scattering-aware Texture Reproduction for 3D Printing : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-E485-1
%R 10.1145/3130800.3130890
%7 2017
%D 2017
%J ACM Transactions on Graphics
%O TOG
%V 36
%N 6
%Z sequence number: 241
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH Asia 2017
%O ACM SIGGRAPH Asia 2017 Bangkok, Thailand, 27 - 30 November 2017

Article

D4D2

A. Elhayek, E. de Aguiar, A. Jain, J. Tompson, L. Pishchulin, M. Andriluka, C. Bregler, B. Schiele, and C. Theobalt

“MARCOnI -- ConvNet-Based MARker-Less Motion Capture in Outdoor and Indoor Scenes,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 3, 2017.

@article{Elhayek2017,
TITLE = {{MARCOnI} -- {ConvNet}-Based {MARker}-Less Motion Capture in Outdoor and Indoor Scenes},
AUTHOR = {Elhayek, Ahmed and de Aguiar, Edilson and Jain, Arjun and Tompson, Jonathan and Pishchulin, Leonid and Andriluka, Mykhaylo and Bregler, Christoph and Schiele, Bernt and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0162-8828},
DOI = {10.1109/TPAMI.2016.2557779},
PUBLISHER = {IEEE Computer Society.},
ADDRESS = {New York},
YEAR = {2017},
DATE = {2017},
JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
VOLUME = {39},
NUMBER = {3},
PAGES = {501--514},
}

Endnote

%0 Journal Article
%A Elhayek, Ahmed
%A de Aguiar, Edilson
%A Jain, Arjun
%A Tompson, Jonathan
%A Pishchulin, Leonid
%A Andriluka, Mykhaylo
%A Bregler, Christoph
%A Schiele, Bernt
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T MARCOnI -- ConvNet-Based MARker-Less Motion Capture in Outdoor and Indoor Scenes : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002E-05D5-0
%R 10.1109/TPAMI.2016.2557779
%7 2017
%D 2017
%J IEEE Transactions on Pattern Analysis and Machine Intelligence
%O IEEE Trans. Pattern Anal. Mach. Intell. PAMI
%V 39
%N 3
%& 501
%P 501 - 514
%I IEEE Computer Society.
%C New York
%@ false

Report

G. Fox, A. Meka, M. Zollhöfer, C. Richardt, and C. Theobalt

“Live User-guided Intrinsic Video For Static Scenes,” Max-Planck-Institut für Informatik, Saarbrücken, MPI-I-2017-4-001, 2017.

Abstract

We present a novel real-time approach for user-guided intrinsic decomposition

of static scenes captured by an RGB-D sensor. In the

first step, we acquire a three-dimensional representation of the scene

using a dense volumetric reconstruction framework. The obtained

reconstruction serves as a proxy to densely fuse reflectance estimates

and to store user-provided constraints in three-dimensional space.

User constraints, in the form of constant shading and reflectance

strokes, can be placed directly on the real-world geometry using

an intuitive touch-based interaction metaphor, or using interactive

mouse strokes. Fusing the decomposition results and constraints in

three-dimensional space allows for robust propagation of this information

to novel views by re-projection.We leverage this information

to improve on the decomposition quality of existing intrinsic video

decomposition techniques by further constraining the ill-posed decomposition

problem. In addition to improved decomposition quality,

we show a variety of live augmented reality applications such as

recoloring of objects, relighting of scenes and editing of material

appearance.

BibTeX

@techreport{Report2017-4-001,
TITLE = {Live User-guided Intrinsic Video For Static Scenes},
AUTHOR = {Fox, Gereon and Meka, Abhimitra and Zollh{\"o}fer, Michael and Richardt, Christian and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0946-011X},
NUMBER = {MPI-I-2017-4-001},
INSTITUTION = {Max-Planck-Institut f{\"u}r Informatik},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2017},
ABSTRACT = {We present a novel real-time approach for user-guided intrinsic decomposition of static scenes captured by an RGB-D sensor. In the first step, we acquire a three-dimensional representation of the scene using a dense volumetric reconstruction framework. The obtained reconstruction serves as a proxy to densely fuse reflectance estimates and to store user-provided constraints in three-dimensional space. User constraints, in the form of constant shading and reflectance strokes, can be placed directly on the real-world geometry using an intuitive touch-based interaction metaphor, or using interactive mouse strokes. Fusing the decomposition results and constraints in three-dimensional space allows for robust propagation of this information to novel views by re-projection.We leverage this information to improve on the decomposition quality of existing intrinsic video decomposition techniques by further constraining the ill-posed decomposition problem. In addition to improved decomposition quality, we show a variety of live augmented reality applications such as recoloring of objects, relighting of scenes and editing of material appearance.},
TYPE = {Research Report},
}

Endnote

%0 Report
%A Fox, Gereon
%A Meka, Abhimitra
%A Zollh&#246;fer, Michael
%A Richardt, Christian
%A Theobalt, Christian
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Live User-guided Intrinsic Video For Static Scenes : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002C-5DA7-3
%Y Max-Planck-Institut f&#252;r Informatik
%C Saarbr&#252;cken
%D 2017
%P 12 p.
%X We present a novel real-time approach for user-guided intrinsic decomposition
of static scenes captured by an RGB-D sensor. In the
first step, we acquire a three-dimensional representation of the scene
using a dense volumetric reconstruction framework. The obtained
reconstruction serves as a proxy to densely fuse reflectance estimates
and to store user-provided constraints in three-dimensional space.
User constraints, in the form of constant shading and reflectance
strokes, can be placed directly on the real-world geometry using
an intuitive touch-based interaction metaphor, or using interactive
mouse strokes. Fusing the decomposition results and constraints in
three-dimensional space allows for robust propagation of this information
to novel views by re-projection.We leverage this information
to improve on the decomposition quality of existing intrinsic video
decomposition techniques by further constraining the ill-posed decomposition
problem. In addition to improved decomposition quality,
we show a variety of live augmented reality applications such as
recoloring of objects, relighting of scenes and editing of material
appearance.
%B Research Report
%@ false

Thesis

D4IMPR-CS

P. Garrido

“High-quality face capture, animation and editing from monocular video,” Universität des Saarlandes, Saarbrücken, 2017.

@phdthesis{Garridophd17,
TITLE = {High-quality face capture, animation and editing from monocular video},
AUTHOR = {Garrido, Pablo},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291-scidok-69419},
DOI = {10.22028/D291-26785},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2017},
DATE = {2017},
}

Endnote

%0 Thesis
%A Garrido, Pablo
%Y Theobalt, Christian
%A referee: Perez, Patrick
%A referee: Pauly, Mark
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T High-quality face capture, animation and editing from monocular video : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-D1BC-2
%U urn:nbn:de:bsz:291-scidok-69419
%R 10.22028/D291-26785 
%F OTHER: hdl:20.500.11880/26798
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2017
%P 185 p.
%V phd
%9 phd
%U http://scidok.sulb.uni-saarland.de/volltexte/2017/6941/http://scidok.sulb.uni-saarland.de/doku/lic_ohne_pod.php?la=de

Conference paper

D2D4

S. Georgoulis, K. Rematas, T. Ritschel, M. Fritz, T. Tuytelaars, and L. Van Gool

“What Is Around The Camera?,” in IEEE International Conference on Computer Vision (ICCV 2017), Venice, Italy, 2017.

@inproceedings{stam17iccv,
TITLE = {What Is Around The Camera?},
AUTHOR = {Georgoulis, Stamatios and Rematas, Konstantinos and Ritschel, Tobias and Fritz, Mario and Tuytelaars, Tinne and Van Gool, Luc},
LANGUAGE = {eng},
ISBN = {978-1-5386-1032-9},
DOI = {10.1109/ICCV.2017.553},
PUBLISHER = {IEEE},
YEAR = {2017},
DATE = {2017},
BOOKTITLE = {IEEE International Conference on Computer Vision (ICCV 2017)},
PAGES = {5180--5188},
ADDRESS = {Venice, Italy},
}

Endnote

%0 Conference Proceedings
%A Georgoulis, Stamatios
%A Rematas, Konstantinos
%A Ritschel, Tobias
%A Fritz, Mario
%A Tuytelaars, Tinne
%A Van Gool, Luc
%+ External Organizations
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T What Is Around The Camera? : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-D0CD-4
%R 10.1109/ICCV.2017.553
%D 2017
%B International Conference on Computer Vision
%Z date of event: 2017-10-22 - 2017-10-29
%C Venice, Italy
%B IEEE International Conference on Computer Vision
%P 5180 - 5188
%I IEEE
%@ 978-1-5386-1032-9

Thesis

D4IMPR-CS

Y. Gryaditskaya

“High Dynamic Range Imaging: Problems of Video Exposure Bracketing, Luminance Calibration and Gloss Editing,” Universität des Saarlandes, Saarbrücken, 2017.

Abstract

Two-dimensional, conventional images are gradually losing their hegemony, leaving room for novel formats. Among these, 8 bit images give place to high dynamic range (HDR) image formats, allowing to improve colour gamut and visibility of details in dark and bright areas leading to a more immersive viewing experience. It opens wide opportunities for post-processing, which can be useful for artistic rendering, enhancement of viewing experience or medical applications. Simultaneously, light-field scene representation as well is gaining importance, propelled by the recent reappearance of virtual reality, the improvement of both acquisition techniques, and computational and storage capabilities. Light-field data as well allows to achieve a broad range of effects in post-production: among others, it enables a change of a camera position, an aperture or a focal length. It facilitates object insertions and simplifies visual effects workflow by integrating 3D nature of visual effects with 3D nature of light fields. Content generation is one of the stumbling blocks in these realms. Sensor limitations of a conventional camera do not allow to capture wide dynamic range. This especially is the case for mobile devices, where small sensors are optimised for capturing in high-resolution. The “HDR mode” often encountered on such devices, relies on techniques called “exposure fusion” and allows to partially overcome the limited range of a sensor. The HDR video at the same time remains a challenging problem. We suggest a solution for an HDR video capturing on a mobile device. We analyse dynamic range of motion regions, the regions which are the most prone to reconstruction artefacts, and suggest a real-time exposure selection algorithm. Further, an HDR content visualization task often requires an input to be in absolute values. We address this problem by presenting a calibration algorithm that can be applied to existent imagery and does not require any additional measurement hardware. Finally, as light fields use becomes more common, a key challenge is the ability to edit or modify the appearance of the objects in the light field. To this end, we propose a multidimensional filtering approach in which the specular highlights are filtered in the spatial and angular domains to target a desired increase of the material roughness.

BibTeX

@phdthesis{Gryphd17,
TITLE = {High Dynamic Range Imaging: Problems of Video Exposure Bracketing, Luminance Calibration and Gloss Editing},
AUTHOR = {Gryaditskaya, Yulia},
LANGUAGE = {eng},
URL = {urn:nbn:de:bsz:291-scidok-69296},
DOI = {10.22028/D291-26783},
SCHOOL = {Universit{\"a}t des Saarlandes},
ADDRESS = {Saarbr{\"u}cken},
YEAR = {2017},
DATE = {2017},
ABSTRACT = {Two-dimensional, conventional images are gradually losing their hegemony, leaving room for novel formats. Among these, 8 bit images give place to high dynamic range (HDR) image formats, allowing to improve colour gamut and visibility of details in dark and bright areas leading to a more immersive viewing experience. It opens wide opportunities for post-processing, which can be useful for artistic rendering, enhancement of viewing experience or medical applications. Simultaneously, light-field scene representation as well is gaining importance, propelled by the recent reappearance of virtual reality, the improvement of both acquisition techniques, and computational and storage capabilities. Light-field data as well allows to achieve a broad range of effects in post-production: among others, it enables a change of a camera position, an aperture or a focal length. It facilitates object insertions and simplifies visual effects workflow by integrating 3D nature of visual effects with 3D nature of light fields. Content generation is one of the stumbling blocks in these realms. Sensor limitations of a conventional camera do not allow to capture wide dynamic range. This especially is the case for mobile devices, where small sensors are optimised for capturing in high-resolution. The {\textquotedblleft}HDR mode{\textquotedblright} often encountered on such devices, relies on techniques called {\textquotedblleft}exposure fusion{\textquotedblright} and allows to partially overcome the limited range of a sensor. The HDR video at the same time remains a challenging problem. We suggest a solution for an HDR video capturing on a mobile device. We analyse dynamic range of motion regions, the regions which are the most prone to reconstruction artefacts, and suggest a real-time exposure selection algorithm. Further, an HDR content visualization task often requires an input to be in absolute values. We address this problem by presenting a calibration algorithm that can be applied to existent imagery and does not require any additional measurement hardware. Finally, as light fields use becomes more common, a key challenge is the ability to edit or modify the appearance of the objects in the light field. To this end, we propose a multidimensional filtering approach in which the specular highlights are filtered in the spatial and angular domains to target a desired increase of the material roughness.},
}

Endnote

%0 Thesis
%A Gryaditskaya, Yulia
%Y Seidel, Hans-Peter
%A referee: Myszkowski, Karol
%+ Computer Graphics, MPI for Informatics, Max Planck Society
International Max Planck Research School, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T High Dynamic Range Imaging: Problems of Video Exposure Bracketing, Luminance Calibration and Gloss Editing :
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-ABA6-3
%U urn:nbn:de:bsz:291-scidok-69296
%R 10.22028/D291-26783
%F OTHER: hdl:20.500.11880/26796
%I Universit&#228;t des Saarlandes
%C Saarbr&#252;cken
%D 2017
%P 88 p.
%V phd
%9 phd
%X Two-dimensional, conventional images are gradually losing their hegemony, leaving room for novel formats. Among these, 8 bit images give place to high dynamic range (HDR) image formats, allowing to improve colour gamut and visibility of details in dark and bright areas leading to a more immersive viewing experience. It opens wide opportunities for post-processing, which can be useful for artistic rendering, enhancement of viewing experience or medical applications. Simultaneously, light-field scene representation as well is gaining importance, propelled by the recent reappearance of virtual reality, the improvement of both acquisition techniques, and computational and storage capabilities. Light-field data as well allows to achieve a broad range of effects in post-production: among others, it enables a change of a camera position, an aperture or a focal length. It facilitates object insertions and simplifies visual effects workflow by integrating 3D nature of visual effects with 3D nature of light fields. Content generation is one of the stumbling blocks in these realms. Sensor limitations of a conventional camera do not allow to capture wide dynamic range. This especially is the case for mobile devices, where small sensors are optimised for capturing in high-resolution. The &#8220;HDR mode&#8221; often encountered on such devices, relies on techniques called &#8220;exposure fusion&#8221; and allows to partially overcome the limited range of a sensor. The HDR video at the same time remains a challenging problem. We suggest a solution for an HDR video capturing on a mobile device. We analyse dynamic range of motion regions, the regions which are the most prone to reconstruction artefacts, and suggest a real-time exposure selection algorithm. Further, an HDR content visualization task often requires an input to be in absolute values. We address this problem by presenting a calibration algorithm that can be applied to existent imagery and does not require any additional measurement hardware. Finally, as light fields use becomes more common, a key challenge is the ability to edit or modify the appearance of the objects in the light field. To this end, we propose a multidimensional filtering approach in which the specular highlights are filtered in the spatial and angular domains to target a desired increase of the material roughness.
%U http://scidok.sulb.uni-saarland.de/volltexte/2017/6929/

Article

K. Haubenwallner, H.-P. Seidel, and M. Steinberger

“ShapeGenetics: Using Genetic Algorithms for Procedural Modeling,” Computer Graphics Forum (Proc. EUROGRAPHICS 2017), vol. 36, no. 2, 2017.

@article{haubenwallner2017shapegenetics,
TITLE = {{ShapeGenetics}: {U}sing Genetic Algorithms for Procedural Modeling},
AUTHOR = {Haubenwallner, Karl and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13120},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford},
YEAR = {2017},
DATE = {2017},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {36},
NUMBER = {2},
PAGES = {213--223},
BOOKTITLE = {The European Association for Computer Graphics 38th Annual Conference (EUROGRAPHICS 2017)},
}

Endnote

%0 Journal Article
%A Haubenwallner, Karl
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T ShapeGenetics: Using Genetic Algorithms for Procedural Modeling : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002C-5C69-8
%R 10.1111/cgf.13120
%7 2017
%D 2017
%J Computer Graphics Forum
%V 36
%N 2
%& 213
%P 213 - 223
%I Wiley-Blackwell
%C Oxford
%@ false
%B The European Association for Computer Graphics 38th Annual Conference
%O EUROGRAPHICS 2017 Lyon, France, 24-28 April 2017 EG 2017

Conference paper

C. Jiang and R. Chen

“Polyhedral Meshes with Concave Faces,” in SIGGRAPH Asia 2017 Posters (ACM SIGGRAPH Asia 2017), Bangkok, Thailand, 2017.

@inproceedings{JiangSA2017,
TITLE = {Polyhedral Meshes with Concave Faces},
AUTHOR = {Jiang, Caigui and Chen, Renjie},
LANGUAGE = {eng},
ISBN = {978-1-4503-5405-9},
DOI = {10.1145/3145690.3145694},
PUBLISHER = {ACM},
YEAR = {2017},
BOOKTITLE = {SIGGRAPH Asia 2017 Posters (ACM SIGGRAPH Asia 2017)},
PAGES = {1--2},
EID = {36},
ADDRESS = {Bangkok, Thailand},
}

Endnote

%0 Conference Proceedings
%A Jiang, Caigui
%A Chen, Renjie
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Polyhedral Meshes with Concave Faces : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0000-5FE4-5
%R 10.1145/3145690.3145694
%D 2017
%B ACM SIGGRAPH Asia
%Z date of event: 2017-11-27 - 2017-11-30
%C  Bangkok, Thailand
%B SIGGRAPH Asia 2017 Posters
%P 1 - 2
%Z sequence number: 36
%I ACM
%@ 978-1-4503-5405-9

Article

C. Jiang, C. Tang, H.-P. Seidel, and P. Wonka

“Design and Volume Optimization of Space Structures,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2017), vol. 36, no. 4, 2017.

@article{JiangSIGGRAPH2017,
TITLE = {Design and Volume Optimization of Space Structures},
AUTHOR = {Jiang, Caigui and Tang, Chengcheng and Seidel, Hans-Peter and Wonka, Peter},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3072959.3073619},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {36},
NUMBER = {4},
PAGES = {1--14},
EID = {159},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2017},
}

Endnote

%0 Journal Article
%A Jiang, Caigui
%A Tang, Chengcheng
%A Seidel, Hans-Peter
%A Wonka, Peter
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Design and Volume Optimization of Space Structures : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-7D8E-2
%R 10.1145/3072959.3073619
%7 2017
%D 2017
%J ACM Transactions on Graphics
%V 36
%N 4
%& 1
%P 1 - 14
%Z sequence number: 159
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2017
%O ACM SIGGRAPH 2017 Los Angeles, California, 30 July - 3 August

Article

P. Kellnhofer, P. Didyk, S.-P. Wang, P. Sitthi-Amorn, W. Freeman, F. Durand, and W. Matusik

“3DTV at Home: Eulerian-Lagrangian Stereo-to-Multiview Conversion,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2017), vol. 36, no. 4, 2017.

@article{KellnhoferSIGGRAPH2017,
TITLE = {{3DTV} at Home: {Eulerian}-{Lagrangian} Stereo-to-Multiview Conversion},
AUTHOR = {Kellnhofer, Petr and Didyk, Piotr and Wang, Szu-Po and Sitthi-Amorn, Pitchaya and Freeman, William and Durand, Fredo and Matusik, Wojciech},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3072959.3073617},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {36},
NUMBER = {4},
PAGES = {1--13},
EID = {146},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2017},
}

Endnote

%0 Journal Article
%A Kellnhofer, Petr
%A Didyk, Piotr
%A Wang, Szu-Po
%A Sitthi-Amorn, Pitchaya
%A Freeman, William
%A Durand, Fredo
%A Matusik, Wojciech
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
External Organizations
%T 3DTV at Home: Eulerian-Lagrangian Stereo-to-Multiview Conversion : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-F894-D
%R 10.1145/3072959.3073617
%7 2017
%D 2017
%J ACM Transactions on Graphics
%V 36
%N 4
%& 1
%P 1 - 13
%Z sequence number: 146
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2017
%O ACM SIGGRAPH 2017 Los Angeles, California, 30 July - 3 August

Article

B. Kerbl, M. Kenzel, D. Schmalstieg, H.-P. Seidel, and M. Steinberger

“Hierarchical Bucket Queuing for Fine-Grained Priority Scheduling on the GPU,” Computer Graphics Forum, vol. 36, no. 8, 2017.

@article{Seidel_Steinberger2016,
TITLE = {Hierarchical Bucket Queuing for Fine-Grained Priority Scheduling on the {GPU}},
AUTHOR = {Kerbl, Bernhard and Kenzel, Michael and Schmalstieg, Dieter and Seidel, Hans-Peter and Steinberger, Markus},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13075},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2017},
DATE = {2017},
JOURNAL = {Computer Graphics Forum},
VOLUME = {36},
NUMBER = {8},
PAGES = {232--246},
}

Endnote

%0 Journal Article
%A Kerbl, Bernhard
%A Kenzel, Michael
%A Schmalstieg, Dieter
%A Seidel, Hans-Peter
%A Steinberger, Markus
%+ External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Hierarchical Bucket Queuing for Fine-Grained Priority Scheduling on the GPU

 : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002C-1823-8
%R 10.1111/cgf.13075
%7 2016-12-05
%D 2017
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 36
%N 8
%& 232
%P 232 - 246
%I Blackwell-Wiley
%C Oxford
%@ false

Paper

H. Kim, M. Zollhöfer, A. Tewari, J. Thies, C. Richardt, and C. Theobalt

“InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image,” 2017. [Online]. Available: http://arxiv.org/abs/1703.10956.

Abstract

We introduce InverseFaceNet, a deep convolutional inverse rendering framework

for faces that jointly estimates facial pose, shape, expression, reflectance

and illumination from a single input image in a single shot. By estimating all

these parameters from just a single image, advanced editing possibilities on a

single face image, such as appearance editing and relighting, become feasible.

Previous learning-based face reconstruction approaches do not jointly recover

all dimensions, or are severely limited in terms of visual quality. In

contrast, we propose to recover high-quality facial pose, shape, expression,

reflectance and illumination using a deep neural network that is trained using

a large, synthetically created dataset. Our approach builds on a novel loss

function that measures model-space similarity directly in parameter space and

significantly improves reconstruction accuracy. In addition, we propose an

analysis-by-synthesis breeding approach which iteratively updates the synthetic

training corpus based on the distribution of real-world images, and we

demonstrate that this strategy outperforms completely synthetically trained

networks. Finally, we show high-quality reconstructions and compare our

approach to several state-of-the-art approaches.

BibTeX

@online{DBLP:journals/corr/KimZTTRT17,
TITLE = {{InverseFaceNet}: {D}eep Single-Shot Inverse Face Rendering From A Single Image},
AUTHOR = {Kim, Hyeongwoo and Zollh{\"o}fer, Michael and Tewari, Ayush and Thies, Justus and Richardt, Christian and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1703.10956},
EPRINT = {1703.10956},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {We introduce InverseFaceNet, a deep convolutional inverse rendering framework for faces that jointly estimates facial pose, shape, expression, reflectance and illumination from a single input image in a single shot. By estimating all these parameters from just a single image, advanced editing possibilities on a single face image, such as appearance editing and relighting, become feasible. Previous learning-based face reconstruction approaches do not jointly recover all dimensions, or are severely limited in terms of visual quality. In contrast, we propose to recover high-quality facial pose, shape, expression, reflectance and illumination using a deep neural network that is trained using a large, synthetically created dataset. Our approach builds on a novel loss function that measures model-space similarity directly in parameter space and significantly improves reconstruction accuracy. In addition, we propose an analysis-by-synthesis breeding approach which iteratively updates the synthetic training corpus based on the distribution of real-world images, and we demonstrate that this strategy outperforms completely synthetically trained networks. Finally, we show high-quality reconstructions and compare our approach to several state-of-the-art approaches.},
}

Endnote

%0 Report
%A Kim, Hyeongwoo
%A Zollh&#246;fer, Michael
%A Tewari, Ayush
%A Thies, Justus
%A Richardt, Christian
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T InverseFaceNet: Deep Single-Shot Inverse Face Rendering From A Single Image : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-8BCD-B
%U http://arxiv.org/abs/1703.10956
%D 2017
%X   We introduce InverseFaceNet, a deep convolutional inverse rendering framework
for faces that jointly estimates facial pose, shape, expression, reflectance
and illumination from a single input image in a single shot. By estimating all
these parameters from just a single image, advanced editing possibilities on a
single face image, such as appearance editing and relighting, become feasible.
Previous learning-based face reconstruction approaches do not jointly recover
all dimensions, or are severely limited in terms of visual quality. In
contrast, we propose to recover high-quality facial pose, shape, expression,
reflectance and illumination using a deep neural network that is trained using
a large, synthetically created dataset. Our approach builds on a novel loss
function that measures model-space similarity directly in parameter space and
significantly improves reconstruction accuracy. In addition, we propose an
analysis-by-synthesis breeding approach which iteratively updates the synthetic
training corpus based on the distribution of real-world images, and we
demonstrate that this strategy outperforms completely synthetically trained
networks. Finally, we show high-quality reconstructions and compare our
approach to several state-of-the-art approaches.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Article

T. R. Kol, O. Klehm, H.-P. Seidel, and E. Eisemann

“Expressive Single Scattering for Light Shaft Stylization,” IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 7, 2017.

@article{kol2016expressive,
TITLE = {Expressive Single Scattering for Light Shaft Stylization},
AUTHOR = {Kol, Timothy R. and Klehm, Oliver and Seidel, Hans-Peter and Eisemann, Elmar},
LANGUAGE = {eng},
ISSN = {1077-2626},
DOI = {10.1109/TVCG.2016.2554114},
PUBLISHER = {IEEE Computer Society},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {IEEE Transactions on Visualization and Computer Graphics},
VOLUME = {23},
NUMBER = {7},
PAGES = {1753--1766},
}

Endnote

%0 Journal Article
%A Kol, Timothy R.
%A Klehm, Oliver
%A Seidel, Hans-Peter
%A Eisemann, Elmar
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Expressive Single Scattering for Light Shaft Stylization : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002C-64E7-2
%R 10.1109/TVCG.2016.2554114
%7 2016-04-14
%D 2017
%J IEEE Transactions on Visualization and Computer Graphics
%V 23
%N 7
%& 1753
%P 1753 - 1766
%I IEEE Computer Society
%C New York, NY
%@ false

Article

T. Leimkühler, H.-P. Seidel, and T. Ritschel

“Minimal Warping: Planning Incremental Novel-view Synthesis,” Computer Graphics Forum (Proc. Eurographics Symposium on Rendering 2017), vol. 36, no. 4, 2017.

@article{LeimkuehlerEGSR2017,
TITLE = {Minimal Warping: {P}lanning Incremental Novel-view Synthesis},
AUTHOR = {Leimk{\"u}hler, Thomas and Seidel, Hans-Peter and Ritschel, Tobias},
LANGUAGE = {eng},
ISSN = {0167-7055},
DOI = {10.1111/cgf.13219},
PUBLISHER = {Wiley-Blackwell},
ADDRESS = {Oxford},
YEAR = {2017},
DATE = {2017},
JOURNAL = {Computer Graphics Forum (Proc. Eurographics Symposium on Rendering)},
VOLUME = {36},
NUMBER = {4},
PAGES = {1--14},
BOOKTITLE = {Eurographics Symposium on Rendering 2017},
EDITOR = {Zwicker, Matthias and Sander, Pedro},
}

Endnote

%0 Journal Article
%A Leimk&#252;hler, Thomas
%A Seidel, Hans-Peter
%A Ritschel, Tobias
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
%T Minimal Warping: Planning Incremental Novel-view Synthesis : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-CD7C-D
%R 10.1111/cgf.13219
%7 2017
%D 2017
%J Computer Graphics Forum
%V 36
%N 4
%& 1
%P 1 - 14
%I Wiley-Blackwell
%C Oxford
%@ false
%B Eurographics Symposium on Rendering 2017
%O Eurographics Symposium on Rendering 2017 EGSR 2017 Helsinki, Finland, 19-21 June 2017

Article

K. L. Lurie, R. Angst, D. V. Zlatev, J. C. Liao, and A. K. E. Bowden

“3D Reconstruction of Cystoscopy Videos for Comprehensive Bladder Records,” Biomedical Optics Express, vol. 8, no. 4, 2017.

@article{AngstOSA2017,
TITLE = {{3D} reconstruction of cystoscopy videos for comprehensive bladder records},
AUTHOR = {Lurie, Kristen L. and Angst, Roland and Zlatev, Dimitar V. and Liao, Joseph C. and Bowden, Audrey K. Ellerbee},
LANGUAGE = {eng},
ISSN = {2156-7085},
DOI = {10.1364/BOE.8.002106},
PUBLISHER = {OSA Publishing},
ADDRESS = {Washington, DC},
YEAR = {2017},
JOURNAL = {Biomedical Optics Express},
VOLUME = {8},
NUMBER = {4},
PAGES = {2106--2123},
}

Endnote

%0 Journal Article
%A Lurie, Kristen L.
%A Angst, Roland
%A Zlatev, Dimitar V.
%A Liao, Joseph C.
%A Bowden, Audrey K. Ellerbee
%+ External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
%T 3D Reconstruction of Cystoscopy Videos for Comprehensive Bladder Records : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-BDC5-8
%R 10.1364/BOE.8.002106
%7 2017
%D 2017
%J Biomedical Optics Express
%V 8
%N 4
%& 2106
%P 2106 - 2123
%I OSA Publishing
%C Washington, DC
%@ false

Article

B. Masia, A. Serrano, and D. Gutierrez

“Dynamic Range Expansion Based on Image Statistics,” Multimedia Tools and Applications, vol. 76, no. 1, 2017.

@article{RTM_MMTA2015,
TITLE = {Dynamic Range Expansion Based on Image Statistics},
AUTHOR = {Masia, Belen and Serrano, Ana and Gutierrez, Diego},
LANGUAGE = {eng},
ISSN = {1380-7501},
DOI = {10.1007/s11042-015-3036-0},
PUBLISHER = {Springer},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {Multimedia Tools and Applications},
VOLUME = {76},
NUMBER = {1},
PAGES = {631--648},
}

Endnote

%0 Journal Article
%A Masia, Belen
%A Serrano, Ana
%A Gutierrez, Diego
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Dynamic Range Expansion Based on Image Statistics : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-0029-78ED-4
%R 10.1007/s11042-015-3036-0
%7 2015-11-17
%D 2017
%J Multimedia Tools and Applications
%V 76
%N 1
%& 631
%P 631 - 648
%I Springer
%C New York, NY
%@ false

Conference paper

D. Mehta, H. Rhodin, D. Casas, P. Fua, O. Sotnychenko, W. Xu, and C. Theobalt

“Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision,” in 3DV 2017, International Conference on 3D Vision, Qingdao, China, 2017.

@inproceedings{Mehta_3DV2017,
TITLE = {Monocular {3D} Human Pose Estimation in the Wild Using Improved {CNN} Supervision},
AUTHOR = {Mehta, Dushyant and Rhodin, Helge and Casas, Dan and Fua, Pascal and Sotnychenko, Oleksandr and Xu, Weipeng and Theobalt, Christian},
LANGUAGE = {eng},
ISBN = {978-1-5386-2610-8; 978-1-5386-2611-5},
DOI = {10.1109/3DV.2017.00064},
PUBLISHER = {IEEE},
YEAR = {2017},
DATE = {2017},
BOOKTITLE = {3DV 2017, International Conference on 3D Vision},
PAGES = {506--516},
ADDRESS = {Qingdao, China},
}

Endnote

%0 Conference Proceedings
%A Mehta, Dushyant
%A Rhodin, Helge
%A Casas, Dan
%A Fua, Pascal
%A Sotnychenko, Oleksandr
%A Xu, Weipeng
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Monocular 3D Human Pose Estimation in the Wild Using Improved CNN Supervision : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-5E01-4
%R 10.1109/3DV.2017.00064
%D 2017
%B International Conference on 3D Vision
%Z date of event: 2017-10-10 - 2017-10-12
%C Qingdao, China
%B 3DV 2017
%P 506 - 516
%I IEEE
%@ 978-1-5386-2610-8 978-1-5386-2611-5

Conference paper

D. Mehta, S. Sridhar, O. Sotnychenko, H. Rhodin, F. Mueller, W. Xu, D. Casas, and C. Theobalt

“Demo of VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera,” in CVPR 2017 Demonstrations, Honolulu, HI, USA, 2017.

@inproceedings{VNectDemo_CVPR2017,
TITLE = {Demo of {VNect}: {R}eal-time {3D} Human Pose Estimation with a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sridhar, Srinath and Sotnychenko, Oleksandr and Rhodin, Helge and Mueller, Franziska and Xu, Weipeng and Casas, Dan and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://gvv.mpi-inf.mpg.de/projects/VNectDemo/},
YEAR = {2017},
BOOKTITLE = {CVPR 2017 Demonstrations},
ADDRESS = {Honolulu, HI, USA},
}

Endnote

%0 Conference Proceedings
%A Mehta, Dushyant
%A Sridhar, Srinath
%A Sotnychenko, Oleksandr
%A Rhodin, Helge
%A Mueller, Franziska
%A Xu, Weipeng
%A Casas, Dan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Demo of VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0002-F383-7
%U http://gvv.mpi-inf.mpg.de/projects/VNectDemo/
%D 2017
%B 30th IEEE Conference on Computer Vision and Pattern Recognition 
%Z date of event: 2017-07-22 - 2017-07-25
%C Honolulu, HI, USA
%B CVPR 2017 Demonstrations
%U http://gvv.mpi-inf.mpg.de/projects/VNectDemo/

Paper

D. Mehta, O. Sotnychenko, F. Mueller, W. Xu, S. Sridhar, G. Pons-Moll, and C. Theobalt

“Single-Shot Multi-Person 3D Body Pose Estimation From Monocular RGB Input,” 2017. [Online]. Available: http://arxiv.org/abs/1712.03453.

Abstract

We propose a new efficient single-shot method for multi-person 3D pose
estimation in general scenes from a monocular RGB camera. Our fully
convolutional DNN-based approach jointly infers 2D and 3D joint locations on
the basis of an extended 3D location map supported by body part associations.
This new formulation enables the readout of full body poses at a subset of
visible joints without the need for explicit bounding box tracking. It
therefore succeeds even under strong partial body occlusions by other people
and objects in the scene. We also contribute the first training data set
showing real images of sophisticated multi-person interactions and occlusions.
To this end, we leverage multi-view video-based performance capture of
individual people for ground truth annotation and a new image compositing for
user-controlled synthesis of large corpora of real multi-person images. We also
propose a new video-recorded multi-person test set with ground truth 3D
annotations. Our method achieves state-of-the-art performance on challenging
multi-person scenes.

BibTeX

@online{Mehta1712.03453,
TITLE = {Single-Shot Multi-Person {3D} Body Pose Estimation From Monocular {RGB} Input},
AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Sridhar, Srinath and Pons-Moll, Gerard and Theobalt, Christian},
LANGUAGE = {eng},
URL = {http://arxiv.org/abs/1712.03453},
EPRINT = {1712.03453},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {We propose a new efficient single-shot method for multi-person 3D pose<br>estimation in general scenes from a monocular RGB camera. Our fully<br>convolutional DNN-based approach jointly infers 2D and 3D joint locations on<br>the basis of an extended 3D location map supported by body part associations.<br>This new formulation enables the readout of full body poses at a subset of<br>visible joints without the need for explicit bounding box tracking. It<br>therefore succeeds even under strong partial body occlusions by other people<br>and objects in the scene. We also contribute the first training data set<br>showing real images of sophisticated multi-person interactions and occlusions.<br>To this end, we leverage multi-view video-based performance capture of<br>individual people for ground truth annotation and a new image compositing for<br>user-controlled synthesis of large corpora of real multi-person images. We also<br>propose a new video-recorded multi-person test set with ground truth 3D<br>annotations. Our method achieves state-of-the-art performance on challenging<br>multi-person scenes.<br>},
}

Endnote

%0 Report
%A Mehta, Dushyant
%A Sotnychenko, Oleksandr
%A Mueller, Franziska
%A Xu, Weipeng
%A Sridhar, Srinath
%A Pons-Moll, Gerard
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
%T Single-Shot Multi-Person 3D Body Pose Estimation From Monocular RGB
  Input : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0000-438F-4
%U http://arxiv.org/abs/1712.03453
%D 2017
%X   We propose a new efficient single-shot method for multi-person 3D pose<br>estimation in general scenes from a monocular RGB camera. Our fully<br>convolutional DNN-based approach jointly infers 2D and 3D joint locations on<br>the basis of an extended 3D location map supported by body part associations.<br>This new formulation enables the readout of full body poses at a subset of<br>visible joints without the need for explicit bounding box tracking. It<br>therefore succeeds even under strong partial body occlusions by other people<br>and objects in the scene. We also contribute the first training data set<br>showing real images of sophisticated multi-person interactions and occlusions.<br>To this end, we leverage multi-view video-based performance capture of<br>individual people for ground truth annotation and a new image compositing for<br>user-controlled synthesis of large corpora of real multi-person images. We also<br>propose a new video-recorded multi-person test set with ground truth 3D<br>annotations. Our method achieves state-of-the-art performance on challenging<br>multi-person scenes.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Paper

D. Mehta, S. Sridhar, O. Sotnychenko, H. Rhodin, M. Shafiei, H.-P. Seidel, W. Xu, D. Casas, and C. Theobalt

“VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera,” 2017. [Online]. Available: http://arxiv.org/abs/1705.01583.

Abstract

We present the first real-time method to capture the full global 3D skeletal

pose of a human in a stable, temporally consistent manner using a single RGB

camera. Our method combines a new convolutional neural network (CNN) based pose

regressor with kinematic skeleton fitting. Our novel fully-convolutional pose

formulation regresses 2D and 3D joint positions jointly in real time and does

not require tightly cropped input frames. A real-time kinematic skeleton

fitting method uses the CNN output to yield temporally stable 3D global pose

reconstructions on the basis of a coherent kinematic skeleton. This makes our

approach the first monocular RGB method usable in real-time applications such

as 3D character control---thus far, the only monocular methods for such

applications employed specialized RGB-D cameras. Our method's accuracy is

quantitatively on par with the best offline 3D monocular RGB pose estimation

methods. Our results are qualitatively comparable to, and sometimes better

than, results from monocular RGB-D approaches, such as the Kinect. However, we

show that our approach is more broadly applicable than RGB-D solutions, i.e. it

works for outdoor scenes, community videos, and low quality commodity RGB

cameras.

BibTeX

@online{MehtaArXiv2017,
TITLE = {{VNect}: Real-time {3D} Human Pose Estimation with a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sridhar, Srinath and Sotnychenko, Oleksandr and Rhodin, Helge and Shafiei, Mohammad and Seidel, Hans-Peter and Xu, Weipeng and Casas, Dan and Theobalt, Christian},
URL = {http://arxiv.org/abs/1705.01583},
DOI = {10.1145/3072959.3073596},
EPRINT = {1705.01583},
EPRINTTYPE = {arXiv},
YEAR = {2017},
ABSTRACT = {We present the first real-time method to capture the full global 3D skeletal pose of a human in a stable, temporally consistent manner using a single RGB camera. Our method combines a new convolutional neural network (CNN) based pose regressor with kinematic skeleton fitting. Our novel fully-convolutional pose formulation regresses 2D and 3D joint positions jointly in real time and does not require tightly cropped input frames. A real-time kinematic skeleton fitting method uses the CNN output to yield temporally stable 3D global pose reconstructions on the basis of a coherent kinematic skeleton. This makes our approach the first monocular RGB method usable in real-time applications such as 3D character control---thus far, the only monocular methods for such applications employed specialized RGB-D cameras. Our method's accuracy is quantitatively on par with the best offline 3D monocular RGB pose estimation methods. Our results are qualitatively comparable to, and sometimes better than, results from monocular RGB-D approaches, such as the Kinect. However, we show that our approach is more broadly applicable than RGB-D solutions, i.e. it works for outdoor scenes, community videos, and low quality commodity RGB cameras.},
}

Endnote

%0 Report
%A Mehta, Dushyant
%A Sridhar, Srinath
%A Sotnychenko, Oleksandr
%A Rhodin, Helge
%A Shafiei, Mohammad
%A Seidel, Hans-Peter
%A Xu, Weipeng
%A Casas, Dan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera : 
%U http://hdl.handle.net/11858/00-001M-0000-002D-7D78-3
%R 10.1145/3072959.3073596
%U http://arxiv.org/abs/1705.01583
%D 2017
%X   We present the first real-time method to capture the full global 3D skeletal
pose of a human in a stable, temporally consistent manner using a single RGB
camera. Our method combines a new convolutional neural network (CNN) based pose
regressor with kinematic skeleton fitting. Our novel fully-convolutional pose
formulation regresses 2D and 3D joint positions jointly in real time and does
not require tightly cropped input frames. A real-time kinematic skeleton
fitting method uses the CNN output to yield temporally stable 3D global pose
reconstructions on the basis of a coherent kinematic skeleton. This makes our
approach the first monocular RGB method usable in real-time applications such
as 3D character control---thus far, the only monocular methods for such
applications employed specialized RGB-D cameras. Our method's accuracy is
quantitatively on par with the best offline 3D monocular RGB pose estimation
methods. Our results are qualitatively comparable to, and sometimes better
than, results from monocular RGB-D approaches, such as the Kinect. However, we
show that our approach is more broadly applicable than RGB-D solutions, i.e. it
works for outdoor scenes, community videos, and low quality commodity RGB
cameras.

%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Article

D. Mehta, S. Sridhar, O. Sotnychenko, H. Rhodin, M. Shafiei, H.-P. Seidel, W. Xu, D. Casas, and C. Theobalt

“VNect: Real-Time 3D Human Pose Estimation With a Single RGB Camera,” ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2017), vol. 36, no. 4, 2017.

@article{MehtaSIGGRAPH2017,
TITLE = {{VNect}: {R}eal-Time {3D} Human Pose Estimation With a Single {RGB} Camera},
AUTHOR = {Mehta, Dushyant and Sridhar, Srinath and Sotnychenko, Oleksandr and Rhodin, Helge and Shafiei, Mohammad and Seidel, Hans-Peter and Xu, Weipeng and Casas, Dan and Theobalt, Christian},
LANGUAGE = {eng},
ISSN = {0730-0301},
DOI = {10.1145/3072959.3073596},
PUBLISHER = {ACM},
ADDRESS = {New York, NY},
YEAR = {2017},
DATE = {2017},
JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)},
VOLUME = {36},
NUMBER = {4},
PAGES = {1--14},
EID = {44},
BOOKTITLE = {Proceedings of ACM SIGGRAPH 2017},
}

Endnote

%0 Journal Article
%A Mehta, Dushyant
%A Sridhar, Srinath
%A Sotnychenko, Oleksandr
%A Rhodin, Helge
%A Shafiei, Mohammad
%A Seidel, Hans-Peter
%A Xu, Weipeng
%A Casas, Dan
%A Theobalt, Christian
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
Computer Graphics, MPI for Informatics, Max Planck Society
External Organizations
Computer Graphics, MPI for Informatics, Max Planck Society
%T VNect: Real-Time 3D Human Pose Estimation With a Single RGB Camera : 
%G eng
%U http://hdl.handle.net/11858/00-001M-0000-002D-7D95-0
%R 10.1145/3072959.3073596
%7 2017
%D 2017
%J ACM Transactions on Graphics
%V 36
%N 4
%& 1
%P 1 - 14
%Z sequence number: 44
%I ACM
%C New York, NY
%@ false
%B Proceedings of ACM SIGGRAPH 2017
%O ACM SIGGRAPH 2017 Los Angeles, California, 30 July - 3 August

Article

A. Meka, G. Fox, M. Zollhöfer, C. Richardt, and C. Theobalt

“Live User-guided Intrinsic Video For Static Scenes,” IEEE Transactions on Visualization and Computer Graphics, vol. 23, no. 11, 2017.