Gerard Pons-Moll (Senior Researcher)

Prof. Dr. Gerard Pons-Moll

Address: Max-Planck-Institut für Informatik
Saarland Informatics Campus
Campus E1 4
66123 Saarbrücken
Location: E1 4 - 605
Phone: +49 681 9325 2135
Fax: +49 681 9325 2099
E-mail: Get email via email

Group Homepage

Please visit my group website: http://virtualhumans.mpi-inf.mpg.de

Offers

If you are interested in doing a PhD on related areas, contact me direclty or send your application to d2-application@mpi-inf.mpg.de. We also have projects for bachelor and master theses and research internships of 6 months.

Publications -- MPII only

Xie, X., Bhatnagar, B. L., & Pons-Moll, G. (n.d.). Visibility Aware Human-Object Interaction Tracking from Single RGB Camera. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Vancouver, Canada: IEEE.

(Accepted/in press)

Links

Export

BibTeX

@inproceedings{xie2023vistracker, TITLE = {Visibility Aware Human-Object Interaction Tracking from Single {RGB} Camera}, AUTHOR = {Xie, Xianghui and Bhatnagar, Bharat Lal and Pons-Moll, Gerard}, LANGUAGE = {eng}, PUBLISHER = {IEEE}, YEAR = {2023}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023)}, ADDRESS = {Vancouver, Canada}, }

Endnote

%0 Conference Proceedings %A Xie, Xianghui %A Bhatnagar, Bharat Lal %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Visibility Aware Human-Object Interaction Tracking from Single RGB Camera : %G eng %U http://hdl.handle.net/21.11116/0000-000C-EBF4-8 %D 2023 %B 36th IEEE/CVF Conference on Computer Vision and Pattern Recognition %Z date of event: 2023-06-18 - 2023-06-23 %C Vancouver, Canada %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %I IEEE

Petrov, I. A., Marin, R., Chibane, J., & Pons-Moll, G. (n.d.). Object Pop-Up: Can We Infer 3D Objects and their Poses from Human Interactions Alone? In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023). Vancouver, Canada: IEEE.

(Accepted/in press)

Links

Export

BibTeX

@inproceedings{petrov2020objectpopup, TITLE = {Object Pop-Up: {C}an We Infer {3D} Objects and their Poses from Human Interactions Alone?}, AUTHOR = {Petrov, Ilya A. and Marin, Riccardo and Chibane, Julian and Pons-Moll, Gerard}, LANGUAGE = {eng}, PUBLISHER = {IEEE}, YEAR = {2023}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2023)}, ADDRESS = {Vancouver, Canada}, }

Endnote

%0 Conference Proceedings %A Petrov, Ilya A. %A Marin, Riccardo %A Chibane, Julian %A Pons-Moll, Gerard %+ External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Object Pop-Up: Can We Infer 3D Objects and their Poses from Human Interactions Alone? : %G eng %U http://hdl.handle.net/21.11116/0000-000D-159E-A %D 2023 %B 36th IEEE/CVF Conference on Computer Vision and Pattern Recognition %Z date of event: 2023-06-18 - 2023-06-23 %C Vancouver, Canada %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %I IEEE

Habermann, M., Xu, W., Zollhöfer, M., Pons-Moll, G., & Theobalt, C. (2023). A Deeper Look into DeepCap. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4). doi:10.1109/TPAMI.2021.3093553

Abstract

Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture approaches either required expensive multi-view setups or did not recover dense space-time coherent geometry with frame-to-frame correspondences. We propose a novel deep learning approach for monocular dense human performance capture. Our method is trained in a weakly supervised manner based on multi-view supervision completely removing the need for training data with 3D ground truth annotations. The network architecture is based on two separate networks that disentangle the task into a pose estimation and a non-rigid surface deformation step. Extensive qualitative and quantitative evaluations show that our approach outperforms the state of the art in terms of quality and robustness. This work is an extended version of DeepCap where we provide more detailed explanations, comparisons and results as well as applications.

Links

Export

BibTeX

@article{Habermann2111.10563, TITLE = {A Deeper Look into {DeepCap}}, AUTHOR = {Habermann, Marc and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISSN = {0162-8828}, DOI = {10.1109/TPAMI.2021.3093553}, PUBLISHER = {IEEE}, ADDRESS = {Piscataway, NJ}, YEAR = {2023}, MARGINALMARK = {$\bullet$}, DATE = {2023}, ABSTRACT = {Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture approaches either required expensive multi-view setups or did not recover dense space-time coherent geometry with frame-to-frame correspondences. We propose a novel deep learning approach for monocular dense human performance capture. Our method is trained in a weakly supervised manner based on multi-view supervision completely removing the need for training data with 3D ground truth annotations. The network architecture is based on two separate networks that disentangle the task into a pose estimation and a non-rigid surface deformation step. Extensive qualitative and quantitative evaluations show that our approach outperforms the state of the art in terms of quality and robustness. This work is an extended version of DeepCap where we provide more detailed explanations, comparisons and results as well as applications. }, JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, VOLUME = {45}, NUMBER = {4}, PAGES = {4009--4002}, }

Endnote

%0 Journal Article %A Habermann, Marc %A Xu, Weipeng %A Zollhöfer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T A Deeper Look into DeepCap : (Invited Paper) %G eng %U http://hdl.handle.net/21.11116/0000-0009-8C33-0 %R 10.1109/TPAMI.2021.3093553 %7 2021 %D 2023 %X Human performance capture is a highly important computer vision problem with many applications in movie production and virtual/augmented reality. Many previous performance capture approaches either required expensive multi-view setups or did not recover dense space-time coherent geometry with frame-to-frame correspondences. We propose a novel deep learning approach for monocular dense human performance capture. Our method is trained in a weakly supervised manner based on multi-view supervision completely removing the need for training data with 3D ground truth annotations. The network architecture is based on two separate networks that disentangle the task into a pose estimation and a non-rigid surface deformation step. Extensive qualitative and quantitative evaluations show that our approach outperforms the state of the art in terms of quality and robustness. This work is an extended version of DeepCap where we provide more detailed explanations, comparisons and results as well as applications. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV %J IEEE Transactions on Pattern Analysis and Machine Intelligence %O IEEE Trans. Pattern Anal. Mach. Intell. %V 45 %N 4 %& 4009 %P 4009 - 4002 %I IEEE %C Piscataway, NJ %@ false

Bhatnagar, B. L., Xie, X., Petrov, I., Sminchisescu, C., Theobalt, C., & Pons-Moll, G. (2022). BEHAVE: Dataset and Method for Tracking Human Object Interactions. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022). New Orleans, LA, USA: IEEE. doi:10.1109/CVPR52688.2022.01547

Links

Export

BibTeX

@inproceedings{Bhatnagar_CVPR2022, TITLE = {{BEHAVE}: {D}ataset and Method for Tracking Human Object Interactions}, AUTHOR = {Bhatnagar, Bharat Lal and Xie, Xianghui and Petrov, Ilya and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-6654-6946-3}, DOI = {10.1109/CVPR52688.2022.01547}, PUBLISHER = {IEEE}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)}, PAGES = {15914--15925}, ADDRESS = {New Orleans, LA, USA}, }

Endnote

%0 Conference Proceedings %A Bhatnagar, Bharat Lal %A Xie, Xianghui %A Petrov, Ilya %A Sminchisescu, Cristian %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T BEHAVE: Dataset and Method for Tracking Human Object Interactions : %G eng %U http://hdl.handle.net/21.11116/0000-000A-2889-F %R 10.1109/CVPR52688.2022.01547 %D 2022 %B 35th IEEE/CVF Conference on Computer Vision and Pattern Recognition %Z date of event: 2022-06-19 - 2022-06-24 %C New Orleans, LA, USA %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 15914 - 15925 %I IEEE %@ 978-1-6654-6946-3

Xie, X., Bhatnagar, B. L., & Pons-Moll, G. (2022). CHORE: Contact, Human and Object Reconstruction from a Single RGB Image. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-20086-1_8

Links

Export

BibTeX

@inproceedings{Xie_ECCV2022, TITLE = {{CHORE}: {C}ontact, Human and Object Reconstruction from a Single {RGB} Image}, AUTHOR = {Xie, Xianghui and Bhatnagar, Bharat Lal and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-031-20085-4}, DOI = {10.1007/978-3-031-20086-1_8}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {125--145}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13662}, ADDRESS = {Tel Aviv, Israel}, }

Endnote

%0 Conference Proceedings %A Xie, Xianghui %A Bhatnagar, Bharat Lal %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T CHORE: Contact, Human and Object Reconstruction from a Single RGB Image : %G eng %U http://hdl.handle.net/21.11116/0000-000C-2A81-3 %R 10.1007/978-3-031-20086-1_8 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Cissé, Moustapha; Farinella, Giovanni; Hassner, Tal %P 125 - 145 %I Springer %@ 978-3-031-20085-4 %B Lecture Notes in Computer Science %N 13662 %U https://rdcu.be/c26JY

Chibane, J., Engelmann, F., Tran, A. T., & Pons-Moll, G. (2022). Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation using Bounding Boxes. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-19821-2_39

Links

Export

BibTeX

@inproceedings{Chibane_ECCV2022, TITLE = {{Box2Mask}: {W}eakly Supervised {3D} Semantic Instance Segmentation using Bounding Boxes}, AUTHOR = {Chibane, Julian and Engelmann, Francis and Tran, Anh Tuan and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-031-19820-5}, DOI = {10.1007/978-3-031-19821-2_39}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {681--699}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13691}, ADDRESS = {Tel Aviv, Israel}, }

Endnote

%0 Conference Proceedings %A Chibane, Julian %A Engelmann, Francis %A Tran, Anh Tuan %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation using Bounding Boxes : %G eng %U http://hdl.handle.net/21.11116/0000-000C-2A9A-8 %R 10.1007/978-3-031-19821-2_39 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Cissé, Moustapha; Farinella, Giovanni; Hassner, Tal %P 681 - 699 %I Springer %@ 978-3-031-19820-5 %B Lecture Notes in Computer Science %N 13691 %U https://rdcu.be/c26Up

Lazova, V., Guzov, V., Olszewski, K., Tulyakov, S., & Pons-Moll, G. (2022). Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation. Retrieved from https://arxiv.org/abs/2204.10850

(arXiv: 2204.10850)

Abstract

We present a novel method for performing flexible, 3D-aware image content manipulation while enabling high-quality novel view synthesis. While NeRF-based approaches are effective for novel view synthesis, such models memorize the radiance for every point in a scene within a neural network. Since these models are scene-specific and lack a 3D scene representation, classical editing such as shape manipulation, or combining scenes is not possible. Hence, editing and combining NeRF-based scenes has not been demonstrated. With the aim of obtaining interpretable and controllable scene representations, our model couples learnt scene-specific feature volumes with a scene agnostic neural rendering network. With this hybrid representation, we decouple neural rendering from scene-specific geometry and appearance. We can generalize to novel scenes by optimizing only the scene-specific 3D feature representation, while keeping the parameters of the rendering network fixed. The rendering function learnt during the initial training stage can thus be easily applied to new scenes, making our approach more flexible. More importantly, since the feature volumes are independent of the rendering model, we can manipulate and combine scenes by editing their corresponding feature volumes. The edited volume can then be plugged into the rendering model to synthesize high-quality novel views. We demonstrate various scene manipulations, including mixing scenes, deforming objects and inserting objects into scenes, while still producing photo-realistic results.

Links

Export

BibTeX

@online{Lazova2204.10850, TITLE = {Control-{NeRF}: Editable Feature Volumes for Scene Rendering and Manipulation}, AUTHOR = {Lazova, Verica and Guzov, Vladimir and Olszewski, Kyle and Tulyakov, Sergey and Pons-Moll, Gerard}, LANGUAGE = {eng}, URL = {https://arxiv.org/abs/2204.10850}, EPRINT = {2204.10850}, EPRINTTYPE = {arXiv}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, ABSTRACT = {We present a novel method for performing flexible, 3D-aware image content manipulation while enabling high-quality novel view synthesis. While NeRF-based approaches are effective for novel view synthesis, such models memorize the radiance for every point in a scene within a neural network. Since these models are scene-specific and lack a 3D scene representation, classical editing such as shape manipulation, or combining scenes is not possible. Hence, editing and combining NeRF-based scenes has not been demonstrated. With the aim of obtaining interpretable and controllable scene representations, our model couples learnt scene-specific feature volumes with a scene agnostic neural rendering network. With this hybrid representation, we decouple neural rendering from scene-specific geometry and appearance. We can generalize to novel scenes by optimizing only the scene-specific 3D feature representation, while keeping the parameters of the rendering network fixed. The rendering function learnt during the initial training stage can thus be easily applied to new scenes, making our approach more flexible. More importantly, since the feature volumes are independent of the rendering model, we can manipulate and combine scenes by editing their corresponding feature volumes. The edited volume can then be plugged into the rendering model to synthesize high-quality novel views. We demonstrate various scene manipulations, including mixing scenes, deforming objects and inserting objects into scenes, while still producing photo-realistic results. }, }

Endnote

%0 Report %A Lazova, Verica %A Guzov, Vladimir %A Olszewski, Kyle %A Tulyakov, Sergey %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Control-NeRF: Editable Feature Volumes for Scene Rendering and Manipulation : %G eng %U http://hdl.handle.net/21.11116/0000-000C-2A9F-3 %U https://arxiv.org/abs/2204.10850 %D 2022 %X We present a novel method for performing flexible, 3D-aware image content manipulation while enabling high-quality novel view synthesis. While NeRF-based approaches are effective for novel view synthesis, such models memorize the radiance for every point in a scene within a neural network. Since these models are scene-specific and lack a 3D scene representation, classical editing such as shape manipulation, or combining scenes is not possible. Hence, editing and combining NeRF-based scenes has not been demonstrated. With the aim of obtaining interpretable and controllable scene representations, our model couples learnt scene-specific feature volumes with a scene agnostic neural rendering network. With this hybrid representation, we decouple neural rendering from scene-specific geometry and appearance. We can generalize to novel scenes by optimizing only the scene-specific 3D feature representation, while keeping the parameters of the rendering network fixed. The rendering function learnt during the initial training stage can thus be easily applied to new scenes, making our approach more flexible. More importantly, since the feature volumes are independent of the rendering model, we can manipulate and combine scenes by editing their corresponding feature volumes. The edited volume can then be plugged into the rendering model to synthesize high-quality novel views. We demonstrate various scene manipulations, including mixing scenes, deforming objects and inserting objects into scenes, while still producing photo-realistic results. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Zhang, X., Bhatnagar, B. L., Starke, S., Guzov, V., & Pons-Moll, G. (2022). COUCH: Towards Controllable Human-Chair Interactions. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-20065-6_30

Links

Export

BibTeX

@inproceedings{Zhang_ECCV2022, TITLE = {{COUCH}: {T}owards Controllable Human-Chair Interactions}, AUTHOR = {Zhang, Xiaohan and Bhatnagar, Bharat Lal and Starke, Sebastian and Guzov, Vladimir and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-031-20064-9}, DOI = {10.1007/978-3-031-20065-6_30}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {518--535}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13665}, ADDRESS = {Tel Aviv, Israel}, }

Endnote

%0 Conference Proceedings %A Zhang, Xiaohan %A Bhatnagar, Bharat Lal %A Starke, Sebastian %A Guzov, Vladimir %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T COUCH: Towards Controllable Human-Chair Interactions : %G eng %U http://hdl.handle.net/21.11116/0000-000C-2A92-0 %R 10.1007/978-3-031-20065-6_30 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Cissé, Moustapha; Farinella, Giovanni; Hassner, Tal %P 518 - 535 %I Springer %@ 978-3-031-20064-9 %B Lecture Notes in Computer Science %N 13665 %U https://rdcu.be/c26Qf

Corona, E., Pons-Moll, G., Alenyà, G., & Moreno-Noguer, F. (2022). Learned Vertex Descent: A New Direction for 3D Human Model Fitting. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-20086-1_9

Links

Export

BibTeX

@inproceedings{Corona_ECCV2022, TITLE = {Learned Vertex Descent: {A} New Direction for {3D} Human Model Fitting}, AUTHOR = {Corona, Enric and Pons-Moll, Gerard and Aleny{\`a}, Guillem and Moreno-Noguer, Francesc}, LANGUAGE = {eng}, ISBN = {978-3-031-20085-4}, DOI = {10.1007/978-3-031-20086-1_9}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {145--164}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13662}, ADDRESS = {Tel Aviv, Israel}, }

Endnote

%0 Conference Proceedings %A Corona, Enric %A Pons-Moll, Gerard %A Alenyà, Guillem %A Moreno-Noguer, Francesc %+ External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Learned Vertex Descent: A New Direction for 3D Human Model Fitting : %G eng %U http://hdl.handle.net/21.11116/0000-000C-2A8C-8 %R 10.1007/978-3-031-20086-1_9 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Cissé, Moustapha; Farinella, Giovanni; Hassner, Tal %P 145 - 164 %I Springer %@ 978-3-031-20085-4 %B Lecture Notes in Computer Science %N 13662 %U https://rdcu.be/c26Or

Liao, Z., Yang, J., Saito, J., Pons-Moll, G., & Zhou, Y. (2022). Skeleton-Free Pose Transfer for Stylized 3D Characters. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-20086-1_37

Links

Export

BibTeX

@inproceedings{Liao_ECCV2022, TITLE = {Skeleton-Free Pose Transfer for Stylized {3D} Characters}, AUTHOR = {Liao, Zhouyingcheng and Yang, Jimei and Saito, Jun and Pons-Moll, Gerard and Zhou, Yang}, LANGUAGE = {eng}, ISBN = {978-3-031-20085-4}, DOI = {10.1007/978-3-031-20086-1_37}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {640--656}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13662}, ADDRESS = {Tel Aviv, Israel}, }

Endnote

%0 Conference Proceedings %A Liao, Zhouyingcheng %A Yang, Jimei %A Saito, Jun %A Pons-Moll, Gerard %A Zhou, Yang %+ External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Skeleton-Free Pose Transfer for Stylized 3D Characters : %G eng %U http://hdl.handle.net/21.11116/0000-000C-2A98-A %R 10.1007/978-3-031-20086-1_37 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Cissé, Moustapha; Farinella, Giovanni; Hassner, Tal %P 640 - 656 %I Springer %@ 978-3-031-20085-4 %B Lecture Notes in Computer Science %N 13662 %U https://rdcu.be/c26R4

Guzov, V., Sattler, T., & Pons-Moll, G. (2022). Visually Plausible Human-Object Interaction Capture from Wearable Sensors. Retrieved from https://arxiv.org/abs/2205.02830

(arXiv: 2205.02830)

Abstract

In everyday lives, humans naturally modify the surrounding environment through interactions, e.g., moving a chair to sit on it. To reproduce such interactions in virtual spaces (e.g., metaverse), we need to be able to capture and model them, including changes in the scene geometry, ideally from ego-centric input alone (head camera and body-worn inertial sensors). This is an extremely hard problem, especially since the object/scene might not be visible from the head camera (e.g., a human not looking at a chair while sitting down, or not looking at the door handle while opening a door). In this paper, we present HOPS, the first method to capture interactions such as dragging objects and opening doors from ego-centric data alone. Central to our method is reasoning about human-object interactions, allowing to track objects even when they are not visible from the head camera. HOPS localizes and registers both the human and the dynamic object in a pre-scanned static scene. HOPS is an important first step towards advanced AR/VR applications based on immersive virtual universes, and can provide human-centric training data to teach machines to interact with their surroundings. The supplementary video, data, and code will be available on our project page at http://virtualhumans.mpi-inf.mpg.de/hops/

Links

Export

BibTeX

@online{Guzov2205.02830, TITLE = {Visually Plausible Human-Object Interaction Capture from Wearable Sensors}, AUTHOR = {Guzov, Vladimir and Sattler, Torsten and Pons-Moll, Gerard}, LANGUAGE = {eng}, URL = {https://arxiv.org/abs/2205.02830}, EPRINT = {2205.02830}, EPRINTTYPE = {arXiv}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, ABSTRACT = {In everyday lives, humans naturally modify the surrounding environment through interactions, e.g., moving a chair to sit on it. To reproduce such interactions in virtual spaces (e.g., metaverse), we need to be able to capture and model them, including changes in the scene geometry, ideally from ego-centric input alone (head camera and body-worn inertial sensors). This is an extremely hard problem, especially since the object/scene might not be visible from the head camera (e.g., a human not looking at a chair while sitting down, or not looking at the door handle while opening a door). In this paper, we present HOPS, the first method to capture interactions such as dragging objects and opening doors from ego-centric data alone. Central to our method is reasoning about human-object interactions, allowing to track objects even when they are not visible from the head camera. HOPS localizes and registers both the human and the dynamic object in a pre-scanned static scene. HOPS is an important first step towards advanced AR/VR applications based on immersive virtual universes, and can provide human-centric training data to teach machines to interact with their surroundings. The supplementary video, data, and code will be available on our project page at http://virtualhumans.mpi-inf.mpg.de/hops/ }, }

Endnote

%0 Report %A Guzov, Vladimir %A Sattler, Torsten %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Visually Plausible Human-Object Interaction Capture from Wearable Sensors : %G eng %U http://hdl.handle.net/21.11116/0000-000C-2A9C-6 %U https://arxiv.org/abs/2205.02830 %D 2022 %X In everyday lives, humans naturally modify the surrounding environment through interactions, e.g., moving a chair to sit on it. To reproduce such interactions in virtual spaces (e.g., metaverse), we need to be able to capture and model them, including changes in the scene geometry, ideally from ego-centric input alone (head camera and body-worn inertial sensors). This is an extremely hard problem, especially since the object/scene might not be visible from the head camera (e.g., a human not looking at a chair while sitting down, or not looking at the door handle while opening a door). In this paper, we present HOPS, the first method to capture interactions such as dragging objects and opening doors from ego-centric data alone. Central to our method is reasoning about human-object interactions, allowing to track objects even when they are not visible from the head camera. HOPS localizes and registers both the human and the dynamic object in a pre-scanned static scene. HOPS is an important first step towards advanced AR/VR applications based on immersive virtual universes, and can provide human-centric training data to teach machines to interact with their surroundings. The supplementary video, data, and code will be available on our project page at http://virtualhumans.mpi-inf.mpg.de/hops/ %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Zhou, K., Lal Bhatnagar, B., Lenssen, J. E., & Pons-Moll, G. (2022). TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement. Retrieved from https://arxiv.org/abs/2205.07982

(arXiv: 2205.07982)

Abstract

We present TOCH, a method for refining incorrect 3D hand-object interaction sequences using a data prior. Existing hand trackers, especially those that rely on very few cameras, often produce visually unrealistic results with hand-object intersection or missing contacts. Although correcting such errors requires reasoning about temporal aspects of interaction, most previous work focus on static grasps and contacts. The core of our method are TOCH fields, a novel spatio-temporal representation for modeling correspondences between hands and objects during interaction. The key component is a point-wise object-centric representation which encodes the hand position relative to the object. Leveraging this novel representation, we learn a latent manifold of plausible TOCH fields with a temporal denoising auto-encoder. Experiments demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object interaction models, which are limited to static grasps and contacts. More importantly, our method produces smooth interactions even before and after contact. Using a single trained TOCH model, we quantitatively and qualitatively demonstrate its usefulness for 1) correcting erroneous reconstruction results from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising, and 3) grasp transfer across objects. We will release our code and trained model on our project page at http://virtualhumans.mpi-inf.mpg.de/toch/

Links

Export

BibTeX

@online{Zhou_2205.07982, TITLE = {{TOCH}: Spatio-Temporal Object Correspondence to Hand for Motion Refinement}, AUTHOR = {Zhou, Keyang and Lal Bhatnagar, Bharat and Lenssen, Jan Eric and Pons-Moll, Gerard}, LANGUAGE = {eng}, URL = {https://arxiv.org/abs/2205.07982}, EPRINT = {2205.07982}, EPRINTTYPE = {arXiv}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, ABSTRACT = {We present TOCH, a method for refining incorrect 3D hand-object interaction sequences using a data prior. Existing hand trackers, especially those that rely on very few cameras, often produce visually unrealistic results with hand-object intersection or missing contacts. Although correcting such errors requires reasoning about temporal aspects of interaction, most previous work focus on static grasps and contacts. The core of our method are TOCH fields, a novel spatio-temporal representation for modeling correspondences between hands and objects during interaction. The key component is a point-wise object-centric representation which encodes the hand position relative to the object. Leveraging this novel representation, we learn a latent manifold of plausible TOCH fields with a temporal denoising auto-encoder. Experiments demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object interaction models, which are limited to static grasps and contacts. More importantly, our method produces smooth interactions even before and after contact. Using a single trained TOCH model, we quantitatively and qualitatively demonstrate its usefulness for 1) correcting erroneous reconstruction results from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising, and 3) grasp transfer across objects. We will release our code and trained model on our project page at http://virtualhumans.mpi-inf.mpg.de/toch/ }, }

Endnote

%0 Report %A Zhou, Keyang %A Lal Bhatnagar, Bharat %A Lenssen, Jan Eric %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement : %G eng %U http://hdl.handle.net/21.11116/0000-000A-ACF3-2 %U https://arxiv.org/abs/2205.07982 %D 2022 %X We present TOCH, a method for refining incorrect 3D hand-object interaction sequences using a data prior. Existing hand trackers, especially those that rely on very few cameras, often produce visually unrealistic results with hand-object intersection or missing contacts. Although correcting such errors requires reasoning about temporal aspects of interaction, most previous work focus on static grasps and contacts. The core of our method are TOCH fields, a novel spatio-temporal representation for modeling correspondences between hands and objects during interaction. The key component is a point-wise object-centric representation which encodes the hand position relative to the object. Leveraging this novel representation, we learn a latent manifold of plausible TOCH fields with a temporal denoising auto-encoder. Experiments demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object interaction models, which are limited to static grasps and contacts. More importantly, our method produces smooth interactions even before and after contact. Using a single trained TOCH model, we quantitatively and qualitatively demonstrate its usefulness for 1) correcting erroneous reconstruction results from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising, and 3) grasp transfer across objects. We will release our code and trained model on our project page at http://virtualhumans.mpi-inf.mpg.de/toch/ %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Tiwari, G., Antic, D., Lenssen, J. E., Sarafianos, N., Tung, T., & Pons-Moll, G. (2022). Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-20065-6_33

Links

Export

BibTeX

@inproceedings{Tiwari_ECCV22, TITLE = {Pose-{NDF}: {M}odeling Human Pose Manifolds with Neural Distance Fields}, AUTHOR = {Tiwari, Garvita and Antic, Dimitrije and Lenssen, Jan Eric and Sarafianos, Nikolaos and Tung, Tony and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {10.1007/978-3-031-20065-6{\textunderscore}33}, DOI = {10.1007/978-3-031-20065-6_33}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {572--589}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13665}, ADDRESS = {Tel Aviv, Israel}, }

Endnote

%0 Conference Proceedings %A Tiwari, Garvita %A Antic, Dimitrije %A Lenssen, Jan Eric %A Sarafianos, Nikolaos %A Tung, Tony %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields : %G eng %U http://hdl.handle.net/21.11116/0000-000A-B582-6 %R 10.1007/978-3-031-20065-6_33 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Cissé, Moustapha; Farinella, Giovanni; Hassner, Tal %P 572 - 589 %I Springer %@ 10.1007/978-3-031-20065-6_33 %B Lecture Notes in Computer Science %N 13665 %U https://rdcu.be/c26RY

Zhou, K., Bhatnagar, B. L., Lenssen, J. E., & Pons-Moll, G. (2022). TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-20062-5_1

Links

Export

BibTeX

@inproceedings{Zhou_ECCV2022, TITLE = {{TOCH}: {S}patio-Temporal Object Correspondence to Hand for Motion Refinement}, AUTHOR = {Zhou, Keyang and Bhatnagar, Bharat Lal and Lenssen, Jan Eric and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-031-20061-8}, DOI = {10.1007/978-3-031-20062-5_1}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {1--19}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13663}, ADDRESS = {Tel Aviv, Israel}, }

Endnote

%0 Conference Proceedings %A Zhou, Keyang %A Bhatnagar, Bharat Lal %A Lenssen, Jan Eric %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement : %G eng %U http://hdl.handle.net/21.11116/0000-000A-B586-2 %R 10.1007/978-3-031-20062-5_1 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Cissé, Moustapha; Farinella, Giovanni; Hassner, Tal %P 1 - 19 %I Springer %@ 978-3-031-20061-8 %B Lecture Notes in Computer Science %N 13663 %U https://rdcu.be/c26JY

Saint, A., Kacem, A., Cherenkova, K., Papadopoulos, K., Chibane, J., Pons-Moll, G., … Ottersten, B. (2021). SHARP 2020: The 1st Shape Recovery from Partial Textured 3D Scans Challenge Results. In Computer Vision -- ECCV Workshops 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-66096-3_50

Links

Export

BibTeX

@inproceedings{Saint_ECCV20, TITLE = {{SHARP} 2020: {T}he 1st Shape Recovery from Partial Textured {3D} Scans Challenge Results}, AUTHOR = {Saint, Alexandre and Kacem, Anis and Cherenkova, Kseniya and Papadopoulos, Konstantinos and Chibane, Julian and Pons-Moll, Gerard and Gusev, Gleb and Fofi, David and Aouada, Djamila and Ottersten, Bj{\"o}rn}, LANGUAGE = {eng}, ISBN = {978-3-030-66095-6}, DOI = {10.1007/978-3-030-66096-3_50}, PUBLISHER = {Springer}, YEAR = {2020}, MARGINALMARK = {$\bullet$}, DATE = {2021}, BOOKTITLE = {Computer Vision -- ECCV Workshops 2020}, EDITOR = {Bartoli, Adrien and Fusiello, Andrea}, PAGES = {741--755}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12536}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Saint, Alexandre %A Kacem, Anis %A Cherenkova, Kseniya %A Papadopoulos, Konstantinos %A Chibane, Julian %A Pons-Moll, Gerard %A Gusev, Gleb %A Fofi, David %A Aouada, Djamila %A Ottersten, Björn %+ External Organizations External Organizations External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T SHARP 2020: The 1st Shape Recovery from Partial Textured 3D Scans Challenge Results : %G eng %U http://hdl.handle.net/21.11116/0000-0009-4714-1 %R 10.1007/978-3-030-66096-3_50 %D 2021 %B 16th European Conference on Compute Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV Workshops 2020 %E Bartoli, Adrien; Fusiello, Andrea %P 741 - 755 %I Springer %@ 978-3-030-66095-6 %B Lecture Notes in Computer Science %N 12536

Chibane, J., Bansal, A., Lazova, V., & Pons-Moll, G. (2021). Stereo Radiance Fields (SRF): Learning View Synthesis from Sparse Views of Novel Scenes. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021). Nashville, TN, USA (Virtual): IEEE. doi:10.1109/CVPR46437.2021.00782

Links

Export

BibTeX

@inproceedings{chibane21srf, TITLE = {Stereo Radiance Fields {(SRF)}: {L}earning View Synthesis from Sparse Views of Novel Scenes}, AUTHOR = {Chibane, Julian and Bansal, Aayush and Lazova, Verica and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-6654-4509-2}, DOI = {10.1109/CVPR46437.2021.00782}, PUBLISHER = {IEEE}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)}, PAGES = {7907--7916}, ADDRESS = {Nashville, TN, USA (Virtual)}, }

Endnote

%0 Conference Proceedings %A Chibane, Julian %A Bansal, Aayush %A Lazova, Verica %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Stereo Radiance Fields (SRF): Learning View Synthesis from Sparse Views of Novel Scenes : %G eng %U http://hdl.handle.net/21.11116/0000-0008-326F-4 %R 10.1109/CVPR46437.2021.00782 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Nashville, TN, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 7907 - 7916 %I IEEE %@ 978-1-6654-4509-2

Habermann, M., Liu, L., Xu, W., Zollhöfer, M., Pons-Moll, G., & Theobalt, C. (2021). Real-time Deep Dynamic Characters. ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2021), 40(4). doi:10.1145/3450626.3459749

Links

Export

BibTeX

@article{Habermann2021, TITLE = {Real-time Deep Dynamic Characters}, AUTHOR = {Habermann, Marc and Liu, Lingjie and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISSN = {0730-0301}, DOI = {10.1145/3450626.3459749}, PUBLISHER = {ACM}, ADDRESS = {New York, NY}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)}, VOLUME = {40}, NUMBER = {4}, PAGES = {1--16}, EID = {94}, BOOKTITLE = {Proceedings of ACM SIGGRAPH 2021}, }

Endnote

%0 Journal Article %A Habermann, Marc %A Liu, Lingjie %A Xu, Weipeng %A Zollhöfer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society %T Real-time Deep Dynamic Characters : %G eng %U http://hdl.handle.net/21.11116/0000-0009-2A93-2 %R 10.1145/3450626.3459749 %7 2021 %D 2021 %J ACM Transactions on Graphics %V 40 %N 4 %& 1 %P 1 - 16 %Z sequence number: 94 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2021 %O ACM SIGGRAPH 2021

Guzov, V., Mir, A., Sattler,, T., & Pons-Moll, G. (2021). Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021). Nashville, TN, US (Virtual): IEEE. doi:10.1109/CVPR46437.2021.00430

Links

Export

BibTeX

@inproceedings{Guzov_CVPR21, TITLE = {Human {POSEitioning} System ({HPS}): {3D} Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors}, AUTHOR = {Guzov, Vladimir and Mir, Aymen and Sattler,, Torsten and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-6654-4509-2}, DOI = {10.1109/CVPR46437.2021.00430}, PUBLISHER = {IEEE}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)}, PAGES = {4318--4329}, ADDRESS = {Nashville, TN, US (Virtual)}, }

Endnote

%0 Conference Proceedings %A Guzov, Vladimir %A Mir, Aymen %A Sattler,, Torsten %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors : %G eng %U http://hdl.handle.net/21.11116/0000-0009-8AF2-A %R 10.1109/CVPR46437.2021.00430 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Nashville, TN, US (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 4318 - 4329 %I IEEE %@ 978-1-6654-4509-2

Tiwari, G., Sarafianos, N., Tung, T., & Pons-Moll, G. (2021). Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing. In ICCV 2021, IEEE/CVF International Conference on Computer Vision. Virtual Event: IEEE. doi:10.1109/ICCV48922.2021.01150

Links

Export

BibTeX

@inproceedings{Tiwari_ICCV21, TITLE = {Neural-{GIF}: {N}eural Generalized Implicit Functions for Animating People in Clothing}, AUTHOR = {Tiwari, Garvita and Sarafianos, Nikolaos and Tung, Tony and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-6654-2812-5}, DOI = {10.1109/ICCV48922.2021.01150}, PUBLISHER = {IEEE}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {ICCV 2021, IEEE/CVF International Conference on Computer Vision}, PAGES = {11688--11698}, ADDRESS = {Virtual Event}, }

Endnote

%0 Conference Proceedings %A Tiwari, Garvita %A Sarafianos, Nikolaos %A Tung, Tony %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing : %G eng %U http://hdl.handle.net/21.11116/0000-0009-8C23-2 %R 10.1109/ICCV48922.2021.01150 %D 2021 %B IEEE/CVF International Conference on Computer Vision %Z date of event: 2021-10-11 - 2021-10-17 %C Virtual Event %B ICCV 2021 %P 11688 - 11698 %I IEEE %@ 978-1-6654-2812-5

Pumarola, A., Corona, E., Pons-Moll, G., & Moreno-Noguer, F. (2021). D-NeRF: Neural Radiance Fields for Dynamic Scenes. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021). Nashville, TN, US (Virtual): IEEE. doi:10.1109/CVPR46437.2021.01018

Links

Export

BibTeX

@inproceedings{Pumarolacvpr2021, TITLE = {{D-NeRF}: {N}eural Radiance Fields for Dynamic Scenes}, AUTHOR = {Pumarola, Albert and Corona, Enric and Pons-Moll, Gerard and Moreno-Noguer, Francesc}, LANGUAGE = {eng}, ISBN = {978-1-6654-4509-2}, DOI = {10.1109/CVPR46437.2021.01018}, PUBLISHER = {IEEE}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)}, PAGES = {10318--10327}, ADDRESS = {Nashville, TN, US (Virtual)}, }

Endnote

%0 Conference Proceedings %A Pumarola, Albert %A Corona, Enric %A Pons-Moll, Gerard %A Moreno-Noguer, Francesc %+ External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T D-NeRF: Neural Radiance Fields for Dynamic Scenes : %G eng %U http://hdl.handle.net/21.11116/0000-0007-8333-B %R 10.1109/CVPR46437.2021.01018 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Nashville, TN, US (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 10318 - 10327 %I IEEE %@ 978-1-6654-4509-2

Zhou, K., Bhatnagar, B. L., Schiele, B., & Pons-Moll, G. (2021). Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes. Retrieved from https://arxiv.org/abs/2102.01161

(arXiv: 2102.01161)

Abstract

Most learning methods for 3D data (point clouds, meshes) suffer significant performance drops when the data is not carefully aligned to a canonical orientation. Aligning real world 3D data collected from different sources is non-trivial and requires manual intervention. In this paper, we propose the Adjoint Rigid Transform (ART) Network, a neural module which can be integrated with a variety of 3D networks to significantly boost their performance. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks such as shape reconstruction, interpolation, non-rigid registration, and latent disentanglement. ART achieves this with self-supervision and a rotation equivariance constraint on predicted rotations. The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks. We will release our code and pre-trained models for further research.

Links

Export

BibTeX

@online{Zhou2102.01161, TITLE = {Adjoint Rigid Transform Network: {T}ask-conditioned Alignment of {3D} Shapes}, AUTHOR = {Zhou, Keyang and Bhatnagar, Bharat Lal and Schiele, Bernt and Pons-Moll, Gerard}, LANGUAGE = {eng}, URL = {https://arxiv.org/abs/2102.01161}, EPRINT = {2102.01161}, EPRINTTYPE = {arXiv}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, ABSTRACT = {Most learning methods for 3D data (point clouds, meshes) suffer significant performance drops when the data is not carefully aligned to a canonical orientation. Aligning real world 3D data collected from different sources is non-trivial and requires manual intervention. In this paper, we propose the Adjoint Rigid Transform (ART) Network, a neural module which can be integrated with a variety of 3D networks to significantly boost their performance. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks such as shape reconstruction, interpolation, non-rigid registration, and latent disentanglement. ART achieves this with self-supervision and a rotation equivariance constraint on predicted rotations. The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks. We will release our code and pre-trained models for further research. }, }

Endnote

%0 Report %A Zhou, Keyang %A Bhatnagar, Bharat Lal %A Schiele, Bernt %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes : %G eng %U http://hdl.handle.net/21.11116/0000-0009-80FA-C %U https://arxiv.org/abs/2102.01161 %D 2021 %X Most learning methods for 3D data (point clouds, meshes) suffer significant performance drops when the data is not carefully aligned to a canonical orientation. Aligning real world 3D data collected from different sources is non-trivial and requires manual intervention. In this paper, we propose the Adjoint Rigid Transform (ART) Network, a neural module which can be integrated with a variety of 3D networks to significantly boost their performance. ART learns to rotate input shapes to a learned canonical orientation, which is crucial for a lot of tasks such as shape reconstruction, interpolation, non-rigid registration, and latent disentanglement. ART achieves this with self-supervision and a rotation equivariance constraint on predicted rotations. The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks. We will release our code and pre-trained models for further research. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV

Sattar, H., Krombholz, K., Pons-Moll, G., & Fritz, M. (2021). Body Shape Privacy in Images: Understanding Privacy and Preventing Automatic Shape Extraction. In Computer Vision -- ECCV Workshops 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-68238-5_31

[DOI]

[PuRe]

[BibTeX]

Abstract

Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions. Even from a single image of a clothed person, a realistic looking body shape can be inferred that captures a users' weight group and body shape type well. This opens up a whole spectrum of applications -- in particular in fashion -- where virtual try-on and recommendation systems can make use of these new and automatized cues. However, a realistic depiction of the undressed body is regarded highly private and therefore might not be consented by most people. Hence, we ask if the automatic extraction of such information can be effectively evaded. While adversarial perturbations have been shown to be effective for manipulating the output of machine learning models -- in particular, end-to-end deep learning approaches -- state of the art shape estimation methods are composed of multiple stages. We perform the first investigation of different strategies that can be used to effectively manipulate the automatic shape estimation while preserving the overall appearance of the original image.

Links

[DOI]

[PuRe]

[BibTeX]

Export

BibTeX

@inproceedings{Sattar_ECCV20, TITLE = {Body Shape Privacy in Images: {U}nderstanding Privacy and Preventing Automatic Shape Extraction}, AUTHOR = {Sattar, Hosnieh and Krombholz, Katharina and Pons-Moll, Gerard and Fritz, Mario}, LANGUAGE = {eng}, ISBN = {978-3-030-68237-8}, DOI = {10.1007/978-3-030-68238-5_31}, PUBLISHER = {Springer}, YEAR = {2020}, MARGINALMARK = {$\bullet$}, ABSTRACT = {Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions. Even from a single image of a clothed person, a realistic looking body shape can be inferred that captures a users' weight group and body shape type well. This opens up a whole spectrum of applications -- in particular in fashion -- where virtual try-on and recommendation systems can make use of these new and automatized cues. However, a realistic depiction of the undressed body is regarded highly private and therefore might not be consented by most people. Hence, we ask if the automatic extraction of such information can be effectively evaded. While adversarial perturbations have been shown to be effective for manipulating the output of machine learning models -- in particular, end-to-end deep learning approaches -- state of the art shape estimation methods are composed of multiple stages. We perform the first investigation of different strategies that can be used to effectively manipulate the automatic shape estimation while preserving the overall appearance of the original image. }, BOOKTITLE = {Computer Vision -- ECCV Workshops 2020}, EDITOR = {Bartoli, Adrien and Fusiello, Andrea}, PAGES = {411--428}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12539}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Sattar, Hosnieh %A Krombholz, Katharina %A Pons-Moll, Gerard %A Fritz, Mario %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Body Shape Privacy in Images: Understanding Privacy and Preventing Automatic Shape Extraction : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D755-7 %R 10.1007/978-3-030-68238-5_31 %D 2021 %B 16th European Conference on Compute Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %X Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions. Even from a single image of a clothed person, a realistic looking body shape can be inferred that captures a users' weight group and body shape type well. This opens up a whole spectrum of applications -- in particular in fashion -- where virtual try-on and recommendation systems can make use of these new and automatized cues. However, a realistic depiction of the undressed body is regarded highly private and therefore might not be consented by most people. Hence, we ask if the automatic extraction of such information can be effectively evaded. While adversarial perturbations have been shown to be effective for manipulating the output of machine learning models -- in particular, end-to-end deep learning approaches -- state of the art shape estimation methods are composed of multiple stages. We perform the first investigation of different strategies that can be used to effectively manipulate the automatic shape estimation while preserving the overall appearance of the original image. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Cryptography and Security, cs.CR,Computer Science, Learning, cs.LG %B Computer Vision -- ECCV Workshops 2020 %E Bartoli, Adrien; Fusiello, Andrea %P 411 - 428 %I Springer %@ 978-3-030-68237-8 %B Lecture Notes in Computer Science %N 12539

Pons-Moll, G., Moreno-Noguer, F., Corona, E., Pumarola, A., & Alenyà, G. (2021). SMPLicit: Topology-aware Generative Model for Clothed People. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021). Virtual Conference: IEEE. doi:10.1109/CVPR46437.2021.01170

Links

Export

BibTeX

@inproceedings{corona2021smplicit, TITLE = {{SMPLicit}: {T}opology-aware Generative Model for Clothed People}, AUTHOR = {Pons-Moll, Gerard and Moreno-Noguer, Francesc and Corona, Enric and Pumarola, Albert and Aleny{\`a}, Guillem}, LANGUAGE = {eng}, ISBN = {978-1-6654-4509-2}, DOI = {10.1109/CVPR46437.2021.01170}, PUBLISHER = {IEEE}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021)}, PAGES = {11870--11880}, ADDRESS = {Virtual Conference}, }

Endnote

%0 Conference Proceedings %A Pons-Moll, Gerard %A Moreno-Noguer, Francesc %A Corona, Enric %A Pumarola, Albert %A Alenyà, Guillem %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations External Organizations %T SMPLicit: Topology-aware Generative Model for Clothed People : %G eng %U http://hdl.handle.net/21.11116/0000-0008-3267-C %R 10.1109/CVPR46437.2021.01170 %D 2021 %B 34th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2021-06-19 - 2021-06-25 %C Virtual Conference %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 11870 - 11880 %I IEEE %@ 978-1-6654-4509-2

Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Elgharib, M., Fua, P., … Theobalt, C. (2020). XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera. ACM Transactions on Graphics (Proc. ACM SIGGRAPH 2020), 39(4). doi:10.1145/3386569.3392410

Links

Export

BibTeX

@article{Mehta_TOG2020, TITLE = {{XNect}: {R}eal-time Multi-person {3D} Human Pose Estimation with a Single {RGB} Camera}, AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Elgharib, Mohamed and Fua, Pascal and Seidel, Hans-Peter and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISSN = {0730-0301}, DOI = {10.1145/3386569.3392410}, PUBLISHER = {ACM}, ADDRESS = {New York, NY}, YEAR = {2020}, JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH)}, VOLUME = {39}, NUMBER = {4}, EID = {82}, BOOKTITLE = {Proceedings of ACM SIGGRAPH 2020}, }

Endnote

%0 Journal Article %A Mehta, Dushyant %A Sotnychenko, Oleksandr %A Mueller, Franziska %A Xu, Weipeng %A Elgharib, Mohamed %A Fua, Pascal %A Seidel, Hans-Peter %A Rhodin, Helge %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0007-832D-3 %R 10.1145/3386569.3392410 %7 2020 %D 2020 %J ACM Transactions on Graphics %V 39 %N 4 %Z sequence number: 82 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH 2020 %O ACM SIGGRAPH 2020 Virtual Conference ; 2020, 17-28 August

Zhou, K., Bhatnagar, B. L., & Pons-Moll, G. (2020). Unsupervised Shape and Pose Disentanglement for 3D Meshes. In Computer Vision -- ECCV 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-58542-6_21

Links

Export

BibTeX

@inproceedings{zhou20unsupervised, TITLE = {Unsupervised Shape and Pose Disentanglement for {3D} Meshes}, AUTHOR = {Zhou, Keyang and Bhatnagar, Bharat Lal and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-030-58541-9}, DOI = {10.1007/978-3-030-58542-6_21}, PUBLISHER = {Springer}, YEAR = {2020}, DATE = {2020}, BOOKTITLE = {Computer Vision -- ECCV 2020}, EDITOR = {Vedaldi, A. and Bischof, H. and Brox, Th. and Frahm, J.-M.}, PAGES = {341--357}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12367}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Zhou, Keyang %A Bhatnagar, Bharat Lal %A Pons-Moll, Gerard %+ External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Unsupervised Shape and Pose Disentanglement for 3D Meshes : %G eng %U http://hdl.handle.net/21.11116/0000-0006-E8A7-8 %R 10.1007/978-3-030-58542-6_21 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, A.; Bischof, H.; Brox, Th.; Frahm, J.-M. %P 341 - 357 %I Springer %@ 978-3-030-58541-9 %B Lecture Notes in Computer Science %N 12367

Ma, Q., Yang, J., Ranjan, A., Pujades, S., Pons-Moll, G., Tang, S., & Black, M. J. (2020). Learning to Dress 3D People in Generative Clothing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020). Seattle, WA, USA (Virtual): IEEE. doi:10.1109/CVPR42600.2020.00650

Links

Export

BibTeX

@inproceedings{Ma_CVPR2020, TITLE = {Learning to Dress {3D} People in Generative Clothing}, AUTHOR = {Ma, Qianli and Yang, Jinlong and Ranjan, Anurag and Pujades, Sergi and Pons-Moll, Gerard and Tang, Siyu and Black, Michael J.}, LANGUAGE = {eng}, ISBN = {978-1-7281-7168-5}, DOI = {10.1109/CVPR42600.2020.00650}, PUBLISHER = {IEEE}, YEAR = {2020}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)}, PAGES = {6468--6477}, ADDRESS = {Seattle, WA, USA (Virtual)}, }

Endnote

%0 Conference Proceedings %A Ma, Qianli %A Yang, Jinlong %A Ranjan, Anurag %A Pujades, Sergi %A Pons-Moll, Gerard %A Tang, Siyu %A Black, Michael J. %+ External Organizations External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Learning to Dress 3D People in Generative Clothing : %G eng %U http://hdl.handle.net/21.11116/0000-0005-749D-8 %R 10.1109/CVPR42600.2020.00650 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 6468 - 6477 %I IEEE %@ 978-1-7281-7168-5

Chibane, J., Alldieck, T., & Pons-Moll, G. (2020). Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020). Seattle, WA, USA (Virtual): IEEE. doi:10.1109/CVPR42600.2020.00700

Links

Export

BibTeX

@inproceedings{chibane20ifnet, TITLE = {Implicit Functions in Feature Space for {3D} Shape Reconstruction and Completion}, AUTHOR = {Chibane, Julian and Alldieck, Thiemo and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-7281-7168-5}, DOI = {10.1109/CVPR42600.2020.00700}, PUBLISHER = {IEEE}, YEAR = {2020}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)}, PAGES = {6968--6979}, ADDRESS = {Seattle, WA, USA (Virtual)}, }

Endnote

%0 Conference Proceedings %A Chibane, Julian %A Alldieck, Thiemo %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion : %G eng %U http://hdl.handle.net/21.11116/0000-0007-7023-3 %R 10.1109/CVPR42600.2020.00700 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 6968 - 6979 %I IEEE %@ 978-1-7281-7168-5

Chibane, J., Mir, A., & Pons-Moll, G. (2020). Neural Unsigned Distance Fields for Implicit Function Learning. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Virtual Event: Curran Associates, Inc.

Links

Export

BibTeX

@inproceedings{Chibane_NeurIPs2020, TITLE = {Neural Unsigned Distance Fields for Implicit Function Learning}, AUTHOR = {Chibane, Julian and Mir, Aymen and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {9781713829546}, PUBLISHER = {Curran Associates, Inc.}, YEAR = {2020}, BOOKTITLE = {Advances in Neural Information Processing Systems 33 (NeurIPS 2020)}, EDITOR = {Larochelle, H. and Ranzato, M. and Hadsell, R. and Balcan, M. F. and Lin, H.}, PAGES = {21638--21652}, ADDRESS = {Virtual Event}, }

Endnote

%0 Conference Proceedings %A Chibane, Julian %A Mir, Aymen %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Neural Unsigned Distance Fields for Implicit Function Learning : %G eng %U http://hdl.handle.net/21.11116/0000-0007-6FCB-9 %D 2020 %B 34th Conference on Neural Information Processing Systems %Z date of event: 2020-12-06 - 2020-12-12 %C Virtual Event %B Advances in Neural Information Processing Systems 33 %E Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.; Lin, H. %P 21638 - 21652 %I Curran Associates, Inc. %@ 9781713829546 %U https://papers.nips.cc/paper/2020/file/f69e505b08403ad2298b9f262659929a-Paper.pdf

Mir, A., Alldieck, T., & Pons-Moll, G. (2020). Learning to Transfer Texture from Clothing Images to 3D Humans. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020). Seattle, WA, USA (Virtual): IEEE. doi:10.1109/CVPR42600.2020.00705

Links

Export

BibTeX

@inproceedings{mir20pix2surf, TITLE = {Learning to Transfer Texture from Clothing Images to {3D} Humans}, AUTHOR = {Mir, Aymen and Alldieck, Thiemo and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-7281-7168-5}, DOI = {10.1109/CVPR42600.2020.00705}, PUBLISHER = {IEEE}, YEAR = {2020}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)}, PAGES = {7021--7032}, ADDRESS = {Seattle, WA, USA (Virtual)}, }

Endnote

%0 Conference Proceedings %A Mir, Aymen %A Alldieck, Thiemo %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Learning to Transfer Texture from Clothing Images to 3D Humans : %G eng %U http://hdl.handle.net/21.11116/0000-0007-7026-0 %R 10.1109/CVPR42600.2020.00705 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 7021 - 7032 %I IEEE %@ 978-1-7281-7168-5

Chibane, J., & Pons-Moll, G. (2020). Implicit Feature Networks for Texture Completion from Partial 3D Data. In Computer Vision -- ECCV Workshops 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-66096-3_48

Links

Export

BibTeX

@inproceedings{Chibane_ECCVW20, TITLE = {Implicit Feature Networks for Texture Completion from Partial {3D} Data}, AUTHOR = {Chibane, Julian and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-030-66095-6}, DOI = {10.1007/978-3-030-66096-3_48}, PUBLISHER = {Springer}, YEAR = {2020}, DATE = {2020}, BOOKTITLE = {Computer Vision -- ECCV Workshops 2020}, EDITOR = {Bartoli, Adrien and Fusiello, Andrea}, PAGES = {717--725}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12536}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Chibane, Julian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Implicit Feature Networks for Texture Completion from Partial 3D Data : %G eng %U http://hdl.handle.net/21.11116/0000-0007-D88B-9 %R 10.1007/978-3-030-66096-3_48 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV Workshops 2020 %E Bartoli, Adrien; Fusiello, Andrea %P 717 - 725 %I Springer %@ 978-3-030-66095-6 %B Lecture Notes in Computer Science %N 12536

Tome, D., Alldieck, T., Peluse, P., Pons-Moll, G., Agapito, L., Badino, H., & de la Torre, F. (2020). SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence. doi:10.1109/TPAMI.2020.3029700

Links

Export

BibTeX

@article{SelfPose2020, TITLE = {{SelfPose}: {3D} Egocentric Pose Estimation from a Headset Mounted Camera}, AUTHOR = {Tome, Denis and Alldieck, Thiemo and Peluse, Patrick and Pons-Moll, Gerard and Agapito, Lourdes and Badino, Hernan and de la Torre, Fernando}, LANGUAGE = {eng}, ISSN = {0162-8828}, DOI = {10.1109/TPAMI.2020.3029700}, PUBLISHER = {IEEE}, ADDRESS = {Piscataway, NJ}, YEAR = {2020}, JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, }

Endnote

%0 Journal Article %A Tome, Denis %A Alldieck, Thiemo %A Peluse, Patrick %A Pons-Moll, Gerard %A Agapito, Lourdes %A Badino, Hernan %A de la Torre, Fernando %+ External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations %T SelfPose: 3D Egocentric Pose Estimation from a Headset Mounted Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0007-7008-2 %R 10.1109/TPAMI.2020.3029700 %7 2020 %D 2020 %J IEEE Transactions on Pattern Analysis and Machine Intelligence %O IEEE Trans. Pattern Anal. Mach. Intell. %I IEEE %C Piscataway, NJ %@ false

Tiwari, G., Bhatnagar, B. L., Tung, T., & Pons-Moll, G. (2020). SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing. In Computer Vision -- ECCV 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-58580-8_1

Links

Export

BibTeX

@inproceedings{tiwari20sizer, TITLE = {SIZER: {A} Dataset and Model for Parsing {3D} Clothing and Learning Size Sensitive {3D} Clothing}, AUTHOR = {Tiwari, Garvita and Bhatnagar, Bharat Lal and Tung, Tony and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-030-58579-2}, DOI = {10.1007/978-3-030-58580-8_1}, PUBLISHER = {Springer}, YEAR = {2020}, DATE = {2020}, BOOKTITLE = {Computer Vision -- ECCV 2020}, EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael}, PAGES = {1--18}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12348}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Tiwari, Garvita %A Bhatnagar, Bharat Lal %A Tung, Tony %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T SIZER: A Dataset and Model for Parsing 3D Clothing and Learning Size Sensitive 3D Clothing : %G eng %U http://hdl.handle.net/21.11116/0000-0006-E32D-8 %R 10.1007/978-3-030-58580-8_1 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 1 - 18 %I Springer %@ 978-3-030-58579-2 %B Lecture Notes in Computer Science %N 12348

Bhatnagar, B. L., Sminchisescu, C., Theobalt, C., & Pons-Moll, G. (2020a). LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Virtual Event: Curran Associates, Inc.

Links

Export

BibTeX

@inproceedings{bhatnagar2020loopreg, TITLE = {{LoopReg}: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for {3D} Human Mesh Registration}, AUTHOR = {Bhatnagar, Bharat Lal and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {9781713829546}, PUBLISHER = {Curran Associates, Inc.}, YEAR = {2020}, BOOKTITLE = {Advances in Neural Information Processing Systems 33 (NeurIPS 2020)}, EDITOR = {Larochelle, H. and Ranzato, M. and Hadsell, R. and Balcan, M. F. and Lin, H.}, PAGES = {12909--12922}, ADDRESS = {Virtual Event}, }

Endnote

%0 Conference Proceedings %A Bhatnagar, Bharat Lal %A Sminchisescu, Cristian %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration : %G eng %U http://hdl.handle.net/21.11116/0000-0007-6FD1-1 %D 2020 %B 34th Conference on Neural Information Processing Systems %Z date of event: 2020-12-06 - 2020-12-12 %C Virtual Event %B Advances in Neural Information Processing Systems 33 %E Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M. F.; Lin, H. %P 12909 - 12922 %I Curran Associates, Inc. %@ 9781713829546 %U https://papers.nips.cc/paper/2020/file/970af30e481057c48f87e101b61e6994-Paper.pdf

Deng, B., Lewis, J. P., Jeruzalski, T., Pons-Moll, G., Hinton, G., Norouzi, M., & Tagliasacchi, A. (2020). NASA: Neural Articulated Shape Approximation. In Computer Vision -- ECCV 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-58571-6_36

Links

Export

BibTeX

@inproceedings{deng2019neural, TITLE = {{NASA}: {N}eural Articulated Shape Approximation}, AUTHOR = {Deng, Boyang and Lewis, J. P. and Jeruzalski, Timothy and Pons-Moll, Gerard and Hinton, Geoffrey and Norouzi, Mohammad and Tagliasacchi, Andrea}, LANGUAGE = {eng}, ISBN = {978-3-030-58570-9}, DOI = {10.1007/978-3-030-58571-6_36}, PUBLISHER = {Springer}, YEAR = {2020}, DATE = {2020}, BOOKTITLE = {Computer Vision -- ECCV 2020}, EDITOR = {Vedali, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael}, PAGES = {612--628}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12352}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Deng, Boyang %A Lewis, J. P. %A Jeruzalski, Timothy %A Pons-Moll, Gerard %A Hinton, Geoffrey %A Norouzi, Mohammad %A Tagliasacchi, Andrea %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T NASA: Neural Articulated Shape Approximation : %G eng %U http://hdl.handle.net/21.11116/0000-0007-701D-B %R 10.1007/978-3-030-58571-6_36 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedali, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 612 - 628 %I Springer %@ 978-3-030-58570-9 %B Lecture Notes in Computer Science %N 12352

Bhatnagar, B. L., Sminchisescu, C., Theobalt, C., & Pons-Moll, G. (2020b). Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction. In Computer Vision -- ECCV 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-58536-5_19

Links

Export

BibTeX

@inproceedings{bhatnagar2020ipnet, TITLE = {Combining Implicit Function Learning and Parametric Models for {3D} Human Reconstruction}, AUTHOR = {Bhatnagar, Bharat Lal and Sminchisescu, Cristian and Theobalt, Christian and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-030-58535-8}, DOI = {10.1007/978-3-030-58536-5_19}, PUBLISHER = {Springer}, YEAR = {2020}, DATE = {2020}, BOOKTITLE = {Computer Vision -- ECCV 2020}, EDITOR = {Vedaldi, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael}, PAGES = {311--329}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12347}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Bhatnagar, Bharat Lal %A Sminchisescu, Cristian %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction : %G eng %U http://hdl.handle.net/21.11116/0000-0006-E89E-3 %R 10.1007/978-3-030-58536-5_19 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedaldi, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 311 - 329 %I Springer %@ 978-3-030-58535-8 %B Lecture Notes in Computer Science %N 12347

Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., … Liu, Y. (2020). DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10). doi:10.1109/TPAMI.2019.2928296

Links

Export

BibTeX

@article{DoubleFusion20, TITLE = {{DoubleFusion}: {R}eal-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor}, AUTHOR = {Yu, Tao and Zheng, Zherong and Guo, Kaiwen and Zhao, Jianhui and Dai, Qionghai and Li, Hao and Pons-Moll, Gerard and Liu, Yebin}, LANGUAGE = {eng}, ISSN = {0162-8828}, DOI = {10.1109/TPAMI.2019.2928296}, PUBLISHER = {IEEE}, ADDRESS = {Piscataway, NJ}, YEAR = {2020}, DATE = {2020}, JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, VOLUME = {42}, NUMBER = {10}, PAGES = {2523--2539}, }

Endnote

%0 Journal Article %A Yu, Tao %A Zheng, Zherong %A Guo, Kaiwen %A Zhao, Jianhui %A Dai, Qionghai %A Li, Hao %A Pons-Moll, Gerard %A Liu, Yebin %+ External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor : %G eng %U http://hdl.handle.net/21.11116/0000-0007-11D7-3 %R 10.1109/TPAMI.2019.2928296 %7 2020 %D 2020 %J IEEE Transactions on Pattern Analysis and Machine Intelligence %O IEEE Trans. Pattern Anal. Mach. Intell. %V 42 %N 10 %& 2523 %P 2523 - 2539 %I IEEE %C Piscataway, NJ %@ false

Brazil, G., Pons-Moll, G., Liu, X., & Schiele, B. (2020). Kinematic 3D Object Detection in Monocular Video. In Computer Vision -- ECCV 2020. Glasgow, UK: Springer. doi:10.1007/978-3-030-58592-1_9

Links

Export

BibTeX

@inproceedings{Brazil20Kinematic, TITLE = {Kinematic {3D} Object Detection in Monocular Video}, AUTHOR = {Brazil, Garrick and Pons-Moll, Gerard and Liu, Xiaoming and Schiele, Bernt}, LANGUAGE = {eng}, ISBN = {978-3-030-58591-4}, DOI = {10.1007/978-3-030-58592-1_9}, PUBLISHER = {Springer}, YEAR = {2020}, DATE = {2020}, BOOKTITLE = {Computer Vision -- ECCV 2020}, EDITOR = {Vedali, Andrea and Bischof, Horst and Brox, Thomas and Frahm, Jan-Michael}, PAGES = {135--152}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {12368}, ADDRESS = {Glasgow, UK}, }

Endnote

%0 Conference Proceedings %A Brazil, Garrick %A Pons-Moll, Gerard %A Liu, Xiaoming %A Schiele, Bernt %+ External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Kinematic 3D Object Detection in Monocular Video : %G eng %U http://hdl.handle.net/21.11116/0000-0007-701F-9 %R 10.1007/978-3-030-58592-1_9 %D 2020 %B 16th European Conference on Computer Vision %Z date of event: 2020-08-23 - 2020-08-28 %C Glasgow, UK %B Computer Vision -- ECCV 2020 %E Vedali, Andrea; Bischof, Horst; Brox, Thomas; Frahm, Jan-Michael %P 135 - 152 %I Springer %@ 978-3-030-58591-4 %B Lecture Notes in Computer Science %N 12368

Patel, C., Liao, Z., & Pons-Moll, G. (2020). TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020). Seattle, WA, USA (Virtual): IEEE. doi:10.1109/CVPR42600.2020.00739

Links

Export

BibTeX

@inproceedings{Patel_2020_CVPR, TITLE = {TailorNet: {P}redicting Clothing in {3D} as a Function of Human Pose, Shape and Garment Style}, AUTHOR = {Patel, Chaitanya and Liao, Zhouyingcheng and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-7281-7168-5}, DOI = {10.1109/CVPR42600.2020.00739}, PUBLISHER = {IEEE}, YEAR = {2020}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)}, PAGES = {7363--7373}, ADDRESS = {Seattle, WA, USA (Virtual)}, }

Endnote

%0 Conference Proceedings %A Patel, Chaitanya %A Liao, Zhouyingcheng %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style : %G eng %U http://hdl.handle.net/21.11116/0000-0006-ECF2-F %R 10.1109/CVPR42600.2020.00739 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 7363 - 7373 %I IEEE %@ 978-1-7281-7168-5

Habermann, M., Xu, W., Zollhöfer, M., Pons-Moll, G., & Theobalt, C. (2020). DeepCap: Monocular Human Performance Capture Using Weak Supervision. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020). Seattle, WA, USA (Virtual): IEEE. doi:10.1109/CVPR42600.2020.00510

Links

Export

BibTeX

@inproceedings{deepcap2020, TITLE = {{DeepCap}: {M}onocular Human Performance Capture Using Weak Supervision}, AUTHOR = {Habermann, Marc and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISBN = {978-1-7281-7168-5}, DOI = {10.1109/CVPR42600.2020.00510}, PUBLISHER = {IEEE}, YEAR = {2020}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020)}, PAGES = {5051--5062}, ADDRESS = {Seattle, WA, USA (Virtual)}, }

Endnote

%0 Conference Proceedings %A Habermann, Marc %A Xu, Weipeng %A Zollhöfer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T DeepCap: Monocular Human Performance Capture Using Weak Supervision : %G eng %U http://hdl.handle.net/21.11116/0000-0006-A895-4 %R 10.1109/CVPR42600.2020.00510 %D 2020 %B 33rd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2020-06-14 - 2020-06-19 %C Seattle, WA, USA (Virtual) %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 5051 - 5062 %I IEEE %@ 978-1-7281-7168-5

Habermann, M., Xu, W., Zollhöfer, M., Pons-Moll, G., & Theobalt, C. (2019). LiveCap: Real-time Human Performance Capture from Monocular Video. ACM Transactions on Graphics, 38(2). doi:10.1145/3311970

Links

Export

BibTeX

@article{Habermann_TOG19, TITLE = {{LiveCap}: {R}eal-time Human Performance Capture from Monocular Video}, AUTHOR = {Habermann, Marc and Xu, Weipeng and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISSN = {0730-0301}, DOI = {10.1145/3311970}, PUBLISHER = {ACM}, ADDRESS = {New York, NY}, YEAR = {2019}, DATE = {2019}, JOURNAL = {ACM Transactions on Graphics}, VOLUME = {38}, NUMBER = {2}, EID = {14}, }

Endnote

%0 Journal Article %A Habermann, Marc %A Xu, Weipeng %A Zollhöfer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T LiveCap: Real-time Human Performance Capture from Monocular Video : %G eng %U http://hdl.handle.net/21.11116/0000-0002-B947-E %R 10.1145/3311970 %7 2019 %D 2019 %J ACM Transactions on Graphics %V 38 %N 2 %Z sequence number: 14 %I ACM %C New York, NY %@ false

Sattar, H., Krombholz, K., Pons-Moll, G., & Fritz, M. (2019). Shape Evasion: Preventing Body Shape Inference of Multi-Stage Approaches. Retrieved from http://arxiv.org/abs/1905.11503

(arXiv: 1905.11503)

Abstract

Links

Export

BibTeX

@online{Sattar_arXiv1905.11503, TITLE = {Shape Evasion: Preventing Body Shape Inference of Multi-Stage Approaches}, AUTHOR = {Sattar, Hosnieh and Krombholz, Katharina and Pons-Moll, Gerard and Fritz, Mario}, LANGUAGE = {eng}, URL = {http://arxiv.org/abs/1905.11503}, EPRINT = {1905.11503}, EPRINTTYPE = {arXiv}, YEAR = {2019}, ABSTRACT = {Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions. Even from a single image of a clothed person, a realistic looking body shape can be inferred that captures a users' weight group and body shape type well. This opens up a whole spectrum of applications -- in particular in fashion -- where virtual try-on and recommendation systems can make use of these new and automatized cues. However, a realistic depiction of the undressed body is regarded highly private and therefore might not be consented by most people. Hence, we ask if the automatic extraction of such information can be effectively evaded. While adversarial perturbations have been shown to be effective for manipulating the output of machine learning models -- in particular, end-to-end deep learning approaches -- state of the art shape estimation methods are composed of multiple stages. We perform the first investigation of different strategies that can be used to effectively manipulate the automatic shape estimation while preserving the overall appearance of the original image. }, }

Endnote

%0 Report %A Sattar, Hosnieh %A Krombholz, Katharina %A Pons-Moll, Gerard %A Fritz, Mario %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Shape Evasion: Preventing Body Shape Inference of Multi-Stage Approaches : %G eng %U http://hdl.handle.net/21.11116/0000-0003-B2E5-1 %U http://arxiv.org/abs/1905.11503 %D 2019 %X Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions. Even from a single image of a clothed person, a realistic looking body shape can be inferred that captures a users' weight group and body shape type well. This opens up a whole spectrum of applications -- in particular in fashion -- where virtual try-on and recommendation systems can make use of these new and automatized cues. However, a realistic depiction of the undressed body is regarded highly private and therefore might not be consented by most people. Hence, we ask if the automatic extraction of such information can be effectively evaded. While adversarial perturbations have been shown to be effective for manipulating the output of machine learning models -- in particular, end-to-end deep learning approaches -- state of the art shape estimation methods are composed of multiple stages. We perform the first investigation of different strategies that can be used to effectively manipulate the automatic shape estimation while preserving the overall appearance of the original image. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Cryptography and Security, cs.CR,Computer Science, Learning, cs.LG

Alldieck, T., Pons-Moll, G., Theobalt, C., & Magnor, M. A. (2019). Tex2Shape: Detailed Full Human Body Geometry from a Single Image. In International Conference on Computer Vision (ICCV 2019). Seoul, Korea: IEEE. doi:10.1109/ICCV.2019.00238

Abstract

We present a simple yet effective method to infer detailed full human body shape from only a single photograph. Our model can infer full-body shape including face, hair, and clothing including wrinkles at interactive frame-rates. Results feature details even on parts that are occluded in the input image. Our main idea is to turn shape regression into an aligned image-to-image translation problem. The input to our method is a partial texture map of the visible region obtained from off-the-shelf methods. From a partial texture, we estimate detailed normal and vector displacement maps, which can be applied to a low-resolution smooth body model to add detail and clothing. Despite being trained purely with synthetic data, our model generalizes well to real-world photographs. Numerous results demonstrate the versatility and robustness of our method.

Links

Export

BibTeX

@inproceedings{Alldieck_ICCV2019, TITLE = {{Tex2Shape}: Detailed Full Human Body Geometry from a Single Image}, AUTHOR = {Alldieck, Thiemo and Pons-Moll, Gerard and Theobalt, Christian and Magnor, Marcus A.}, LANGUAGE = {eng}, ISBN = {978-1-7281-4803-8}, DOI = {10.1109/ICCV.2019.00238}, PUBLISHER = {IEEE}, YEAR = {2019}, DATE = {2019}, ABSTRACT = {We present a simple yet effective method to infer detailed full human body shape from only a single photograph. Our model can infer full-body shape including face, hair, and clothing including wrinkles at interactive frame-rates. Results feature details even on parts that are occluded in the input image. Our main idea is to turn shape regression into an aligned image-to-image translation problem. The input to our method is a partial texture map of the visible region obtained from off-the-shelf methods. From a partial texture, we estimate detailed normal and vector displacement maps, which can be applied to a low-resolution smooth body model to add detail and clothing. Despite being trained purely with synthetic data, our model generalizes well to real-world photographs. Numerous results demonstrate the versatility and robustness of our method. }, BOOKTITLE = {International Conference on Computer Vision (ICCV 2019)}, PAGES = {2293--2303}, ADDRESS = {Seoul, Korea}, }

Endnote

%0 Conference Proceedings %A Alldieck, Thiemo %A Pons-Moll, Gerard %A Theobalt, Christian %A Magnor, Marcus A. %+ External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations %T Tex2Shape: Detailed Full Human Body Geometry from a Single Image : %G eng %U http://hdl.handle.net/21.11116/0000-0003-ECBE-E %R 10.1109/ICCV.2019.00238 %D 2019 %B International Conference on Computer Vision %Z date of event: 2019-10-27 - 2019-11-02 %C Seoul, Korea %X We present a simple yet effective method to infer detailed full human body shape from only a single photograph. Our model can infer full-body shape including face, hair, and clothing including wrinkles at interactive frame-rates. Results feature details even on parts that are occluded in the input image. Our main idea is to turn shape regression into an aligned image-to-image translation problem. The input to our method is a partial texture map of the visible region obtained from off-the-shelf methods. From a partial texture, we estimate detailed normal and vector displacement maps, which can be applied to a low-resolution smooth body model to add detail and clothing. Despite being trained purely with synthetic data, our model generalizes well to real-world photographs. Numerous results demonstrate the versatility and robustness of our method. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV %B International Conference on Computer Vision %P 2293 - 2303 %I IEEE %@ 978-1-7281-4803-8

Bhatnagar, B. L., Tiwari, G., Theobalt, C., & Pons-Moll, G. (2019). Multi-Garment Net: Learning to Dress 3D People from Images. In International Conference on Computer Vision (ICCV 2019). Seoul, Korea: IEEE. doi:10.1109/ICCV.2019.00552

Links

Export

BibTeX

@inproceedings{bhatnagar_ICCV2019, TITLE = {Multi-Garment {N}et: {L}earning to Dress {3D} People from Images}, AUTHOR = {Bhatnagar, Bharat Lal and Tiwari, Garvita and Theobalt, Christian and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-7281-4803-8}, DOI = {10.1109/ICCV.2019.00552}, PUBLISHER = {IEEE}, YEAR = {2019}, DATE = {2019}, BOOKTITLE = {International Conference on Computer Vision (ICCV 2019)}, PAGES = {5419--5429}, ADDRESS = {Seoul, Korea}, }

Endnote

%0 Conference Proceedings %A Bhatnagar, Bharat Lal %A Tiwari, Garvita %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Multi-Garment Net: Learning to Dress 3D People from Images : %G eng %U http://hdl.handle.net/21.11116/0000-0004-89E8-C %R 10.1109/ICCV.2019.00552 %D 2019 %B International Conference on Computer Vision %Z date of event: 2019-10-27 - 2019-11-02 %C Seoul, Korea %B International Conference on Computer Vision %P 5419 - 5429 %I IEEE %@ 978-1-7281-4803-8

Habermann, M., Xu, W., Rohdin, H., Zollhöfer, M., Pons-Moll, G., & Theobalt, C. (2019). NRST: Non-rigid Surface Tracking from Monocular Video. In Pattern Recognition (GCPR 2018). Stuttgart, Germany: Springer. doi:10.1007/978-3-030-12939-2_23

Links

Export

BibTeX

@inproceedings{Habermann_GVPR18, TITLE = {{NRST}: {N}on-rigid Surface Tracking from Monocular Video}, AUTHOR = {Habermann, Marc and Xu, Weipeng and Rohdin, Helge and Zollh{\"o}fer, Michael and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISBN = {978-3-030-12938-5}, DOI = {10.1007/978-3-030-12939-2_23}, PUBLISHER = {Springer}, YEAR = {2018}, DATE = {2019}, BOOKTITLE = {Pattern Recognition (GCPR 2018)}, EDITOR = {Brox, Thomas and Bruhn, Andr{\'e}s and Fritz, Mario}, PAGES = {335--348}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {11269}, ADDRESS = {Stuttgart, Germany}, }

Endnote

%0 Conference Proceedings %A Habermann, Marc %A Xu, Weipeng %A Rohdin, Helge %A Zollhöfer, Michael %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T NRST: Non-rigid Surface Tracking from Monocular Video : %G eng %U http://hdl.handle.net/21.11116/0000-0002-B94C-9 %R 10.1007/978-3-030-12939-2_23 %D 2019 %B 40th German Conference on Pattern Recognition %Z date of event: 2018-10-09 - 2018-10-12 %C Stuttgart, Germany %B Pattern Recognition %E Brox, Thomas; Bruhn, Andrés; Fritz, Mario %P 335 - 348 %I Springer %@ 978-3-030-12938-5 %B Lecture Notes in Computer Science %N 11269

Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Seidel, H.-P., Fua, P., … Theobalt, C. (2019). XNect Demo (v2): Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera. In CVPR 2019 Demonstrations. Long Beach, CA, USA.

Links

Export

BibTeX

@inproceedings{XNectDemoV2_CVPR2019, TITLE = {Demo of {VNect} (v2): {R}eal-time {3D} Human Pose Estimation with a Single {RGB} Camera}, AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Seidel, Hans-Peter and Fua, Pascal and Elgharib, Mohamed and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, YEAR = {2019}, BOOKTITLE = {CVPR 2019 Demonstrations}, ADDRESS = {Long Beach, CA, USA}, }

Endnote

%0 Conference Proceedings %A Mehta, Dushyant %A Sotnychenko, Oleksandr %A Mueller, Franziska %A Xu, Weipeng %A Seidel, Hans-Peter %A Fua, Pascal %A Elgharib, Mohamed %A Rhodin, Helge %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T XNect Demo (v2): Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0004-71DB-6 %D 2019 %B 32nd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2019-06-16 - 2019-06-20 %C Long Beach, CA, USA %B CVPR 2019 Demonstrations %U http://gvv.mpi-inf.mpg.de/projects/XNectDemoV2/http://gvv.mpi-inf.mpg.de/projects/XNectDemoV2/

Lazova, V., Insafutdinov, E., & Pons-Moll, G. (2019). 360-Degree Textures of People in Clothing from a Single Image. In International Conference on 3D Vision. Québec City, Canada: IEEE. doi:10.1109/3DV.2019.00076

Links

Export

BibTeX

@inproceedings{Lazova_3DV2019, TITLE = {360-Degree Textures of People in Clothing from a Single Image}, AUTHOR = {Lazova, Verica and Insafutdinov, Eldar and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-7281-3131-3}, DOI = {10.1109/3DV.2019.00076}, PUBLISHER = {IEEE}, YEAR = {2019}, DATE = {2019}, BOOKTITLE = {International Conference on 3D Vision}, PAGES = {643--653}, ADDRESS = {Qu{\'e}bec City, Canada}, }

Endnote

%0 Conference Proceedings %A Lazova, Verica %A Insafutdinov, Eldar %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T 360-Degree Textures of People in Clothing from a Single Image : %G eng %U http://hdl.handle.net/21.11116/0000-0005-46EF-0 %R 10.1109/3DV.2019.00076 %D 2019 %B International Conference on 3D Vision %Z date of event: 2019-09-16 - 2019-09-19 %C Québec City, Canada %B International Conference on 3D Vision %P 643 - 653 %I IEEE %@ 978-1-7281-3131-3

Habibie, I., Xu, W., Mehta, D., Pons-Moll, G., & Theobalt, C. (2019). In the Wild Human Pose Estimation using Explicit 2D Features and Intermediate 3D Representations. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019). Long Beach, CA, USA: IEEE. doi:10.1109/CVPR.2019.01116

Links

Export

BibTeX

@inproceedings{habibieCVPR19, TITLE = {In the Wild Human Pose Estimation using Explicit {2D} Features and Intermediate {3D} Representations}, AUTHOR = {Habibie, Ikhsanul and Xu, Weipeng and Mehta, Dushyant and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISBN = {978-1-7281-3293-8}, DOI = {10.1109/CVPR.2019.01116}, PUBLISHER = {IEEE}, YEAR = {2019}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)}, PAGES = {10897--10906}, ADDRESS = {Long Beach, CA, USA}, }

Endnote

%0 Conference Proceedings %A Habibie, Ikhsanul %A Xu, Weipeng %A Mehta, Dushyant %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T In the Wild Human Pose Estimation using Explicit 2D Features and Intermediate 3D Representations : %G eng %U http://hdl.handle.net/21.11116/0000-0003-6520-7 %R 10.1109/CVPR.2019.01116 %D 2019 %B 32nd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2019-06-15 - 2019-06-20 %C Long Beach, CA, USA %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 10897 - 10906 %I IEEE %@ 978-1-7281-3293-8

Sattar, H., Pons-Moll, G., & Fritz, M. (2019). Fashion is Taking Shape: Understanding Clothing Preference Based on Body Shape From Online Sources. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV 2019). Waikoloa Village, HI, USA: IEEE. doi:10.1109/WACV.2019.00108

Links

Export

BibTeX

@inproceedings{sattar19wacv, TITLE = {Fashion is Taking Shape: {U}nderstanding Clothing Preference Based on Body Shape From Online Sources}, AUTHOR = {Sattar, Hosnieh and Pons-Moll, Gerard and Fritz, Mario}, LANGUAGE = {eng}, ISBN = {978-1-7281-1975-5}, DOI = {10.1109/WACV.2019.00108}, PUBLISHER = {IEEE}, YEAR = {2019}, BOOKTITLE = {2019 IEEE Winter Conference on Applications of Computer Vision (WACV 2019)}, PAGES = {968--977}, ADDRESS = {Waikoloa Village, HI, USA}, }

Endnote

%0 Conference Proceedings %A Sattar, Hosnieh %A Pons-Moll, Gerard %A Fritz, Mario %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Fashion is Taking Shape: Understanding Clothing Preference Based on Body Shape From Online Sources : %G eng %U http://hdl.handle.net/21.11116/0000-0001-B309-B %R 10.1109/WACV.2019.00108 %D 2019 %B IEEE Winter Conference on Applications of Computer Vision %Z date of event: 2019-01-08 - 2019-01-10 %C Waikoloa Village, HI, USA %B 2019 IEEE Winter Conference on Applications of Computer Vision %P 968 - 977 %I IEEE %@ 978-1-7281-1975-5

Alldieck, T., Magnor, M. A., Bhatnagar, B. L., Theobalt, C., & Pons-Moll, G. (2019). Learning to Reconstruct People in Clothing from a Single RGB Camera. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019). Long Beach, CA, USA: IEEE. doi:10.1109/CVPR.2019.00127

Links

Export

BibTeX

@inproceedings{alldieck19cvpr, TITLE = {Learning to Reconstruct People in Clothing from a Single {RGB} Camera}, AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Bhatnagar, Bharat Lal and Theobalt, Christian and Pons-Moll, Gerard}, ISBN = {978-1-7281-3293-8}, DOI = {10.1109/CVPR.2019.00127}, PUBLISHER = {IEEE}, YEAR = {2019}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)}, PAGES = {1175--1186}, ADDRESS = {Long Beach, CA, USA}, }

Endnote

%0 Conference Proceedings %A Alldieck, Thiemo %A Magnor, Marcus A. %A Bhatnagar, Bharat Lal %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Learning to Reconstruct People in Clothing from a Single RGB Camera : %U http://hdl.handle.net/21.11116/0000-0003-5F97-9 %R 10.1109/CVPR.2019.00127 %D 2019 %B 32nd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2019-06-15 - 2019-06-20 %C Long Beach, CA, USA %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 1175 - 1186 %I IEEE %@ 978-1-7281-3293-8

Mahmood, N., Ghorbani, N., Troje, N. F., Pons-Moll, G., & Black, M. J. (2019). AMASS: Archive of Motion Capture as Surface Shapes. In International Conference on Computer Vision (ICCV 2019). Seoul, Korea: IEEE. doi:10.1109/ICCV.2019.00554

Links

Export

BibTeX

@inproceedings{Mahmood_ICCV2019, TITLE = {{AMASS}: Archive of Motion Capture as Surface Shapes}, AUTHOR = {Mahmood, Naureen and Ghorbani, Nima and Troje, Nikolaus F. and Pons-Moll, Gerard and Black, Michael J.}, LANGUAGE = {eng}, ISBN = {978-1-7281-4803-8}, DOI = {10.1109/ICCV.2019.00554}, PUBLISHER = {IEEE}, YEAR = {2019}, DATE = {2019}, BOOKTITLE = {International Conference on Computer Vision (ICCV 2019)}, PAGES = {5441--5450}, ADDRESS = {Seoul, Korea}, }

Endnote

%0 Conference Proceedings %A Mahmood, Naureen %A Ghorbani, Nima %A Troje, Nikolaus F. %A Pons-Moll, Gerard %A Black, Michael J. %+ External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T AMASS: Archive of Motion Capture as Surface Shapes : %G eng %U http://hdl.handle.net/21.11116/0000-0003-ECAB-3 %R 10.1109/ICCV.2019.00554 %D 2019 %B International Conference on Computer Vision %Z date of event: 2019-10-27 - 2019-11-02 %C Seoul, Korea %B International Conference on Computer Vision %P 5441 - 5450 %I IEEE %@ 978-1-7281-4803-8

Yu, T., Zheng, Z., Zhong, Y., Zhao, J., Quionhai, D., Pons-Moll, G., & Liu, Y. (2019). SimulCap : Single-View Human Performance Capture with Cloth Simulation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019). Long Beach, CA, USA: IEEE. doi:10.1109/CVPR.2019.00565

Links

Export

BibTeX

@inproceedings{SimulCap19, TITLE = {{SimulCap} : {S}ingle-View Human Performance Capture with Cloth Simulation}, AUTHOR = {Yu, Tao and Zheng, Zerong and Zhong, Yuan and Zhao, Jianhui and Quionhai, Dai and Pons-Moll, Gerard and Liu, Yebin}, LANGUAGE = {eng}, ISBN = {978-1-7281-3293-8}, DOI = {10.1109/CVPR.2019.00565}, PUBLISHER = {IEEE}, YEAR = {2019}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2019)}, PAGES = {5499--5509}, ADDRESS = {Long Beach, CA, USA}, }

Endnote

%0 Conference Proceedings %A Yu, Tao %A Zheng, Zerong %A Zhong, Yuan %A Zhao, Jianhui %A Quionhai, Dai %A Pons-Moll, Gerard %A Liu, Yebin %+ External Organizations External Organizations External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T SimulCap : Single-View Human Performance Capture with Cloth Simulation : %G eng %U http://hdl.handle.net/21.11116/0000-0003-651E-B %R 10.1109/CVPR.2019.00565 %D 2019 %B 32nd IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2019-06-16 - 2019-06-20 %C Long Beach, CA, USA %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 5499 - 5509 %I IEEE %@ 978-1-7281-3293-8

Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Elgharib, M., Fua, P., … Theobalt, C. (2019). XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera. Retrieved from http://arxiv.org/abs/1907.00837

(arXiv: 1907.00837)

Abstract

We present a real-time approach for multi-person 3D motion capture at over 30 fps using a single RGB camera. It operates in generic scenes and is robust to difficult occlusions both by other people and objects. Our method operates in subsequent stages. The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy. In the second stage, a fully-connected neural network turns the possibly partial (on account of occlusion) 2D pose and 3D pose features for each subject into a complete 3D pose estimate per individual. The third stage applies space-time skeletal model fitting to the predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose, and enforce temporal coherence. Our method returns the full skeletal pose in joint angles for each subject. This is a further key distinction from previous work that neither extracted global body positions nor joint angle results of a coherent skeleton in real time for multi-person scenes. The proposed system runs on consumer hardware at a previously unseen speed of more than 30 fps given 512x320 images as input while achieving state-of-the-art accuracy, which we will demonstrate on a range of challenging real-world scenes.

Links

Export

BibTeX

@online{Mehta_arXiv1907.00837, TITLE = {{XNect}: Real-time Multi-person {3D} Human Pose Estimation with a Single {RGB} Camera}, AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Elgharib, Mohamed and Fua, Pascal and Seidel, Hans-Peter and Rhodin, Helge and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, URL = {http://arxiv.org/abs/1907.00837}, EPRINT = {1907.00837}, EPRINTTYPE = {arXiv}, YEAR = {2019}, ABSTRACT = {We present a real-time approach for multi-person 3D motion capture at over 30 fps using a single RGB camera. It operates in generic scenes and is robust to difficult occlusions both by other people and objects. Our method operates in subsequent stages. The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy. In the second stage, a fully-connected neural network turns the possibly partial (on account of occlusion) 2D pose and 3D pose features for each subject into a complete 3D pose estimate per individual. The third stage applies space-time skeletal model fitting to the predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose, and enforce temporal coherence. Our method returns the full skeletal pose in joint angles for each subject. This is a further key distinction from previous work that neither extracted global body positions nor joint angle results of a coherent skeleton in real time for multi-person scenes. The proposed system runs on consumer hardware at a previously unseen speed of more than 30 fps given 512x320 images as input while achieving state-of-the-art accuracy, which we will demonstrate on a range of challenging real-world scenes. }, }

Endnote

%0 Report %A Mehta, Dushyant %A Sotnychenko, Oleksandr %A Mueller, Franziska %A Xu, Weipeng %A Elgharib, Mohamed %A Fua, Pascal %A Seidel, Hans-Peter %A Rhodin, Helge %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0003-FE21-A %U http://arxiv.org/abs/1907.00837 %D 2019 %X We present a real-time approach for multi-person 3D motion capture at over 30 fps using a single RGB camera. It operates in generic scenes and is robust to difficult occlusions both by other people and objects. Our method operates in subsequent stages. The first stage is a convolutional neural network (CNN) that estimates 2D and 3D pose features along with identity assignments for all visible joints of all individuals. We contribute a new architecture for this CNN, called SelecSLS Net, that uses novel selective long and short range skip connections to improve the information flow allowing for a drastically faster network without compromising accuracy. In the second stage, a fully-connected neural network turns the possibly partial (on account of occlusion) 2D pose and 3D pose features for each subject into a complete 3D pose estimate per individual. The third stage applies space-time skeletal model fitting to the predicted 2D and 3D pose per subject to further reconcile the 2D and 3D pose, and enforce temporal coherence. Our method returns the full skeletal pose in joint angles for each subject. This is a further key distinction from previous work that neither extracted global body positions nor joint angle results of a coherent skeleton in real time for multi-person scenes. The proposed system runs on consumer hardware at a previously unseen speed of more than 30 fps given 512x320 images as input while achieving state-of-the-art accuracy, which we will demonstrate on a range of challenging real-world scenes. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Graphics, cs.GR

Alldieck, T., Magnor, M. A., Xu, W., Theobalt, C., & Pons-Moll, G. (2018a). Video Based Reconstruction of 3D People Models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018). Salt Lake City, UT, USA: IEEE. doi:10.1109/CVPR.2018.00875

Links

Export

BibTeX

@inproceedings{alldieck2018video, TITLE = {Video Based Reconstruction of {3D} People Models}, AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Xu, Weipeng and Theobalt, Christian and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-5386-6420-9}, DOI = {10.1109/CVPR.2018.00875}, PUBLISHER = {IEEE}, YEAR = {2018}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)}, PAGES = {8387--8397}, ADDRESS = {Salt Lake City, UT, USA}, }

Endnote

%0 Conference Proceedings %A Alldieck, Thiemo %A Magnor, Marcus A. %A Xu, Weipeng %A Theobalt, Christian %A Pons-Moll, Gerard %+ External Organizations External Organizations External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Video Based Reconstruction of 3D People Models : %G eng %U http://hdl.handle.net/21.11116/0000-0001-1E24-6 %R 10.1109/CVPR.2018.00875 %D 2018 %B 31st IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2018-06-18 - 2018-06-22 %C Salt Lake City, UT, USA %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 8387 - 8397 %I IEEE %@ 978-1-5386-6420-9

Mehta, D., Sotnychenko, O., Mueller, F., Rhodin, H., Xu, W., Pons-Moll, G., & Theobalt, C. (2018). Demo of XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera. In ECCV 2018 Demo Sessions. Munich, Germany. Retrieved from http://gvv.mpi-inf.mpg.de/projects/XNectDemo/

Links

Export

BibTeX

@inproceedings{XNectDemo_ECCV2018, TITLE = {Demo of {XNect}: Real-time Multi-person {3D} Human Pose Estimation with a Single {RGB} Camera}, AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Rhodin, Helge and Xu, Weipeng and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, URL = {http://gvv.mpi-inf.mpg.de/projects/XNectDemo/}, YEAR = {2018}, BOOKTITLE = {ECCV 2018 Demo Sessions}, ADDRESS = {Munich, Germany}, }

Endnote

%0 Conference Proceedings %A Mehta, Dushyant %A Sotnychenko, Oleksandr %A Mueller, Franziska %A Rhodin, Helge %A Xu, Weipeng %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Demo of XNect: Real-time Multi-person 3D Human Pose Estimation with a Single RGB Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0002-F4DC-3 %U http://gvv.mpi-inf.mpg.de/projects/XNectDemo/ %D 2018 %B European Conference on Computer Vision %Z date of event: 2018-09-08 - 2018-09-14 %C Munich, Germany %B ECCV 2018 Demo Sessions %U http://gvv.mpi-inf.mpg.de/projects/XNectDemo/

Mehta, D., Sotnychenko, O., Mueller, F., Xu, W., Sridhar, S., Pons-Moll, G., & Theobalt, C. (2018). Single-Shot Multi-person 3D Pose Estimation from Monocular RGB. In 3DV 2018 , International Conference on 3D Vision. Verona, Italy: IEEE. doi:10.1109/3DV.2018.00024

Links

Export

BibTeX

@inproceedings{Mehta_3DV2018, TITLE = {Single-Shot Multi-person {3D} Pose Estimation from Monocular {RGB}}, AUTHOR = {Mehta, Dushyant and Sotnychenko, Oleksandr and Mueller, Franziska and Xu, Weipeng and Sridhar, Srinath and Pons-Moll, Gerard and Theobalt, Christian}, LANGUAGE = {eng}, ISBN = {978-1-5386-8425-2 ; 978-1-5386-8426-9}, DOI = {10.1109/3DV.2018.00024}, PUBLISHER = {IEEE}, YEAR = {2018}, DATE = {2018}, BOOKTITLE = {3DV 2018 , International Conference on 3D Vision}, PAGES = {120--130}, ADDRESS = {Verona, Italy}, }

Endnote

%0 Conference Proceedings %A Mehta, Dushyant %A Sotnychenko, Oleksandr %A Mueller, Franziska %A Xu, Weipeng %A Sridhar, Srinath %A Pons-Moll, Gerard %A Theobalt, Christian %+ Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society %T Single-Shot Multi-person 3D Pose Estimation from Monocular RGB : %G eng %U http://hdl.handle.net/21.11116/0000-0002-5C46-9 %R 10.1109/3DV.2018.00024 %D 2018 %B International Conference on 3D Vision %Z date of event: 2018-09-05 - 2018-09-08 %C Verona, Italy %B 3DV 2018 %P 120 - 130 %I IEEE %@ 978-1-5386-8425-2 978-1-5386-8426-9

Yu, T., Zheng, Z., Guo, K., Zhao, J., Dai, Q., Li, H., … Liu, Y. (2018). DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018). Salt Lake City, UT, USA: IEEE. doi:10.1109/CVPR.2018.00761

Links

Export

BibTeX

@inproceedings{DoubleFusion, TITLE = {{DoubleFusion}: {R}eal-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor}, AUTHOR = {Yu, Tao and Zheng, Zherong and Guo, Kaiwen and Zhao, Jianhui and Dai, Qionghai and Li, Hao and Pons-Moll, Gerard and Liu, Yebin}, LANGUAGE = {eng}, ISBN = {978-1-5386-6420-9}, DOI = {10.1109/CVPR.2018.00761}, PUBLISHER = {IEEE}, YEAR = {2018}, BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018)}, PAGES = {7287--7296}, ADDRESS = {Salt Lake City, UT, USA}, }

Endnote

%0 Conference Proceedings %A Yu, Tao %A Zheng, Zherong %A Guo, Kaiwen %A Zhao, Jianhui %A Dai, Qionghai %A Li, Hao %A Pons-Moll, Gerard %A Liu, Yebin %+ External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations %T DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensor : %G eng %U http://hdl.handle.net/21.11116/0000-0001-1E30-8 %R 10.1109/CVPR.2018.00761 %D 2018 %B 31st IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2018-06-18 - 2018-06-22 %C Salt Lake City, UT, USA %B IEEE/CVF Conference on Computer Vision and Pattern Recognition %P 7287 - 7296 %I IEEE %@ 978-1-5386-6420-9

Omran, M., Lassner,, C., Pons-Moll, G., Gehler, P., & Schiele, B. (2018). Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation. In 3DV 2018 , International Conference on 3D Vision. Verona, Italy: IEEE. doi:10.1109/3DV.2018.00062

Links

Export

BibTeX

@inproceedings{omran2018nbf, TITLE = {Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation}, AUTHOR = {Omran, Mohamed and Lassner,, Christoph and Pons-Moll, Gerard and Gehler, Peter and Schiele, Bernt}, LANGUAGE = {eng}, ISBN = {978-1-5386-8425-2 ; 978-1-5386-8426-9}, DOI = {10.1109/3DV.2018.00062}, PUBLISHER = {IEEE}, YEAR = {2018}, DATE = {2018}, BOOKTITLE = {3DV 2018 , International Conference on 3D Vision}, PAGES = {484--494}, ADDRESS = {Verona, Italy}, }

Endnote

%0 Conference Proceedings %A Omran, Mohamed %A Lassner,, Christoph %A Pons-Moll, Gerard %A Gehler, Peter %A Schiele, Bernt %+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Neural Body Fitting: Unifying Deep Learning and Model Based Human Pose and Shape Estimation : %G eng %U http://hdl.handle.net/21.11116/0000-0001-E564-C %R 10.1109/3DV.2018.00062 %D 2018 %B International Conference on 3D Vision %Z date of event: 2018-09-05 - 2018-09-08 %C Verona, Italy %B 3DV 2018 %P 484 - 494 %I IEEE %@ 978-1-5386-8425-2 978-1-5386-8426-9

Huang, Y., Kaufmann, M., Aksan, E., Black, M. J., Hilliges, O., & Pons-Moll, G. (2018). Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time. ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia 2018), 37(6). doi:10.1145/3272127.3275108

Links

Export

BibTeX

@article{DIP:SIGGRAPHAsia:2018, TITLE = {Deep Inertial Poser: {L}earning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time}, AUTHOR = {Huang, Yinghao and Kaufmann, Manuel and Aksan, Emre and Black, Michael J. and Hilliges, Otmar and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISSN = {0730-0301}, ISBN = {978-1-4503-6008-1}, DOI = {10.1145/3272127.3275108}, PUBLISHER = {ACM}, ADDRESS = {New York, NY}, YEAR = {2018}, DATE = {2018}, JOURNAL = {ACM Transactions on Graphics (Proc. ACM SIGGRAPH Asia)}, VOLUME = {37}, NUMBER = {6}, EID = {185}, BOOKTITLE = {Proceedings of ACM SIGGRAPH Asia 2018}, }

Endnote

%0 Journal Article %A Huang, Yinghao %A Kaufmann, Manuel %A Aksan, Emre %A Black, Michael J. %A Hilliges, Otmar %A Pons-Moll, Gerard %+ External Organizations External Organizations External Organizations External Organizations External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Deep Inertial Poser: Learning to Reconstruct Human Pose from Sparse Inertial Measurements in Real Time : %G eng %U http://hdl.handle.net/21.11116/0000-0002-9C1E-E %R 10.1145/3272127.3275108 %7 2018 %D 2018 %J ACM Transactions on Graphics %O TOG %V 37 %N 6 %Z sequence number: 185 %I ACM %C New York, NY %@ false %B Proceedings of ACM SIGGRAPH Asia 2018 %O ACM SIGGRAPH Asia 2018 Tokyo, Japan, December 04 - 07, 2018 SA'18 SA 2018 %@ 978-1-4503-6008-1

Von Marcard, T., Henschel, R., Black, M. J., Rosenhahn, B., & Pons-Moll, G. (2018). Recovering Accurate {3D} Human Pose in the Wild Using {IMUs} and a Moving Camera. In Computer Vision -- ECCV 2018. Munich, Germany: Springer. doi:10.1007/978-3-030-01249-6_37

Links

Export

BibTeX

@inproceedings{Marcard_ECCV2018, TITLE = {Recovering Accurate {\textbraceleft}{3D}{\textbraceright} Human Pose in the Wild Using {\textbraceleft}{IMUs}{\textbraceright} and a Moving Camera}, AUTHOR = {von Marcard, Timo and Henschel, Roberto and Black, Michael J. and Rosenhahn, Bodo and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-3-030-01248-9}, DOI = {10.1007/978-3-030-01249-6_37}, PUBLISHER = {Springer}, YEAR = {2018}, DATE = {2018}, BOOKTITLE = {Computer Vision -- ECCV 2018}, PAGES = {614--631}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {11214}, ADDRESS = {Munich, Germany}, }

Endnote

%0 Conference Proceedings %A von Marcard, Timo %A Henschel, Roberto %A Black, Michael J. %A Rosenhahn, Bodo %A Pons-Moll, Gerard %+ External Organizations External Organizations External Organizations External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Recovering Accurate {3D} Human Pose in the Wild Using {IMUs} and a Moving Camera : %G eng %U http://hdl.handle.net/21.11116/0000-0002-5B61-B %R 10.1007/978-3-030-01249-6_37 %D 2018 %B 15th European Conference on Computer Vision %Z date of event: 2018-09-08 - 2018-09-14 %C Munich, Germany %B Computer Vision -- ECCV 2018 %P 614 - 631 %I Springer %@ 978-3-030-01248-9 %B Lecture Notes in Computer Science %N 11214

Alldieck, T., Magnor, M. A., Xu, W., Theobalt, C., & Pons-Moll, G. (2018b). Detailed Human Avatars from Monocular Video. In 3DV 2018 , International Conference on 3D Vision. Verona, Italy: IEEE. doi:10.1109/3DV.2018.00022

Links

Export

BibTeX

@inproceedings{Alldieck_3DV2018, TITLE = {Detailed Human Avatars from Monocular Video}, AUTHOR = {Alldieck, Thiemo and Magnor, Marcus A. and Xu, Weipeng and Theobalt, Christian and Pons-Moll, Gerard}, LANGUAGE = {eng}, ISBN = {978-1-5386-8425-2 ; 978-1-5386-8426-9}, DOI = {10.1109/3DV.2018.00022}, PUBLISHER = {IEEE}, YEAR = {2018}, DATE = {2018}, BOOKTITLE = {3DV 2018 , International Conference on 3D Vision}, PAGES = {98--109}, ADDRESS = {Verona, Italy}, }

Endnote

%0 Conference Proceedings %A Alldieck, Thiemo %A Magnor, Marcus A. %A Xu, Weipeng %A Theobalt, Christian %A Pons-Moll, Gerard %+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations Computer Graphics, MPI for Informatics, Max Planck Society Computer Graphics, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Detailed Human Avatars from Monocular Video : %G eng %U http://hdl.handle.net/21.11116/0000-0002-5C40-F %R 10.1109/3DV.2018.00022 %D 2018 %B International Conference on 3D Vision %Z date of event: 2018-09-05 - 2018-09-08 %C Verona, Italy %B 3DV 2018 %P 98 - 109 %I IEEE %@ 978-1-5386-8425-2 978-1-5386-8426-9