Jan Eric Lenssen (Senior Researcher)

Dr. Jan Eric Lenssen

Address: Max-Planck-Institut für Informatik
Saarland Informatics Campus
Campus E1 4
66123 Saarbrücken
Location: E1 4 - 618
Phone: +49 681 9325 2141
Fax: +49 681 9325 2099
E-mail: Get email via email

Personal Information

Please visit my group website for further information!

Selected Publications

For a full publication list, visit my group website!

2024

Article

D2D6

R. Yunus, J. E. Lenssen, M. Niemeyer, Y. Liao, C. Rupprecht, C. Theobalt, G. Pons-Moll, J.-B. Huang, V. Golyanik, and E. Ilg

“Recent Trends in 3D Reconstruction of General Non-Rigid Scenes,” Computer Graphics Forum (Proc. EUROGRAPHICS 2024), vol. 43.

@article{yunus24star,
TITLE = {Recent Trends in {3D} Reconstruction of General Non-Rigid Scenes},
AUTHOR = {Yunus, Raza and Lenssen, Jan Eric and Niemeyer, Michael and Liao, Yiyi and Rupprecht, Christian and Theobalt, Christian and Pons-Moll, Gerard and Huang, Jia-Bin and Golyanik, Vladislav and Ilg, Eddy},
LANGUAGE = {eng},
ISSN = {0167-7055},
PUBLISHER = {Blackwell-Wiley},
ADDRESS = {Oxford},
YEAR = {2024},
PUBLREMARK = {Accepted},
JOURNAL = {Computer Graphics Forum (Proc. EUROGRAPHICS)},
VOLUME = {43},
BOOKTITLE = {EUROGRAPHICS 2024 STAR -- State of The Art Reports},
}

Endnote

%0 Journal Article
%A Yunus, Raza
%A Lenssen, Jan Eric
%A Niemeyer, Michael
%A Liao, Yiyi
%A Rupprecht, Christian
%A Theobalt, Christian
%A Pons-Moll, Gerard
%A Huang, Jia-Bin
%A Golyanik, Vladislav
%A Ilg, Eddy
%+ External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Visual Computing and Artificial Intelligence, MPI for Informatics, Max Planck Society
External Organizations
%T Recent Trends in 3D Reconstruction of General Non-Rigid Scenes : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-B1CE-2
%D 2024
%J Computer Graphics Forum
%O Computer Graphics Forum : journal of the European Association for Computer Graphics Comput. Graph. Forum
%V 43
%I Blackwell-Wiley
%C Oxford
%@ false
%B EUROGRAPHICS 2024 STAR &#8211; State of The Art Reports
%O EUROGRAPHICS 2024 EG 2024 45th Annual Conference of the European Association for Computer Graphics ; Limassol, Cyprus, April 22-26

Conference paper

D. Das, C. Wewer, R. Yunus, E. Ilg, and J. E. Lenssen

“Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), Seattle, WA, USA.

@inproceedings{2312.01196,
TITLE = {Neural Parametric {G}aussians for Monocular Non-Rigid Object Reconstruction},
AUTHOR = {Das, Devikalyan and Wewer, Christopher and Yunus, Raza and Ilg, Eddy and Lenssen, Jan Eric},
LANGUAGE = {eng},
PUBLISHER = {IEEE},
YEAR = {2024},
PUBLREMARK = {Accepted},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)},
ADDRESS = {Seattle, WA, USA},
}

Endnote

%0 Conference Proceedings
%A Das, Devikalyan
%A Wewer, Christopher
%A Yunus, Raza
%A Ilg, Eddy
%A Lenssen, Jan Eric
%+ External Organizations
External Organizations
External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Neural Parametric Gaussians for Monocular Non-Rigid Object
  Reconstruction : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-B10E-B
%D 2024
%B 37th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2024-06-17 - 2024-06-21
%C Seattle, WA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%I IEEE

Conference paper

Y. He, G. Tiwari, T. Birdal, J. E. Lenssen, and G. Pons-Moll

“NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priors,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), Seattle, WA, USA.

@inproceedings{he24nrdf,
TITLE = {{NRDF}: {N}eural {R}iemannian Distance Fields for Learning Articulated Pose Priors},
AUTHOR = {He, Yannan and Tiwari, Garvita and Birdal, Tolga and Lenssen, Jan Eric and Pons-Moll, Gerard},
LANGUAGE = {eng},
PUBLISHER = {IEEE},
YEAR = {2024},
PUBLREMARK = {Accepted},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)},
ADDRESS = {Seattle, WA, USA},
}

Endnote

%0 Conference Proceedings
%A He, Yannan
%A Tiwari, Garvita
%A Birdal, Tolga
%A Lenssen, Jan Eric
%A Pons-Moll, Gerard
%+ External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priors : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-B120-5
%D 2024
%B 37th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2024-06-17 - 2024-06-21
%C Seattle, WA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%I IEEE

Conference paper

P. Schröppel, C. Wewer, J. E. Lenssen, E. Ilg, and T. Brox

“Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), Seattle, WA, USA.

@inproceedings{schroppel24npcd,
TITLE = {Neural Point Cloud Diffusion for Disentangled {3D} Shape and Appearance Generation},
AUTHOR = {Schr{\"o}ppel, Philipp and Wewer, Christopher and Lenssen, Jan Eric and Ilg, Eddy and Brox, Thomas},
LANGUAGE = {eng},
YEAR = {2024},
PUBLREMARK = {Accepted},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)},
ADDRESS = {Seattle, WA, USA},
}

Endnote

%0 Conference Proceedings
%A Schr&#246;ppel, Philipp
%A Wewer, Christopher
%A Lenssen, Jan Eric
%A Ilg, Eddy
%A Brox, Thomas
%+ External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-B0D8-7
%D 2024
%B 37th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2024-06-17 - 2024-06-21
%C Seattle, WA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition

Conference paper

X. Xie, B. L. Bhatnagar, J. E. Lenssen, and G. Pons-Moll

“Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), Seattle, WA, USA.

@inproceedings{xie24hdm,
TITLE = {Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation},
AUTHOR = {Xie, Xianghui and Bhatnagar, Bharat Lal and Lenssen, Jan Eric and Pons-Moll, Gerard},
LANGUAGE = {eng},
PUBLISHER = {IEEE},
YEAR = {2024},
PUBLREMARK = {Accepted},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)},
ADDRESS = {Seattle, WA, USA},
}

Endnote

%0 Conference Proceedings
%A Xie, Xianghui
%A Bhatnagar, Bharat Lal
%A Lenssen, Jan Eric
%A Pons-Moll, Gerard
%+ External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
%T Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-B0C2-F
%D 2024
%B 37th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2024-06-17 - 2024-06-21
%C Seattle, WA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%I IEEE
%U https://virtualhumans.mpi-inf.mpg.de/procigen-hdm

Conference paper

K. Zhou, B. L. Bhatnagar, J. E. Lenssen, and G. Pons-Moll

“GEARS: Local Geometry-aware Hand-object Interaction Synthesis,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), Seattle, WA, USA.

@inproceedings{zhou24gears,
TITLE = {{GEARS}: {L}ocal Geometry-aware Hand-object Interaction Synthesis},
AUTHOR = {Zhou, Keyang and Bhatnagar, Bharat Lal and Lenssen, Jan Eric and Pons-Moll, Gerard},
LANGUAGE = {eng},
PUBLISHER = {IEEE},
YEAR = {2024},
PUBLREMARK = {Accepted},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)},
ADDRESS = {Seattle, WA, USA},
}

Endnote

%0 Conference Proceedings
%A Zhou, Keyang
%A Bhatnagar, Bharat Lal
%A Lenssen, Jan Eric
%A Pons-Moll, Gerard
%+ External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
%T GEARS: Local Geometry-aware Hand-object Interaction Synthesis : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000E-B1C4-C
%D 2024
%B 37th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2024-06-17 - 2024-06-21
%C Seattle, WA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%I IEEE

2023

Conference paper

C. Wewer, E. Ilg, B. Schiele, and J. E. Lenssen

“SimNP: Learning Self-Similarity Priors Between Neural Points,” in IEEE/CVF International Conference on Computer Vision (ICCV 2023), Paris, France, 2023.

@inproceedings{Wewer_ICCV23,
TITLE = {{SimNP}: {L}earning Self-Similarity Priors Between Neural Points},
AUTHOR = {Wewer, Christopher and Ilg, Eddy and Schiele, Bernt and Lenssen, Jan Eric},
LANGUAGE = {eng},
ISBN = {979-8-3503-0718-4},
DOI = {10.1109/ICCV51070.2023.00812},
PUBLISHER = {IEEE},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {IEEE/CVF International Conference on Computer Vision (ICCV 2023)},
PAGES = {8807--8818},
ADDRESS = {Paris, France},
}

Endnote

%0 Conference Proceedings
%A Wewer, Christopher
%A Ilg, Eddy
%A Schiele, Bernt
%A Lenssen, Jan Eric
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T SimNP: Learning Self-Similarity Priors Between Neural Points : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000D-C95E-8
%R 10.1109/ICCV51070.2023.00812
%D 2023
%B IEEE/CVF International Conference on Computer Vision
%Z date of event: 2023-10-02 - 2023-10-06
%C Paris, France
%B IEEE/CVF International Conference on Computer Vision
%P 8807 - 8818
%I IEEE
%@ 979-8-3503-0718-4

2022

Conference paper

G. Tiwari, D. Antic, J. E. Lenssen, N. Sarafianos, T. Tung, and G. Pons-Moll

“Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields,” in Computer Vision -- ECCV 2022, Tel Aviv, Israel, 2022.

@inproceedings{Tiwari_ECCV22,
TITLE = {Pose-{NDF}: {M}odeling Human Pose Manifolds with Neural Distance Fields},
AUTHOR = {Tiwari, Garvita and Antic, Dimitrije and Lenssen, Jan Eric and Sarafianos, Nikolaos and Tung, Tony and Pons-Moll, Gerard},
LANGUAGE = {eng},
ISBN = {10.1007/978-3-031-20065-6{\textunderscore}33},
DOI = {10.1007/978-3-031-20065-6_33},
PUBLISHER = {Springer},
YEAR = {2022},
MARGINALMARK = {$\bullet$},
DATE = {2022},
BOOKTITLE = {Computer Vision -- ECCV 2022},
EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal},
PAGES = {572--589},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {13665},
ADDRESS = {Tel Aviv, Israel},
}

Endnote

%0 Conference Proceedings
%A Tiwari, Garvita
%A Antic, Dimitrije
%A Lenssen, Jan Eric
%A Sarafianos, Nikolaos
%A Tung, Tony
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fields : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-B582-6
%R 10.1007/978-3-031-20065-6_33
%D 2022
%B 17th European Conference on Computer Vision
%Z date of event: 2022-10-23 - 2022-10-27
%C Tel Aviv, Israel
%B Computer Vision -- ECCV 2022
%E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni; Hassner, Tal
%P 572 - 589
%I Springer
%@ 10.1007/978-3-031-20065-6_33
%B Lecture Notes in Computer Science
%N 13665
%U https://rdcu.be/c26RY

Conference paper

K. Zhou, B. L. Bhatnagar, J. E. Lenssen, and G. Pons-Moll

“TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement,” in Computer Vision -- ECCV 2022, Tel Aviv, Israel, 2022.

@inproceedings{Zhou_ECCV2022,
TITLE = {{TOCH}: {S}patio-Temporal Object Correspondence to Hand for Motion Refinement},
AUTHOR = {Zhou, Keyang and Bhatnagar, Bharat Lal and Lenssen, Jan Eric and Pons-Moll, Gerard},
LANGUAGE = {eng},
ISBN = {978-3-031-20061-8},
DOI = {10.1007/978-3-031-20062-5_1},
PUBLISHER = {Springer},
YEAR = {2022},
MARGINALMARK = {$\bullet$},
DATE = {2022},
BOOKTITLE = {Computer Vision -- ECCV 2022},
EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal},
PAGES = {1--19},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {13663},
ADDRESS = {Tel Aviv, Israel},
}

Endnote

%0 Conference Proceedings
%A Zhou, Keyang
%A Bhatnagar, Bharat Lal
%A Lenssen, Jan Eric
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T TOCH: Spatio-Temporal Object Correspondence to Hand for Motion
  Refinement : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-B586-2
%R 10.1007/978-3-031-20062-5_1
%D 2022
%B 17th European Conference on Computer Vision
%Z date of event: 2022-10-23 - 2022-10-27
%C Tel Aviv, Israel
%B Computer Vision -- ECCV 2022
%E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni; Hassner, Tal
%P 1 - 19
%I Springer
%@ 978-3-031-20061-8
%B Lecture Notes in Computer Science
%N 13663
%U https://rdcu.be/c26JY

Paper

K. Zhou, B. Lal Bhatnagar, J. E. Lenssen, and G. Pons-Moll

“TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement,” 2022. [Online]. Available: https://arxiv.org/abs/2205.07982.

Abstract

We present TOCH, a method for refining incorrect 3D hand-object interaction
sequences using a data prior. Existing hand trackers, especially those that
rely on very few cameras, often produce visually unrealistic results with
hand-object intersection or missing contacts. Although correcting such errors
requires reasoning about temporal aspects of interaction, most previous work
focus on static grasps and contacts. The core of our method are TOCH fields, a
novel spatio-temporal representation for modeling correspondences between hands
and objects during interaction. The key component is a point-wise
object-centric representation which encodes the hand position relative to the
object. Leveraging this novel representation, we learn a latent manifold of
plausible TOCH fields with a temporal denoising auto-encoder. Experiments
demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object
interaction models, which are limited to static grasps and contacts. More
importantly, our method produces smooth interactions even before and after
contact. Using a single trained TOCH model, we quantitatively and qualitatively
demonstrate its usefulness for 1) correcting erroneous reconstruction results
from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising,
and 3) grasp transfer across objects. We will release our code and trained
model on our project page at virtualhumans.mpi-inf.mpg.de/toch/

BibTeX

@online{Zhou_2205.07982,
TITLE = {{TOCH}: Spatio-Temporal Object Correspondence to Hand for Motion Refinement},
AUTHOR = {Zhou, Keyang and Lal Bhatnagar, Bharat and Lenssen, Jan Eric and Pons-Moll, Gerard},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2205.07982},
EPRINT = {2205.07982},
EPRINTTYPE = {arXiv},
YEAR = {2022},
MARGINALMARK = {$\bullet$},
ABSTRACT = {We present TOCH, a method for refining incorrect 3D hand-object interaction<br>sequences using a data prior. Existing hand trackers, especially those that<br>rely on very few cameras, often produce visually unrealistic results with<br>hand-object intersection or missing contacts. Although correcting such errors<br>requires reasoning about temporal aspects of interaction, most previous work<br>focus on static grasps and contacts. The core of our method are TOCH fields, a<br>novel spatio-temporal representation for modeling correspondences between hands<br>and objects during interaction. The key component is a point-wise<br>object-centric representation which encodes the hand position relative to the<br>object. Leveraging this novel representation, we learn a latent manifold of<br>plausible TOCH fields with a temporal denoising auto-encoder. Experiments<br>demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object<br>interaction models, which are limited to static grasps and contacts. More<br>importantly, our method produces smooth interactions even before and after<br>contact. Using a single trained TOCH model, we quantitatively and qualitatively<br>demonstrate its usefulness for 1) correcting erroneous reconstruction results<br>from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising,<br>and 3) grasp transfer across objects. We will release our code and trained<br>model on our project page at http://virtualhumans.mpi-inf.mpg.de/toch/<br>},
}

Endnote

%0 Report
%A Zhou, Keyang
%A Lal Bhatnagar, Bharat
%A Lenssen, Jan Eric
%A Pons-Moll, Gerard
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T TOCH: Spatio-Temporal Object Correspondence to Hand for Motion
  Refinement : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-ACF3-2
%U https://arxiv.org/abs/2205.07982
%D 2022
%X   We present TOCH, a method for refining incorrect 3D hand-object interaction<br>sequences using a data prior. Existing hand trackers, especially those that<br>rely on very few cameras, often produce visually unrealistic results with<br>hand-object intersection or missing contacts. Although correcting such errors<br>requires reasoning about temporal aspects of interaction, most previous work<br>focus on static grasps and contacts. The core of our method are TOCH fields, a<br>novel spatio-temporal representation for modeling correspondences between hands<br>and objects during interaction. The key component is a point-wise<br>object-centric representation which encodes the hand position relative to the<br>object. Leveraging this novel representation, we learn a latent manifold of<br>plausible TOCH fields with a temporal denoising auto-encoder. Experiments<br>demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object<br>interaction models, which are limited to static grasps and contacts. More<br>importantly, our method produces smooth interactions even before and after<br>contact. Using a single trained TOCH model, we quantitatively and qualitatively<br>demonstrate its usefulness for 1) correcting erroneous reconstruction results<br>from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising,<br>and 3) grasp transfer across objects. We will release our code and trained<br>model on our project page at http://virtualhumans.mpi-inf.mpg.de/toch/<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV