Xinting Hu (Postdoc)

Personal Information

Publications

2025

Conference paper

X. Hu, H. Wang, J. E. Lenssen, and B. Schiele

“PersonaHOI: Effortlessly Improving Personalized Face with Human-Object Interaction Generation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025), Nashville, TN, USA.

Abstract

We introduce PersonaHOI, a training- and tuning-free framework that fuses a
general StableDiffusion model with a personalized face diffusion (PFD) model to
generate identity-consistent human-object interaction (HOI) images. While
existing PFD models have advanced significantly, they often overemphasize
facial features at the expense of full-body coherence, PersonaHOI introduces an
additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By
incorporating cross-attention constraints in the PFD branch and spatial merging
at both latent and residual levels, PersonaHOI preserves personalized facial
details while ensuring interactive non-facial regions. Experiments, validated
by a novel interaction alignment metric, demonstrate the superior realism and
scalability of PersonaHOI, establishing a new standard for practical
personalized face with HOI generation. Our code will be available at
github.com/JoyHuYY1412/PersonaHOI

BibTeX

@inproceedings{Hu_CVPR25,
TITLE = {{PersonaHOI}: {E}ffortlessly Improving Personalized Face with Human-Object Interaction Generation},
AUTHOR = {Hu, Xinting and Wang, Haoran and Lenssen, Jan Eric and Schiele, Bernt},
LANGUAGE = {eng},
PUBLISHER = {IEEE},
YEAR = {2025},
PUBLREMARK = {Accepted},
MARGINALMARK = {$\bullet$},
ABSTRACT = {We introduce PersonaHOI, a training- and tuning-free framework that fuses a<br>general StableDiffusion model with a personalized face diffusion (PFD) model to<br>generate identity-consistent human-object interaction (HOI) images. While<br>existing PFD models have advanced significantly, they often overemphasize<br>facial features at the expense of full-body coherence, PersonaHOI introduces an<br>additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By<br>incorporating cross-attention constraints in the PFD branch and spatial merging<br>at both latent and residual levels, PersonaHOI preserves personalized facial<br>details while ensuring interactive non-facial regions. Experiments, validated<br>by a novel interaction alignment metric, demonstrate the superior realism and<br>scalability of PersonaHOI, establishing a new standard for practical<br>personalized face with HOI generation. Our code will be available at<br>https://github.com/JoyHuYY1412/PersonaHOI<br>},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)},
ADDRESS = {Nashville, TN, USA},
}

Endnote

%0 Conference Proceedings
%A Hu, Xinting
%A Wang, Haoran
%A Lenssen, Jan Eric
%A Schiele, Bernt
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T PersonaHOI: Effortlessly Improving Personalized Face with Human-Object
  Interaction Generation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-793D-3
%D 2025
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2025-06-11 - 2025-06-15
%C Nashville, TN, USA
%X   We introduce PersonaHOI, a training- and tuning-free framework that fuses a<br>general StableDiffusion model with a personalized face diffusion (PFD) model to<br>generate identity-consistent human-object interaction (HOI) images. While<br>existing PFD models have advanced significantly, they often overemphasize<br>facial features at the expense of full-body coherence, PersonaHOI introduces an<br>additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By<br>incorporating cross-attention constraints in the PFD branch and spatial merging<br>at both latent and residual levels, PersonaHOI preserves personalized facial<br>details while ensuring interactive non-facial regions. Experiments, validated<br>by a novel interaction alignment metric, demonstrate the superior realism and<br>scalability of PersonaHOI, establishing a new standard for practical<br>personalized face with HOI generation. Our code will be available at<br>https://github.com/JoyHuYY1412/PersonaHOI<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%I IEEE

Conference paper

Y. Wu, X. Hu, Y. Sun, Y. Zhou, W. Zhu, F. Rao, B. Schiele, and X. Yang

“Number it: Temporal Grounding Videos like Flipping Manga,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025), Nashville, TN, USA.

Abstract

BibTeX

@inproceedings{Wu_CVPR25,
TITLE = {Number it: {T}emporal Grounding Videos like Flipping {M}anga},
AUTHOR = {Wu, Yongliang and Hu, Xinting and Sun, Yuyang and Zhou, Yizhou and Zhu, Wenbo and Rao, Fengyun and Schiele, Bernt and Yang, Xu},
LANGUAGE = {eng},
PUBLISHER = {IEEE},
YEAR = {2025},
PUBLREMARK = {Accepted},
MARGINALMARK = {$\bullet$},
ABSTRACT = {We introduce PersonaHOI, a training- and tuning-free framework that fuses a<br>general StableDiffusion model with a personalized face diffusion (PFD) model to<br>generate identity-consistent human-object interaction (HOI) images. While<br>existing PFD models have advanced significantly, they often overemphasize<br>facial features at the expense of full-body coherence, PersonaHOI introduces an<br>additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By<br>incorporating cross-attention constraints in the PFD branch and spatial merging<br>at both latent and residual levels, PersonaHOI preserves personalized facial<br>details while ensuring interactive non-facial regions. Experiments, validated<br>by a novel interaction alignment metric, demonstrate the superior realism and<br>scalability of PersonaHOI, establishing a new standard for practical<br>personalized face with HOI generation. Our code will be available at<br>https://github.com/JoyHuYY1412/PersonaHOI<br>},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025)},
ADDRESS = {Nashville, TN, USA},
}

Endnote

%0 Conference Proceedings
%A Wu, Yongliang
%A Hu, Xinting
%A Sun, Yuyang
%A Zhou, Yizhou
%A Zhu, Wenbo
%A Rao, Fengyun
%A Schiele, Bernt
%A Yang, Xu
%+ External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
%T Number it: Temporal Grounding Videos like Flipping Manga : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-0D30-9
%D 2025
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2025-06-11 - 2025-06-15
%C Nashville, TN, USA
%X   We introduce PersonaHOI, a training- and tuning-free framework that fuses a<br>general StableDiffusion model with a personalized face diffusion (PFD) model to<br>generate identity-consistent human-object interaction (HOI) images. While<br>existing PFD models have advanced significantly, they often overemphasize<br>facial features at the expense of full-body coherence, PersonaHOI introduces an<br>additional StableDiffusion (SD) branch guided by HOI-oriented text inputs. By<br>incorporating cross-attention constraints in the PFD branch and spatial merging<br>at both latent and residual levels, PersonaHOI preserves personalized facial<br>details while ensuring interactive non-facial regions. Experiments, validated<br>by a novel interaction alignment metric, demonstrate the superior realism and<br>scalability of PersonaHOI, establishing a new standard for practical<br>personalized face with HOI generation. Our code will be available at<br>https://github.com/JoyHuYY1412/PersonaHOI<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%I IEEE

Paper

Y. Wu, Z. Li, X. Hu, X. Ye, X. Zeng, G. Yu, W. Zhu, B. Schiele, M.-H. Yang, and X. Yang

“KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models,” 2025. [Online]. Available: https://arxiv.org/abs/2505.16707.

Abstract

Recent advances in multi-modal generative models have enabled significant
progress in instruction-based image editing. However, while these models
produce visually plausible outputs, their capacity for knowledge-based
reasoning editing tasks remains under-explored. In this paper, we introduce
KRIS-Bench (Knowledge-based Reasoning in Image-editing Systems Benchmark), a
diagnostic benchmark designed to assess models through a cognitively informed
lens. Drawing from educational theory, KRIS-Bench categorizes editing tasks
across three foundational knowledge types: Factual, Conceptual, and Procedural.
Based on this taxonomy, we design 22 representative tasks spanning 7 reasoning
dimensions and release 1,267 high-quality annotated editing instances. To
support fine-grained evaluation, we propose a comprehensive protocol that
incorporates a novel Knowledge Plausibility metric, enhanced by knowledge hints
and calibrated through human studies. Empirical results on 10 state-of-the-art
models reveal significant gaps in reasoning performance, highlighting the need
for knowledge-centric benchmarks to advance the development of intelligent
image editing systems.

BibTeX

@online{Wu_2505.16707,
TITLE = {{KRIS}-Bench: Benchmarking Next-Level Intelligent Image Editing Models},
AUTHOR = {Wu, Yongliang and Li, Zonghui and Hu, Xinting and Ye, Xinyu and Zeng, Xianfang and Yu, Gang and Zhu, Wenbo and Schiele, Bernt and Yang, Ming-Hsuan and Yang, Xu},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2505.16707},
EPRINT = {2505.16707},
EPRINTTYPE = {arXiv},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Recent advances in multi-modal generative models have enabled significant<br>progress in instruction-based image editing. However, while these models<br>produce visually plausible outputs, their capacity for knowledge-based<br>reasoning editing tasks remains under-explored. In this paper, we introduce<br>KRIS-Bench (Knowledge-based Reasoning in Image-editing Systems Benchmark), a<br>diagnostic benchmark designed to assess models through a cognitively informed<br>lens. Drawing from educational theory, KRIS-Bench categorizes editing tasks<br>across three foundational knowledge types: Factual, Conceptual, and Procedural.<br>Based on this taxonomy, we design 22 representative tasks spanning 7 reasoning<br>dimensions and release 1,267 high-quality annotated editing instances. To<br>support fine-grained evaluation, we propose a comprehensive protocol that<br>incorporates a novel Knowledge Plausibility metric, enhanced by knowledge hints<br>and calibrated through human studies. Empirical results on 10 state-of-the-art<br>models reveal significant gaps in reasoning performance, highlighting the need<br>for knowledge-centric benchmarks to advance the development of intelligent<br>image editing systems.<br>},
}

Endnote

%0 Report
%A Wu, Yongliang
%A Li, Zonghui
%A Hu, Xinting
%A Ye, Xinyu
%A Zeng, Xianfang
%A Yu, Gang
%A Zhu, Wenbo
%A Schiele, Bernt
%A Yang, Ming-Hsuan
%A Yang, Xu
%+ External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
External Organizations
External Organizations
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
External Organizations
External Organizations
%T KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Models : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0011-4672-E
%U https://arxiv.org/abs/2505.16707
%D 2025
%X   Recent advances in multi-modal generative models have enabled significant<br>progress in instruction-based image editing. However, while these models<br>produce visually plausible outputs, their capacity for knowledge-based<br>reasoning editing tasks remains under-explored. In this paper, we introduce<br>KRIS-Bench (Knowledge-based Reasoning in Image-editing Systems Benchmark), a<br>diagnostic benchmark designed to assess models through a cognitively informed<br>lens. Drawing from educational theory, KRIS-Bench categorizes editing tasks<br>across three foundational knowledge types: Factual, Conceptual, and Procedural.<br>Based on this taxonomy, we design 22 representative tasks spanning 7 reasoning<br>dimensions and release 1,267 high-quality annotated editing instances. To<br>support fine-grained evaluation, we propose a comprehensive protocol that<br>incorporates a novel Knowledge Plausibility metric, enhanced by knowledge hints<br>and calibrated through human studies. Empirical results on 10 state-of-the-art<br>models reveal significant gaps in reasoning performance, highlighting the need<br>for knowledge-centric benchmarks to advance the development of intelligent<br>image editing systems.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV

2024

Conference paper

A. Das, X. Hu, L. Jiang, and B. Schiele

“MTA-CLIP: Language-Guided Semantic Segmentation with Mask-Text Alignment,” in Computer Vision -- ECCV 2024, Milano, Italy, 2024.

@inproceedings{DasECCV24,
TITLE = {{MTA-CLIP}: {L}anguage-Guided Semantic Segmentation with Mask-Text Alignment},
AUTHOR = {Das, Anurag and Hu, Xinting and Jiang, Li and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {978-3-031-72948-5},
DOI = {10.1007/978-3-031-72949-2_3},
PUBLISHER = {Springer},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {Computer Vision -- ECCV 2024},
EDITOR = {Leonardis, Ale{\v s} and Ricci, Elisa and Roth, Stefan and Russakovsky, Olga and Sattler, Torsten and Varol, G{\"u}l},
PAGES = {39--56},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {15112},
ADDRESS = {Milano, Italy},
}

Endnote

%0 Conference Proceedings
%A Das, Anurag
%A Hu, Xinting
%A Jiang, Li
%A Schiele, Bernt
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T MTA-CLIP: Language-Guided Semantic Segmentation with Mask-Text Alignment : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-D869-8
%R 10.1007/978-3-031-72949-2_3
%D 2024
%B 18th European Conference on Computer Vision 
%Z date of event: 2024-09-29 - 2024-10-04
%C Milano, Italy
%B Computer Vision -- ECCV 2024
%E Leonardis, Ale&#353; ; Ricci, Elisa; Roth, Stefan; Russakovsky, Olga; Sattler, Torsten; Varol, G&#252;l
%P 39 - 56
%I Springer
%@ 978-3-031-72948-5
%B Lecture Notes in Computer Science
%N 15112

Conference paper

X. Hu, L. Jiang, and B. Schiele

“Training Vision Transformers for Semi-Supervised Semantic Segmentation,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), Seattle, WA, USA, 2024.

@inproceedings{Hu_CVPR24,
TITLE = {Training Vision Transformers for Semi-Supervised Semantic Segmentation},
AUTHOR = {Hu, Xinting and Jiang, Li and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {979-8-3503-5300-6},
DOI = {10.1109/CVPR52733.2024.00384},
PUBLISHER = {IEEE},
YEAR = {2024},
MARGINALMARK = {$\bullet$},
DATE = {2024},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024)},
PAGES = {4007--4017},
ADDRESS = {Seattle, WA, USA},
}

Endnote

%0 Conference Proceedings
%A Hu, Xinting
%A Jiang, Li
%A Schiele, Bernt
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Training Vision Transformers for Semi-Supervised Semantic Segmentation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0010-1036-F
%R 10.1109/CVPR52733.2024.00384
%D 2024
%B 41st IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2024-06-17 - 2024-06-21
%C Seattle, WA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 4007 - 4017
%I IEEE
%@ 979-8-3503-5300-6