D2
Computer Vision and Machine Learning

Margret Keuper (Research Leader)

Prof. Dr. Margret Keuper

Address
Max-Planck-Institut für Informatik
Saarland Informatics Campus
Campus E1 4
66123 Saarbrücken
Standort
E1 4 - 617
Telefon
+49 681 9325 2117
Fax
+49 681 9325 2099

Publications

Jung, S., Lukasik, J., & Keuper, M. (n.d.). Neural Architecture Design and Robustness: A Dataset. In Eleventh International Conference on Learning Representations (ICLR 2023). Kigali, Rwanda: OpenReview.net.
(Accepted/in press)
Abstract
Deep learning models have proven to be successful in a wide <br>range of machine learning tasks. Yet, they are often highly sensitive to <br>perturbations on the input data which can lead to incorrect decisions <br>with high confidence, hampering their deployment for practical <br>use-cases. Thus, finding architectures that are (more) robust against <br>perturbations has received much attention in recent years. Just like the <br>search for well-performing architectures in terms of clean accuracy, <br>this usually involves a tedious trial-and-error process with one <br>additional challenge: the evaluation of a network's robustness is <br>significantly more expensive than its evaluation for clean accuracy. <br>Thus, the aim of this paper is to facilitate better streamlined research <br>on architectural design choices with respect to their impact on <br>robustness as well as, for example, the evaluation of surrogate measures <br>for robustness. We therefore borrow one of the most commonly considered <br>search spaces for neural architecture search for image classification, <br>NAS-Bench-201, which contains a manageable size of 6466 non-isomorphic <br>network designs. We evaluate all these networks on a range of common <br>adversarial attacks and corruption types and introduce a database on <br>neural architecture design and robustness evaluations. We further <br>present three exemplary use cases of this dataset, in which we (i) <br>benchmark robustness measurements based on Jacobian and Hessian matrices <br>for their robustness predictability, (ii) perform neural architecture <br>search on robust accuracies, and (iii) provide an initial analysis of <br>how architectural design choices affect robustness. We find that <br>carefully crafting the topology of a network can have substantial impact <br>on its robustness, where networks with the same parameter count range in <br>mean adversarial robust accuracy from 20%-41%.
Export
BibTeX
@inproceedings{Jung_ICLR23, TITLE = {Neural Architecture Design and Robustness: {A} Dataset}, AUTHOR = {Jung, Steffen and Lukasik, Jovita and Keuper, Margret}, LANGUAGE = {eng}, PUBLISHER = {OpenReview.net}, YEAR = {2023}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, ABSTRACT = {Deep learning models have proven to be successful in a wide <br>range of machine learning tasks. Yet, they are often highly sensitive to <br>perturbations on the input data which can lead to incorrect decisions <br>with high confidence, hampering their deployment for practical <br>use-cases. Thus, finding architectures that are (more) robust against <br>perturbations has received much attention in recent years. Just like the <br>search for well-performing architectures in terms of clean accuracy, <br>this usually involves a tedious trial-and-error process with one <br>additional challenge: the evaluation of a network's robustness is <br>significantly more expensive than its evaluation for clean accuracy. <br>Thus, the aim of this paper is to facilitate better streamlined research <br>on architectural design choices with respect to their impact on <br>robustness as well as, for example, the evaluation of surrogate measures <br>for robustness. We therefore borrow one of the most commonly considered <br>search spaces for neural architecture search for image classification, <br>NAS-Bench-201, which contains a manageable size of 6466 non-isomorphic <br>network designs. We evaluate all these networks on a range of common <br>adversarial attacks and corruption types and introduce a database on <br>neural architecture design and robustness evaluations. We further <br>present three exemplary use cases of this dataset, in which we (i) <br>benchmark robustness measurements based on Jacobian and Hessian matrices <br>for their robustness predictability, (ii) perform neural architecture <br>search on robust accuracies, and (iii) provide an initial analysis of <br>how architectural design choices affect robustness. We find that <br>carefully crafting the topology of a network can have substantial impact <br>on its robustness, where networks with the same parameter count range in <br>mean adversarial robust accuracy from 20%-41%.}, BOOKTITLE = {Eleventh International Conference on Learning Representations (ICLR 2023)}, ADDRESS = {Kigali, Rwanda}, }
Endnote
%0 Conference Proceedings %A Jung, Steffen %A Lukasik, Jovita %A Keuper, Margret %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Neural Architecture Design and Robustness: A Dataset : %G eng %U http://hdl.handle.net/21.11116/0000-000C-738F-2 %D 2023 %B Eleventh International Conference on Learning Representations %Z date of event: 2023-05-01 - 2023-05-05 %C Kigali, Rwanda %X Deep learning models have proven to be successful in a wide <br>range of machine learning tasks. Yet, they are often highly sensitive to <br>perturbations on the input data which can lead to incorrect decisions <br>with high confidence, hampering their deployment for practical <br>use-cases. Thus, finding architectures that are (more) robust against <br>perturbations has received much attention in recent years. Just like the <br>search for well-performing architectures in terms of clean accuracy, <br>this usually involves a tedious trial-and-error process with one <br>additional challenge: the evaluation of a network's robustness is <br>significantly more expensive than its evaluation for clean accuracy. <br>Thus, the aim of this paper is to facilitate better streamlined research <br>on architectural design choices with respect to their impact on <br>robustness as well as, for example, the evaluation of surrogate measures <br>for robustness. We therefore borrow one of the most commonly considered <br>search spaces for neural architecture search for image classification, <br>NAS-Bench-201, which contains a manageable size of 6466 non-isomorphic <br>network designs. We evaluate all these networks on a range of common <br>adversarial attacks and corruption types and introduce a database on <br>neural architecture design and robustness evaluations. We further <br>present three exemplary use cases of this dataset, in which we (i) <br>benchmark robustness measurements based on Jacobian and Hessian matrices <br>for their robustness predictability, (ii) perform neural architecture <br>search on robust accuracies, and (iii) provide an initial analysis of <br>how architectural design choices affect robustness. We find that <br>carefully crafting the topology of a network can have substantial impact <br>on its robustness, where networks with the same parameter count range in <br>mean adversarial robust accuracy from 20%-41%. %B Eleventh International Conference on Learning Representations %I OpenReview.net
Li, Y., Zhang, D., Keuper, M., & Khoreva, A. (n.d.). Intra-Source Style Augmentation for Improved Domain Generalization. In 2023 IEEE Winter Conference on Applications of Computer Vision (WACV 2023). Waikoloa Village, HI, USA: IEEE.
(Accepted/in press)
Export
BibTeX
@inproceedings{Li_WACV23, TITLE = {Intra-Source Style Augmentation for Improved Domain Generalization}, AUTHOR = {Li, Yumeng and Zhang, Dan and Keuper, Margret and Khoreva, Anna}, LANGUAGE = {eng}, PUBLISHER = {IEEE}, YEAR = {2023}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {2023 IEEE Winter Conference on Applications of Computer Vision (WACV 2023)}, ADDRESS = {Waikoloa Village, HI, USA}, }
Endnote
%0 Conference Proceedings %A Li, Yumeng %A Zhang, Dan %A Keuper, Margret %A Khoreva, Anna %+ External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Intra-Source Style Augmentation for Improved Domain Generalization : %G eng %U http://hdl.handle.net/21.11116/0000-000B-67FC-6 %D 2022 %B IEEE Winter Conference on Applications of Computer Vision %Z date of event: 2023-01-03 - 2023-01-07 %C Waikoloa Village, HI, USA %B 2023 IEEE Winter Conference on Applications of Computer Vision %I IEEE
Levinkov, E., Kardoost, A., Andres, B., & Keuper, M. (2023). Higher-Order Multicuts for Geometric Model Fitting and Motion Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1). doi:10.1109/TPAMI.2022.3148795
Abstract
Minimum cost lifted multicut problem is a generalization of the multicut problem and is a means to optimizing a decomposition of a graph w.r.t. both positive and negative edge costs. Its main advantage is that multicut-based formulations do not require the number of components given a priori; instead, it is deduced from the solution. However, the standard multicut cost function is limited to pairwise relationships between nodes, while several important applications either require or can benefit from a higher-order cost function, i.e. hyper-edges. In this paper, we propose a pseudo-boolean formulation for a multiple model fitting problem. It is based on a formulation of any-order minimum cost lifted multicuts, which allows to partition an undirected graph with pairwise connectivity such as to minimize costs defined over any set of hyper-edges. As the proposed formulation is NP-hard and the branch-and-bound algorithm is too slow in practice, we propose an efficient local search algorithm for inference into resulting problems. We demonstrate versatility and effectiveness of our approach in several applications: geometric multiple model fitting, homography and motion estimation, motion segmentation.
Export
BibTeX
@article{Keuper22, TITLE = {Higher-Order Multicuts for Geometric Model Fitting and Motion Segmentation}, AUTHOR = {Levinkov, Evgeny and Kardoost, Amirhossein and Andres, Bjoern and Keuper, Margret}, LANGUAGE = {eng}, ISSN = {0162-8828}, DOI = {10.1109/TPAMI.2022.3148795}, PUBLISHER = {IEEE}, ADDRESS = {Piscataway, NJ}, YEAR = {2023}, MARGINALMARK = {$\bullet$}, DATE = {2023}, ABSTRACT = {Minimum cost lifted multicut problem is a generalization of the multicut problem and is a means to optimizing a decomposition of a graph w.r.t. both positive and negative edge costs. Its main advantage is that multicut-based formulations do not require the number of components given a priori; instead, it is deduced from the solution. However, the standard multicut cost function is limited to pairwise relationships between nodes, while several important applications either require or can benefit from a higher-order cost function, i.e. hyper-edges. In this paper, we propose a pseudo-boolean formulation for a multiple model fitting problem. It is based on a formulation of any-order minimum cost lifted multicuts, which allows to partition an undirected graph with pairwise connectivity such as to minimize costs defined over any set of hyper-edges. As the proposed formulation is NP-hard and the branch-and-bound algorithm is too slow in practice, we propose an efficient local search algorithm for inference into resulting problems. We demonstrate versatility and effectiveness of our approach in several applications: geometric multiple model fitting, homography and motion estimation, motion segmentation.}, JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, VOLUME = {45}, NUMBER = {1}, PAGES = {608--622}, }
Endnote
%0 Journal Article %A Levinkov, Evgeny %A Kardoost, Amirhossein %A Andres, Bjoern %A Keuper, Margret %+ External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Higher-Order Multicuts for Geometric Model Fitting and Motion Segmentation : %G eng %U http://hdl.handle.net/21.11116/0000-0009-F784-B %R 10.1109/TPAMI.2022.3148795 %7 2022 %D 2023 %X Minimum cost lifted multicut problem is a generalization of the multicut problem and is a means to optimizing a decomposition of a graph w.r.t. both positive and negative edge costs. Its main advantage is that multicut-based formulations do not require the number of components given a priori; instead, it is deduced from the solution. However, the standard multicut cost function is limited to pairwise relationships between nodes, while several important applications either require or can benefit from a higher-order cost function, i.e. hyper-edges. In this paper, we propose a pseudo-boolean formulation for a multiple model fitting problem. It is based on a formulation of any-order minimum cost lifted multicuts, which allows to partition an undirected graph with pairwise connectivity such as to minimize costs defined over any set of hyper-edges. As the proposed formulation is NP-hard and the branch-and-bound algorithm is too slow in practice, we propose an efficient local search algorithm for inference into resulting problems. We demonstrate versatility and effectiveness of our approach in several applications: geometric multiple model fitting, homography and motion estimation, motion segmentation. %J IEEE Transactions on Pattern Analysis and Machine Intelligence %O IEEE Trans. Pattern Anal. Mach. Intell. %V 45 %N 1 %& 608 %P 608 - 622 %I IEEE %C Piscataway, NJ %@ false
Grabinski, J., Gavrikov, P., Keuper, J., & Keuper, M. (n.d.). Robust Models are less Over-Confident. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022). New Orleans, LO.
(Accepted/in press)
Export
BibTeX
@inproceedings{Grabinski_Neurips22, TITLE = {Robust Models are less Over-Confident}, AUTHOR = {Grabinski, Julia and Gavrikov, Paul and Keuper, Janis and Keuper, Margret}, LANGUAGE = {eng}, YEAR = {2022}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)}, EDITOR = {Oh, Alice H. and Agarwal, Alekh and Belgrave, Danielle and Cho, Kyunghyun}, ADDRESS = {New Orleans, LO}, }
Endnote
%0 Conference Proceedings %A Grabinski, Julia %A Gavrikov, Paul %A Keuper, Janis %A Keuper, Margret %+ External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Robust Models are less Over-Confident : %G eng %U http://hdl.handle.net/21.11116/0000-000B-67C7-1 %D 2022 %B 36th Conference on Neural Information Processing Systems %Z date of event: 2022-11-28 - 2022-12-09 %C New Orleans, LO %B Advances in Neural Information Processing Systems 35 %E Oh, Alice H.; Agarwal, Alekh; Belgrave, Danielle; Cho, Kyunghyun %U https://openreview.net/forum?id=5K3uopkizS
Saseendran, A., Skubch, K., & Keuper, M. (n.d.). Trading off Image Quality for Robustness is not Necessary with Regularized Deterministic Autoencoders. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022). New Orleans, LO.
(Accepted/in press)
Export
BibTeX
@inproceedings{Saseendran_Neurips22, TITLE = {Trading off Image Quality for Robustness is not Necessary with Regularized Deterministic Autoencoders}, AUTHOR = {Saseendran, Amrutha and Skubch, Kathrin and Keuper, Margret}, LANGUAGE = {eng}, YEAR = {2022}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {Advances in Neural Information Processing Systems 35 (NeurIPS 2022)}, EDITOR = {Oh, Alice H. and Agarwal, Alekh and Belgrave, Danielle and Cho, Kyunghyun}, ADDRESS = {New Orleans, LO}, }
Endnote
%0 Conference Proceedings %A Saseendran, Amrutha %A Skubch, Kathrin %A Keuper, Margret %+ External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Trading off Image Quality for Robustness is not Necessary with Regularized Deterministic Autoencoders : %G eng %U http://hdl.handle.net/21.11116/0000-000B-67CC-C %D 2022 %B 36th Conference on Neural Information Processing Systems %Z date of event: 2022-11-28 - 2022-12-09 %C New Orleans, LO %B Advances in Neural Information Processing Systems 35 %E Oh, Alice H.; Agarwal, Alekh; Belgrave, Danielle; Cho, Kyunghyun %U https://openreview.net/forum?id=9YasTgzma8c
Jung, S., & Keuper, M. (n.d.-a). Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2022). Grenoble, France.
(Accepted/in press)
Abstract
The minimum cost multicut problem is the NP-hard/APX-hard combinatorial<br>optimization problem of partitioning a real-valued edge-weighted graph such as<br>to minimize the total cost of the partition. While graph convolutional neural<br>networks (GNN) have proven to be promising in the context of combinatorial<br>optimization, most of them are only tailored to or tested on positive-valued<br>edge weights, i.e. they do not comply to the nature of the multicut problem. We<br>therefore adapt various GNN architectures including Graph Convolutional<br>Networks, Signed Graph Convolutional Networks and Graph Isomorphic Networks to<br>facilitate the efficient encoding of real-valued edge costs. Moreover, we<br>employ a reformulation of the multicut ILP constraints to a polynomial program<br>as loss function that allows to learn feasible multicut solutions in a scalable<br>way. Thus, we provide the first approach towards end-to-end trainable<br>multicuts. Our findings support that GNN approaches can produce good solutions<br>in practice while providing lower computation times and largely improved<br>scalability compared to LP solvers and optimized heuristics, especially when<br>considering large instances.<br>
Export
BibTeX
@inproceedings{Jung_ECML22, TITLE = {Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks}, AUTHOR = {Jung, Steffen and Keuper, Margret}, LANGUAGE = {eng}, YEAR = {2022}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, ABSTRACT = {The minimum cost multicut problem is the NP-hard/APX-hard combinatorial<br>optimization problem of partitioning a real-valued edge-weighted graph such as<br>to minimize the total cost of the partition. While graph convolutional neural<br>networks (GNN) have proven to be promising in the context of combinatorial<br>optimization, most of them are only tailored to or tested on positive-valued<br>edge weights, i.e. they do not comply to the nature of the multicut problem. We<br>therefore adapt various GNN architectures including Graph Convolutional<br>Networks, Signed Graph Convolutional Networks and Graph Isomorphic Networks to<br>facilitate the efficient encoding of real-valued edge costs. Moreover, we<br>employ a reformulation of the multicut ILP constraints to a polynomial program<br>as loss function that allows to learn feasible multicut solutions in a scalable<br>way. Thus, we provide the first approach towards end-to-end trainable<br>multicuts. Our findings support that GNN approaches can produce good solutions<br>in practice while providing lower computation times and largely improved<br>scalability compared to LP solvers and optimized heuristics, especially when<br>considering large instances.<br>}, BOOKTITLE = {Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2022)}, ADDRESS = {Grenoble, France}, }
Endnote
%0 Conference Proceedings %A Jung, Steffen %A Keuper, Margret %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Learning to solve Minimum Cost Multicuts efficiently using Edge-Weighted Graph Convolutional Neural Networks : %G eng %U http://hdl.handle.net/21.11116/0000-000A-C01E-C %D 2022 %B European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases %Z date of event: 2022-09-19 - 2022-09-23 %C Grenoble, France %X The minimum cost multicut problem is the NP-hard/APX-hard combinatorial<br>optimization problem of partitioning a real-valued edge-weighted graph such as<br>to minimize the total cost of the partition. While graph convolutional neural<br>networks (GNN) have proven to be promising in the context of combinatorial<br>optimization, most of them are only tailored to or tested on positive-valued<br>edge weights, i.e. they do not comply to the nature of the multicut problem. We<br>therefore adapt various GNN architectures including Graph Convolutional<br>Networks, Signed Graph Convolutional Networks and Graph Isomorphic Networks to<br>facilitate the efficient encoding of real-valued edge costs. Moreover, we<br>employ a reformulation of the multicut ILP constraints to a polynomial program<br>as loss function that allows to learn feasible multicut solutions in a scalable<br>way. Thus, we provide the first approach towards end-to-end trainable<br>multicuts. Our findings support that GNN approaches can produce good solutions<br>in practice while providing lower computation times and largely improved<br>scalability compared to LP solvers and optimized heuristics, especially when<br>considering large instances.<br> %B Machine Learning and Knowledge Discovery in Databases
Grabinski, J., Keuper, J., & Keuper, M. (2022). Aliasing and Adversarial Robust Generalization of CNNs. Machine Learning, 111. doi:10.1007/s10994-022-06222-8
Export
BibTeX
@article{Grabinski22a, TITLE = {Aliasing and adversarial robust generalization of {CNNs}}, AUTHOR = {Grabinski, Julia and Keuper, Janis and Keuper, Margret}, LANGUAGE = {eng}, ISSN = {0885-6125}, DOI = {10.1007/s10994-022-06222-8}, PUBLISHER = {Springer}, ADDRESS = {Dordrecht}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, JOURNAL = {Machine Learning}, VOLUME = {111}, PAGES = {3925--3951}, }
Endnote
%0 Journal Article %A Grabinski, Julia %A Keuper, Janis %A Keuper, Margret %+ External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Aliasing and Adversarial Robust Generalization of CNNs : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1BA7-A %R 10.1007/s10994-022-06222-8 %7 2022 %D 2022 %J Machine Learning %V 111 %& 3925 %P 3925 - 3951 %I Springer %C Dordrecht %@ false
Zhou, Y., Li, C., Cheng, Z.-Q., Geng, Y., Xie, X., & Keuper, M. (2022). Hypergraph Transformer for Skeleton-based Action Recognition. Retrieved from https://arxiv.org/abs/2211.09590
(arXiv: 2211.09590)
Abstract
Skeleton-based action recognition aims to predict human actions given human<br>joint coordinates with skeletal interconnections. To model such off-grid data<br>points and their co-occurrences, Transformer-based formulations would be a<br>natural choice. However, Transformers still lag behind state-of-the-art methods<br>using graph convolutional networks (GCNs). Transformers assume that the input<br>is permutation-invariant and homogeneous (partially alleviated by positional<br>encoding), which ignores an important characteristic of skeleton data, i.e.,<br>bone connectivity. Furthermore, each type of body joint has a clear physical<br>meaning in human motion, i.e., motion retains an intrinsic relationship<br>regardless of the joint coordinates, which is not explored in Transformers. In<br>fact, certain re-occurring groups of body joints are often involved in specific<br>actions, such as the subconscious hand movement for keeping balance. Vanilla<br>attention is incapable of describing such underlying relations that are<br>persistent and beyond pair-wise. In this work, we aim to exploit these unique<br>aspects of skeleton data to close the performance gap between Transformers and<br>GCNs. Specifically, we propose a new self-attention (SA) extension, named<br>Hypergraph Self-Attention (HyperSA), to incorporate inherently higher-order<br>relations into the model. The K-hop relative positional embeddings are also<br>employed to take bone connectivity into account. We name the resulting model<br>Hyperformer, and it achieves comparable or better performance w.r.t. accuracy<br>and efficiency than state-of-the-art GCN architectures on NTU RGB+D, NTU RGB+D<br>120, and Northwestern-UCLA datasets. On the largest NTU RGB+D 120 dataset, the<br>significantly improved performance reached by our Hyperformer demonstrates the<br>underestimated potential of Transformer models in this field.<br>
Export
BibTeX
@online{Zhou2211.09590, TITLE = {Hypergraph Transformer for Skeleton-based Action Recognition}, AUTHOR = {Zhou, Yuxuan and Li, Chao and Cheng, Zhi-Qi and Geng, Yifeng and Xie, Xuansong and Keuper, Margret}, LANGUAGE = {eng}, URL = {https://arxiv.org/abs/2211.09590}, EPRINT = {2211.09590}, EPRINTTYPE = {arXiv}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, ABSTRACT = {Skeleton-based action recognition aims to predict human actions given human<br>joint coordinates with skeletal interconnections. To model such off-grid data<br>points and their co-occurrences, Transformer-based formulations would be a<br>natural choice. However, Transformers still lag behind state-of-the-art methods<br>using graph convolutional networks (GCNs). Transformers assume that the input<br>is permutation-invariant and homogeneous (partially alleviated by positional<br>encoding), which ignores an important characteristic of skeleton data, i.e.,<br>bone connectivity. Furthermore, each type of body joint has a clear physical<br>meaning in human motion, i.e., motion retains an intrinsic relationship<br>regardless of the joint coordinates, which is not explored in Transformers. In<br>fact, certain re-occurring groups of body joints are often involved in specific<br>actions, such as the subconscious hand movement for keeping balance. Vanilla<br>attention is incapable of describing such underlying relations that are<br>persistent and beyond pair-wise. In this work, we aim to exploit these unique<br>aspects of skeleton data to close the performance gap between Transformers and<br>GCNs. Specifically, we propose a new self-attention (SA) extension, named<br>Hypergraph Self-Attention (HyperSA), to incorporate inherently higher-order<br>relations into the model. The K-hop relative positional embeddings are also<br>employed to take bone connectivity into account. We name the resulting model<br>Hyperformer, and it achieves comparable or better performance w.r.t. accuracy<br>and efficiency than state-of-the-art GCN architectures on NTU RGB+D, NTU RGB+D<br>120, and Northwestern-UCLA datasets. On the largest NTU RGB+D 120 dataset, the<br>significantly improved performance reached by our Hyperformer demonstrates the<br>underestimated potential of Transformer models in this field.<br>}, }
Endnote
%0 Report %A Zhou, Yuxuan %A Li, Chao %A Cheng, Zhi-Qi %A Geng, Yifeng %A Xie, Xuansong %A Keuper, Margret %+ External Organizations External Organizations External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Hypergraph Transformer for Skeleton-based Action Recognition : %G eng %U http://hdl.handle.net/21.11116/0000-000C-1BC0-D %U https://arxiv.org/abs/2211.09590 %D 2022 %X Skeleton-based action recognition aims to predict human actions given human<br>joint coordinates with skeletal interconnections. To model such off-grid data<br>points and their co-occurrences, Transformer-based formulations would be a<br>natural choice. However, Transformers still lag behind state-of-the-art methods<br>using graph convolutional networks (GCNs). Transformers assume that the input<br>is permutation-invariant and homogeneous (partially alleviated by positional<br>encoding), which ignores an important characteristic of skeleton data, i.e.,<br>bone connectivity. Furthermore, each type of body joint has a clear physical<br>meaning in human motion, i.e., motion retains an intrinsic relationship<br>regardless of the joint coordinates, which is not explored in Transformers. In<br>fact, certain re-occurring groups of body joints are often involved in specific<br>actions, such as the subconscious hand movement for keeping balance. Vanilla<br>attention is incapable of describing such underlying relations that are<br>persistent and beyond pair-wise. In this work, we aim to exploit these unique<br>aspects of skeleton data to close the performance gap between Transformers and<br>GCNs. Specifically, we propose a new self-attention (SA) extension, named<br>Hypergraph Self-Attention (HyperSA), to incorporate inherently higher-order<br>relations into the model. The K-hop relative positional embeddings are also<br>employed to take bone connectivity into account. We name the resulting model<br>Hyperformer, and it achieves comparable or better performance w.r.t. accuracy<br>and efficiency than state-of-the-art GCN architectures on NTU RGB+D, NTU RGB+D<br>120, and Northwestern-UCLA datasets. On the largest NTU RGB+D 120 dataset, the<br>significantly improved performance reached by our Hyperformer demonstrates the<br>underestimated potential of Transformer models in this field.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV
Müller, P., Braun, A., & Keuper, M. (2022). Impact of Realistic Properties of the Point Spread Function on Classification Tasks to Reveal a Possible Distribution Shift. In NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications (NeurIPS 2022 Workshop DistShift). New Orelans, LO: OpenReview.net. Retrieved from https://openreview.net/forum?id=r7WJpE3oy0
Export
BibTeX
@inproceedings{Mueller_NEURIPSW22, TITLE = {Impact of Realistic Properties of the Point Spread Function on Classification Tasks to Reveal a Possible Distribution Shift}, AUTHOR = {M{\"u}ller, Patrick and Braun, Alexander and Keuper, Margret}, LANGUAGE = {eng}, URL = {https://openreview.net/forum?id=r7WJpE3oy0}, PUBLISHER = {OpenReview.net}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications (NeurIPS 2022 Workshop DistShift)}, ADDRESS = {New Orelans, LO}, }
Endnote
%0 Conference Proceedings %A M&#252;ller, Patrick %A Braun, Alexander %A Keuper, Margret %+ External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Impact of Realistic Properties of the Point Spread Function on Classification Tasks to Reveal a Possible Distribution Shift : %G eng %U http://hdl.handle.net/21.11116/0000-000B-67EE-6 %U https://openreview.net/forum?id=r7WJpE3oy0 %D 2022 %B NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications %Z date of event: 2022-12-03 - 2022-12-03 %C New Orelans, LO %B NeurIPS 2022 Workshop on Distribution Shifts: Connecting Methods and Applications %I OpenReview.net %U https://openreview.net/forum?id=r7WJpE3oy0
Zhou, Y., Xiang, W., Li, C., Wang, B., Wei, X., Zhang, L., … Hua, X. (2022). SP-ViT: Learning 2D Spatial Priors for Vision Transformers. In 33rd British Machine Vision Conference (BMVC 2022). London, UK: BMVA Press. Retrieved from https://bmvc2022.mpi-inf.mpg.de/564/
Export
BibTeX
@inproceedings{ZhouBMVC22, TITLE = {{SP-ViT}: {L}earning {2D} Spatial Priors for Vision Transformers}, AUTHOR = {Zhou, Yuxuan and Xiang, Wangmeng and Li, Chao and Wang, Biao and Wei, Xihan and Zhang, Lei and Keuper, Margret and Hua, Xiansheng}, LANGUAGE = {eng}, URL = {https://bmvc2022.mpi-inf.mpg.de/564/}, PUBLISHER = {BMVA Press}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {33rd British Machine Vision Conference (BMVC 2022)}, EID = {564}, ADDRESS = {London, UK}, }
Endnote
%0 Conference Proceedings %A Zhou, Yuxuan %A Xiang, Wangmeng %A Li, Chao %A Wang, Biao %A Wei, Xihan %A Zhang, Lei %A Keuper, Margret %A Hua, Xiansheng %+ External Organizations External Organizations External Organizations External Organizations External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T SP-ViT: Learning 2D Spatial Priors for Vision Transformers : %G eng %U http://hdl.handle.net/21.11116/0000-000B-680B-5 %U https://bmvc2022.mpi-inf.mpg.de/564/ %D 2022 %B 33rd British Machine Vision Conference %Z date of event: 2022-11-21 - 2022-11-24 %C London, UK %B 33rd British Machine Vision Conference %Z sequence number: 564 %I BMVA Press
Grabinski, J., Jung, S., Keuper, J., & Keuper, M. (2022). FrequencyLowCut Pooling - Plug & Play against Catastrophic Overfitting. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-19781-9_3
Export
BibTeX
@inproceedings{Grabinski_ECCV2022, TITLE = {{FrequencyLowCut} pooling -- Plug {\&} Play against Catastrophic Overfitting}, AUTHOR = {Grabinski, Julia and Jung, Steffen and Keuper, Janis and Keuper, Margret}, LANGUAGE = {eng}, ISBN = {978-3-031-19780-2}, DOI = {10.1007/978-3-031-19781-9_3}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {36--57}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13674}, ADDRESS = {Tel Aviv, Israel}, }
Endnote
%0 Conference Proceedings %A Grabinski, Julia %A Jung, Steffen %A Keuper, Janis %A Keuper, Margret %+ External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T FrequencyLowCut Pooling - Plug & Play against Catastrophic Overfitting : %G eng %U http://hdl.handle.net/21.11116/0000-000A-C016-4 %R 10.1007/978-3-031-19781-9_3 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni; Hassner, Tal %P 36 - 57 %I Springer %@ 978-3-031-19780-2 %B Lecture Notes in Computer Science %N 13674
Lukasik, J., Jung, S., & Keuper, M. (2022). Learning Where To Look - Generative NAS is Surprisingly Efficient. In Computer Vision -- ECCV 2022. Tel Aviv, Israel: Springer. doi:10.1007/978-3-031-20050-2_16
Export
BibTeX
@inproceedings{Lukasik_ECCV2022, TITLE = {Learning Where To Look -- Generative {NAS} is Surprisingly Efficient}, AUTHOR = {Lukasik, Jovita and Jung, Steffen and Keuper, Margret}, LANGUAGE = {eng}, ISBN = {978-3-031-20049-6}, DOI = {10.1007/978-3-031-20050-2_16}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, BOOKTITLE = {Computer Vision -- ECCV 2022}, EDITOR = {Avidan, Shai and Brostow, Gabriel and Ciss{\'e}, Moustapha and Farinella, Giovanni and Hassner, Tal}, PAGES = {257--273}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13683}, ADDRESS = {Tel Aviv, Israel}, }
Endnote
%0 Conference Proceedings %A Lukasik, Jovita %A Jung, Steffen %A Keuper, Margret %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Learning Where To Look - Generative NAS is Surprisingly Efficient : %G eng %U http://hdl.handle.net/21.11116/0000-000A-C00C-0 %R 10.1007/978-3-031-20050-2_16 %D 2022 %B 17th European Conference on Computer Vision %Z date of event: 2022-10-23 - 2022-10-27 %C Tel Aviv, Israel %B Computer Vision -- ECCV 2022 %E Avidan, Shai; Brostow, Gabriel; Ciss&#233;, Moustapha; Farinella, Giovanni; Hassner, Tal %P 257 - 273 %I Springer %@ 978-3-031-20049-6 %B Lecture Notes in Computer Science %N 13683 %U https://github.com/jovitalukasik/AG-Net
Jung, S., Ziegler, S., Kardoost, A., & Keuper, M. (2022). Optimizing Edge Detection for Image Segmentation with Multicut Penalties. In Pattern Recognition (DAGM GCPR 2022). Konstanz, Germany: Springer. doi:10.1007/978-3-031-16788-1_12
Abstract
The Minimum Cost Multicut Problem (MP) is a popular way for obtaining a graph<br>decomposition by optimizing binary edge labels over edge costs. While the<br>formulation of a MP from independently estimated costs per edge is highly<br>flexible and intuitive, solving the MP is NP-hard and time-expensive. As a<br>remedy, recent work proposed to predict edge probabilities with awareness to<br>potential conflicts by incorporating cycle constraints in the prediction<br>process. We argue that such formulation, while providing a first step towards<br>end-to-end learnable edge weights, is suboptimal, since it is built upon a<br>loose relaxation of the MP. We therefore propose an adaptive CRF that allows to<br>progressively consider more violated constraints and, in consequence, to issue<br>solutions with higher validity. Experiments on the BSDS500 benchmark for<br>natural image segmentation as well as on electron microscopic recordings show<br>that our approach yields more precise edge detection and image segmentation.<br>
Export
BibTeX
@inproceedings{Jung_GCPR2022, TITLE = {Optimizing Edge Detection for Image Segmentation with Multicut Penalties}, AUTHOR = {Jung, Steffen and Ziegler, Sebastian and Kardoost, Amirhossein and Keuper, Margret}, LANGUAGE = {eng}, ISBN = {978-3-031-16787-4}, DOI = {10.1007/978-3-031-16788-1_12}, PUBLISHER = {Springer}, YEAR = {2022}, MARGINALMARK = {$\bullet$}, DATE = {2022}, ABSTRACT = {The Minimum Cost Multicut Problem (MP) is a popular way for obtaining a graph<br>decomposition by optimizing binary edge labels over edge costs. While the<br>formulation of a MP from independently estimated costs per edge is highly<br>flexible and intuitive, solving the MP is NP-hard and time-expensive. As a<br>remedy, recent work proposed to predict edge probabilities with awareness to<br>potential conflicts by incorporating cycle constraints in the prediction<br>process. We argue that such formulation, while providing a first step towards<br>end-to-end learnable edge weights, is suboptimal, since it is built upon a<br>loose relaxation of the MP. We therefore propose an adaptive CRF that allows to<br>progressively consider more violated constraints and, in consequence, to issue<br>solutions with higher validity. Experiments on the BSDS500 benchmark for<br>natural image segmentation as well as on electron microscopic recordings show<br>that our approach yields more precise edge detection and image segmentation.<br>}, BOOKTITLE = {Pattern Recognition (DAGM GCPR 2022)}, EDITOR = {Andres, Bj{\"o}rn and Bernard, Florian and Cremers, Daniel and Frintrop, Simone and Goldl{\"u}cke, Bastian and Ihrke, Ivo}, PAGES = {182--197}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {13485}, ADDRESS = {Konstanz, Germany}, }
Endnote
%0 Conference Proceedings %A Jung, Steffen %A Ziegler, Sebastian %A Kardoost, Amirhossein %A Keuper, Margret %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T Optimizing Edge Detection for Image Segmentation with Multicut Penalties : %G eng %U http://hdl.handle.net/21.11116/0000-000A-C025-3 %R 10.1007/978-3-031-16788-1_12 %D 2022 %B 44th German Conference on Pattern Recognition %Z date of event: 2022-09-27 - 2022-09-30 %C Konstanz, Germany %X The Minimum Cost Multicut Problem (MP) is a popular way for obtaining a graph<br>decomposition by optimizing binary edge labels over edge costs. While the<br>formulation of a MP from independently estimated costs per edge is highly<br>flexible and intuitive, solving the MP is NP-hard and time-expensive. As a<br>remedy, recent work proposed to predict edge probabilities with awareness to<br>potential conflicts by incorporating cycle constraints in the prediction<br>process. We argue that such formulation, while providing a first step towards<br>end-to-end learnable edge weights, is suboptimal, since it is built upon a<br>loose relaxation of the MP. We therefore propose an adaptive CRF that allows to<br>progressively consider more violated constraints and, in consequence, to issue<br>solutions with higher validity. Experiments on the BSDS500 benchmark for<br>natural image segmentation as well as on electron microscopic recordings show<br>that our approach yields more precise edge detection and image segmentation.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV %B Pattern Recognition %E Andres, Bj&#246;rn; Bernard, Florian; Cremers, Daniel; Frintrop, Simone; Goldl&#252;cke, Bastian; Ihrke, Ivo %P 182 - 197 %I Springer %@ 978-3-031-16787-4 %B Lecture Notes in Computer Science %N 13485
Jung, S., & Keuper, M. (n.d.-b). Internalized Biases in Fréchet Inception Distance. In NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications (NeurIPS 2021 Workshop DistShift). Virtual: OpenReview.net. Retrieved from https://openreview.net/forum?id=mLG96UpmbYz; https://openreview.net/group?id=NeurIPS.cc/2021/Workshop/DistShift
(Accepted/in press)
Export
BibTeX
@inproceedings{Jung_NEURIPSW21, TITLE = {Internalized Biases in{F}r\'{e}chet Inception Distance}, AUTHOR = {Jung, Steffen and Keuper, Margret}, LANGUAGE = {eng}, URL = {https://openreview.net/forum?id=mLG96UpmbYz; https://openreview.net/group?id=NeurIPS.cc/2021/Workshop/DistShift}, PUBLISHER = {OpenReview.net}, YEAR = {2021}, PUBLREMARK = {Accepted}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications (NeurIPS 2021 Workshop DistShift)}, ADDRESS = {Virtual}, }
Endnote
%0 Conference Proceedings %A Jung, Steffen %A Keuper, Margret %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Internalized Biases in Fr&#233;chet Inception Distance : %G eng %U http://hdl.handle.net/21.11116/0000-0009-9CA2-0 %U https://openreview.net/forum?id=mLG96UpmbYz %D 2021 %B NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications %Z date of event: 2021-12-13 - 2021-12-13 %C Virtual %B NeurIPS 2021 Workshop on Distribution Shifts: Connecting Methods and Applications %I OpenReview.net %U https://openreview.net/pdf?id=mLG96UpmbYz
Jung, S., & Keuper, M. (2021). Spectral Distribution Aware Image Generation. In Thirty-Fifth AAAI Conference on Artificial Intelligence. Virtual Conference: AAAI.
Export
BibTeX
@inproceedings{Jung_AAAI21, TITLE = {Spectral Distribution Aware Image Generation}, AUTHOR = {Jung, Steffen and Keuper, Margret}, LANGUAGE = {eng}, PUBLISHER = {AAAI}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {Thirty-Fifth AAAI Conference on Artificial Intelligence}, ADDRESS = {Virtual Conference}, }
Endnote
%0 Conference Proceedings %A Jung, Steffen %A Keuper, Margret %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations %T Spectral Distribution Aware Image Generation : %G eng %U http://hdl.handle.net/21.11116/0000-0007-A808-3 %D 2021 %B Thirty-Fifth AAAI Conference on Artificial Intelligence %Z date of event: 2021-02-02 - 2021-02-09 %C Virtual Conference %B Thirty-Fifth AAAI Conference on Artificial Intelligence %I AAAI
Geiping, J., Lukasik, J., Keuper, M., & Moeller, M. (2021). DARTS for Inverse Problems: a Study on Stability. In NeurIPS 2021 Workshop on Deep Learning and Inverse Problems (NeurIPS 2021 Deep Inverse Workshop). Virtual: OpenReview.net. Retrieved from https://openreview.net/forum?id=ty5XCitJfLA; https://openreview.net/group?id=NeurIPS.cc/2021/Workshop/Deep_Inverse
Export
BibTeX
@inproceedings{Geiping_NEURIPSW21, TITLE = {{DARTS} for Inverse Problems: {A} Study on Stability}, AUTHOR = {Geiping, Jonas and Lukasik, Jovita and Keuper, Margret and Moeller, Michael}, LANGUAGE = {eng}, URL = {https://openreview.net/forum?id=ty5XCitJfLA; https://openreview.net/group?id=NeurIPS.cc/2021/Workshop/Deep_Inverse}, PUBLISHER = {OpenReview.net}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {NeurIPS 2021 Workshop on Deep Learning and Inverse Problems (NeurIPS 2021 Deep Inverse Workshop)}, ADDRESS = {Virtual}, }
Endnote
%0 Conference Proceedings %A Geiping, Jonas %A Lukasik, Jovita %A Keuper, Margret %A Moeller, Michael %+ External Organizations External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations %T DARTS for Inverse Problems: a Study on Stability : %G eng %U http://hdl.handle.net/21.11116/0000-0009-87A6-3 %U https://openreview.net/forum?id=ty5XCitJfLA %D 2021 %B NeurIPS 2021 Workshop on Deep Learning and Inverse Problems %Z date of event: 2021-12-10 - 2021-12-10 %C Virtual %B NeurIPS 2021 Workshop on Deep Learning and Inverse Problems %I OpenReview.net
He, Y., Yu, N., Keuper, M., & Fritz, M. (2021). Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI 2021). Montreal, Canada: IJCAI. doi:10.24963/ijcai.2021/349
Export
BibTeX
@inproceedings{He_IJCAI2021, TITLE = {Beyond the Spectrum: {D}etecting Deepfakes via Re-Synthesis}, AUTHOR = {He, Yang and Yu, Ning and Keuper, Margret and Fritz, Mario}, LANGUAGE = {eng}, ISBN = {978-0-9992411-9-6}, DOI = {10.24963/ijcai.2021/349}, PUBLISHER = {IJCAI}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI 2021)}, EDITOR = {Zhou, Zhi-Hua}, PAGES = {2534--2541}, ADDRESS = {Montreal, Canada}, }
Endnote
%0 Conference Proceedings %A He, Yang %A Yu, Ning %A Keuper, Margret %A Fritz, Mario %+ External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations External Organizations %T Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis : %G eng %U http://hdl.handle.net/21.11116/0000-0009-8833-4 %R 10.24963/ijcai.2021/349 %D 2021 %B Thirtieth International Joint Conference on Artificial Intelligence %Z date of event: 2021-08-19 - 2021-08-27 %C Montreal, Canada %B Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence %E Zhou, Zhi-Hua %P 2534 - 2541 %I IJCAI %@ 978-0-9992411-9-6 %U https://www.ijcai.org/proceedings/2021/0349.pdf
Saseendran, A., Skubch, K., Falkner, S., & Keuper, M. (2021). Shape your Space: A Gaussian Mixture Regularization Approach to Deterministic Autoencoders. In Advances in Neural Information Processing Systems 34 pre-proceedings (NeurIPS 2021). Virtual Event: Curran Associates, Inc.
Export
BibTeX
@inproceedings{Saseendran_NeurIPs2021, TITLE = {Shape your Space: {A} {G}aussian Mixture Regularization Approach to Deterministic Autoencoders}, AUTHOR = {Saseendran, Amrutha and Skubch, Kathrin and Falkner, Stefan and Keuper, Margret}, LANGUAGE = {eng}, PUBLISHER = {Curran Associates, Inc.}, YEAR = {2021}, MARGINALMARK = {$\bullet$}, BOOKTITLE = {Advances in Neural Information Processing Systems 34 pre-proceedings (NeurIPS 2021)}, EDITOR = {Ranzato, M. and Beygelzimer, A. and Liang, P. S. and Vaughan, J. W. and Dauphin, Y.}, ADDRESS = {Virtual Event}, }
Endnote
%0 Conference Proceedings %A Saseendran, Amrutha %A Skubch, Kathrin %A Falkner, Stefan %A Keuper, Margret %+ External Organizations External Organizations External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Shape your Space: A Gaussian Mixture Regularization Approach to Deterministic Autoencoders : %G eng %U http://hdl.handle.net/21.11116/0000-0009-882C-D %D 2021 %B 35th Conference on Neural Information Processing Systems %Z date of event: - %C Virtual Event %B Advances in Neural Information Processing Systems 34 pre-proceedings %E Ranzato, M.; Beygelzimer, A.; Liang, P. S.; Vaughan, J. W.; Dauphin, Y. %I Curran Associates, Inc. %U https://papers.nips.cc/paper/2021/file/3c057cb2b41f22c0e740974d7a428918-Paper.pdf
He, Y., Chiu, W.-C., Keuper, M., & Fritz, M. (2017). STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017). Honolulu, HI, USA: IEEE Computer Society. doi:10.1109/CVPR.2017.757
Export
BibTeX
@inproceedings{yang_cvpr17, TITLE = {{STD2P}: {RGBD} Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling}, AUTHOR = {He, Yang and Chiu, Wei-Chen and Keuper, Margret and Fritz, Mario}, LANGUAGE = {eng}, ISBN = {978-1-5386-0458-8}, DOI = {10.1109/CVPR.2017.757}, PUBLISHER = {IEEE Computer Society}, YEAR = {2017}, DATE = {2017}, BOOKTITLE = {30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017)}, PAGES = {7158--7167}, ADDRESS = {Honolulu, HI, USA}, }
Endnote
%0 Conference Proceedings %A He, Yang %A Chiu, Wei-Chen %A Keuper, Margret %A Fritz, Mario %+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling : %G eng %U http://hdl.handle.net/11858/00-001M-0000-002D-B8E6-C %R 10.1109/CVPR.2017.757 %D 2017 %B 30th IEEE Conference on Computer Vision and Pattern Recognition %Z date of event: 2017-07-22 - 2017-07-25 %C Honolulu, HI, USA %B 30th IEEE Conference on Computer Vision and Pattern Recognition %P 7158 - 7167 %I IEEE Computer Society %@ 978-1-5386-0458-8
He, Y., Keuper, M., Schiele, B., & Fritz, M. (2017). Learning Dilation Factors for Semantic Segmentation of Street Scenes. In Pattern Recognition (GCPR 2017). Basel, Switzerland: Springer. doi:10.1007/978-3-319-66709-6_4
Export
BibTeX
@inproceedings{he17gcpr, TITLE = {Learning Dilation Factors for Semantic Segmentation of Street Scenes}, AUTHOR = {He, Yang and Keuper, Margret and Schiele, Bernt and Fritz, Mario}, LANGUAGE = {eng}, ISBN = {978-3-319-66708-9}, DOI = {10.1007/978-3-319-66709-6_4}, PUBLISHER = {Springer}, YEAR = {2017}, DATE = {2017}, BOOKTITLE = {Pattern Recognition (GCPR 2017)}, EDITOR = {Roth, Volker and Vetter, Thomas}, PAGES = {41--51}, SERIES = {Lecture Notes in Computer Science}, VOLUME = {10496}, ADDRESS = {Basel, Switzerland}, }
Endnote
%0 Conference Proceedings %A He, Yang %A Keuper, Margret %A Schiele, Bernt %A Fritz, Mario %+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Learning Dilation Factors for Semantic Segmentation of Street Scenes : %G eng %U http://hdl.handle.net/11858/00-001M-0000-002D-B8F6-8 %R 10.1007/978-3-319-66709-6_4 %D 2017 %B 39th German Conference on Pattern Recognition %Z date of event: 2017-09-13 - 2017-09-15 %C Basel, Switzerland %B Pattern Recognition %E Roth, Volker; Vetter, Thomas %P 41 - 51 %I Springer %@ 978-3-319-66708-9 %B Lecture Notes in Computer Science %N 10496
He, Y., Chiu, W.-C., Keuper, M., & Fritz, M. (2016). RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling. Retrieved from http://arxiv.org/abs/1604.02388
(arXiv: 1604.02388)
Abstract
Beyond the success in classification, neural networks have recently shown strong results on pixel-wise prediction tasks like image semantic segmentation on RGBD data. However, the commonly used deconvolutional layers for upsampling intermediate representations to the full-resolution output still show different failure modes, like imprecise segmentation boundaries and label mistakes in particular on large, weakly textured objects (e.g. fridge, whiteboard, door). We attribute these errors in part to the rigid way, current network aggregate information, that can be either too local (missing context) or too global (inaccurate boundaries). Therefore we propose a data-driven pooling layer that integrates with fully convolutional architectures and utilizes boundary detection from RGBD image segmentation approaches. We extend our approach to leverage region-level correspondences across images with an additional temporal pooling stage. We evaluate our approach on the NYU-Depth-V2 dataset comprised of indoor RGBD video sequences and compare it to various state-of-the-art baselines. Besides a general improvement over the state-of-the-art, our approach shows particularly good results in terms of accuracy of the predicted boundaries and in segmenting previously problematic classes.
Export
BibTeX
@online{He_arXiv2016, TITLE = {{RGBD} Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling}, AUTHOR = {He, Yang and Chiu, Wei-Chen and Keuper, Margret and Fritz, Mario}, LANGUAGE = {eng}, URL = {http://arxiv.org/abs/1604.02388}, EPRINT = {1604.02388}, EPRINTTYPE = {arXiv}, YEAR = {2016}, ABSTRACT = {Beyond the success in classification, neural networks have recently shown strong results on pixel-wise prediction tasks like image semantic segmentation on RGBD data. However, the commonly used deconvolutional layers for upsampling intermediate representations to the full-resolution output still show different failure modes, like imprecise segmentation boundaries and label mistakes in particular on large, weakly textured objects (e.g. fridge, whiteboard, door). We attribute these errors in part to the rigid way, current network aggregate information, that can be either too local (missing context) or too global (inaccurate boundaries). Therefore we propose a data-driven pooling layer that integrates with fully convolutional architectures and utilizes boundary detection from RGBD image segmentation approaches. We extend our approach to leverage region-level correspondences across images with an additional temporal pooling stage. We evaluate our approach on the NYU-Depth-V2 dataset comprised of indoor RGBD video sequences and compare it to various state-of-the-art baselines. Besides a general improvement over the state-of-the-art, our approach shows particularly good results in terms of accuracy of the predicted boundaries and in segmenting previously problematic classes.}, }
Endnote
%0 Report %A He, Yang %A Chiu, Wei-Chen %A Keuper, Margret %A Fritz, Mario %+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling : %G eng %U http://hdl.handle.net/11858/00-001M-0000-002B-063C-5 %U http://arxiv.org/abs/1604.02388 %D 2016 %X Beyond the success in classification, neural networks have recently shown strong results on pixel-wise prediction tasks like image semantic segmentation on RGBD data. However, the commonly used deconvolutional layers for upsampling intermediate representations to the full-resolution output still show different failure modes, like imprecise segmentation boundaries and label mistakes in particular on large, weakly textured objects (e.g. fridge, whiteboard, door). We attribute these errors in part to the rigid way, current network aggregate information, that can be either too local (missing context) or too global (inaccurate boundaries). Therefore we propose a data-driven pooling layer that integrates with fully convolutional architectures and utilizes boundary detection from RGBD image segmentation approaches. We extend our approach to leverage region-level correspondences across images with an additional temporal pooling stage. We evaluate our approach on the NYU-Depth-V2 dataset comprised of indoor RGBD video sequences and compare it to various state-of-the-art baselines. Besides a general improvement over the state-of-the-art, our approach shows particularly good results in terms of accuracy of the predicted boundaries and in segmenting previously problematic classes. %K Computer Science, Computer Vision and Pattern Recognition, cs.CV
Keuper, M., Levinkov, E., Bonneel, N., Layoue, G., Brox, T., & Andres, B. (2015). Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts. In ICCV 2015, IEEE International Conference on Computer Vision. Santiago, Chile: IEEE. doi:10.1109/ICCV.2015.204
Export
BibTeX
@inproceedings{keuper-2015a, TITLE = {Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts}, AUTHOR = {Keuper, Margret and Levinkov, Evgeny and Bonneel, Nicolas and Layoue, Guilaume and Brox, Thomas and Andres, Bjoern}, LANGUAGE = {eng}, ISBN = {978-1-4673-8390-5}, DOI = {10.1109/ICCV.2015.204}, PUBLISHER = {IEEE}, YEAR = {2015}, DATE = {2015}, BOOKTITLE = {ICCV 2015, IEEE International Conference on Computer Vision}, PAGES = {1751--1759}, ADDRESS = {Santiago, Chile}, }
Endnote
%0 Conference Proceedings %A Keuper, Margret %A Levinkov, Evgeny %A Bonneel, Nicolas %A Layoue, Guilaume %A Brox, Thomas %A Andres, Bjoern %+ External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations External Organizations External Organizations Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society %T Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts : %G eng %U http://hdl.handle.net/11858/00-001M-0000-0028-DC59-C %R 10.1109/ICCV.2015.204 %D 2015 %B IEEE International Conference on Computer Vision %Z date of event: 2015-12-13 - 2015-12-16 %C Santiago, Chile %B ICCV 2015 %P 1751 - 1759 %I IEEE %@ 978-1-4673-8390-5
Keuper, M., Andres, B., & Brox, T. (2015). Motion Trajectory Segmentation via Minimum Cost Multicuts. In ICCV 2015, IEEE International Conference on Computer Vision. Santiago, Chile: IEEE. doi:10.1109/ICCV.2015.374
Export
BibTeX
@inproceedings{keuper-2015b, TITLE = {Motion Trajectory Segmentation via Minimum Cost Multicuts}, AUTHOR = {Keuper, Margret and Andres, Bjoern and Brox, Thomas}, LANGUAGE = {eng}, ISBN = {1-4673-8390-5}, DOI = {10.1109/ICCV.2015.374}, PUBLISHER = {IEEE}, YEAR = {2015}, DATE = {2015}, BOOKTITLE = {ICCV 2015, IEEE International Conference on Computer Vision}, PAGES = {3271--3279}, ADDRESS = {Santiago, Chile}, }
Endnote
%0 Conference Proceedings %A Keuper, Margret %A Andres, Bjoern %A Brox, Thomas %+ Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society External Organizations %T Motion Trajectory Segmentation via Minimum Cost Multicuts : %G eng %U http://hdl.handle.net/11858/00-001M-0000-0028-DC42-F %R 10.1109/ICCV.2015.374 %D 2015 %B IEEE International Conference on Computer Vision %Z date of event: 2015-12-13 - 2015-12-16 %C Santiago, Chile %B ICCV 2015 %P 3271 - 3279 %I IEEE %@ 1-4673-8390-5