Latent Embeddings for Zero-shot Classification

Yongqin Xian , Zeynep Akata , Gaurav Sharma,

Quynh NguyenMatthias Hein and Bernt Schiele

 

We present a novel latent embedding model for learning a compatibility function between image and class embeddings, in the context of zero-shot classification. The proposed method augments the state-of-the-art bilinear compatibility model by incorporating latent variables. Instead of learning a single bilinear map, it learns a collection of latent variable maps with the selection of which map to use being a latent variable for the current image-class pair. We train the model with a ranking based objective function which penalizes incorrect rankings of the true class for a given image. We empirically validate that our model improves the state-of-the-art for various class embeddings consistently on three challenging publicly available datasets for the zero-shot setting. Moreover, our method leads to visually highly interpretable results with clear clusters of different fine-grained object properties that correspond to different latent variable maps.

Paper, Code and Data 

  • If you use our code, please cite:
@inproceedings {XASNHS16, 	
title = {Latent Embeddings for Zero-shot Classification},
booktitle = {IEEE Computer Vision and Pattern Recognition (CVPR)},
year = {2016},
author = {Yongqin Xian and Zeynep Akata and Gaurav Sharma and Quynh Nguyen and Matthias Hein and Bernt Schiele}
}