Joint object and scene segmentation with dynamic CRFs

A Dynamic Conditional Random Field Model for Joint Labeling of Object and Scene Classes [1]

Christian Wojek, Bernt Schiele
in European Conference on Computer Vision (ECCV 2008), Part IV, pp. 733-747, Oct. 13-16 2008, Marseille, France


Object detection and pixel-wise scene labeling have both been active research areas in recent years and impressive results have been reported for both tasks separately. The integration of these different types of approaches should boost performance for both tasks as object detection can profit from powerful scene labeling and also pixel-wise scene labeling can profit from powerful object detection. Consequently, first approaches have been proposed that aim to integrate both object detection and scene labeling in one framework. This paper proposes a novel approach based on conditional random field (CRF) models that extends existing work by 1) formulating the integration as a joint labeling problem of object and scene classes and 2) by systematically integrating dynamic information for the object detection task as well as for the scene labeling task. As a result, the approach is applicable to highly dynamic scenes including both fast camera and object movements. Experiments show the applicability of the novel approach to challenging real-world video sequences and systematically analyze the contribution of different system components to the overall performance.




Training data for car front and read view detector                                             detector-train.tar.gz (54MB)

Training data for unary texture classifier                                                             texture-train.tar.gz (39MB)

Training data for CRF models                                                                              crf-train.tar.gz (826MB)

Test data for CRF models                                                                                     crf-test.tar.gz (823MB)

For a description on formats please read the README file in the root folder.

In case the data set is used for publications we ask the authors to refer to the above ECCV 2008 publication.



For further information and questions please contact Christian Wojek or Bernt Schiele.


[1] A Dynamic Conditional Random Field Model for Joint Labeling of Object and Scene Classes, C. Wojek and B. Schiele, European Conference on Computer Vision (ECCV), October, (2008)