Seminar
Texture Synthesis and Analysis Techniques for Images and 3D Geometry - from Automatic Photo Retouch to Inverse Procedural Modeling
Overview
Topics:
The seminar addresses the topic of texture synthesis: the automatic creation of images by statistical techniques. Texture synthesis techniques have become quite popular recently for a number of applications, such as texturing of 2D and 3D objects based on small examples, semi-automatic retouch of photographs, or even the automatic creation of complex images and 3D models from example data. Much of the facination of this technique is due to the rather simple formal framework (and often really simple algorithms) that are able to create amazing results (if you are not convinced - check this out, in particular the texture-by-numbers application).
The seminar will discuss the theoretical foundations as well as practical implementations of different variants of texture synthesis, ranging from classical results to recent research papers.
Goals:
By participating in the seminar, you can learn at least four different things:
Specific skills:
- An introduction to the theory of non-parametric texture- and geometry synthesis, which is an important current research area.
- Gain some practical experience in this area by a small, guided implementation exercise.
General skills:
- Gain experience in reading and understanding current research literature.
- Practice presentation of scientific topics.
|
 |
Schedule
Rooms and Dates:
- The seminar will take place in the winter break.
- First meeting: Friday, October 22nd 2010, 14h,
Campus E 1 4 (MPI Building), Room 024.
- Second meeting: (last meeting before the block seminar)
Friday, October 29th 2010, 14h,
Campus E 1 4 (MPI Building), Room 024.
- All talks will be held in one week (February 21st to 25th 2011).
- Assignments of topics to advisors will be announced in the Friday meeting (Oct/29th).
- Please register yourself in the HISPOS system before Monday February 7th 2011.
Requirements
Each participant has to:
- Give a talk based on one or more recent research publications.
The focus should be on presenting the major concepts and ideas rather than enumerating details. Each talk should take about 30 minutes (+ 5min, see below).
- Prepare an implementation of a small “toy problem” related to the topic area (there is one and the same practical problem assignment for several talks of similar topics, as listed below, to allow for easy problems and thus reduce the implementation effort). The result of the practical experiment should be presented within the talk, in an additional 5 minutes short presentation.
- Write a summary of approximtely 10 pagers that presents the main ideas of the paper, discusses the outcomes of the practical experiments, and relates the experiments to the broader context. (10% of the summary write-up should concern the practical experiments, i.e. one page).
Follow-ups:
- Good starting point for a Bachelor / Master thesis.
Contact
Instructor:
Teaching Assistants:
Software Framework
Optional Software Package for the Practical Experiments:
- Download the Texton Package
(this software is open source in terms of the GNU Public License Version 2)
This software is based on GeoX, the open source teaching environment our group uses at MPI Informatik but has been modified to work for texture synthesis experiments.
The original GeoX source is available here.
- Documentation: An Introduction to the GeoX System (PDF)
This document describes the GeoX system, which
is very similar. Please talk to your advisor for details.
- This C++ library can be used to conduct the practical experiments, but you are free to use other toolkits/languages.
|
 |
Topics & Literature
Seminar Slides
State of the Art Report
- Reading this STAR-report is recommended for all participants:
Li-Yi Wei, Sylvain Lefebvre, Vivek Kwatra, Greg Turk: State of the Art in Example-based Texture Synthesis. In: Eurographics 2009, State of the Art Report. Link to paper and project page
Basic Texture Synthesis in Images
- Topic 1 – Introduction of the method
- Efros A. A., Leung T. K.: Texture synthesis by non-parametric sampling. In: IEEE International Conference on Computer Vision 1999. Paper link
- Wei L.-Y., Levoy M.: Fast texture synthesis using tree-structured vector quantization. In: In: Proc. SIGGRAPH 2000. Paper link
- Topic 2 – Coherence enhanced
- Ashikhmin M. : Synthesizing natural textures. In: In: Proc. I3D 2001 Paper link
- Topic 3 – Textons (taken, Advisor: Martin Bokeloh)
- Thomas Leung and Jitendra Malik: Representing and recognizing the visual appearance of materials using 3d textons. In: International Journal of Computer Vision, 2001. Paper link
- Topic 4 – Patch-based synthesis (taken, Advisor: Silke Jansen)
- Efros A. A., Freeman W. T. : Image quilting for texture synthesis and transfer. In: Proc. SIGGRAPH 2001 Paper link
- Kwatra V., Schödl A., Essa I., Turk G., Bobick A. : Graphcut textures: image and video synthesis using graph cuts. In: Proc. SIGGRAPH 2003 Paper link
Improving Speed and Quality:
- Topic 5 – Pixel-based global optimization (taken, Advisor: Silke Jansen)
- Kwatra V., Essa I., Bobick A., Kwatra N.: Texture optimization for example-based synthesis. In: Proc. SIGGRAPH 2005. Paper link
- Topic 6 – Real time texture synthesis – pixel based (taken, Advisor: Martin Bokeloh)
- Lefebvre S., Hoppe H. :Parallel controllable texture synthesis. In: Proc. SIGGRAPH 2005. Paper link
- Topic 7 – Real time texture synthesis – patch based (taken, Advisor: Martin Bokeloh)
- Cohen M. F., Shade J., Hiller S., Deussen O. :Wang tiles for image and texture generation. In: Proc. SIGGRAPH 2003. Paper link
- Lagae A., Dutré P. :An alternative for Wang tiles: Colored edges versus colored corners. In: ACM Transactions on Graphics 25, 4 (2006). Paper link
Applications:
- Topic 8 – Solid Texture Synthesis (taken, Advisor: Silke Jansen)
- Kopf J., Fu C.-W., Cohen-Or D., Deussen O., Lischinski D., Wong T.-T. :: Solid texture synthesis from 2d exemplars. In: In: Proc. SIGGRAPH (2007). Paper link
- Pietroni N., Otaduy M. A., Bickel B., Ganovelli F.,Gross M.: Texturing internal surfaces from a few cross sections. In: Computer Graphics Forum 26, (2007). Paper link
- Topic 9 – Image analogies (taken, Advisor: Martin Sunkel)
- Hertzmann A., Jacobs C. E., Oliver N., Curless B.,Salesin D. H.: Image analogies. In: : Proc. SIGGRAPH 2001. Paper link
- Topic 10 – Inverse Texture Synthesis (taken, Advisor: Jens Kerber)
- Wei L.-Y., Han J., Zhou K., Bao H., Guo, B., Shum H.-Y.: :Inverse texture synthesis. In: Proc. SIGGRAPH 2008. Paper link
- Denis Simakov, Yaron Caspi, Eli Shechtman, Michal Irani:Summarizing Visual Data Using Bidirectional Similarity. In: Proc. CVPR 2008. Paper link
- Topic 11 – Image completion (taken, Advisor: Jens Kerber)
- Drori I., Cohen-Or D., Yeshurun H.: Fragment-based image completion. In: In: Proc. SIGGRAPH 2003. Paper link
- Topic 12 – Structure propagation (taken, Advisor: Martin Sunkel)
- Sun J., Yuan L., Jia J., Shum H.-Y.: Image completion with structure propagation. In: Proc. SIGGRAPH 2005. Paper link
- Topic 13 – Many input images (taken, Advisor: Martin Sunkel)
- Hays J., Efros A. A.:Scene completion using millions of photographs. In: Proc. SIGGRAPH 2007. Paper link
- Topic 14 – Non local denoising (taken, Advisor: Martin Sunkel)
- A. Buades, B. Coll, J.M. Morel:A non local algorithm for image denoising. In: Proc. CVPR 2005. Paper link
Geometry Synthesis:
- Topic 15 – Surface texture synthesis (taken, Advisor: Jens Kerber)
- Turk G.: Texture synthesis on surfaces. In: Proc. SIGGRAPH 2001. Paper link
- Topic 16 – Geometry texture synthesis (taken, Advisor: Jens Kerber)
- Bhat P., Ingram S., Turk G.: .: Geometric texture synthesis by example. In: Proc. SGP 2004. Paper link
- Topic 17 – Mesh quilting (taken, Advisor: Silke Jansen)
- Zhou K., Huang X., Wang X., Tong Y., Desbrun M.,Guo B., Shum H.-Y.: Mesh quilting for geometric texture synthesis. In: Proc. SIGGRAPH 2006. Paper link
- Topic 18 – Model synthesis I
- Merrell P.:Example-based model synthesis. In: Proc. I3D 2007. Paper link
- Topic 19 – Model synthesis II
- Merrell P., Manocha D.: Continuous model synthesis. In: Proc. SIGGRAPH Asia 2008. Paper link
- Topic 20 – Inverse procedural modeling by symmetry detection (taken, Advisor: Martin Bokeloh)
- Bokeloh M., Wand M., Seidel H.-P.: A Connection between Partial Symmetry and Inverse Procedural Modeling. In: Proc. SIGGRAPH 2010. Paper link
- Topic 21 – Structural Image Editing
- C. Barnes, E. Shechtman, A. Finkelstein, D. Goldman: PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing. In: : Proc. Siggraph 2009. Paper link
- Topic 22 – Automatic generation of L-systems
- O. Stava, B. Beneš, R. Mech, D. Aliaga, P. Kristof: Inverse Procedural Modeling by Automatic Generation of L-systems. . In: Proc. Eurographics 2010. Paper link
