Machine Learning

Announcements:

  • Re-Exam results
  • Exam results
  • There will be no lectures on the 28th and 30th of January. Furthermore no tutorials will take place in the same week.
  • The registration for the exercise groups has closed. Find your choice in this list: Slot assignments
  • The exercise groups have started on Wednesday, the 24th of October
  • Please subscribe to our machine learning google group, which will be used as a means to discuss the exercise content and we are likely going to post course updates there: Google group.

Overview

In a broader perspective machine learning tries to automatize the process of empirical sciences – namely extracting knowledge about natural phenomena from measured data with the goal to either understand better the underlying processes or to make good predictions. Machine learning methods are therefore widely used in different fields: bioinformatics, computer vision, information retrieval, computer linguistics, robotics,…

The lecture gives a broad introduction into machine learning methods. After the lecture the students should be able to solve and analyze learning problems. The lecture is based on the machine learning lecture of  Matthias Hein. 

Course Information

  • Semester: winter term
  • Year: 2018/2019
  • Type: Core lecture (Stammvorlesung), 9 credit points
  • Time and Location:
    • Lectures
      • Monday 2 pm - 4 pm, lecture hall E 1.3 HS002
      • Wednesday 10 am - 12 am, lecture hall E 1.3 HS002
    • Exercise groups
      • Wednesday 8 am - 10 am, E 1.4 room 024
      • Thursday 8 am - 10 am, E 1.4 room 024
      • Friday 4 pm - 6 pm, E 1.4 room 024
    • Exams
      • 26.02.19 10:00 - 13:00, Günter Hotz Hörsaal E 2.2 0.01 and E 2.5 HS I
      • 28.03.19 10:00 - 13:00, Günter Hotz Hörsaal E 2.2 0.01

List of topics (tentative)

  • Reminder of probability theory
  • Maximum Likelihood/Maximum A Posteriori Estimators
  • Bayesian decision theory
  • Linear classification and regression
  • Kernel methods
  • Model selection and evaluation of learning methods
  • Feature selection
  • Nonparametric methods
  • Boosting, Decision trees
  • Neural networks
  • Structured Output
  • Semi-supervised learning
  • Unsupervised learning (Clustering, Independent Component Analysis)
  • Dimensionality Reduction and Manifold Learning
  • Statistical learning theory

Previous knowledge of machine learning is not required. The participants should be familiar with linear algebra, analysis and probability theory on the level of the local `Mathematics for Computer Scienticists I-III’ lectures. In particular, attendees should be familiar with

  • Discrete and continuous probability theory (marginals, conditional probability, random variables, expectation etc.)
    The first three chapters of: L. Wasserman: All of Statistics, Springer, (2004) provide the necessary background
  • Linear algebra (rank, linear systems, eigenvalues, eigenvectors (in particular for symmetric matrices), singular values, determinant)
    A quick reminder of the basic ideas of linear algebra can be found in the tutorial  of Mark Schmidt (I did not check it for correctness!). Apart from the LU factorization this summarizes all what is used in the lecture in a non-formal way. You might also find the following sheets on matrix identities and gaussian identities useful from Sam Roweis useful.
  • Multivariate analysis (integrals, gradient, Hessian, extrema of multivariate functions)

Literature

The lecture will be partially based on the following books and partially on recent research papers:

  • R.O. Duda, P.E. Hart, and D.G.Stork: Pattern Classification, Wiley, (2000).
  • B. Schoelkopf and A. J. Smola: Learning with Kernels, MIT Press, (2002).
  • J. Shawe-Taylor and N. Christianini: Kernel Methods for Pattern Analysis, Cambridge University Press, (2004).
  • C. M. Bishop: Pattern recognition and Machine Learning, Springer, (2006).
  • T. Hastie, R. Tibshirani, J. Friedman: The Elements of Statistical Learning, Springer, second edition, (2008).
  • L. Devroye, L. Gyoerfi, G. Lugosi: A Probabilistic Theory of Pattern Recognition, Springer, (1996).
  • L. Wasserman: All of Statistics, Springer, (2004).
  • S. Boyd and L. Vandenberghe: Convex Optimization, Cambridge University Press, (2004).

Other resources