Students should have a solid basis in probability and calculus, as well as pleasure with mathematics. Having taken a course on Digital Communications / Communication Theory is also helpful.

This course teaches the fundamentals of Information Theory. Students will acquire a profound understanding of: - Information-theoretic quantities, such as entropy, Kullback-Leibler divergence, and mutual information. - Mathematical tools commonly used in Information Theory, such as Jensen's inequality. - The concepts and fundamental theorems of data compression. - The application of Information Theory in Machine Learning.

This course teaches the fundamentals of Information Theory. The topics covered in this course are as follows: 1) Fundamental quantities and concepts in Information Theory: entropy, Kullback-Leibler divergence, mutual information and Jensen's inequality. 2) Lossless data compression: uniquely decodable and instantaneous source codes, Kraft's inequality, analysis of the optimal codeword length, Huffman codes, and universal compression. 3) Information theory and machine learning: Generalization error, empirical risk minimization, classical statistical learning generalization guarantees, information theoretic generalization bounds.

Lectures: The basic concepts will be mainly taught at the blackboard. We will follow closely the book "Elements of Information Theory" by Cover & Thomas (see Basic Bibliography). Exercises: In order to deepen the understanding of the taught material, every two weeks students have to hand in the solutions to a set of problems. These solutions will be graded from 1 to 10, the average grade over the whole semester will constitute part of the grade of the continuous assessment.