Students should have a solid basis in probability and calculus, as well as pleasure with mathematics. Having taken a course on Digital Communications / Communication Theory is also helpful.

This course teaches the fundamentals of Information Theory. Students will acquire a profound understanding of: - Information-theoretic quantities, such as entropy, Kullback-Leibler divergence, and mutual information. - Mathematical tools commonly used in Information Theory, such as Jensen's inequality. - The concepts and fundamental theorems of data compression. - The application of Information Theory in Machine Learning.

This course teaches the fundamentals of Information Theory. The topics covered in this course are as follows: 1) Fundamental quantities and concepts in Information Theory: entropy, Kullback-Leibler divergence, mutual information and Jensen's inequality, Fisher information and the Cramer-Rao bound. 2) Lossless data compression: uniquely decodable and instantaneous source codes, Kraft's inequality, analysis of the optimal codeword length, Huffman codes, and universal compression. 3) Information theory and machine learning: EM algorithm, variational autoencoders, diffusion models, decision trees.

LECTURES The basic concepts will be mainly taught at the blackboard. We will follow closely the book "Elements of Information Theory" by Cover & Thomas (see Basic Bibliography). EXERCISES In order to deepen the understanding of the taught material, every two weeks students have to hand in the solutions to a set of problems. These solutions will be graded from 1 to 10. LABORATORIES In order to reinforce the theoretical concepts learned in class, a laboratory exercise will be carried out on topics related to machine learning. TUTORING We establish 2 hours per week of tutoring sessions where the professor is available in his office.