Students should have a solid basis in probability and calculus, as well as pleasure with mathematics. Having taken a course on Digital Communications / Communication Theory is also helpful.

This course teaches the fundamentals of Information Theory, including the basic source coding theorems. Students will acquire a profound understanding of: - the concepts of data compression. - the fundamental limits of source codes. - information-theoretic quantities, such as entropy, Kullback-Leibler divergence, and mutual information. - mathematical tools commonly used in Information Theory, such as Jensen's inequality.

This course teaches the fundamentals of Information Theory, which concerns data compression and transmission in digital communication systems. The topics covered in this course are as follows: 1) Fundamental quantities and concepts in Information Theory: entropy, Kullback-Leibler divergence, mutual information and Jensen's inequality. 2) Lossless data compression: uniquely decodable and instantaneous source codes, Kraft's inequality, analysis of the optimal codeword length, Huffman codes, universal compression, and arithmetic coding. 3) Lossy data compression: the rate-distortion theorem and properties of the rate-distortion function.

AF3 Theoretical and practical lessons - 33.5 hours AF4 Lab sessions - 10.5 hours AF5 Office hours - 6 hours AF6 Group work - 30 hours AF7 Individual student work - 62 hours AF8 Continuous and final assessments - 4 hours Lectures (AF3): The basic concepts will be mainly taught at the blackboard. We will follow closely the book "Elements of Information Theory" by Cover & Thomas (see Basic Bibliography). Exercises (AF6/AF7): In order to deepen the understanding of the taught material, every two weeks students have to hand in the solutions to a set of problems. These solutions will be graded from 1 to 10, the average grade over the whole semester will constitute the grade of the continuous assessment. Laboratory Classes (AF4): There will be laboratory classes where students have the opportunity to deepen the concepts learned in class by means of computer exercises. Laboratory classes will also be used to discuss the homework exercises.