STUDENTS ARE EXPECTED TO HAVE COMPLETED Students should have a solid basis in probability and calculus, as well as pleasure with mathematics. Having taken a course on Digital Communications / Communication Theory is also helpful.

COMPETENCES AND SKILLS THAT WILL BE ACQUIRED AND LEARNING RESULTS. This course teaches the fundamentals of Information Theory, including the basic source coding and channel coding theorems. Students will acquire a profound understanding of: - the concepts of data compression/transmission in digital communication systems. - the fundamental limits of source codes and error correcting codes. - information-theoretic quantities, such as entropy, Kullback-Leibler divergence, and mutual information. - mathematical tools/concepts commonly used in Information Theory, such as Jensen's inequality, Fano's inequality, and the Asymptotic Equipartition Property (AEP).

DESCRIPTION OF CONTENTS: PROGRAMME This course teaches the fundamentals of Information Theory, which concerns data compression and transmission in digital communication systems. The topics covered in this course are as follows: 1) Fundamental quantities and concepts in Information Theory: entropy, Kullback-Leibler divergence, mutual information, Jensen's inequality, Fano's inequality, Asymptotic Equipartition Property (AEP), method of types. 2) Data compression: uniquely decodable and instantaneous source codes, Kraft's inequality, analysis of the optimal codeword length, Huffman codes, almost lossless source coding. 3) Data transmission: description of the information-theoretic communication system, channel capacity, Kuhn-Tucker conditions, the channel coding theorem, the joint source-channel coding theorem. 4) Data transmission over the Gaussian channel: differential entropy, entropy-maximizing property of Gaussian random variables, the channel capacity of the Gaussian channel.

LEARNING ACTIVITIES AND METHODOLOGY Lectures: The basic concepts will be mainly taught at the blackboard. We will follow closely the book "Elements of Information Theory" by Cover & Thomas (see Basic Bibliography). Exercises: In order to deepen the understanding of the taught material, every two weeks students have to hand in the solutions to a set of problems. These solutions will be graded from 1 to 10, the average grade over the whole semester will constitute the grade of the continuous assessment. Both lectures and exercises will be held in English.