This course teaches the fundamentals of Information Theory, which concerns data compression and transmission in digital communication systems. The topics covered in this course are as follows:
1) Fundamental quantities and concepts in Information Theory: entropy, Kullback-Leibler divergence, mutual information, and Jensen's inequality.
2) Lossless data compression: uniquely decodable and instantaneous source codes, Kraft's inequality, analysis of the optimal codeword length, Huffman codes, universal compression, and arithmetic coding.
3) Lossy data compression: the rate-distortion theorem and properties of the rate-distortion function.