This course teaches the fundamentals of Information Theory, which concerns data compression and transmission in digital communication systems. The topics covered in this course are as follows:
1) Fundamental quantities and concepts in Information Theory: entropy, Kullback-Leibler divergence, mutual information, Jensen's inequality, Fano's inequality, and method of types.
2) Lossless data compression: uniquely decodable and instantaneous source codes, Kraft's inequality, analysis of the optimal codeword length, Huffman codes, universal compression, and arithmetic coding.
3) Lossy data compression: the rate-distortion theorem and properties of the rate-distortion function.
4) Vector quantization: fixed-rate versus variable-rate quantization, dithered quantization, scalar and lattice quantization.