Introduction to the Theory of Computation

Entropy, relative entropy. The second law of thermodynamics. «Asymptotic equipartition» property. Entropy and stochastic processes. Data compression. Optimal codes, Huffman code, Shannon-Fano-Elias code. Kolmogorov algorithmic complexity. Channel capacity. Shannon’s fundamental theorem. Differential entropy. Gauss bus. Information theory and advanced statistical issues. Maximum Entropy. Source coding. Approximation by Markov process. Galois bodies. Rate-loss function. Signals and noise. Error Correction Codes. Hamming codes, Reed-Muller codes. Information Theory Applications in investment theory.