ECE 563
ECE 563 - Information Theory
See full schedule from Course Explorer
Subject Area
- Communications
- AI: Mathematical Foundations
Course Director
Description
Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, rate-distortion theory.
Topics
- Entropy, relative entropy, mutual information
- Asymptotic equipartition property
- Entropy rates of a stochastic process
- Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
- Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
- Differential entropy
- Gaussian channels
- Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion
Detailed Description and Outline
Topics:
- Entropy, relative entropy, mutual information
- Asymptotic equipartition property
- Entropy rates of a stochastic process
- Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
- Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
- Differential entropy
- Gaussian channels
- Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion
Texts
T. Cover and J. Thomas, Elements of Information Theory, Wiley, 1991.
Last updated
9/19/2025