ECE 563
ECE 563 - Information Theory
Fall 2025
| Title | Rubric | Section | CRN | Type | Hours | Times | Days | Location | Instructor |
|---|---|---|---|---|---|---|---|---|---|
| Information Theory | ECE563 | A | 37140 | DIS | 4 | 1230 - 1350 | T R | 3081 Electrical & Computer Eng Bldg | Olgica Milenkovic |
See full schedule from Course Explorer
Official Description
Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, and rate-distortion theory. Course Information: Prerequisite: One of ECE 534, MATH 464, MATH 564.
Subject Area
- Communications
- AI: Mathematical Foundations
Course Director
Description
Mathematical models for channels and sources; entropy, information, data compression, channel capacity, Shannon's theorems, rate-distortion theory.
Topics
- Entropy, relative entropy, mutual information
- Asymptotic equipartition property
- Entropy rates of a stochastic process
- Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
- Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
- Differential entropy
- Gaussian channels
- Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion
Detailed Description and Outline
Topics:
- Entropy, relative entropy, mutual information
- Asymptotic equipartition property
- Entropy rates of a stochastic process
- Lossless data compression (Huffman, Ziv-Lempel, Arithmetic, Shannon-Fano codes): Kraft inequality, Shannon's source coding theorem
- Channel capacity: jointly typical sequences, Fano's inequality, Shannon's channel coding theorem and its converse
- Differential entropy
- Gaussian channels
- Rate-distortion theory: Shannon's source coding theorem relative to a fidelity criterion
Texts
T. Cover and J. Thomas, Elements of Information Theory, Wiley, 1991.
Last updated
9/19/2025