ECE 598 PM

Subject offerings of new and developing areas of knowledge in electrical and computer engineering intended to augment the existing curriculum. See Class Schedule or departmental course information for topics and prerequisites. Course Information: May be repeated in the same or separate terms if topics vary.

Prerequisites: ECE 490, ECE 534. Computational inference and machine learning have seen a surge of interest in the last 15 years, motivated by applications as diverse as computer vision, speech recognition, analysis of networks and distributed systems, big-data analytics, large-scale computer simulations, and indexing and searching of very large databases. This course introduces the mathematical and computational methods that enable such applications. Topics include computational methods for statistical inference, sparsity analysis, approximate inference and search, and fast optimization.

Computational inference and machine learning have seen a surge of interest in the last 15 years, motivated by applications as diverse as computer vision, speech recognition, analysis of networks and distributed systems, big-data analytics, large-scale computer simulations, and indexing and searching of very large databases. This course introduces the mathematical and computational methods that enable such applications. Topics include computational methods for statistical inference, sparsity analysis, approximate inference and search, and fast optimization.

Lecture 1: Introduction, information-theoretic functionals.

Lecture 2: Bayes inference, maximum likelihood principle, Maximum A Priori (MAP) estimation, Minimum Mean Squared Error (MMSE) estimation

Lecture 3: Empirical Risk Minimization

Lectures 4, 5: Stochastic Approximation and Stochastic Gradient Descent

Lecture 6: Statistical performance analysis via Monte Carlo methods and importance sampling

Lecture 7: Bootstrap

Lecture 8: Bayesian recursive estimation using Particle Filtering

Lectures 9-11: Parameter estimation via Expectation Maximization (EM) algorithm

Lectures 12, 13: Hidden Markov Models, Viterbi algorithm, Baum-Welch learning

Lectures 14, 15: Linear Dynamical Systems, RTS smoother

Lectures 16-18: Graphical models, belief propagation, approximate inference

Lectures 19-22: Approximate inference using variational inference and mean-field techniques

Lectures 23-25: L1-penalized least squares minimization

Lectures 26-28: Reconstruction of sparse signals using Compressive Sensing

Lecture 29: Dimensionality reduction using random projections; hashing

Grading will be based on homeworks (20%), a midterm exam (40%), and a final project (40%).

Notes from the instructor and articles from scientific journals.

2/11/2016