ECE 598 PM
ECE 598 PM - Computational Inference and Learning
Computational inference and machine learning have seen a surge of interest in the last 15 years, motivated by applications as diverse as computer vision, speech recognition, analysis of networks and distributed systems, big-data analytics, large-scale computer simulations, and indexing and searching of very large databases. This course introduces the mathematical and computational methods that enable such applications. Topics include computational methods for statistical inference, sparsity analysis, approximate inference and search, and fast optimization.
Detailed Description and Outline
Lecture 1: Introduction, information-theoretic functionals.
Lecture 2: Bayes inference, maximum likelihood principle, Maximum A Priori (MAP) estimation, Minimum Mean Squared Error (MMSE) estimation
Lecture 3: Empirical Risk Minimization
Lectures 4, 5: Stochastic Approximation and Stochastic Gradient Descent
Lecture 6: Statistical performance analysis via Monte Carlo methods and importance sampling
Lecture 7: Bootstrap
Lecture 8: Bayesian recursive estimation using Particle Filtering
Lectures 9-11: Parameter estimation via Expectation Maximization (EM) algorithm
Lectures 12, 13: Hidden Markov Models, Viterbi algorithm, Baum-Welch learning
Lectures 14, 15: Linear Dynamical Systems, RTS smoother
Lectures 16-18: Graphical models, belief propagation, approximate inference
Lectures 19-22: Approximate inference using variational inference and mean-field techniques
Lectures 23-25: L1-penalized least squares minimization
Lectures 26-28: Reconstruction of sparse signals using Compressive Sensing
Lecture 29: Dimensionality reduction using random projections; hashing
Grading will be based on homeworks (20%), a midterm exam (40%), and a final project (40%).
Notes from the instructor and articles from scientific journals.