Professor Rebecca Willett
willett@discovery.wisc.edu
3537 Engineering Hall
Class logistics:
Class location: 1140 Gymnasium/Natatorium
Class time: 11-12:15pm Mondays and Wednesdays
Office hours: Tuesdays 3-4pm in the WID when classes are in session (see Piazza for specific room locations)
761 office hours with Prof. Nowak 12:15-1:15 Mondays in 1140 Gymnasium/Natatorium
TA Daniel Alarcon (pimentelalar@wisc.edu), office hours Tuesdays 5-6pm in 3355 Engineering Hall
Course Topics and Prerequisites:
This course is focused on statistical learning, estimation, decision theory. Topics include detection theory, likelihood ratio tests, Neyman-Pearson detectors, multiple hypothesis testing, generalized likelihood ratio testing, maximum likelihood estimation, Bayesian inference, empirical risk minimization, concentration inequalities, PAC learning, nonparametric inference. The material is intended for people who have a technical background in engineering, computer science, or mathematics. Students should have knowledge of basic linear algebra, probability, and statistics, as well as some programming experience (MATLAB or Python experience is helpful).
Student evaluation:
Grades: 25% midterm, 25% final, 50% homework
Midterm: March 8, 7:15-9:15pm, in 1106 ME
Final: Friday, May 13, 12:45pm
Topics covered:
Part 1: Basic Theory (Probabilistic Modeling, Probabilistic Decision-Making, Probabilistic Inference, Statistical Learning Theory)
Part 2: Methods, Algorithms and Applications (possible topics include sparsity and compressed sensing, graphical models, kernels, SGD, particle filters, EM algorithm, dimensionality reduction / unsupervised learning, matrix completion)
Selected lecture notes:
- Intro
- Likelihood ratio tests, Neyman-Pearson detectors, ROC curves, and sufficient statistics
- Composite Hypothesis Tests
- T tests and p-values
- Linear Models
- Bayesian Methods
- Bayesian Methods, Part 2
- Structural Risk Minimization
- SRM and the Kraft Inequality
Reference texts:
Shai Shalev-Shwartz, Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, 2014.
Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Second Edition, 2009.
Fundamentals of Statistical Signal Processing (Volumes I and II) by Steven Kay, Prentice Hall, 1993
Statistical Signal Processing by Louis L. Scharf, Addison-Wesley, 1991
Kevin Murphy, Machine Learning: a Probabilistic Perspective, 2012.
Larry Wasserman, All of Statistics: A Concise Course in Statistical Inference. Springer, 2003.
Christopher M. Bishop, Pattern Recognition and Machine Learning. Springer Verlag, 2006.
T. K. Moon and W. C. Stirling, Mathematical Methods and Algorithms for Signal Processing, Prentice Hall, 2000
H. Vincent Poor, An introduction to signal detection and estimation, Springer-Verlag, 1988
Harry L. Van Trees, Detection, estimation, and modulation theory, Wiley, 2001
Robert M. Gray, Lee D.Davisson, An introduction to statistical signal processing, Cambridge University Press, 2004
Academic integrity:
Students are strongly encouraged to work together on homework assignments, but each student must submit his or her own writeup. Plagiarism of material written by classmates, book or article authors, or web posters is prohibited. Students must work independently on exams. Academic integrity will be strictly enforced. http://students.wisc.edu/doso/acadintegrity.html