Professor Rebecca Willett

willett@discovery.wisc.edu
3537 Engineering Hall

Class logistics:

Canvas link: https://canvas.wisc.edu/courses/24608
Class location: 3534 Engineering Hall
Class time: 11-12:15pm Mondays and Wednesdays
Office hours:  12:15-1pm Mondays and 10:30-11:15am Tuesdays when classes are in session

Course Topics and Prerequisites:

This course is focused on statistical learning, estimation, decision theory. Topics include detection theory, likelihood ratio tests, Neyman-Pearson detectors, multiple hypothesis testing, generalized likelihood ratio testing, maximum likelihood estimation, Bayesian inference, empirical risk minimization, concentration inequalities, PAC learning, nonparametric inference. The material is intended for people who have a technical background in engineering, computer science, or mathematics. Students should have knowledge of basic linear algebra, probability, and statistics, as well as some programming experience (MATLAB or Python experience is helpful).

Student evaluation:

Grades: 30% midterm, 25% final project, 25% homework, 10% quizzes, 10% class participation
Midterm: March 8, in class

Topics covered:

• Detection and Classification
  • Hypothesis Testing (Simple Binary, Composite, Multiple)
  • Detection of Signals in Noise, Energy and Subspace Detection
  • Asymptotics, Kullback-Leibler Divergence
  • Changepoint detection
• Estimation Theory
  • Maximum Likelihood Estimation (Analysis, Application, Computational Methods)
  • Bias-Variance Tradeoffs
  • Bayesian Signal Processing
  • Inverse Problems, Sparsity, Compressed Sensing, LASSO
  • Nonparametric Estimation
• Signal Representations
  • Parametric Models and Sufficient Statistics
  • Fourier, Wavelet, and Spectral Representations
• Stochastic Filtering
  • State Space Modeling
  • Wiener and Kalman Filtering, Density Propagation, Particle Filtering

Class reading materials:

• Intro, Linear algebra review slides and reading,
  Probability and statistics review slides and reading.
• Intro to detection theory slides and notes; see also Kay Chapter 3 and Scharf 4.1-4.2
• Likelihood ratio tests, Neyman Pearson detectors, ROC curves, and sufficient statistics.
• Decision Making with Uncertainty
• Multiple testing; read about fMRI data on Canvas
• Sequential Testing
• Changepoint Detection; see “Using the generalized likelihood ratio statistic for sequential detection of a change-point” by Siegmund and Venkatraman
• Linear Models and Maximum Likelihood Estimation; see Kay chapters 4 and 7 and Scharf chapter 6
• Cramer Rao Lower Bounds
• Bayesian Estimation, Ridge Regression, and LASSO
• Stochastic Filtering

Reference texts:

• Statistical Signal Processing by Louis L. Scharf, Addison-Wesley, 1991
• Fundamentals of Statistical Signal Processing (Volumes I and II) by Steven Kay, Prentice Hall, 1993
• Larry Wasserman, All of Statistics: A Concise Course in Statistical Inference. Springer, 2003.
• T. K. Moon and W. C. Stirling, Mathematical Methods and Algorithms for Signal Processing, Prentice Hall, 2000
• H. Vincent Poor, An introduction to signal detection and estimation, Springer-Verlag, 1988
• Harry L. Van Trees, Detection, estimation, and modulation theory, Wiley, 2001
• Robert M. Gray and Lee D. Davisson, An introduction to statistical signal processing, Cambridge University Press, 2004
• Christopher M. Bishop, Pattern Recognition and Machine Learning. Springer Verlag, 2006.

Academic integrity:

Students are strongly encouraged to work together on homework assignments, but each student must submit his or her own writeup. Plagiarism of material written by classmates, book or article authors, or web posters is prohibited. Students must work independently on exams. Academic integrity will be strictly enforced. http://students.wisc.edu/doso/acadintegrity.html