STAT 31140: Computational Imaging — Theory and Methods

Outline:

Computational imaging refers to the process of forming images from data where computation plays an integral role. This course will cover basic principles of computational imaging, including image denoising, regularization techniques, linear inverse problems and optimization-based solvers, and data acquisition models associated with tomography and interferometry. Specific topics may include patch-based denoising, sparse coding, total variation, dictionary learning, computational photography, compressive imaging, inpainting, and deep learning for image reconstruction.

Logistics:

Class place and time: Mondays and Wednesdays, 9-10:20am in Jones 226

Instructor: Rebecca Willett

E-mail: willett@uchicago.edu

Office: 112B Jones

Office hours: Wednesdays, 10:20-11:20am when classes are in session

Prerequisites: Students are expected to have taken a course in calculus and have exposure to numerical computing (e.g. Matlab, Python, Julia, or R).

Evaluation:

Homework: 35%. There are roughly weekly homework assignments (about 8 total). Homework problems include both mathematical derivations and proofs as well as more applied problems that involve writing code and working with real or synthetic data sets.

Midterm exam: 35%
Wednesday, May. 1, 9-11am

Paper Presentation and Class Participation: 30%.

Letter grades will be assigned using the following hard cutoffs:

A: 93% or higher
A-: 90% or higher
B+: 87% or higher
B: 83% or higher
B-: 80% or higher
C+: 77% or higher
C: 60% or higher
D: 50% or higher
F: less than 50%

We reserve the right to curve the grades, but only in a fashion that would improve the grade earned by the stated rubric.

Tentative schedule:

Lectures 1-3: Denoising

  • Kernel methods — smoothing, bilateral filter, non-local means
  • BM3D
  • wavelets and soft thresholding

Lectures 4-5: Deblurring

  • Fourier transforms of images
  • Least squares vs. Tikhinov regularization
  • Gradient descent

Lectures 6-7: Computed Tomography

  • Fourier slice theorem
  • Filtered backprojection

Lectures 8-10: Regularization and optimization

  • Wavelets and L1 regularization
  • Total variation
  • Proximal-gradient algorithms
  • ADMM

Lecture 11: Learned reconstruction

  • Patch-based regularization
  • K-SVD
  • Dictionary Learning

Class Presentations:

Small teams of students will present seminal and cutting-edge papers in computational imaging.

Reading:

    • Chambolle, A., Caselles, V., Cremers, D., Novaga, M., & Pock, T. (2010). An introduction to total variation for image analysis. Theoretical foundations and numerical methods for sparse recovery, 9(263-340), 227. https://hal.archives-ouvertes.fr/hal-00437581/document
    • Bredies, K., Kunisch, K., & Pock, T. (2010). Total generalized variation. SIAM Journal on Imaging Sciences, 3(3), 492-526. https://epubs.siam.org/doi/abs/10.1137/090769521
    • Aharon, M., Elad, M., & Bruckstein, A. (2006). K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing, 54(11), 4311.
    • Milanfar, P. (2013). A tour of modern image filtering: New insights and methods, both practical and theoretical. IEEE Signal Processing Magazine, 30(1), 106-128.
    • Chatterjee, P., & Milanfar, P. (2010). Is denoising dead?. IEEE Transactions on Image Processing, 19(4), 895-911.
    • Venkatakrishnan, S. V., Bouman, C. A., & Wohlberg, B. (2013). Plug-and-play priors for model based reconstruction. In 2013 IEEE Global Conference on Signal and Information Processing (pp. 945-948).
    • Romano, Y., Elad, M., & Milanfar, P. (2017). The little engine that could: Regularization by denoising (RED). SIAM Journal on Imaging Sciences, 10(4), 1804-1844.
    • Chen, Y., Yu, W., & Pock, T. (2015). On learning optimized reaction diffusion processes for effective image restoration. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5261-5269).
    • Holloway, J., Sankaranarayanan, A. C., Veeraraghavan, A., & Tambe, S. (2012, April). Flutter shutter video camera for compressive sensing of videos. In 2012 IEEE International Conference on Computational Photography (ICCP) (pp. 1-9). IEEE.
    • Gottesman, S. R., & Fenimore, E. E. (1989). New family of binary arrays for coded aperture imaging. Applied optics, 28(20), 4344-4352.
    • Duarte, M. F., Davenport, M. A., Takhar, D., Laska, J. N., Sun, T., Kelly, K. F., & Baraniuk, R. G. (2008). Single-pixel imaging via compressive sampling. IEEE signal processing magazine, 25(2), 83-91.
    • Tian, L., Li, X., Ramchandran, K., & Waller, L. (2014). Multiplexed coded illumination for Fourier Ptychography with an LED array microscope. Biomedical optics express, 5(7), 2376-2389.
    • Podilchuk, C. I., & Delp, E. J. (2001). Digital watermarking: algorithms and applications. IEEE signal processing Magazine, 18(4), 33-46.
    • Gustafsson, M. G. (2005). Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. Proceedings of the National Academy of Sciences, 102(37), 13081-13086.
    • Singer, A., & Shkolnisky, Y. (2011). Three-dimensional structure determination from common lines in cryo-EM by eigenvectors and semidefinite programming. SIAM journal on imaging sciences, 4(2), 543-572.
    • Chen, Yunjin, and Thomas Pock. “Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration.” IEEE transactions on pattern analysis and machine intelligence 39.6 (2016): 1256-1272.