Mini-course: Advanced statistical methods. Vladimir Spokoiny (HSE, Russia/Weierstrass Institute, Germany)

Advanced statistical methods

Vladimir Spokoiny (HSE, Russia/Weierstrass Institute, Germany)

Course Schedule:

12, 19, 26 February 18:10 – 21:00, room 205 (3, Kochnovsky Proezd)
13, 20, 27 February 18:10 – 21:00, room 509 (3, Kochnovsky Proezd)

Pass to HSE building can be ordered at: kkorshunova@hse.ru

Short Course Description

This course introduces the main notions, approaches, and methods of nonparametric statistics. The main topics include smoothing and regularization, model selection and parameter tuning, structural inference, efficiency and rate efficiency, local and sieve parametric approaches. The study is mainly limited to regression and density models. The topics of this course form an essential basis for working with complex data structures using modern statistical tools.

Course structure: lectures, seminars, exam.

Prerequisites

Probability theory, linear algebra, mathematical analysis.

Nonparametric Regression

• Regression models: design, errors, and response function.
• Projection estimation. The case of orthogonal design.
• Bias, variance, risk of estimation, rate and accuracy, smoothness classes.

Regularization and roughness penalty

Ridge regression, roughness penalty, penalized maximum likelihood estimation, impact of regularization, modeling bias, complexity, bias-variance trade-off.

Model selection by SURE and AIC, Cross validation

Problem of model choice. Penalized model selection. Akaike criterion. Stein unbiased risk estimation. Parameter tuning by cross-validation.

Model section by smallest accepted

Ordered Model selection and multiple testing. Smallest accepted rule. Parameter tuning by propagation condition and multiplicity correction.

Wavelet methods: hard and soft thresholding

Sequence space model. Wavelet decomposition. Nonlinear wavelet estimation. Hard and soft thresholding.

Density model: Kernel and projection methods

Nadaraya-Watson estimator, high-order kernels. Rate of estimation. Reduction to sequence space model. Wavelet density estimation.

List of References Required or Recommended for the Course

Course materials/Textbooks:

1. Wassermann, L. "All of nonparametric statistics." (2006). Springer
2. Rohde, Charles A. Introductory statistical inference with the likelihood function. Springer International Publishing, 2014.
3. Tsybakov, Alexandre B. "Introduction to nonparametric estimation" (2009).
4. Spokoiny, V. and Dickhaus, T. “Basics of moderm mathematical Statistics” (2015) Springer

Additional sources:

1. Stéphane Boucheron, Olivier Bousquet and Gábor Lugosi “Theory of Classification: a Survey of Some Recent Advances”, ESAIM: P&S, 2005, Vol. 9, p. 323-375
2. Bartlett, Peter L., Boucheron, Stephane, Gábor Lugosi (2002) “Model Selection and Error Estimation”, Machine Learning (48), 85-113