• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site
ФКН
Contacts

109028, Moscow,
11, Pokrovsky boulevard

Phone: +7 (495) 531-00-00 *27254

Email: computerscience@hse.ru

 

Administration
First Deputy Dean Tamara Voznesenskaya
Deputy Dean for Research and International Relations Sergei Obiedkov
Deputy Dean for Methodical and Educational Work Ilya Samonenko
Deputy Dean for Development, Finance and Administration Irina Plisetskaya
Book chapter
Towards Understanding and Answering Comparative Questions

Bondarenko A., Ajjour Y., Dittmar V. et al.

In bk.: WSDM 2022 - Proceedings of the 15th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery (ACM), 2022. P. 66-74.

Article
Empirical Variance Minimization with Applications in Variance Reduction and Optimal Control

Belomestny Denis, Iosipoi L., Paris Q. et al.

Bernoulli: a journal of mathematical statistics and probability. 2022. Vol. 28. No. 2. P. 1382-1407.

Book chapter
Exponential savings in agnostic active learning through abstention

Puchkin N., Zhivotovskiy N.

In bk.: Proceedings of Machine Learning Research. Vol. 134: Conference on Learning Theory. PMLR, 2021. P. 3806-3832.

Article
Measurement of the W boson mass

Derkach D., Maevskiy A., Karpov M. et al.

Journal of High Energy Physics. 2022. P. 1-38.

Book chapter
Empirical Study of Transformers for Source Code

Chirkova N., Troshin S.

In bk.: ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. Association for Computing Machinery (ACM), 2021. P. 703-715.

Colloquium "Approximation with neural networks of minimal size: exotic regimes and superexpressive activations"

Event ended

Dmitry Yarotsky
Skoltech

When: February 1, 16:20
Speaker: Dmitry Yarotsky, Skoltech
Where: Online
Title: Approximation with neural networks of minimal size: exotic regimes and superexpressive activations
Abstract: I will discuss some "exotic" regimes arising in theoretical studies of function approximation by neural networks of minimal size. The classical theory predicts specific power laws relating the model complexity to the approximation accuracy for functions of given smoothness, under the assumption of continuous parameter selection. It turns out that these power laws can break down if we use very deep narrow networks and don't impose the said assumption. This effect is observed for networks with common activation functions, e.g. ReLU. Moreover, there exist some "superexpressive" collections of activation functions that theoretically allow to approximate any continuous function with arbitrary accuracy using a network with a fixed number of neurons, i.e. only by suitably adjusting the weights without increasing the number of neurons. This result is closely connected to the Kolmogorov(-Arnold) Superposition Theorem. An example of superexpressive collection is {sin, arcsin}. At the same time, the commonly used activations are not superexpressive.