Yandex Laboratory

Publications

• Book

  Razzhigaev A., Voronov A., Kaznacheev A. et al.

  Proceedings of the First Workshop on Performance and Interpretability Evaluations of Multimodal, Multipurpose, Massive-Scale Models (MMMPIE 2022)

  Pixel-level autoregression with Transformer models (Image GPT or iGPT) is one of the recent approaches to image generation that has not received massive attention and elaboration due to quadratic complexity of attention as it imposes huge memory requirements and thus restricts the resolution of the generated images. In this paper, we propose to tackle this problem by adopting Byte-Pair-Encoding (BPE) originally proposed for text processing to the image domain to drastically reduce the length of the modeled sequence. The obtained results demonstrate that it is possible to decrease the amount of computation required to generate images pixel-by-pixel while preserving their quality and the expressiveness of the features extracted from the model. Our results show that there is room for improvement for iGPT-like models with more thorough research on the way to the optimal sequence encoding techniques for images.

  International Conference on Computational Linguistics, 2022.

• Article

  Rogozin A., Beznosikov A., Dvinskikh D. et al.

  Decentralized saddle point problems via non-Euclidean mirror prox

  We consider smooth convex-concave saddle point problems in the decentralized distributed setting, where a finite-sum objective is distributed among the nodes of a computational network. At each node, the local objective depends on the groups of local and global variables. For such problems, we propose a decentralized distributed algorithm with O(ϵ−1) communication and oracle calls complexities to achieve accuracy ε in terms of the duality gap and in terms of consensus between nodes. Further, we prove lower bounds for the communication and oracle calls complexities and show that our algorithm matches these bounds, i.e. it is optimal. In contrast to existing decentralized algorithms, our algorithm admits non-euclidean proximal setup, including, e.g. entropic. We illustrate the work of the proposed algorithm on the prominent problem of computing Wasserstein barycenters (WB), where a non-euclidean proximal setup arises naturally in a bilinear saddle point reformulation of the WB problem.

  Optimization Methods and Software. 2025. Vol. 40. No. 5. P. 1127-1152.

• Book chapter

  Nikita Starodubcev, Ilya Drobyshevskiy, Denis Kuznedelev et al.

  Scale-wise Distillation of Diffusion Models

  Recent diffusion distillation methods have achieved remarkable progress, enabling high-quality -step sampling for large-scale text-conditional image and video diffusion models (DMs). However, further reducing the number of sampling steps becomes more and more challenging, suggesting that efficiency gains may be better mined along other model axes. Motivated by this perspective, we introduce SwD, a scale-wise diffusion distillation framework that equips few-step models with progressive generation, avoiding redundant computations at intermediate diffusion timesteps. Beyond efficiency, SwD enriches the family of distribution matching distillation approaches by introducing a simple distillation objective based on kernel Maximum Mean Discrepancy (MMD). This loss significantly improves the convergence of existing distillation methods and performs surprisingly well in isolation, offering a competitive baseline for diffusion distillation. Applied to state-of-the-art text-to-image/video diffusion models, SwD approaches the sampling speed of two full-resolution steps and largely outperforms alternatives under the same compute budget, as evidenced by automatic metrics and human preference studies.

  In bk.: The Fourteenth International Conference on Learning Representations (ICLR 2026). ICLR, 2026.