Yandex Laboratory

Publications

• Book

  Razzhigaev A., Voronov A., Kaznacheev A. et al.

  Proceedings of the First Workshop on Performance and Interpretability Evaluations of Multimodal, Multipurpose, Massive-Scale Models (MMMPIE 2022)

  Pixel-level autoregression with Transformer models (Image GPT or iGPT) is one of the recent approaches to image generation that has not received massive attention and elaboration due to quadratic complexity of attention as it imposes huge memory requirements and thus restricts the resolution of the generated images. In this paper, we propose to tackle this problem by adopting Byte-Pair-Encoding (BPE) originally proposed for text processing to the image domain to drastically reduce the length of the modeled sequence. The obtained results demonstrate that it is possible to decrease the amount of computation required to generate images pixel-by-pixel while preserving their quality and the expressiveness of the features extracted from the model. Our results show that there is room for improvement for iGPT-like models with more thorough research on the way to the optimal sequence encoding techniques for images.

  International Conference on Computational Linguistics, 2022.

• Article

  Rogozin A., Beznosikov A., Dvinskikh D. et al.

  Decentralized saddle point problems via non-Euclidean mirror prox

  We consider smooth convex-concave saddle point problems in the decentralized distributed setting, where a finite-sum objective is distributed among the nodes of a computational network. At each node, the local objective depends on the groups of local and global variables. For such problems, we propose a decentralized distributed algorithm with O(ϵ−1) communication and oracle calls complexities to achieve accuracy ε in terms of the duality gap and in terms of consensus between nodes. Further, we prove lower bounds for the communication and oracle calls complexities and show that our algorithm matches these bounds, i.e. it is optimal. In contrast to existing decentralized algorithms, our algorithm admits non-euclidean proximal setup, including, e.g. entropic. We illustrate the work of the proposed algorithm on the prominent problem of computing Wasserstein barycenters (WB), where a non-euclidean proximal setup arises naturally in a bilinear saddle point reformulation of the WB problem.

  Optimization Methods and Software. 2025. Vol. 40. No. 5. P. 1127-1152.

• Book chapter

  Nikita Starodubcev, Ilya Drobyshevskiy, Denis Kuznedelev et al.

  Scale-wise Distillation of Diffusion Models

  Recent diffusion distillation methods have achieved remarkable progress, enabling high-quality -step sampling for large-scale text-conditional image and video diffusion models (DMs). However, further reducing the number of sampling steps becomes more and more challenging, suggesting that efficiency gains may be better mined along other model axes. Motivated by this perspective, we introduce SwD, a scale-wise diffusion distillation framework that equips few-step models with progressive generation, avoiding redundant computations at intermediate diffusion timesteps. Beyond efficiency, SwD enriches the family of distribution matching distillation approaches by introducing a simple distillation objective based on kernel Maximum Mean Discrepancy (MMD). This loss significantly improves the convergence of existing distillation methods and performs surprisingly well in isolation, offering a competitive baseline for diffusion distillation. Applied to state-of-the-art text-to-image/video diffusion models, SwD approaches the sampling speed of two full-resolution steps and largely outperforms alternatives under the same compute budget, as evidenced by automatic metrics and human preference studies.

  In bk.: The Fourteenth International Conference on Learning Representations (ICLR 2026). ICLR, 2026.

• Working paper

  Bazhenov G., Platonov O., Prokhorenkova L.

  TabGraphs: A Benchmark and Strong Baselines for Learning on Graphs with Tabular Node Features

  Tabular machine learning is an important field for industry and science. In this f ield, table rows are typically treated as independent data samples, but additional information about the relations between these samples is sometimes available and can be used to improve predictive performance. Such information can be naturally modeled with a graph, hence tabular machine learning may benefit from graph machine learning methods. However, graph machine learning models are typically evaluated on datasets with homogeneous, most often text-based node features, which are very different from heterogeneous mixtures of numerical and categorical features present in tabular datasets. Thus, there is a critical difference between the data used in tabular and graph machine learning studies, which does not allow one to understand how successfully graph models can be transferred to tabular data. To bridge this gap, we propose a new benchmark of diverse graphs with heterogeneous tabular node features and realistic prediction tasks. We use this benchmark to evaluate a vast set of models, including simple methods previously overlooked in the literature. Our experiments show that graph neural networks indeed can often bring gains in predictive performance for tabular data, but standard tabular models can also be adapted to work with graph data by using simple graph-based feature augmentation, which sometimes enables them to compete with and even outperform graph neural models. Based on our empirical study, we provide insights for researchers and practitioners in both tabular and graph machine learning fields. Our benchmark and the code for reproducing our experiments are available at https://github.com/yandex-research/tabgraphs.

  arXiv:2409.14500 [cs.LG]. arXiv:2409.14500. arXiv:2409.14500 [cs.LG], 2025