LRI Academic Workshop – AI Data Science
Abstract
A half-day workshop focused on the data, methods, and evaluation behind modern large-scale AI. Talks cover vision-language pre-training and object grounding, deepfake detection, LLM-driven competitive programming, LLM reasoning in STEM, stochastic optimization for Deep Learning (including parameter-free stepsize methods for cross-entropy training), and concept-based interpretability. The program offers a focused, discussion-oriented look at how large models are built, where they succeed, and where they still fall short.
Speakers
Bruno Bauwens
HSE University
The stochastic Polyak stepsize for cross entropy revisited
In order to obtain a parameter free algorithm optimization algorithm, Polyak proposed to derive a stepsize for gradient descent for linearization. This stepsize equals the ratio "loss"/"square of gradient norm". In practice, this stepsize does not work well in the stochastic setting with minibatches, mostly because cross entropy can be quite concave and give very large estimations of a suitable step. There exist many articles that propose corrections, often justified by theoretical results in convex optimization. We propose some more corrections. The literature focusses on training Resnets from scratch on Cifar10 and 100. Our results show that methods do not improve test error, but can speed up learning time on smaller models.
In order to obtain a parameter free algorithm optimization algorithm, Polyak proposed to derive a stepsize for gradient descent for linearization. This stepsize equals the ratio "loss"/"square of gradient norm". In practice, this stepsize does not work well in the stochastic setting with minibatches, mostly because cross entropy can be quite concave and give very large estimations of a suitable step. There exist many articles that propose corrections, often justified by theoretical results in convex optimization. We propose some more corrections. The literature focusses on training Resnets from scratch on Cifar10 and 100. Our results show that methods do not improve test error, but can speed up learning time on smaller models.
Denis Derkach
HSE University
Postdoctoral-Level Large Language Models in STEM: The Potential Impact of AI on Researchers
The acceleration of research has long been a primary objective for both individual scientists and scientific institutions. Each historical paradigm shift in scientific development has promised either fully automated research or, at the very least, significantly expedited progress. With the advancement of large language models and recent notable breakthroughs, a new paradigm is emerging, characterized by promising trajectories and potential structural shifts. This presentation will examine the current state of LLM-driven scientific inquiry, analyze the limitations of this approach, and outline prospective future directions.
The acceleration of research has long been a primary objective for both individual scientists and scientific institutions. Each historical paradigm shift in scientific development has promised either fully automated research or, at the very least, significantly expedited progress. With the advancement of large language models and recent notable breakthroughs, a new paradigm is emerging, characterized by promising trajectories and potential structural shifts. This presentation will examine the current state of LLM-driven scientific inquiry, analyze the limitations of this approach, and outline prospective future directions.
Vasiliy Gromov
HSE University
Coarse-graining structure of semantic spaces
We investigate Hailonakea – a geometric object that represents language in the semantic space. We employ various methods, such as topological data analysis, clustering, intrinsic dimensions estimation algorithms, entropy-complexity (to analyse the semantic trajectories). Our research shows that natural languages are self-organised critical systems, the semantic trajectories are chaotic time series and there are characteristics that help to separate artificial language Esperanto from natural languages. We extract coarse-grained structure of the languages, analyse and compare it to the structure of the semantic space of bots (language models). Additionally, our findings suggest that some characteristics of Hailonakea do not agree with common classifications (genetic, areal), allowing possibility of a new language classification.
We investigate Hailonakea – a geometric object that represents language in the semantic space. We employ various methods, such as topological data analysis, clustering, intrinsic dimensions estimation algorithms, entropy-complexity (to analyse the semantic trajectories). Our research shows that natural languages are self-organised critical systems, the semantic trajectories are chaotic time series and there are characteristics that help to separate artificial language Esperanto from natural languages. We extract coarse-grained structure of the languages, analyse and compare it to the structure of the semantic space of bots (language models). Additionally, our findings suggest that some characteristics of Hailonakea do not agree with common classifications (genetic, areal), allowing possibility of a new language classification.
Dmitry Ilvovsky
HSE University
Solving International Olympiad-Level Programming Problems Using LLMs
This talk will provide an overview of some of the most promising methods for solving competitive programming problems—such as those found in programming contests like ICPC — using modern large language models and agent-based systems built upon them. We will examine popular open benchmarks, discuss the key concepts and limitations of several existing approaches, and explore potential ways to overcome these challenges.
This talk will provide an overview of some of the most promising methods for solving competitive programming problems—such as those found in programming contests like ICPC — using modern large language models and agent-based systems built upon them. We will examine popular open benchmarks, discuss the key concepts and limitations of several existing approaches, and explore potential ways to overcome these challenges.
Iaroslav Koshelev
LRI
Advancing Object Grounding in PanguVL Pre-Training
Object grounding is a foundational capability for Large Vision-Language Models that allows them to comprehend the visual modality. Robust learning of this capability in billion-parameter models at the pre-training stage requires a huge corpus of synthetic data and depends heavily on the quality of its object-level supervision. In this talk, we present a comprehensive methodology for data-driven improvement of this capability. We start by studying the effects of different data formats on downstream grounding benchmarks, and find grounded captions to be the best format for the pre-training stage. This is followed by our novel high-quality 2D object grounding data sampling pipeline from single images, whose core idea is data self-filtering based on the cycle-consistency prior. We further discuss data quality evaluation strategies and build an automatic benchmark whose score correlates heavily with human annotations and allows automatic grid search of the best data pipeline hyperparameters. Overall, these advances allowed us to boost PanguVL grounding capabilities.
Object grounding is a foundational capability for Large Vision-Language Models that allows them to comprehend the visual modality. Robust learning of this capability in billion-parameter models at the pre-training stage requires a huge corpus of synthetic data and depends heavily on the quality of its object-level supervision. In this talk, we present a comprehensive methodology for data-driven improvement of this capability. We start by studying the effects of different data formats on downstream grounding benchmarks, and find grounded captions to be the best format for the pre-training stage. This is followed by our novel high-quality 2D object grounding data sampling pipeline from single images, whose core idea is data self-filtering based on the cycle-consistency prior. We further discuss data quality evaluation strategies and build an automatic benchmark whose score correlates heavily with human annotations and allows automatic grid search of the best data pipeline hyperparameters. Overall, these advances allowed us to boost PanguVL grounding capabilities.
Sergei Kuznetsov
HSE University
Concept-based Interpretable Learning
This talk is based on research that uses formal models of concepts, such as Formal Concept Analysis , Pattern Structures, and Concept Whitening to provide explanation of black-box models by aligning their output with symbolically represented inherently interpretable concepts.
Two types of pattern concepts are consdiered: graph patterns with application to NLP problems, and interval-tuple patterns with applications to the analysis of numerical data.
This talk is based on research that uses formal models of concepts, such as Formal Concept Analysis , Pattern Structures, and Concept Whitening to provide explanation of black-box models by aligning their output with symbolically represented inherently interpretable concepts.
Two types of pattern concepts are consdiered: graph patterns with application to NLP problems, and interval-tuple patterns with applications to the analysis of numerical data.
Andrey Savchenko
Sber AI Lab, HSE University
Deepfake detection of face videos in online conferences
The rise of AI-driven live impersonation in video conferencing poses a growing threat that current detection methods, developed mainly for social-media and broadcast content, do not directly address. In this talk, I briefly overview existing techniques that can used to solve this task. In addition, I describe the possibilities to create the dataset that can be used to train the accurate detection model.
The rise of AI-driven live impersonation in video conferencing poses a growing threat that current detection methods, developed mainly for social-media and broadcast content, do not directly address. In this talk, I briefly overview existing techniques that can used to solve this task. In addition, I describe the possibilities to create the dataset that can be used to train the accurate detection model.