Семинар HDI&TFAIM Lab «GFlowNets for Scientific Discovery: From Mathematical Reasoning to Biological Sequence Design»

This talk presents two recent applications of Generative Flow Networks to scientific discovery problems where diversity is essential for success.First, we address mathematical reasoning in large language models. We develop step-level GFlowNets paired with automatically-trained Process Reward Models (PRMs). Unlike standard RL fine-tuning which suffers from mode collapse, our approach maintains solution diversity while achieving superior generalization, showing a +9.4% improvement on out-of-distribution SAT MATH problems with a 3B parameter model. The innovation is a similarity-based data augmentation technique that transforms 100k MCTS rollouts into 2.1M training examples by reusing intermediate steps, enabling robust step-level reward modeling without any human annotation. Our PRM outperforms existing open-source alternatives while our GFlowNet trains 2× faster than PPO.Second, we tackle mRNA sequence design, where codon redundancy creates astronomical combinatorial spaces (>10^632 sequences for typical proteins). We introduce Curriculum-Augmented GFlowNets (CAGFN) that dynamically adapt training difficulty based on learning progress, enabling efficient exploration while simultaneously optimizing multiple biological objectives (codon adaptation index, minimum free energy, GC content). CAGFN trains 4× faster than non-curriculum baselines, achieves superior Pareto-front coverage (up to 22% of generated sequences), and generalizes effectively to proteins of varying lengths.Both applications demonstrate that GFlowNets' reward-proportional sampling provides major advantages over optimization-only methods in scientific domains where robustness and exploration are paramount.

По всем вопросам обращайтесь к Зеленовой Карине Михайловне kzelenova@hse.ru или к Горностаевой Екатерине Дмитриевне egornostaeva@hse.ru