Seminars 2024
The Lab helds invited talks on NLP, Recommender Systems, Data Mining, and related topics twice a month. For more details see the Russian page.
Seminar Automatic Processing and Analysis of Texts & rdquo; dedicated to various processing tasks (tokenization, recovery segmentation, part-of-speech markup and syntactic parsing) and textual information parsing (extraction tasks information, construction and use of knowledge graphs, construction of question-answer systems, text classification, etc.).
Seminar: "Investigation of transformer modifications for various long documents processing tasks"
Date: February 8, 2024
Speaker: Arij Al Adel, 4th year graduate student, Physics and Technology School of Radio Engineering and Computer Technologies, MIPT.
Annotation:
The following research presentation delves into the study aimed at proposing modifications to the attention mechanism of the encoder-decoder transformer. The proposed enhancements seek to overcome the limitations of attention while processing texts that exceed 512 tokens, with the ultimate goal of expanding the transformer input to thousands of tokens. This study gradually progresses from the initial stages of model design to the current, improved design, with experiments conducted on different natural language processing (NLP) tasks such as translation, Masked Language Modeling (MLM), Question Answering (QA), and summarization. The presentation concludes stage by stage with an overview of each stage's outcomes.
Record the seminar via the link
Presentation
Have you spotted a typo?
Highlight it, click Ctrl+Enter and send us a message. Thank you for your help!
To be used only for spelling or punctuation mistakes.