New machine learning techniques for estimating and interpreting language models

Project goal

is to investigate new approaches to evaluate the quality, efficiency and performance of neural network language models.

Relevance

Currently, the possibilities for the evaluation and interpretation of neural network language models appear to be limited. Existing evaluation and interpretation methods are tied to specific target tasks or evaluate the sensitivity of language models to stand-alone linguistic phenomena (e.g., only to syntactic features of a particular language, typo resistance, or only to the ability to detect toxic comments).

Advantages

• The evaluation and interpretation apparatus developed in the project is compatible with architectures of various models (GPT, BERT, T5, etc.).
• The proposed apparatus makes it possible to identify the shortcomings of existing language models and sets new directions for the development of methods for teaching and training language models.
• The methodology of evaluation and interpretation of language models will allow to create a wide range of automated, objective standardised tests applicable to language models.
• Selection of optimal models is carried out using multi-criteria selection theory.

Scenario of application in practice:

1.   Faced with the need to select a language model for his needs, the user browses through a catalogue of models and their characteristics and independently selects the most suitable model.
2. The user specifies his requirements in the language model in a formalised form.
3. The developed platform provides the user with a list of models that fulfil his requirements.