Tech Talk

I am glad to announce that on October 27 I will give a tech talk at Pi School.

Title: Interpreting Neural Language Models

Abstract: The field of Natural Language Processing (NLP) has seen an unprecedented progress in the last few years. Much of this progress is due to the replacement of traditional systems with newer and more powerful algorithms based on neural networks and deep learning. This improvement, however, comes at the cost of interpretability, since deep neural models offer little transparency about their inner workings and their abilities. Therefore, in the last few years, an increasingly large body of work has been devoted to the analysis and interpretation of these models. This talk will be divided into two parts. In the first part, we will briefly introduce Neural Language Models (NLMs) and the main techniques developed for interpreting their decisions and their inner linguistic knowledge. In the second part, we will see how to fine-tune one of the most popular NLM and then analyze its decisions according to two different interpretability methods: integrated gradients and analysis of attention matrices.

The materials on the talk are available here: Slides and Code.