CLiC-it 2021 Papers

Two papers accepted at CLiC-it 2021! In ‘Probing Tasks Under Stress’ (with Chiara Alzetta, Dominique Brunato, Felice Dell’Orletta and Giulia Venturi) we introduced a new approach to put increasingly under pressure the effectiveness of a suite of probing tasks to test the linguistic knowledge implicitly encoded by a BERT Italian model. To achieve this goal, we set up a number of experiments aimed at comparing the performance of a regression model trained with BERT representations to predict the values of a set of linguistic properties extracted from the Italian Universal Dependency Treebank and from a suite of control datasets we specifically built for the purpose of this study.
In ‘On the role of Textual Connectives in Sentence Comprehension: a new Dataset for Italian’ (with Giorgia Albertin, Alessio Miaschi and Dominique Brunato) we presented a new evaluation resource for Italian aimed at assessing the role of textual connectives in the comprehension of the meaning of a sentence. The resource is arranged in two sections, corresponding to a distinct challenge task conceived to test how subtle modifications involving connectives in real usage sentences influence the perceived acceptability of the sentence by native speakers and Neural Language Models (NLMs). Although the main focus is the presentation of the dataset, we also provided some preliminary data comparing human judgments and NLMs performance in the two tasks.