CLiC-it 2024 Paper

Our paper ‘Controllable Text Generation To Evaluate Linguistic Abilities of Italian LLMs’ (with Cristiano Ciaccio, Felice Dell’Orletta and Giulia Venturi) has been accepted to CLiC-it 2024! In this paper we propose a new evaluation framework leveraging the potentialities of Controllable Text Generation. Our approach evaluates the models' capacity to generate sentences that adhere to specific linguistic constraints and their ability to recognize the linguistic properties of their own generated sentences, also in terms of consistency with the specified constraints. We tested our approach on six Italian LLMs using various linguistic constraints.