Projects · Gabriele Sarti

Aletheia's Quest: An AI Lie Detection Challenge

A competition to build general-purpose lie detectors for LLMs

Ongoing

Aletheia's Quest: An AI Lie Detection Challenge project visual

Explore

I collaborated in the organization of Aletheia's Quest, a competition designed to move AI lie detection beyond methods tailored to a single model or behavior. In the first phase, pre-selected red teams produced diverse deception datasets and trained model organisms to lie in specific undisclosed scenarios. During the main competition, blue teams developed unified detectors for deception detection with either black-box access or privileged white-box access to weights and activations through NNsight and NDIF. Reproducible baselines and a public validation leaderboard allowed participants to obtain live feedback for their submissions. At the end of the competition, final methods were evaluated on held-out red teams datasets using OOD AUROC as the main success metric. The competition adopted explicit safeguards against organism-specific shortcuts and additional recognition for scientific novelty and computational scalability.

The NDIF Ecosystem: Open-source Tools and Infrastructure for Interpretability Research

A suite of tools for supporting interpretability research on deep neural networks

Ongoing

The NDIF Ecosystem: Open-source Tools and Infrastructure for Interpretability Research project visual

Explore

As a member of the National Deep Inference Fabric (NDIF) team, I collaborated to the development of a suite of open-source tools and infrastructure to support large-scale interpretability research on LLMs. The NDIF ecosystem is based on NNsight, a library built on PyTorch to simplify internal inspection and interventions on deep learning models, with first-class support for LLMs and remote execution capabilities on the NDIF server. NNterp extends NNsight with standardized naming and efficient implementations of common methods in interpretability research, and the NDIF Workbench is a web-based interface for prototyping and exploration, supporting interpretability educators. I personally contributed to the initial version of NDIF Skills to facilitate AI-assisted interpretability research, and to the development of the Interpretability Interchange Format (INIF) to enhance the reproducibility and portability of interpretability artifacts.

Unsupervised Machine Translation Quality Estimation

Estimating translation quality from the internal confidence of translation models

Past

Unsupervised Machine Translation Quality Estimation project visual

This project, marking the end of my PhD journey, investigated whether word-level translation errors can be identified directly from a model's uncertainty and internal signals. The QE4PE study analyzed translations by 42 professional translators across two language directions, releasing their behavioral logs and fine-grained error annotations collected with our custom GroTE post-editing interface. The results showed that domain, language and individual baselines were more predictive of post-editing speed than differences between human-made and automatic error span highlights. We benchmarked 14 supervised and unsupervised metrics across 12 translation directions and multiple annotators, finding substantial potential in efficient uncertainty- and interpretability-based methods while showing that supervised systems and single-annotator evaluations can be brittle under label disagreement (EMNLP 2025).

Culturally-grounded Reasoning for Italian Language Games

Language games as a testbed bridging logical reasoning and cultural understanding in LLMs

Past

Culturally-grounded Reasoning for Italian Language Games project visual

This project used Italian rebuses and crosswords as culturally-grounded benchmark requiring sequential reasoning and sociolinguistic knowledge to evaluate LLM abilities in Italian. We introduced EurekaRebus, a large collection of verbalized Italian rebuses, and showed that state-of-the-art LLMs struggle to solve them. The dataset later became part of the Italian CALAMITA challenge. The project later expanded to crosswords: Crossword Space introduced dual encoders that learn a shared latent space between clues and answers, winning the CLiC-it 2025 Best Paper Award, while Cruciverb-IT was the first shared task on Italian crossword solving, releasing a large dataset of clue-answer pairs and generated grids for evaluating answer generation and end-to-end solving.

Attributing Context Usage in Language Models

Tools and methods for scalable input attribution in language models

Past

Attributing Context Usage in Language Models project visual

Explore

This project was the first and most consistent part of my PhD thesis. The main deliverables were PECoRe, a scalable framework to identify and attribute context usage in LLM generations through a combination of information-theoretic measures and contrastive gradient attribution, and the Inseq library to democratize access to attribution methods for generative language models. While originally tested on neural machine translation models, we validated the PECoRe framework's effectiveness on LLMs for question answering with retrieval-augmented generation (MIRAGE), showing it can outperform self-citation approaches and lead to improved faithfulness to actual context usage. The framework is integrated in Inseq, supporting execution on all Hugging Face-compatible LLMs (inseq attribute-context).

Foundation Models for Italian NLP

Developing and evaluating Italian foundation models for text and vision

Past

Foundation Models for Italian NLP project visual

This project addressed the shortage of open foundation models and evaluation resources designed specifically for Italian across both text and vision. The main text deliverable was IT5, the first family of Italian encoder-decoder Transformers: we cleaned a 41-billion-word Italian split of mC4, pretrained four model sizes, and introduced the ItaGen benchmark for Italian understanding and generation. IT5 achieved a stronger scale-to-performance ratio than multilingual alternatives and set a new state of the art for Italian language generation. In parallel, CLIP-Italian was trained on more than 1.4 million Italian image-text pairs and outperformed multilingual CLIP on image retrieval and zero-shot classification. We released the IT5 and CLIP-Italian models alongside their training code, datasets and interactive demos.

Scholar Monitor

Active

A customizable Google Scholar personal monitor to improve citation tracking and impact analysis.

Website Code

Interpretability Interchange Format (INIF)

WIP

INIF is a standardized format for the representation of interpretability artifacts.

Code Documentation

NDIF Skills

WIP

NDIF Skills for Claude Code and Codex, supporting AI-assisted interpretability research.

Code

labl

Archived

labl is a minimal Python library for human-friendly span labeling and edit evaluation.

Code

Citation

Gabriele Sarti, Vilém Zouhar, Malvina Nissim, and Arianna Bisazza. 2025. Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.

      Gabriele Sarti, Vilém Zouhar, Malvina Nissim, and Arianna Bisazza. 2025. [Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement](https://doi.org/10.18653/v1/2025.emnlp-main.924). In *Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*. Association for Computational Linguistics.

[Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement](https://doi.org/10.18653/v1/2025.emnlp-main.924) (Sarti et al., EMNLP 2025)

      [Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement](https://doi.org/10.18653/v1/2025.emnlp-main.924) (Sarti et al., EMNLP 2025)

@inproceedings{sarti2025unsupervised,
  title = "Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement",
  author = "Sarti, Gabriele  and
      Zouhar, Vil{\'e}m  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
  pages = "18320--18337",
  publisher = "Association for Computational Linguistics",
  year = "2025",
  doi = "10.18653/v1/2025.emnlp-main.924",
  url = "https://doi.org/10.18653/v1/2025.emnlp-main.924",
}

      @inproceedings{sarti2025unsupervised,
  title = "Unsupervised Word-level Quality Estimation for Machine Translation Through the Lens of Annotators (Dis)agreement",
  author = "Sarti, Gabriele  and
      Zouhar, Vil{\'e}m  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
  pages = "18320--18337",
  publisher = "Association for Computational Linguistics",
  year = "2025",
  doi = "10.18653/v1/2025.emnlp-main.924",
  url = "https://doi.org/10.18653/v1/2025.emnlp-main.924",
}

Inseq

Active

Inseq is a Python library for democratizing access to attributional analyses of generative language models, supporting a wide range of attribution methods and model architectures.

Code Documentation

Citation

Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, and Arianna Bisazza. 2023. Inseq: An Interpretability Toolkit for Sequence Generation Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). Association for Computational Linguistics.

Gabriele Sarti, Nils Feldhus, Jirui Qi, Malvina Nissim, and Arianna Bisazza. 2024. Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit. In Proceedings of the 2nd World Conference on Explainable AI: System Demonstrations.

      Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal, Malvina Nissim, and Arianna Bisazza. 2023. [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://aclanthology.org/2023.acl-demo.40/). In *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)*. Association for Computational Linguistics.

Gabriele Sarti, Nils Feldhus, Jirui Qi, Malvina Nissim, and Arianna Bisazza. 2024. [Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit](https://ceur-ws.org/Vol-3793/paper_37.pdf). In *Proceedings of the 2nd World Conference on Explainable AI: System Demonstrations*.

[Inseq: An Interpretability Toolkit for Sequence Generation Models](https://aclanthology.org/2023.acl-demo.40/) (Sarti et al., ACL 2023)

[Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit](https://ceur-ws.org/Vol-3793/paper_37.pdf) (Sarti et al., xAI 2024)

      [Inseq: An Interpretability Toolkit for Sequence Generation Models](https://aclanthology.org/2023.acl-demo.40/) (Sarti et al., ACL 2023)

[Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit](https://ceur-ws.org/Vol-3793/paper_37.pdf) (Sarti et al., xAI 2024)

@inproceedings{sarti2023inseq,
  title = "Inseq: An Interpretability Toolkit for Sequence Generation Models",
  author = "Sarti, Gabriele  and
      Feldhus, Nils  and
      Sickert, Ludwig  and
      Wal, Oskar van der  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",
  pages = "421--435",
  publisher = "Association for Computational Linguistics",
  year = "2023",
  doi = "10.18653/v1/2023.acl-demo.40",
  url = "https://aclanthology.org/2023.acl-demo.40/",
}

@inproceedings{sarti2024democratizing,
  title = "Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit",
  author = "Sarti, Gabriele  and
      Feldhus, Nils  and
      Qi, Jirui  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 2nd World Conference on Explainable AI: System Demonstrations",
  year = "2024",
  url = "https://ceur-ws.org/Vol-3793/paper\_37.pdf",
}

      @inproceedings{sarti2023inseq,
  title = "Inseq: An Interpretability Toolkit for Sequence Generation Models",
  author = "Sarti, Gabriele  and
      Feldhus, Nils  and
      Sickert, Ludwig  and
      Wal, Oskar van der  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)",
  pages = "421--435",
  publisher = "Association for Computational Linguistics",
  year = "2023",
  doi = "10.18653/v1/2023.acl-demo.40",
  url = "https://aclanthology.org/2023.acl-demo.40/",
}

@inproceedings{sarti2024democratizing,
  title = "Democratizing Advanced Attribution Analyses of Generative Language Models with the Inseq Toolkit",
  author = "Sarti, Gabriele  and
      Feldhus, Nils  and
      Qi, Jirui  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 2nd World Conference on Explainable AI: System Demonstrations",
  year = "2024",
  url = "https://ceur-ws.org/Vol-3793/paper\_37.pdf",
}

GroTE

Archived

The Groningen Translation Environment (GroTE) is a Gradio post-editing interface supporting the editing of translations with highlighted spans for the study of word-level quality estimation.

Code Demo

Citation

Gabriele Sarti, Vilém Zouhar, Grzegorz Chrupała, Ana Guerberof-Arenas, Malvina Nissim, and Arianna Bisazza. 2025. QE4PE: Word-level Quality Estimation for Human Post-Editing. Transactions of the Association for Computational Linguistics. MIT Press.

      Gabriele Sarti, Vilém Zouhar, Grzegorz Chrupała, Ana Guerberof-Arenas, Malvina Nissim, and Arianna Bisazza. 2025. [QE4PE: Word-level Quality Estimation for Human Post-Editing](https://doi.org/10.1162/TACL.a.46). *Transactions of the Association for Computational Linguistics*. MIT Press.

[QE4PE: Word-level Quality Estimation for Human Post-Editing](https://doi.org/10.1162/TACL.a.46) (Sarti et al., TACL 2025)

      [QE4PE: Word-level Quality Estimation for Human Post-Editing](https://doi.org/10.1162/TACL.a.46) (Sarti et al., TACL 2025)

@article{sarti2025qe4pe,
  title = "QE4PE: Word-level Quality Estimation for Human Post-Editing",
  author = "Sarti, Gabriele  and
      Zouhar, Vil{\'e}m  and
      Chrupała, Grzegorz  and
      Guerberof-Arenas, Ana  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  journal = "Transactions of the Association for Computational Linguistics",
  volume = "13",
  pages = "1410--1435",
  publisher = "MIT Press",
  year = "2025",
  doi = "10.1162/tacl.a.46",
  url = "https://doi.org/10.1162/TACL.a.46",
}

      @article{sarti2025qe4pe,
  title = "QE4PE: Word-level Quality Estimation for Human Post-Editing",
  author = "Sarti, Gabriele  and
      Zouhar, Vil{\'e}m  and
      Chrupała, Grzegorz  and
      Guerberof-Arenas, Ana  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  journal = "Transactions of the Association for Computational Linguistics",
  volume = "13",
  pages = "1410--1435",
  publisher = "MIT Press",
  year = "2025",
  doi = "10.1162/tacl.a.46",
  url = "https://doi.org/10.1162/TACL.a.46",
}

Italian Crossword Space

A CLIP-style dual encoder for Italian crossword clues and answers, enabling semantic retrieval.

Model Code

Citation

Cristiano Ciaccio, Gabriele Sarti, Alessio Miaschi, and Felice Dell'Orletta. 2025. Crossword Space: Latent Manifold Learning for Italian Crosswords and Beyond. In Proceedings of the 11th Italian Conference on Computational Linguistics. CEUR Workshop Proceedings.

Cristiano Ciaccio, Gabriele Sarti, Alessio Miaschi, Felice Dell'Orletta, and Malvina Nissim. 2026. Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task. In Proceedings of the 9th Evaluation Campaign of Natural Language Processing Tools for Italian.

      Cristiano Ciaccio, Gabriele Sarti, Alessio Miaschi, and Felice Dell'Orletta. 2025. [Crossword Space: Latent Manifold Learning for Italian Crosswords and Beyond](https://aclanthology.org/2025.clicit-1.26/). In *Proceedings of the 11th Italian Conference on Computational Linguistics*. CEUR Workshop Proceedings.

Cristiano Ciaccio, Gabriele Sarti, Alessio Miaschi, Felice Dell'Orletta, and Malvina Nissim. 2026. [Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task](https://apa.dipsco.unitn.it/evalita2026/62.pdf). In *Proceedings of the 9th Evaluation Campaign of Natural Language Processing Tools for Italian*.

[Crossword Space: Latent Manifold Learning for Italian Crosswords and Beyond](https://aclanthology.org/2025.clicit-1.26/) (Ciaccio et al., CLiC-it 2025)

[Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task](https://apa.dipsco.unitn.it/evalita2026/62.pdf) (Ciaccio et al., EVALITA 2026)

      [Crossword Space: Latent Manifold Learning for Italian Crosswords and Beyond](https://aclanthology.org/2025.clicit-1.26/) (Ciaccio et al., CLiC-it 2025)

[Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task](https://apa.dipsco.unitn.it/evalita2026/62.pdf) (Ciaccio et al., EVALITA 2026)

@inproceedings{ciaccio2025crossword,
  title = "Crossword Space: Latent Manifold Learning for Italian Crosswords and Beyond",
  author = "Ciaccio, Cristiano  and
      Sarti, Gabriele  and
      Miaschi, Alessio  and
      Dell'Orletta, Felice",
  booktitle = "Proceedings of the 11th Italian Conference on Computational Linguistics",
  pages = "245--255",
  publisher = "CEUR Workshop Proceedings",
  year = "2025",
  url = "https://aclanthology.org/2025.clicit-1.26/",
}

@inproceedings{ciaccio2026cruciverb,
  title = "Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task",
  author = "Ciaccio, Cristiano  and
      Sarti, Gabriele  and
      Miaschi, Alessio  and
      Dell'Orletta, Felice  and
      Nissim, Malvina",
  booktitle = "Proceedings of the 9th Evaluation Campaign of Natural Language Processing Tools for Italian",
  year = "2026",
  url = "https://apa.dipsco.unitn.it/evalita2026/62.pdf",
}

      @inproceedings{ciaccio2025crossword,
  title = "Crossword Space: Latent Manifold Learning for Italian Crosswords and Beyond",
  author = "Ciaccio, Cristiano  and
      Sarti, Gabriele  and
      Miaschi, Alessio  and
      Dell'Orletta, Felice",
  booktitle = "Proceedings of the 11th Italian Conference on Computational Linguistics",
  pages = "245--255",
  publisher = "CEUR Workshop Proceedings",
  year = "2025",
  url = "https://aclanthology.org/2025.clicit-1.26/",
}

@inproceedings{ciaccio2026cruciverb,
  title = "Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task",
  author = "Ciaccio, Cristiano  and
      Sarti, Gabriele  and
      Miaschi, Alessio  and
      Dell'Orletta, Felice  and
      Nissim, Malvina",
  booktitle = "Proceedings of the 9th Evaluation Campaign of Natural Language Processing Tools for Italian",
  year = "2026",
  url = "https://apa.dipsco.unitn.it/evalita2026/62.pdf",
}

Italian Verbalized Rebus Solvers

LLMs trained to solve verbalized rebus puzzles in Italian, as part of the EurekaRebus project.

Models Code Demo

Citation

Gabriele Sarti, Tommaso Caselli, Malvina Nissim, and Arianna Bisazza. 2024. Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses. In Proceedings of the 10th Italian Conference on Computational Linguistics. CEUR Workshop Proceedings.

Gabriele Sarti, Tommaso Caselli, Arianna Bisazza, and Malvina Nissim. 2024. EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge. In Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024). CEUR Workshop Proceedings.

      Gabriele Sarti, Tommaso Caselli, Malvina Nissim, and Arianna Bisazza. 2024. [Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses](https://aclanthology.org/2024.clicit-1.96/). In *Proceedings of the 10th Italian Conference on Computational Linguistics*. CEUR Workshop Proceedings.

Gabriele Sarti, Tommaso Caselli, Arianna Bisazza, and Malvina Nissim. 2024. [EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge](https://aclanthology.org/2024.clicit-1.132/). In *Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)*. CEUR Workshop Proceedings.

[Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses](https://aclanthology.org/2024.clicit-1.96/) (Sarti et al., CLiC-it 2024)

[EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge](https://aclanthology.org/2024.clicit-1.132/) (Sarti et al., CLiC-it 2024)

      [Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses](https://aclanthology.org/2024.clicit-1.96/) (Sarti et al., CLiC-it 2024)

[EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge](https://aclanthology.org/2024.clicit-1.132/) (Sarti et al., CLiC-it 2024)

@inproceedings{sarti2024verbis,
  title = "Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses",
  author = "Sarti, Gabriele  and
      Caselli, Tommaso  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 10th Italian Conference on Computational Linguistics",
  pages = "888--897",
  publisher = "CEUR Workshop Proceedings",
  year = "2024",
  url = "https://aclanthology.org/2024.clicit-1.96/",
}

@inproceedings{sarti2024eurekarebus,
  title = "EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge",
  author = "Sarti, Gabriele  and
      Caselli, Tommaso  and
      Bisazza, Arianna  and
      Nissim, Malvina",
  booktitle = "Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)",
  pages = "1202--1208",
  publisher = "CEUR Workshop Proceedings",
  year = "2024",
  url = "https://aclanthology.org/2024.clicit-1.132/",
}

      @inproceedings{sarti2024verbis,
  title = "Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses",
  author = "Sarti, Gabriele  and
      Caselli, Tommaso  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 10th Italian Conference on Computational Linguistics",
  pages = "888--897",
  publisher = "CEUR Workshop Proceedings",
  year = "2024",
  url = "https://aclanthology.org/2024.clicit-1.96/",
}

@inproceedings{sarti2024eurekarebus,
  title = "EurekaRebus - Verbalized Rebus Solving with LLMs: A CALAMITA Challenge",
  author = "Sarti, Gabriele  and
      Caselli, Tommaso  and
      Bisazza, Arianna  and
      Nissim, Malvina",
  booktitle = "Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)",
  pages = "1202--1208",
  publisher = "CEUR Workshop Proceedings",
  year = "2024",
  url = "https://aclanthology.org/2024.clicit-1.132/",
}

IT5

A suite of Italian T5 models pre-trained on the mC4-IT corpus.

Models Code Demo

Citation

Gabriele Sarti and Malvina Nissim. 2024. IT5: Text-to-text Pretraining for Italian Language Understanding and Generation. In Joint International Conference on Computational Linguistics, Language Resources and Evaluation. ELRA and ICCL.

      Gabriele Sarti and Malvina Nissim. 2024. [IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://aclanthology.org/2024.lrec-main.823/). In *Joint International Conference on Computational Linguistics, Language Resources and Evaluation*. ELRA and ICCL.

[IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://aclanthology.org/2024.lrec-main.823/) (Sarti et al., LREC-COLING 2024)

      [IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://aclanthology.org/2024.lrec-main.823/) (Sarti et al., LREC-COLING 2024)

@inproceedings{sarti2024text,
  title = "IT5: Text-to-text Pretraining for Italian Language Understanding and Generation",
  author = "Sarti, Gabriele  and
      Nissim, Malvina",
  booktitle = "Joint International Conference on Computational Linguistics, Language Resources and Evaluation",
  pages = "9422--9433",
  publisher = "ELRA and ICCL",
  year = "2024",
  url = "https://aclanthology.org/2024.lrec-main.823/",
}

      @inproceedings{sarti2024text,
  title = "IT5: Text-to-text Pretraining for Italian Language Understanding and Generation",
  author = "Sarti, Gabriele  and
      Nissim, Malvina",
  booktitle = "Joint International Conference on Computational Linguistics, Language Resources and Evaluation",
  pages = "9422--9433",
  publisher = "ELRA and ICCL",
  year = "2024",
  url = "https://aclanthology.org/2024.lrec-main.823/",
}

CLIP-Italian

A CLIP-style dual encoder for Italian text and images.

Model Code Demo Video Blog

Citation

Federico Bianchi, Giuseppe Attanasio, Raphael Pisoni, Silvia Terragni, Gabriele Sarti, and Dario Balestri. 2023. Contrastive Language-Image Pre-training for the Italian Language. In Proceedings of the 9th Italian Conference on Computational Linguistics. CEUR Workshop Proceedings.

      Federico Bianchi, Giuseppe Attanasio, Raphael Pisoni, Silvia Terragni, Gabriele Sarti, and Dario Balestri. 2023. [Contrastive Language-Image Pre-training for the Italian Language](https://aclanthology.org/2023.clicit-1.11/). In *Proceedings of the 9th Italian Conference on Computational Linguistics*. CEUR Workshop Proceedings.

[Contrastive Language-Image Pre-training for the Italian Language](https://aclanthology.org/2023.clicit-1.11/) (Bianchi et al., CLiC-it 2023)

      [Contrastive Language-Image Pre-training for the Italian Language](https://aclanthology.org/2023.clicit-1.11/) (Bianchi et al., CLiC-it 2023)

@inproceedings{bianchi2023contrastive,
  title = "Contrastive Language-Image Pre-training for the Italian Language",
  author = "Bianchi, Federico  and
      Attanasio, Giuseppe  and
      Pisoni, Raphael  and
      Terragni, Silvia  and
      Sarti, Gabriele  and
      Balestri, Dario",
  booktitle = "Proceedings of the 9th Italian Conference on Computational Linguistics",
  pages = "78--85",
  publisher = "CEUR Workshop Proceedings",
  year = "2023",
  url = "https://aclanthology.org/2023.clicit-1.11/",
}

      @inproceedings{bianchi2023contrastive,
  title = "Contrastive Language-Image Pre-training for the Italian Language",
  author = "Bianchi, Federico  and
      Attanasio, Giuseppe  and
      Pisoni, Raphael  and
      Terragni, Silvia  and
      Sarti, Gabriele  and
      Balestri, Dario",
  booktitle = "Proceedings of the 9th Italian Conference on Computational Linguistics",
  pages = "78--85",
  publisher = "CEUR Workshop Proceedings",
  year = "2023",
  url = "https://aclanthology.org/2023.clicit-1.11/",
}

Cruciverb-IT

A dataset of Italian crossword clues and answers for research on reasoning in language games.

EVALITA '26 Dataset Augmented Dataset Code

Citation

      Cristiano Ciaccio, Gabriele Sarti, Alessio Miaschi, Felice Dell'Orletta, and Malvina Nissim. 2026. [Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task](https://apa.dipsco.unitn.it/evalita2026/62.pdf). In *Proceedings of the 9th Evaluation Campaign of Natural Language Processing Tools for Italian*.

[Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task](https://apa.dipsco.unitn.it/evalita2026/62.pdf) (Ciaccio et al., EVALITA 2026)

      [Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task](https://apa.dipsco.unitn.it/evalita2026/62.pdf) (Ciaccio et al., EVALITA 2026)

@inproceedings{ciaccio2026cruciverb,
  title = "Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task",
  author = "Ciaccio, Cristiano  and
      Sarti, Gabriele  and
      Miaschi, Alessio  and
      Dell'Orletta, Felice  and
      Nissim, Malvina",
  booktitle = "Proceedings of the 9th Evaluation Campaign of Natural Language Processing Tools for Italian",
  year = "2026",
  url = "https://apa.dipsco.unitn.it/evalita2026/62.pdf",
}

      @inproceedings{ciaccio2026cruciverb,
  title = "Cruciverb-IT@ EVALITA 2026: Overview of the Crossword Solving in Italian Task",
  author = "Ciaccio, Cristiano  and
      Sarti, Gabriele  and
      Miaschi, Alessio  and
      Dell'Orletta, Felice  and
      Nissim, Malvina",
  booktitle = "Proceedings of the 9th Evaluation Campaign of Natural Language Processing Tools for Italian",
  year = "2026",
  url = "https://apa.dipsco.unitn.it/evalita2026/62.pdf",
}

QE4PE

Professional MT post-editing with word-level error spans, to study word-level quality estimation.

Dataset Code

Citation

      Gabriele Sarti, Vilém Zouhar, Grzegorz Chrupała, Ana Guerberof-Arenas, Malvina Nissim, and Arianna Bisazza. 2025. [QE4PE: Word-level Quality Estimation for Human Post-Editing](https://doi.org/10.1162/TACL.a.46). *Transactions of the Association for Computational Linguistics*. MIT Press.

[QE4PE: Word-level Quality Estimation for Human Post-Editing](https://doi.org/10.1162/TACL.a.46) (Sarti et al., TACL 2025)

      [QE4PE: Word-level Quality Estimation for Human Post-Editing](https://doi.org/10.1162/TACL.a.46) (Sarti et al., TACL 2025)

@article{sarti2025qe4pe,
  title = "QE4PE: Word-level Quality Estimation for Human Post-Editing",
  author = "Sarti, Gabriele  and
      Zouhar, Vil{\'e}m  and
      Chrupała, Grzegorz  and
      Guerberof-Arenas, Ana  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  journal = "Transactions of the Association for Computational Linguistics",
  volume = "13",
  pages = "1410--1435",
  publisher = "MIT Press",
  year = "2025",
  doi = "10.1162/tacl.a.46",
  url = "https://doi.org/10.1162/TACL.a.46",
}

      @article{sarti2025qe4pe,
  title = "QE4PE: Word-level Quality Estimation for Human Post-Editing",
  author = "Sarti, Gabriele  and
      Zouhar, Vil{\'e}m  and
      Chrupała, Grzegorz  and
      Guerberof-Arenas, Ana  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  journal = "Transactions of the Association for Computational Linguistics",
  volume = "13",
  pages = "1410--1435",
  publisher = "MIT Press",
  year = "2025",
  doi = "10.1162/tacl.a.46",
  url = "https://doi.org/10.1162/TACL.a.46",
}

EurekaRebus

A collection of more than 200k verbalized rebus puzzles in Italian for research on reasoning in language games.

Dataset Code Demo

Citation

      Gabriele Sarti, Tommaso Caselli, Malvina Nissim, and Arianna Bisazza. 2024. [Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses](https://aclanthology.org/2024.clicit-1.96/). In *Proceedings of the 10th Italian Conference on Computational Linguistics*. CEUR Workshop Proceedings.

[Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses](https://aclanthology.org/2024.clicit-1.96/) (Sarti et al., CLiC-it 2024)

      [Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses](https://aclanthology.org/2024.clicit-1.96/) (Sarti et al., CLiC-it 2024)

@inproceedings{sarti2024verbis,
  title = "Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses",
  author = "Sarti, Gabriele  and
      Caselli, Tommaso  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 10th Italian Conference on Computational Linguistics",
  pages = "888--897",
  publisher = "CEUR Workshop Proceedings",
  year = "2024",
  url = "https://aclanthology.org/2024.clicit-1.96/",
}

      @inproceedings{sarti2024verbis,
  title = "Non Verbis, Sed Rebus: Large Language Models Are Weak Solvers of Italian Rebuses",
  author = "Sarti, Gabriele  and
      Caselli, Tommaso  and
      Nissim, Malvina  and
      Bisazza, Arianna",
  booktitle = "Proceedings of the 10th Italian Conference on Computational Linguistics",
  pages = "888--897",
  publisher = "CEUR Workshop Proceedings",
  year = "2024",
  url = "https://aclanthology.org/2024.clicit-1.96/",
}

DivEMT

Machine translation post-editing with behavioral logs across six typologically diverse target languages.

Dataset Code

mC4 IT

A cleaned Italian split of mC4 containing ~41 billion words, used to pretrain the IT5 model suite.

Dataset Code

Citation

      Gabriele Sarti and Malvina Nissim. 2024. [IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://aclanthology.org/2024.lrec-main.823/). In *Joint International Conference on Computational Linguistics, Language Resources and Evaluation*. ELRA and ICCL.

[IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://aclanthology.org/2024.lrec-main.823/) (Sarti et al., LREC-COLING 2024)

      [IT5: Text-to-text Pretraining for Italian Language Understanding and Generation](https://aclanthology.org/2024.lrec-main.823/) (Sarti et al., LREC-COLING 2024)

@inproceedings{sarti2024text,
  title = "IT5: Text-to-text Pretraining for Italian Language Understanding and Generation",
  author = "Sarti, Gabriele  and
      Nissim, Malvina",
  booktitle = "Joint International Conference on Computational Linguistics, Language Resources and Evaluation",
  pages = "9422--9433",
  publisher = "ELRA and ICCL",
  year = "2024",
  url = "https://aclanthology.org/2024.lrec-main.823/",
}

      @inproceedings{sarti2024text,
  title = "IT5: Text-to-text Pretraining for Italian Language Understanding and Generation",
  author = "Sarti, Gabriele  and
      Nissim, Malvina",
  booktitle = "Joint International Conference on Computational Linguistics, Language Resources and Evaluation",
  pages = "9422--9433",
  publisher = "ELRA and ICCL",
  year = "2024",
  url = "https://aclanthology.org/2024.lrec-main.823/",
}

Current Projects

Explore

Explore

Past Projects

Explore

Tools

Scholar Monitor

Interpretability Interchange Format (INIF)

NDIF Skills

labl

Inseq

GroTE

Models

Italian Crossword Space

Italian Verbalized Rebus Solvers

IT5

CLIP-Italian

Datasets

Cruciverb-IT

QE4PE

EurekaRebus

DivEMT

mC4 IT