Research interests
- Faithful LLM Reasoning
- Mechanistic Interpretability
- User Modeling and Personalization
- White-box Agent Auditing
Gabriele Sarti
Postdoc in LLM Interpretability
Bau Lab, Khoury College of Computer SciencesNortheastern University
Welcome to my website! 👋 I am a postdoc in the BauLab at Northeastern University, working on interpretability interfaces and scalable white-box monitoring in the National Deep Inference Fabric (NDIF) project.
I completed my PhD in NLP at the University of Groningen, where I worked on actionable interpretability for machine translation as a member of InCLoW, GroNLP and the InDeep consortium. Before that, I was a applied scientist intern at Amazon Translate and a research scientist at Aindo.
My research focuses on open tools, methods and interface to bridge the gap between interpretability research and model evaluations, and make white-box auditing a practical part of how we evaluate frontier AI. This is especially important as behavioural tests fail to surface unverbalized behaviors, and are increasingly inadequate as models get more capable.
Your (anonymous) constructive feedback is always welcome! 🙂
Previous Experience
Northeastern University2026 - OngoingPostdoctoral Researcher
Amazon Web Services AI Lab2022Applied Scientist Intern, Amazon Translate
Aindo2020 - 2021Research Scientist, Generative AI Systems
Education
University of Groningen2021 - 2025PhD in NLP InterpretabilityThesis: From Insights to Impact: Actionable Interpretability for Neural Machine Translation
University of Trieste & SISSA2018 - 2020MSc. in Data ScienceThesis: Interpreting Neural Language Models for Linguistic Complexity Assessment
News
All newsJune 2026
I am co-organizing the BlackboxNLP Workshop at EMNLP 2026! Participate in our reproducibility challenge! 🔍
June 2026
I worked with talented mentees as part of the Spring'26 SPAR Program on the project "Monitoring and Attributing Implicit Personalization in Conversational Agents, and will be a mentor for the CBAI Summer Fellowship as well! 🌱
April 2026
My PhD thesis on Actionable Interpretability for Machine Translation was awarded the 2025 Best Dissertation Award by the European Association for Machine Translation! Excited to present my work in Tilburg in June 🏆
January 2026
I started a postdoctoral position at Northeastern University as a member of the BauLab. Very excited to work with the NDIF team on building cutting-edge tools for interpretability research! 🔍
December 2025
I graduated cum laude with my PhD in Natural Language Processing from the University of Groningen! 🎓 My thesis on Actionable Interpretability for Machine Translation is now available online. Huge thanks to my supervisors Arianna Bisazza, Malvina Nissim and Grzegorz Chrupała for their support during these years.
February 2025
I am visiting the IRT Saint-Exupéry in Toulouse, France, to collaborate on an interpretability project with the DEEL team.
Selected publications
All publicationsBeyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
We estimate CoT steps causal importance via early exit and use this measure to study how answers form across the reasoning traces of several model families. Across diverse tasks, we find that reasoning typically crosses a commitment boundary—a sharp transition from transient intermediate guesses to a stable, high-confidence answer. This transition often happens in a single step, well before the model's reasoning block ends, and is followed by epiphenomenal CoT steps that leave the final answer probability unaltered. Using attention probes, we show that answer-formation stages can be linearly decoded from intermediate reasoning steps with high accuracy and generalize robustly to unseen reasoning tasks. We exploit this signal to early-exit reasoning blocks at the commitment boundary, reducing the length of CoTs up to 55% on average with negligible impact on model performance.
Predicting Future Behaviors in Reasoning Models Enables Better Steering
Prior steering work implicitly relies on internal features that detect behavior in already generated text, and these are poor predictors of future behavioral outcomes. Instead, we train activation probes to predict future behavior likelihoods from intermediate reasoning steps. These probes predict the most likely behavior with 64%–91% accuracy, revealing a separate type of internal prediction features. Building on these prediction features, we introduce a text-level steering method, Future Probe Controlled Generation. FPCG samples multiple candidate sentences and chooses the best one according to a probe predicting the future behavior likelihood. This enables steering with almost no output quality degradation. FPCG also enables steering in several evaluations where activation steering fails.
Agents of Chaos
Agents of Chaos is an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live lab environment with persistent memory, email accounts, messaging applications access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings.
A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents
We propose a framework for evaluating goal-directedness that integrates behavioural evaluation with interpretability-based analyses of models' internal representations. As a case study, we examine an LLM agent navigating a 2D grid world toward a goal state. Behaviourally, we evaluate the agent against an optimal policy across varying grid sizes, obstacle densities, and goal structures, finding that performance scales with task difficulty while remaining robust to difficulty-preserving transformations and complex goal structures. We then use probing methods to decode the agent's internal representations of the environment state and its multi-step action plans. We find that the LLM agent non-linearly encodes a coarse spatial map of the environment, preserving approximate task-relevant cues about its position and the goal location; that its actions are broadly consistent with these internal representations; and that reasoning reorganises them, shifting from broader environment structural cues toward information supporting immediate action selection.
Steering Large Language Models for Machine Translation Personalization
We explore various strategies for personalizing automatically generated translations when few examples are available, with a focus on the challenging domain of literary translation. After showing that classifiers can detect personalized translations with high accuracy, we evaluate various prompting strategies and inference-time interventions for steering model generations towards a personalized style, with a particular focus on contrastive steering with sparse autoencoder (SAE) latents to identify salient personalization properties. We demonstrate that contrastive SAE steering yields robust style conditioning and translation quality, resulting in higher inference-time computational efficiency than prompting approaches. We further examine the impact of steering on model activations, finding that layers encoding personalization properties are impacted similarly by prompting and SAE steering, suggesting a similar mechanism at play.
Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation
We present MIRAGE — Model Internals-based RAG Explanations — a plug-and-play approach using model internals for faithful answer attribution in RAG applications. MIRAGE detects context-sensitive answer tokens and pairs them with retrieved documents contributing to their prediction via saliency methods. We evaluate our proposed approach on a multilingual extractive QA dataset, finding high agreement with human answer attribution. On open-ended QA, MIRAGE achieves citation quality and efficiency comparable to self-citation while also allowing for a finer-grained control of attribution parameters. Our qualitative evaluation highlights the faithfulness of MIRAGE's attributions and underscores the promising application of model internals for RAG answer attribution.
Quantifying the Plausibility of Context Reliance in Neural Machine Translation
We introduce Plausibility Evaluation of Context Reliance (PECoRe), an end-to-end interpretability framework designed to quantify context usage in language models' generations. Our approach leverages model internals to (i) contrastively identify context-sensitive target tokens in generated texts and (ii) link them to contextual cues justifying their prediction. We use PECoRe to quantify the plausibility of context-aware machine translation models, comparing model rationales with human annotations across several discourse-level phenomena. Finally, we apply our method to unannotated model translations to identify context-mediated predictions and highlight instances of (im)plausible context usage throughout generation.
A Primer on the Inner Workings of Transformer-based Language Models
The rapid progress of research aimed at interpreting the inner workings of advanced language models has highlighted a need for contextualizing the insights gained from years of work in this area. This primer provides a concise technical introduction to the current techniques used to interpret the inner workings of Transformer-based language models, focusing on the generative decoder-only architecture. We conclude by presenting a comprehensive overview of the known internal mechanisms implemented by these models, uncovering connections across popular approaches and active research directions in this area.
Inseq: An Interpretability Toolkit for Sequence Generation Models
Inseq is a Python library to democratize access to interpretability analyses of sequence generation models. It enables intuitive and optimized extraction of models' internal information and feature importance scores for popular decoder-only and encoder-decoder Transformers architectures. We showcase its potential by adopting it to highlight gender biases in machine translation models and locate factual knowledge inside GPT-2. Thanks to its extensible interface supporting cutting-edge techniques such as contrastive feature attribution, Inseq can drive future advances in explainable natural language generation, centralizing good practices and enabling fair and reproducible model evaluations.
From Insights to Impact: Actionable Interpretability for Neural Machine Translation
Ph.D. Thesis, University of Groningen · Cum Laude, EAMT Best Thesis Award
This dissertation aims to bridge the gap between method-centric interpretability research and outcome-centric real-world machine translation applications. We develop novel methods to understand and control language model generation, then study how to integrate these advances effectively into human translation workflows. Our research spans three interconnected macro-themes: understanding how language models exploit contextual information during generation, controlling model generation for personalized translation outputs, and integrating interpretability insights into human translation workflows.