Three Papers Accepted at EMNLP & ArabicNLP 2025

Thrilled to announce that we have two papers have been accepted at EMNLP 2025, and one in the co-located ArabicNLP 2025 conference.

Beyond the Leaderboard: Understanding Performance Disparities in Large Language Models via Model Diffing

Fine-tuning LLMs is the go-to way to improve them, but most evaluations only tell you that a model got better, not why. Our paper takes a different approach using model diffing in order to compare the internal representations of two models. We specifically choose SimPO as a case study, and use crosscoders to find and categorize the latent concepts that differentiate the original Gemma-2-9b-it from its fine-tuned version. By looking at the mechanistic changes, we can attribute performance gains to concrete capabilities. This gives us a much richer, more actionable picture of what fine-tuning actually does.

• PDF
• Poster
• Video

Editing Across Languages: A Survey of Multilingual Knowledge Editing

Knowledge editing is the ability to update facts in an LLM after training. While this has been extensively studied in English, our survey is the first to systematically map out how this research extends to multilingual settings. This work consolidates a rapidly evolving area and lays the groundwork for building LLMs that can be edited reliably across languages.

• PDF
• Poster
• Video

An Exploration of Knowledge Editing for Arabic

This is the first study of knowledge editing for Arabic, a morphologically rich language that has been largely overlooked in this space. We tested four knowledge editing methods (ROME, MEMIT, ICE, and LTE) on Arabic translations of the ZsRE and Counterfact benchmarks, using Llama-2-7B-chat as our base model. We looked at both multilingual (editing in Arabic, verifying in Arabic) and cross-lingual (editing in English, verifying in Arabic) settings.

Key findings:

• Parameter-based methods (ROME, MEMIT) struggle significantly with cross-lingual generalization — editing a fact in English doesn’t reliably update it in Arabic
• Instruction-tuned methods (ICE, LTE) perform more robustly across languages
• Joint Arabic-English training improves both editability in Arabic and transfer between languages
• We extended Learning-To-Edit (LTE) to a multilingual setting

To help the community build on this work, we’re releasing Arabic knowledge editing benchmarks and multilingual training data for LTE on Hugging Face.

• PDF
• Poster
• Data
• Video

Congratulations to all the authors on these accepted papers!