https://fdalvi.github.io/tags/speech/ Recent content in Speech on Fahim Dalvi Hugo en Fri, 15 Aug 2025 13:00:00 +0300 https://fdalvi.github.io/blog/2025-08-15-from-words-to-waves-interspeech-2025/ Fri, 15 Aug 2025 13:00:00 +0300 https://fdalvi.github.io/blog/2025-08-15-from-words-to-waves-interspeech-2025/ <p>Our paper <a href="https://doi.org/10.21437/Interspeech.2025-2180">From Words to Waves: Analyzing Concept Formation in Speech and Text-Based Foundation Models</a> has been accepted at <a href="https://interspeech2025.org/">Interspeech 2025</a>.</p> <p>LLMs have shown that text-only training can give models remarkable reasoning abilities and abstract semantic understanding. This raises a fascinating question: <strong>do speech models develop similar conceptual structures when trained only on audio?</strong> And when models are trained on both speech and text together, do they build a richer understanding?</p> <p>We used <strong>Latent Concept Analysis</strong> from our prior work on interpretability to examine how semantic abstractions form across modalities, and find lots of interesting differences on how speech and text modalities differ in their internal representations. We released our code and a curated audio version of the SST-2 dataset on <a href="https://github.com/shammur/MultimodalXplain">GitHub</a> and <a href="https://huggingface.co/collections/QCRI/multimodalxplain">Hugging Face</a> to support reproducibility.</p>