INTERSPEECH 2026
Deserved a Better Recap
Notes from INTERSPEECH 2026 in Sydney, including talks and posters on speech representations, speech LLMs, clinical speech, and deepfakes.
Four days at INTERSPEECH 2026 in Sydney were enough to fill my notes, but nowhere near enough to see the whole program. Between the talks and poster sessions, there was plenty to learn, including in areas outside my usual work. There was also time to catch up with colleagues, talk through ideas, and see a little of Sydney.
These are a few moments from the week and some papers that interested me, mostly on speech representations, speech language models, clinical speech, and deepfakes. To everyone whose work I missed, I wish I had made it to more sessions. This is one person’s experience of a very good program, not an attempt to cover it all.
I took photos along the way. Some posters are tilted and a few speakers are blurry, but the paper links should make up for my photography.
Sydney in four days
Monday began with John Hansen’s keynote on balancing machine learning with speech, language, and hearing science. His emphasis on knowing the problem and the data was a good opening to the conference. With so much attention on larger models, there is still a lot to learn from how people speak and listen.
Later that morning I was at our poster on unsupervised audio retrieval, having the kind of conversations that make poster sessions so enjoyable. Someone comes with a question, stays for a second one, and leaves thinking about how it connects to their own work.
Sabato Marco Siniscalchi gave the survey talk Monaural Speech Enhancement: From Shallow Networks to Speech Foundation Models (slides). Marco gave a clear account of decades of research and how the field got here, from early neural networks through mapping and masking approaches to generative and foundation models. What I appreciated was the care in connecting those steps, explaining what each approach was trying to solve rather than simply moving through a list of architectures. That kind of synthesis takes a lot of work, and makes it much easier to understand where the open questions are.
In the evening, we walked along the harbour to the Welcome Reception. It was a chance to catch up with friends and talk through research ideas in a more informal setting, without watching the time for the next session. Sydney gave us a beautiful setting for those conversations. At the reception, Australian wildlife joined the program too, including a small crocodile and a very relaxed marsupial.
Tuesday was another day of talks and posters, with topics ranging from universal audio encoders to deepfake detection and the effect of a cold on someone’s voice. In the afternoon, Jinyu Li’s survey talk, End-to-End Automatic Speech Recognition, gave us a clear view of what is established and what still needs work. His closing takeaways covered the maturity of encoder-adapter-LLM systems, the need to demonstrate what the LLM actually contributes, and the challenges of long recordings and efficient decoding. I appreciated that balance between progress and practical questions. A survey is especially useful when it helps us decide what to investigate next, and I made sure to photograph that final slide.
On Wednesday, I spent time with work on speech language models and clinical speech, and presented our oral on etiology-aware dysarthric speech recognition in the afternoon. In the evening, Alkis Koudounas presented our synthetic pathological speech poster. The photos from that day end at Darling Harbour, after the sessions.
Thursday morning began with Yu Tsao’s survey talk, Bio-Informed Speech Processing, Modeling, and Generation (slides). This was a highlight for me. Yu connected speech generation with the biology of producing and hearing speech, bringing in articulatory movement, muscle activity, and the listener’s hearing profile. The range was impressive, but the direction stayed clear: information about the speaker and listener can help us build speech systems that better serve them. It was a generous overview of the research, with a strong sense of what these connections could make possible.
Junichi Yamagishi’s keynote then focused on countermeasures against the misuse of speech generative AI, connecting detection with proactive protection. Alkis also presented SHALLOW on the main stage that day. A full final day, with plenty still to think about on the way home.
Papers from the week
These are some of the papers I had the chance to spend time with during the talks and poster sessions. There was a lot of thoughtful work across these areas, and I wanted to share a few of the ideas that interested me, with links for anyone who would like to read more.
Speech representations
I enjoyed the different ways these papers approached a shared problem: which parts of an audio signal should a model preserve? Some aim to isolate linguistic content, others to cover more kinds of sound or understand what a model has already learned. Together, they give a good sense of how much there is still to explore in audio representations.
PINT has a simple idea: train a tokenizer on different speakers saying the same words under different conditions. The shared content helps it preserve the words while reducing speaker information. In the reported experiments, speaker identity becomes harder to recover, while phonetic discrimination and language modeling improve. I would be curious to see what a speech language model learns on top of these tokens.
USAD 2.0 takes on the ambitious task of building one encoder for speech, environmental sound, and music. It brings together knowledge from specialist models through distillation. I like the effort to make those different kinds of expertise useful within a shared encoder, rather than treating each audio domain separately.
Three more papers added useful perspectives:
- ProLAP makes room for the fact that the same sound can have several valid descriptions. Representing audio and text as distributions lets it capture different levels of specificity, such as “a dog barking” and “an animal sound”. It is a thoughtful way to account for ambiguity rather than force a single match.
- DiscoPhon provides a multilingual benchmark for phoneme discovery from just ten hours of speech in an unseen language. I appreciated the clear linguistic question behind it: how well do the units we learn actually correspond to phonemes across languages?
- TIMBRE puts substantial experimental effort into understanding which layers transfer best for speech emotion recognition across languages. Its strongest cross-corpus transfer comes around one third of the way through the model. That systematic analysis makes the paper a useful reference when deciding which layers to use.
Speech language models
There was plenty of energy around speech language models, and I liked seeing work on both making them more practical and understanding their behavior. These papers look at efficiency, reasoning, and evaluation, with concrete questions behind each.
AdaTS offers a practical way to reduce the number of speech tokens sent to the language model. Instead of downsampling at a fixed rate, it merges tokens where the signal is less informative. The paper reports at least a halving of the input token sequence while retaining strong performance across recognition, question answering, and translation. What I like is that it uses the audio itself to decide where the model needs more detail.
SPARE examines a less intuitive result: asking an audio model to reason step by step can make its answers worse. The authors hypothesize that long reasoning sequences shift attention away from the audio, and train a summary-aligned internal token to help guide the reasoning. It is good to see this behavior investigated directly, with an approach that tries to keep the reasoning connected to what the model “heard”.
LlaSep uses language modeling for speaker separation, generating a codec-token stream for each speaker from a mixture. I found this an interesting use of the same approach: the answer is separated audio, not a written description of it.
Preference-ASR gives proper attention to a practical part of transcription: whether the recognizer produces the format the user requested. Numbers, names, disfluencies, and capitalization involve choices that ordinary normalization can erase. Model rankings change with those preferences. I liked seeing those choices treated as something worth evaluating, rather than just cleaning up afterward.
MSU-Bench brings multi-speaker conversations into the evaluation, testing whether models can connect speakers to what they said and reason about the dialogue. Its focus on who is speaking adds an important dimension beyond recognizing the words in isolation.
Speech and health
Clinical speech is close to my own work, and it was good to see the range of questions being addressed, from temporary changes in the voice to ways of learning with limited clinical data.
The Common Cold Corpus follows the same people in healthy and cold-affected conditions, using recordings made remotely on their own devices. Modern speaker embeddings shift consistently between the two conditions, even when changes in conventional acoustic features are less clear. A familiar experience, sounding different when we have a cold, becomes a concrete question about the stability of speaker representations.
Augmenting Dysarthric Speech Severity Assessment with MOS Supervision interested me for its choice of training data. Human ratings collected to evaluate synthetic speech help assess dysarthric speech, improving intelligibility and naturalness prediction through fine-tuning. With clinical annotations often scarce, finding useful supervision in a different research area is an appealing idea.
Speech graphs for dementia detection look at interpretable features of speech structure. I appreciate work that keeps the description of the signal understandable, especially in a clinical setting.
CLEAR uses a pretrained Whisper encoder as part of a system for reconstructing noisy heart sounds. It was interesting to see knowledge learned from speech put to work on a different kind of clinical audio.
Audio deepfakes
Following Yamagishi’s keynote, it was interesting to see papers turn some of those broader security questions into specific evaluation and modeling problems. Both of these look beyond giving a whole recording a single real-or-fake label.
FakeSound2 evaluates localization, source tracing, and generalization to unseen sources. I appreciated the broader view of what a detector should tell us: where a manipulation is, what produced it, and whether the system still works when the source changes.
Temporal Transition-Aware Multi-Head Modeling concentrates on the boundaries of partially spoofed audio. It models transitions from real to fake and back, alongside frame-level authenticity, to locate short manipulations inside otherwise genuine speech. The focus on those transitions is a neat way to use the structure of the problem.
Our papers
We brought some work to Sydney too. It was good to share it in person and discuss the questions behind it. Here is a short introduction to each paper, with code links if you would like to take a closer look:
- Etiology-Aware Speech Language Models for Dysarthric Speech Recognition. We asked whether predicting the cause of dysarthria could help a model transcribe the speech. Predicting the condition before the transcript worked best, without needing clinical labels at inference. A separate classifier was better at identifying the condition but did not improve recognition, which made the way we used that information especially interesting. Code.
- SSL-based Sequence Matching for Unsupervised Audio Retrieval. Here we looked at matching audio without labels, using humming and spoken queries. The two tasks favored different approaches: continuous sequences for humming, and discrete units with text retrieval methods for speech. Code.
- Synthetic Pathological Speech at Scale. We explored whether generating healthy and pathological vowel sounds could help with the shortage of clinical data. In the evaluated settings, classifiers trained on synthetic samples outperformed real-only baselines. Code.
- Hallucination Benchmark for Speech Foundation Models. With SHALLOW, we wanted a clearer picture of what goes wrong when a recognizer hallucinates. The benchmark examines four dimensions of hallucination, helping distinguish error patterns that a single word error rate can hide. Code.
- RT-SEMamba. This work looks at real-time speech enhancement with Mamba, using progressive distillation to compress an eight-layer model into a one-layer student. The aim is to keep more of the larger model’s quality within a smaller system. Code.
Coming home
Looking back through the photos, I am glad the week included both work close to mine and ideas I would not normally spend an afternoon with. A program this varied gives you plenty of reasons to try another session. Conversations with the people behind the papers make it easier to follow up afterward.
There was a lot to enjoy in Sydney beyond the sessions too. The harbour walks gave us time to talk, think about possible work together, and enjoy being somewhere new. I am grateful to the colleagues who shared the week, both inside the conference and outside it.
Thank you to everyone who made INTERSPEECH 2026 happen, from the organizers and volunteers to the session chairs and everyone who shared their work. It was a great week of speech research and good company.