Skip to content

Speech Enhancement Models 2026

This is a short review of the frontier of SE models in June of 2026.

State-Space Machines

In 2024, the SEMamba model was published, which introduces Mamba as a backbone in the MP-SENet, and brought SSMs to the forefront of speech processing. The model placed second at the URGENT 2024 challenge. Several models build on top of SEMamba:

  • MC-SEMamba Simple multi-channel adaption of SEMamba. The first convolutional layer in the SEMamba has two input channels, for magnitude and phase each. For MC-SEMamba, this is simply scaled by the number of input channels. The output is single channel, the magnitude mask and phase are predicted for a single reference microphone.
  • USEMamba Iteration on SEMamba, that simplifies losses and makes the model a hybrid of generative and discriminative, improving on tasks like deverberation and bandwith extension.
  • AVSEMamba Audio-visual version of SEMamba. A pretrained ResNet-18 extracts spatio-temporal features from full face video and stacks it with the magnitude and phase as an extra input channel to SEMamba.
  • SEMamba++ This model uses Mamba only for the time axis, for the frequency axis blocks based on [[Fourier Analysis Networks]] are used that better adapt to the periodic nature of the data. Furthermore, multi-timescale processing is introduced to better capture global features of the speech. The adapted frequency blocks better generalize, and the model outperforms USEMamba and other SOTA SSMs
  • MambAttention Based on the observation, that hybrid approaches of Mamba and Multihead Attention improve performance and generalization, this model precedes Mamba with a Transformer pass, improving out-of-domain performance significantly.
  • RWSA-MambaUNet Follow-up to MambAttention that incorporates Hybrid blocks into U-Net architecture. The downand upsampling improves performance and generalization.
  • oSpatialNet-Mamba Incorporates Mamba in SpatialNet and uses causal convolutions of online multi-channel streaming. Mamba outperforms retention networks and MSA attention.
  • SBMamba Incorporates oSpatialNet architecture into Schrödinger Bridge optimal transport regime. The authors propose SSMs as a learned optimal control solution to the Markov-style Schrödinger System. SBMamba achieves SOTA streaming performance at very low parameters counts with a single inference step.

An alternative branch of SSM-based SE are LTI-SSMs for low-latency speech processing, introduced by SaShiMi in 2022. SaShiMi works on the raw waveform and uses an hour-glass downand upsampling architecture, similar to U-Net but using simple reshape+linear projections instead of convolutions. The upsampling path is interleaved with [[S4]] blocks. The model outperformed WaveNet with very large context windows.

  • S4ND U-Net An iteration on SaShiMi, replacing S4 with S4ND layers. S4ND runs the SISO S4 model over mulitple channels in parallel, achieving mamba 1-sytle MIMO capabilities. With much fewer parameters, S4ND matched and outperformed other SOTA time-domain models. They also found that the architecture worked better in the time than in the time-frequency domain.
  • aTTENuate Another variation of SaShiMi but with a architecture U-Net and more SSMs, including non-linearities and pre-convolutions. This model achieves very low latency and decent PESQ on VB DMD.