USEMamba

USEMamba is an iteration upon SEMamba.

Loss Terms¶

The authors simplify the loss terms to - L1 between the waveforms - A Mutli-Resolution STFT loss - and the anti-wrapping phase loss.

They remove the consistency loss, metric loss, and complex loss.

Mapping-based Magnitude Prediction¶

SEMamba used a masking approach for the magnitude. The authors note that, while working well for additive noise, this approach falls short for deverberation, packet-loss and bandwith-extension. So the masking-based prediction is swapped for mapping-based prediction, which is harder to train but also allows for generative approaches to work.

Sampling Frequency Independent STFT¶

The SFI STFT is a adjusted FFT based on sampling rate for a fixed duration of feature frames across sampling rates. This allows the model to generalize to other sampling rates without retraining.

Generative Modeling¶

The paper also introduces a generative variant of USEMamba, based on flow-based modeling: At training, the magnitude-phase input of the corrupted speech is stacked with a noise corrupted version of the clean speech \(x_{\tau}\). During training, adaptive normalization layers feed the time embedding \(\tau\) in between the bidirectional Mamba blocks. This results in a hybrid model that is predictive as well as generative. The authors show that the hybrid approach performs similar to the discriminative approach, but is also capable of bandwith extension.