Skip to content

MP SENet

MP-SENet is a pioneering architecture for speech enhancement. It models magnitude and phase jointly in a dense representation. It also introduces a GAN-style metric loss, which yielded very strong PESQ values.

Architecture

MP-SENet uses a dialated DenseNet to encode STFT magnitude and phase (T, F, 2) into a dense 3D representation (T,F//2,C). This dense representation is then processed by a [[Conformer]] model, before being decoded by two decoders, for magnitude mask and phase each. Both decoders are based on dialated DenseNet as well. For the magnitude mask, outputs are passed through a learned sigmoid activation. For phase, the output is passed through an arctan2 activation. The input magnitudes are masked and combined with phase, before being passed through the inverse STFT.

Loss

The model uses several losses between prediction and target:

  • L1 loss on the waveform
  • L2 loss between magnitudes
  • L2 between real + L2 between imaginary components
  • Anti-wrapping phase loss
  • GAN-style metric loss (PESQ)