Ideas
SSM-based Architectures¶
Oscillatory SSMs for Phase-Aware SE¶
Grown into a full project: Oscillatory SSM for Phase Accurate Speech Enhancement.
Complex SSMs excel in frequency-based domains like vision and audio (see Mamba 3, p16). We also observe cortical oscillations playing a critical part in auditory cortical processing. Since SEMamba, Mamba has been the de-facto model incorporated for any kind of speech processing task. This invites the question of whether oscillatory dynamics might improve speech enhancement compared to other non-selective and selective SSM baselines.
Schrödinger Bridge U-Net Mamba¶
SBMamba shows that single-step generative inference is highly feasible with selective SSMs like Mamba. MambAttention proves that interleaving Mamba and Attention layers improves performance and generalization. RWSA-MambaUNet incorporates these hybrid layers into a U-Net architecture to achieve SOTA performance on the URGENT 2026 dataset. The ideas neatly combine: use RWSA-MambaUNet as the backbone for Schrödinger Bridge training.
- Potentially even incorporate the cross-band blocks from SBMamba/SpatialNet for multi-channel SE.
- If Selective LRU proves successful, also check its incorporation instead of Mamba.
- In the June 2026 SEMamba experiments, Selective LRU performed within ~0.1 PESQ of Mamba while preserving phase slightly better (see Results).
- Same U-Net architecture could be incorporated into the SGMSE framework?
Predictive Coding¶
Neural Oscillation and Global Neuronal Workspace (GNW) are basic mechanisms in the brain, however incorporating them in signal processing algorithms just because they exist in the brain is ill-posed. Predictive Coding on the other hand is a powerful neuro-computational framework embedded in the [[Bayesian Probability]] theory. Bayesian approaches are fundamental to statistical signal processing and have also been a well-suited model of human psychophysical behavior. Thus, predictive coding may yield insights for improving the design of deep neural networks.
Predictive coding poses that the human brain is constantly generating and updating a mental model of the environment from prior beliefs, while sensory observations are constantly compared to this model. The brain then tries to minimize the error between mental model, informed top-down from prior beliefs, and observation, informed by bottom-up sensory signals. The difference between model and observation is called "free-energy".