Predictive Coding
Predictive coding is a theory of brain function in which the brain continuously generates top-down predictions of its sensory input from an internal generative model of the world. Bottom-up signals mainly carry prediction errors, the mismatch between prediction and observation. Perception is then the process of updating the internal model to minimize these errors. The theory was formulated computationally for the visual cortex by Rao & Ballard (1999) and generalized under the free-energy principle by Friston (2005, 2010).
Hierarchical Formulation¶
The cortex is modeled as a hierarchy of levels \(l=1,\ldots,L\). Each level holds a representation \(\mu_{l}\) that predicts the activity of the level below through a generative mapping \(g_{l}\), with the sensory input as the lowest level \(\mu_{0}\). The prediction error at each level is
Assuming Gaussian distributions, the representations are inferred by minimizing the precision-weighted sum of squared prediction errors
for example by gradient descent \(\dot{\mu}_{l}\propto-\frac{\partial F}{\partial\mu_{l}}\). Each level is thus pulled both by the error it receives from below (bottom-up) and by the prediction from above (top-down). Slowly learning the parameters of \(g_{l}\) on the same objective corresponds to learning the generative model.
Under the free-energy principle, \(F\) is the variational free energy, the negative of the Evidence Lower Bound. Minimizing it performs approximate Bayesian inference, which connects predictive coding to latent-variable Generative Modeling like the Variational Autoencoder.
Precision Weighting¶
The precision \(\Pi_{l}\) (inverse variance) expresses how reliable the prediction errors at a level are expected to be. Errors from reliable sources have more influence on inference than errors from noisy ones. Attention has been interpreted as the optimization of precision: attending to a stimulus increases the gain of its prediction errors. This offers a computational account of selective attention, e.g. to a target speaker in the Cocktail Party Problem.
Relevance for Speech Enhancement¶
Predictive coding frames perception as inference with a generative prior, which matches the statistical view of speech enhancement as estimating clean speech from a noisy observation. Ideas for using it in the CHAIS project are collected in Ideas#Predictive Coding.