Efficient coding hypothesis

The efficient coding hypothesis was proposed by Horace Barlow in 1961 as a theoretical model of sensory coding in the brain. Within the brain, neurons often communicate with one another by sending electrical impulses referred to as action potentials or spikes. One goal of sensory neuroscience is to decipher the meaning of these spikes in order to understand how the brain represents and processes information about the outside world. Barlow hypothesized that the spikes in the sensory system formed a neural code for efficiently representing sensory information. By efficient Barlow meant that the code minimized the number of spikes needed to transmit a given signal. This is somewhat analogous to transmitting information across the internet, where different file formats can be used to transmit a given image. Different file formats require different number of bits for representing the same image at given distortion level, and some are better suited for representing certain classes of images than others. According to this model, the brain is thought to use a code which is suited for representing visual and audio information representative of an organism's natural environment.

Efficient coding and information theory

The development of the Barlow's hypothesis was influenced by information theory introduced by Claude Shannon only a decade before. Information theory provides the mathematical framework for analyzing communication systems. It formally defines concepts such as information, channel capacity, and redundancy. Barlow's model treats the sensory pathway as a communications channel where neuronal spiking is an efficient code for representing sensory signals. The spiking code aims to maximize available channel capacity by minimizing the redundancy between representational units.

A key prediction of the efficient coding hypothesis is that sensory processing in the brain should be adapted to natural stimuli. Neurons in the visual (auditory) system should be optimized for coding images (sounds) representative of those found in nature. Researchers have shown that filters optimized for coding natural images lead to filters which resemble the receptive fields of simple-cells in V1. In the auditory domain, optimizing a network for coding natural sounds leads to filters which resemble the impulse response of cochlear filters found in the inner ear.

References

* Barlow, H. (1961) 'Possible principles underlying the transformation of sensory messages' in Sensory Communication, MIT Press

* Lewicki, M.S. (2002) Efficient coding of natural sounds. Nature: Neuroscience, 5(4):356–363

* Olshausen, B. A. and Field, D.J. (1997) Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37(23):3311–3325