In what ways do our brains use compression?

Question

I am reading a book on information theory and one line in the book states that "Brains are the ultimate compression and communication systems." However, there is no further elaboration on the matter. I am curious to know what specifically this sentence may be referring to. In particular, how does the brain demonstrate compression?

Answer 1

Say for example, you know what a particular type of flower looks like. You know it has purple petals and a white center. Now, unbeknownst to you, this flower has been slowly changing (evolving) to have a different shape, size, and even shade over the years since you first had seen one of these flowers. All the while, you have been around this flower your whole life. I mean, it grows in your current backyard, and it also grew in the backyard of the house you grew up in. In this scenario, you may have dreams about when you were a kid playing in the backyard. Instead of seeing what the flowers used to look like (when you actually were a kid), you will most likely see them as they appear to you today. Although your brain has memories of the presence of these flowers in the yard when you were a kid, their actual defining details have been rewritten over time with their current appearance. The hypothesis here is that your brain is trying to conserve capacity by linking all recorded encounters with this type of flower so that they all share the same "image file" in your brain. Witness testimonies have a similar problem -- where data received after the fact can change a person's memory of what happened and how things and people actually looked. I recommend you research the problems known about witness testimony.

Answer 2

Check out:

1.6. Bayesian Hierarchical predictive coding approach to explanation of the findings

from this thesis:

http://discovery.ucl.ac.uk/1435549/

There's a comparison between image compression algorithms and what the brain does.

Answer 3

Horace Barlow was probably the first to elaborate on the idea of compression in sensory systems:

Barlow, Possible principles underlying the transformation of sensory messages. Sensory Communication, 1961.

It is interesting to get his perspective 40 years later from:

Barlow, Redundancy reduction revisited., Network, 2001.

More recently, Bialek and colleagues have done a lot of research on this (again, primarily in sensory systems). For example, here is a recent study where they quantify the amount of predictive information (about the future state of a stimulus) compressed into retinal ganglion signalling (and downstream):

Palmer et al., Predictive information in a sensory population, PNAS, 2015.

In this case, they observe that the retina encodes predictive information about the future state of an external stimulus at a rate that approaches a theoretic limit (according to their model based on the Information Bottleneck principle) thus exhibiting (roughly) maximal compression.

Answer 4

The Semantic Pointer Architecture (SPA), which was used to built the biologically plausible cognitive-model Spaun, is a demonstration of the utility of compression in the brain.

In Spaun uses compression to relate concepts in a unified and grounded manner. For example, for number concepts in Spaun's "copying task", Spaun first gets the concept of the number by gradually extracting larger and larger features until from a visual hierarchy. This is analogous to the Deep Learning's Restricted Boltzmann Machines. This compressed representation can than be decompressed into individual motor actions and sent back down the motor hierarchy.

To summarize, the perception problem is the dual of the action problem and they are unified my compression, as visualised in the image below taken from the presentation found here.

This compression has also been used in terms of scalable knowledge representations.

For further information on the concepts behind SPA and Spaun, beyond the context of compression, see "How to Build a Brain" by Chris Eliasmith.