I have just started reading about the SPA (semantic pointer architecture) and wondering how it relates to deep learning concepts.
Deep Learning (which is now supposed to be called Differentiable Programming?) is a means of doing inference over a set of data.
SPA is a Vector Symbolic Architecture (VSA) implemented using Holographic Reduced Representations (HRR). That is to say, sparse vectors represent symbols and there is a set of operations that can be used to combine them.
Consequently, SPA can be used as a sort of prior for Deep Learning to provide constraints on the representation it's learning.
For example, SPA could be used to represent the output of a visual system, which would then be used by other symbolic systems. In the case of Spaun, Deep Learning is used in the vision system to map images onto known symbols. These known symbols can then be manipulated in ways that would be impossible for Deep Learning to perform. For example, Rapid Variable Creation used for solving Raven's Progressive Matrices is impossible for Deep Learning to perform without any structural priors.