What is the difference between attractor and recurrent network?

How are attractor and recurrent networks related? Is attractor network is private case of recurrent? Is there a mathematical formalism that defines both rigorously as graphs?

Many recurrent networks have attractors. For example, the BAM (bidirectional associative network) is a recurrent network: one input is presented and the network cycles until no more changes are seen in the connection weights. The BAM can be understood as developing attractor states: Seen in n-dimensions (in which n is the number of units in the model), the state can be seen as a position. Whenever the same stimuli is presented, the state of the network will stabilize to the same position.

One reference is Chartier S, Boukadoum M. (2006) A bidirectional heteroassociative memory for binary and grey-level patterns. IEEE Transactions on Neural Networks, 17(2):385-96.

As explained in Denis Cousineau's answer, there are many recurrent neural networks who's activities can be described as attractors. However, I would like to highlight two cases in particular, the Neural Engineering Framework, Conceptors and FORCE trained networks.

In the Neural Engineering Framework case, a recurrent neural network can approximate any dynamical system, including chaotic attractors. See these notes for a derivation and a demonstration. The networks can achieve certain stable states, for example when emulating a memory.

Conceptors are similar to the Neural Engineering Framework, except they questionably swap out recurrent weights to generate different patterns.

In FORCE training, a chaotically spiking reservoir can be trained to approximate seemingly any desired dynamical system.

In all these cases, assuming you have a system that can be seen as a transition between various stable states, I don't see why each state wouldn't be possible to map onto the node of a graph.

Only a nonlinear recurrent system can have interesting attractors: i.e., feedforward networks cannot have any attractor, and linear dynamical system can only have one (stable) fixed point (i.e., point attractor).

The relevant area of math is dynamical systems theory.

• Can you please elaborate on the connection between the chaos's attractor and the one context of neuroscience?
– 0x90
Feb 5 '17 at 8:41
• @0x90 I think that should be a separate question. Feb 5 '17 at 11:22