How are attractor and recurrent networks related? Is attractor network is private case of recurrent? Is there a mathematical formalism that defines both rigorously as graphs?
0
Add a comment
|
$\begingroup$
$\endgroup$
0
Many recurrent networks have attractors. For example, the BAM (bidirectional associative network) is a recurrent network: one input is presented and the network cycles until no more changes are seen in the connection weights. The BAM can be understood as developing attractor states: Seen in n-dimensions (in which n is the number of units in the model), the state can be seen as a position. Whenever the same stimuli is presented, the state of the network will stabilize to the same position.
One reference is Chartier S, Boukadoum M. (2006) A bidirectional heteroassociative memory for binary and grey-level patterns. IEEE Transactions on Neural Networks, 17(2):385-96.
$\begingroup$
$\endgroup$
As explained in Denis Cousineau's answer, there are many recurrent neural networks who's activities can be described as attractors. However, I would like to highlight two cases in particular, the Neural Engineering Framework, Conceptors and FORCE trained networks.
In the Neural Engineering Framework case, a recurrent neural network can approximate any dynamical system, including chaotic attractors. See these notes for a derivation and a demonstration. The networks can achieve certain stable states, for example when emulating a memory.
Conceptors are similar to the Neural Engineering Framework, except they questionably swap out recurrent weights to generate different patterns.
In FORCE training, a chaotically spiking reservoir can be trained to approximate seemingly any desired dynamical system.
In all these cases, assuming you have a system that can be seen as a transition between various stable states, I don't see why each state wouldn't be possible to map onto the node of a graph.
$\begingroup$
$\endgroup$
2
Only a nonlinear recurrent system can have interesting attractors: i.e., feedforward networks cannot have any attractor, and linear dynamical system can only have one (stable) fixed point (i.e., point attractor).
The relevant area of math is dynamical systems theory.
