I have recently discovered that both Hamming and Hopfield networks are both content addressable memories. However, what is the difference between these two networks designs and their performance?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA182255 - Hamming networks require fewer connections and often outperform Hopfield nets. $\endgroup$– Wuschelbeutel KartoffelhuhnCommented Aug 19, 2016 at 23:17
-
$\begingroup$ @WuschelbeutelKartoffelhuhn yep, that's the paper I was using to craft my own answer :) $\endgroup$– Seanny123Commented Aug 20, 2016 at 2:49
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Hopfield and Hamming networks are older connectionist networks used to classify noisy binary input data.
Hopfield networks are the basis for Hamming networks, so let's focus on those first.
A Hopfield network is a set of neurons that do classification via mutual inhibition, as shown in the figure below from Wikipedia:
Note the neurons are not like your typical biologically plausible neurons, they have two states "+1" and "-1". Their update rule, which forces them into an output pattern, enables these two states.
A Hamming network is a two-layer network, as shown in this figure (taken from here) where the data is flowing from the bottom:
As you can see, the second layer, sometimes called the MaxNet layer, is identical to the Hopfield Network, however the input layer resembles a classic single-layer perceptron. This single-layer perceptron is train according to the Hamming Distances of the inputs.
Performance wise, as shown in the conclusion "A Comparison of Hamming and Hopfield Neural Nets for Pattern Classification" by Lippmann et al., Hamming nets generally out-perform Hopfield nets.