I have recently discovered that both Hamming and Hopfield networks are both content addressable memories. However, what is the difference between these two networks designs and their performance?

  • $\begingroup$ www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA182255 - Hamming networks require fewer connections and often outperform Hopfield nets. $\endgroup$ – Wuschelbeutel Kartoffelhuhn Aug 19 '16 at 23:17
  • $\begingroup$ @WuschelbeutelKartoffelhuhn yep, that's the paper I was using to craft my own answer :) $\endgroup$ – Seanny123 Aug 20 '16 at 2:49

Hopfield and Hamming networks are older connectionist networks used to classify noisy binary input data.

Hopfield networks are the basis for Hamming networks, so let's focus on those first.

A Hopfield network is a set of neurons that do classification via mutual inhibition, as shown in the figure below from Wikipedia:


Note the neurons are not like your typical biologically plausible neurons, they have two states "+1" and "-1". Their update rule, which forces them into an output pattern, enables these two states.

A Hamming network is a two-layer network, as shown in this figure (taken from here) where the data is flowing from the bottom:


As you can see, the second layer, sometimes called the MaxNet layer, is identical to the Hopfield Network, however the input layer resembles a classic single-layer perceptron. This single-layer perceptron is train according to the Hamming Distances of the inputs.

Performance wise, as shown in the conclusion "A Comparison of Hamming and Hopfield Neural Nets for Pattern Classification" by Lippmann et al., Hamming nets generally out-perform Hopfield nets.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.