3
$\begingroup$

Local learning rules like Contrastive Hebbian Learning, XCAL, etc. are based on the idea of strengthening edges when the neurons they connect fire simultaneously. This causes frequent patterns in the input stream to be encoded in the layers.

I understand how this attracts similar input patterns into learned activation states. However, I fail to see how this memory can be used to predict the future input. How is it possible for neurons to predict future input streams under local learning rules?

Initial and final states in Hebbian Learning

From: Introduction Hebbian learning (2005)

Improve this question
$\endgroup$
1
  • $\begingroup$ What do you mean by "future input"? Do you mean: If you had a system whose input was a grid of pixels and the output was the number the pixels were representing, how does the network generalise from the training examples to the testing examples? $\endgroup$
    – Seanny123
    Mar 7 '16 at 1:27
1
$\begingroup$

see Lisman, J. E. and Otmakhova, N. A. (2001), Storage, recall, and novelty detection of sequences by the hippocampus: Elaborating on the SOCRATIC model to account for normal and aberrant effects of dopamine. Hippocampus, 11: 551–568. doi: 10.1002/hipo.1071: http://wwww.bio.brandeis.edu/lismanlab/pdf/socratic.pdf for one proposed mechanism.

in short, local learning rules can develop connections between items that occur sequentially. if you have one network that pattern completes (i.e. identifies) the input, you can pass that identification to a second network which learns temporal associations between items.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Can you improve your language a bit and go into a bit more detail? $\endgroup$
    – jona
    Jan 14 '16 at 19:43
  • $\begingroup$ what more detail would you like? $\endgroup$
    – honi
    Jan 15 '16 at 0:34
  • $\begingroup$ Thanks for the edit. This sounds very reasonable. Together with STDP this answers my (not very precise) question. $\endgroup$
    – danijar
    Mar 7 '16 at 8:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.