5
$\begingroup$

The primary learning mechanism of artificial neural networks (ANN) is back-propagation, which is not biologically plausible [2].

Trevor Berkolay created an alternative to this learning with the Neurological Engineering Framework (NEF) and Nengo called hPES (Homeostatic Prescribed Error Sensitivity) [1]. But how does it's learning capabilities compare to the standard supervised and unsupervised learning of ANNs in terms of computational power required and speed of learning?

[1] See also, "How to Build a Brain" by Chris Eliasmith chapter 6.4

[2] Further details around this claim can be found in the question "Is back-prop biologically plausible?"

Note: This question portrayed ANNs and the NEF as adversarial, which really isn't the case. Spaun the brain model, which is kind of the poster child of the NEF, uses ANNs (Convolutional Neural Networks specifically) converted into spiking neurons in it's vision system.

$\endgroup$
3
$\begingroup$

According to the paper, the advantage of this new approach over conventional ANNs, Deep Belief Networks (DBN) and Self-Organising Networks (SON) are:

  1. Remains functional during online learning.
  2. Requires only two layers connected with simultaneous supervised and unsupervised learning
  3. Employs spiking neuron models to reproduce central features of biological learning, such as spike-timing dependent plasticity (STDP)

So arguably, hPES is superior to ANNs in terms of capability, but in terms of performance, you'll have to compare the code from his experiments with ANNs meant to accomplish the same task, but since both methodologies are designed with different purposes in mind, it might not be worth comparing. Additionally, it should be noted that although the author claims to have created a SON, the validation of this claim in the paper (and his master's thesis) is pretty weak. In the paper he only proves that the SON increases sparsity, which is not what SONs (for example Kohonen networks) are generally used for.

Finally, note that although hPES is more biologically plausible, it still has some of the same problems that ANNs have. Namely, the parameters that it takes in must be optimized for the specific task it's learning (some approaches for solving this with ANNs include Genetic Algorithms) even if it is less sensitive to parameter modification. The author mentions this investigation of parameters as being part of future work.

$\endgroup$
  • $\begingroup$ Doesn't hPES only deal with rate codes, and not spike times? A better question would be how does hPES compare to STDP. $\endgroup$ – Matt Way Jan 29 '15 at 23:01
  • $\begingroup$ @MattWay hPES can be used with spiking and rate neurons, like everything in Nengo $\endgroup$ – Seanny123 Jan 30 '15 at 1:59
  • $\begingroup$ I didn't say it couldn't. $\endgroup$ – Matt Way Jan 30 '15 at 2:11
  • $\begingroup$ @MattWay reading this again, I'm sorry I mis-understood you. If you consult Trevor Bekolay's Master's thesis you see that hPES captures all aspects of STDP. $\endgroup$ – Seanny123 Sep 15 '15 at 23:18
  • $\begingroup$ @MattWay years later, I have a VERY brief summary of how it matches in this answer $\endgroup$ – Seanny123 Jun 20 '18 at 19:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.