0
$\begingroup$

First some background. The field of artificial neural networks was inspired by early brain neuron research, such as that by McCulloch and Pitts. However, the early ANN research stalled due to the XOR problem and no effective training algorithm. Multilayer neural networks solved the first problem, but then that led to the training problem, since a single layer perceptron could be trained with the pocket algorithm, but that was no longer possible with a multilayer perceptron. Finally, backpropagation was discovered, and that lead to the modern ANN renaissance.

Now, back to the question. From the history of ANN it seems that backpropagation is essential to make ANNs learn. No other effective approach has been discovered.

At the same time, the brain does not seem to exhibit anything like backpropagation. And even if it did, the signal propagation seems too noisy and not granular enough for backpropagation to work.

So, since our best attempts to replication the brain's ability to learn synthetically results in a construct that has no biological corrollary, how is it possible for the brain to learn, and extremely effectively at that? Do we have any idea what sort of algorithm the brain uses to learn, if it can't use backpropagation? Is it like a global metaheurastic such as simulated annealing or evolutionary algorithms?

Improve this question
$\endgroup$
11
  • 2
    $\begingroup$ Possible duplicate of: How do neurons decide how to alter their output signals? Also see: Biological plausibility of RBMs. $\endgroup$
    – Arnon Weinberg ♦
    Commented Dec 10 at 18:57
  • $\begingroup$ @ArnonWeinberg thanks, the first question seems related. However, Hebbian learning is not a very good learning algorithm. It seems implausible the brain uses Hebbian learning, since the brain seems to learn things that even our best neural networks cannot learn yet, and SOTA neural networks use much, much more effective algorithms than Hebbian learning. $\endgroup$
    – yters
    Commented Dec 11 at 13:28
  • 1
    $\begingroup$ The brain is massively parallel and recurrent, it doesn't make much sense to discuss algorithms for feedforward ANNs in the context of real brain circuitry. $\endgroup$
    – Bryan Krause ♦
    Commented Dec 11 at 20:25
  • $\begingroup$ @BryanKrause Feedforward ANNs are massively parallel (see NVIDIA's stock boom), and variants are recurrent. However, recurrency dramatically impacts the learning rate in a negative way, which is why the transformer architecture was adopted instead of recurrent neural network. So, still it is very mysterious how the brain learns much more effectively than modern neural networks, yet modern neural networks rejected all the potential learning mechanisms we've identified in the brain as being much to inefficient to learn even in a massively parallel format. $\endgroup$
    – yters
    Commented Dec 12 at 2:58
  • $\begingroup$ Here's a good related question: psychology.stackexchange.com/questions/26883/… $\endgroup$
    – yters
    Commented Dec 12 at 3:00

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.