0
$\begingroup$

First some background. The field of artificial neural networks was inspired by early brain neuron research, such as that by McCulloch and Pitts. However, the early ANN research stalled due to the XOR problem and no effective training algorithm. Multilayer neural networks solved the first problem, but then that led to the training problem, since a single layer perceptron could be trained with the pocket algorithm, but that was no longer possible with a multilayer perceptron. Finally, backpropagation was discovered, and that lead to the modern ANN renaissance.

Now, back to the question. From the history of ANN it seems that backpropagation is essential to make ANNs learn. No other effective approach has been discovered.

At the same time, the brain does not seem to exhibit anything like backpropagation. And even if it did, the signal propagation seems too noisy and not granular enough for backpropagation to work.

So, since our best attempts to replication the brain's ability to learn synthetically results in a construct that has no biological corrollary, how is it possible for the brain to learn, and extremely effectively at that? Do we have any idea what sort of algorithm the brain uses to learn, if it can't use backpropagation? Is it like a global metaheurastic such as simulated annealing or evolutionary algorithms?

$\endgroup$
11
  • 2
    $\begingroup$ Possible duplicate of: How do neurons decide how to alter their output signals? Also see: Biological plausibility of RBMs. $\endgroup$
    – Arnon Weinberg
    Commented Dec 10 at 18:57
  • $\begingroup$ @ArnonWeinberg thanks, the first question seems related. However, Hebbian learning is not a very good learning algorithm. It seems implausible the brain uses Hebbian learning, since the brain seems to learn things that even our best neural networks cannot learn yet, and SOTA neural networks use much, much more effective algorithms than Hebbian learning. $\endgroup$
    – yters
    Commented Dec 11 at 13:28
  • 1
    $\begingroup$ The brain is massively parallel and recurrent, it doesn't make much sense to discuss algorithms for feedforward ANNs in the context of real brain circuitry. $\endgroup$
    – Bryan Krause
    Commented Dec 11 at 20:25
  • $\begingroup$ @BryanKrause Feedforward ANNs are massively parallel (see NVIDIA's stock boom), and variants are recurrent. However, recurrency dramatically impacts the learning rate in a negative way, which is why the transformer architecture was adopted instead of recurrent neural network. So, still it is very mysterious how the brain learns much more effectively than modern neural networks, yet modern neural networks rejected all the potential learning mechanisms we've identified in the brain as being much to inefficient to learn even in a massively parallel format. $\endgroup$
    – yters
    Commented Dec 12 at 2:58
  • $\begingroup$ Here's a good related question: psychology.stackexchange.com/questions/26883/… $\endgroup$
    – yters
    Commented Dec 12 at 3:00

1 Answer 1

0
$\begingroup$

Learning something can be said to be a process where you have made changes to your brain to the point of now being able to produce new thoughts in response to internal thoughts or external impressions. These thoughts produced will be perceived by you and conscious ones will carry some meaning.

Your question asks how we are able to create and edit these pathways which provide the framework for generating these thoughts/ideas, which has been the purpose of the learning.

Neuron activation is the most prevalent and known activity in the brain which relates to the phenomenon of having thoughts. When a neural is activated, its dendrites and axon terminals prime for connection with other neurons, they do this by physically extending its axons out into space in an effort to encounter another brain cell reaching out for connections. In this case, both neurons are activated and this is causing them to reach out for connections. Because both neurons are reaching out, naturally, there is a higher chance that they come close to each other, are attracted to each other, and as a result, bind together, forming a connection between them. This connection now means that when one neuron is activated, the other will react in some way, which is what facilitates thought.

This process does not involve back propagation. It is an absolute bond which forms in real time, of which can be said to have had an absolute point in time when that connection was established, and this connection has one effect, to allow for transmission of chemicals between the connected neurons which allows for them to have an effect on each other. Back propagation involves many nodes in a neural network and allows for the editing of multiple nodes in one sweep, which is not similar to how the brain functions.

Researchers postulate how the brain learns without back propagation, but the brain and a computer, which a neural network runs on, are different, therefore its wrong to assume brains need back propagation to learn just because current forms of neural networks need back propagation to learn. A computer which a neural network runs on is a bunch of transistors, and gates, which perform operations for the purpose of turning transistors either on or off. The brain is a network which runs on atoms and molecules which are constantly affecting each other in a dynamic ever changing way, with other avenues for altering its structure. A neural network is a purposefully constructed network which has absolute mechanisms for altering itself which have been programmatically implemented to be its primary way of learning.

Ultimately, the brain does not conduct itself, and is not constructed, upon the same underlying structure as a neural network, which is what allows it to learn in ways that don't require back propagation.

New contributor
Mike Wazowski is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.