What do the weights of an artificial neural network represent in biological neurons?

Question

In artificial neural networks the connections between neurons are a assigned numbers called "weights" or "parameters". As new data is fed into the neural net, these weights change. This is how the neural net "learns".

I'm trying to understand what these weights might correspond to in a real brain. The idea occurred to me that it might represent the strength of the myelination on the connections between real neurons.

Is this a correct understanding?

Does the degree of myelineation along an axon vary over time as the neural net processes more experiences?

Answer 1

The weights in an artificial neural network are an approximation of multiple processes combined that take place in biological neurons. Myelination plays a role, but not a major one. Weights in artificial neural networks can be positive or negative numbers.

Weight magnitude. The magnitude of a weight is analogous to a combination of increased dendritic connections between neurons, number of synapses between their dendrites, density of neurotransmitter receptors on the post-synaptic terminals, as well as increased neurotransmitter vesicle formation and fusion on the pre-synaptic terminals.

Positive weights. Positive weights are analogous to the pre-synaptic terminals of the synapses releasing excitatory neurotransmitters (i.e. glutamate). They make it more likely the receiving cell will fire an action potential.

Negative weights. Negative weights are analogous to inhibitory neurotransmitters (i.e. GABA) being released at the synapse. They make it less likely the receiving cell will fire an action potential.

Myelination

Myelination increases the distance that action potentials can travel down the axon. The membrane voltage potential decays much closer to the cell body if the axon is not myelinated. One analogy is that of a garden hose. If the axon is not myelinated, then the garden hose is leaky with holes, and less water pressure (that caries water pressure waves, the action potentials) makes it to the end of the hose.

However, increased myelination is not necessarily analogous to an increased weight. For example, if there are few dendrites between two neurons, with few synaptic connections, then myelination would have little effect.

Learning

You also mentioned learning. During learning, the weights in an artificial neuron increase or decrease. In biological neurons, learning takes place at multiple scales and areas (i.e. non-synaptic and synaptic). At first, during increasing connection strength, you may see increased vesicle fusion, then increased neurotransmitter receptors, then new synaptic boutons, and new dendrites. When "unlearning", or forgetting, these processes happen in reverse.

Overall, a weight in artificial neurons clump a lot of biological complexity into one number that only crudely approximates the degree of connection strength between two biological neurons.

Answer 2

The weight matrix is typically considered to be a strength of connectivity metric between nodes in the parallel computronium model that neural networks are based on. That fact is fairly evident when you investigate ANN learning algorithms.

For instance, the backpropagation algorithm for feedforward networks is designed to strengthen a string of associations between layers, thus forming a state vector to represent the optimal solution.

The association seems even more evident in Hopfield networks. A training algorithm in a Hopfield net is designed to adjust the weight matrix until the network state converges to a stability point. Certain stabilities represent certain associative memories; similar to the brain.

However, I think forming too strong of a connection between the current neural network models and the functions of the biological brain might actually be detrimental to your understanding of the former. Neural networks are not designed to work like the brain, they're designed to be a parallel architecture. It just so happens that the brain is also a parallel architecture, but learning in the brain is so different from any ANN model that it makes very little sense to compare the two on such a fundamental level.

Answer 3

Historically, neuroscientists have not thought of myelination to be the biological analogue of weights in an artificial neural network (but see below!).

Instead, the strength of synaptic connections seems to be the most likely analogue. Here, strength refers mostly to the size and of the voltage response in the receiving neuron due to the release of neurotransmitter in the sending neuron. So, for example, Neuron A is connected to Neuron B, and then A releases the neurotransmitter glutamate and causes a 1 millivolt response in B. But then another neuron, Neuron C may be connected to Neuron D and when it releases glutamate it might only cause a 0.5 millivolt response--not as strong as the A->B connection. Importantly, the strengths of synaptic connections change due to experience, and this is thought to (in part) underlie learning, similarly to how changes in weights in an artificial neural network are the mechanism of artificial learning.

All this said, your point about myelination is interesting in light of recent work: an October 2014 paper, published in Science showed that production of myelin (in the form of oligodendrocytes) was necessary for motor learning in mice.