Is there a biological equivalent to the bias term used in artificial neural networks?

Question

For the longest time, I had been thinking that the bias term used in the standard artificial neural network model (the ubiquitous one used in most machine learning implementations) can be interpreted as some kind of 'background' input, maybe similar to LFP.

However, I had previously missed the perspective that the bias term can also be interpreted as a parameter of response selectivity. For instance, a ReLu activated horizontal line detector with 0 bias will fire weakly when given a vertical line input but a sufficiently large negative bias will prevent the detector from firing (which, in the case of a horizontal line detector probably is desirable).

My question is, what would be the neural equivalent of bias and its modification mechanisms? Is the shape of the activation function known to be subject to any learning mechanisms at all?

Answer 1

Let's call "response function" the function that gives a neuron's firing rate (or probability of firing) given its input. Then, different kinds of neurons have different response functions. All of them, as far as I'm aware, are nonlinear, and can usually be experimentally measured.

There are neurons that will fire a little, even for small inputs (so they're basically rectified, nonlinear units) and neurons that need a specific, non-zero threshold of input to be crossed before they fire at all (somewhat analogous to having negative bias). These used to be called Type I and Type II neurons respectively, although I've rarely heard this definition recently. There are also neurons that always fire unless inhibited from doing so ("tonic activity"): you can see this as implementing a positive bias.

About the second part of your question, the "modification mechanisms": there are many factors that influence a neuron's probability of firing which are independent both of the input and of the input weight. You can see these as changes in the bias, either temporary or long-term. The main causes of this are the neuron's recent firing history (usually neurons are less likely to fire if they've just fired, unless they like to burst) and other phenomena usually called 'intrinsic plasticity'.

I hope this gives you a bit more of an idea.