For the longest time, I had been thinking that the bias term used in the standard artificial neural network model (the ubiquitous one used in most machine learning implementations) can be interpreted as some kind of 'background' input, maybe similar to LFP.

However, I had previously missed the perspective that the bias term can also be interpreted as a parameter of response selectivity. For instance, a ReLu activated horizontal line detector with 0 bias will fire weakly when given a vertical line input but a sufficiently large negative bias will prevent the detector from firing (which, in the case of a horizontal line detector probably is desirable).

My question is, what would be the neural equivalent of bias and its modification mechanisms? Is the shape of the activation function known to be subject to any learning mechanisms at all?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.