Biological Plausibility of Back-Prop
No, the algorithm of back-prop (BP) isn't biologically plausible. However, there are other means which involve propagating the error through multiple layers of neurons in a feed-forward network which are biologically plausible. But before we evaluate these substitutions, let's review why back-prop isn't biologically plausible [1]:
1. Weight Transport Problem
BP uses the same weights for forward-pass and backward error propagation. But synapses are uni-directional.
2. Derivative Transport Problem
The derivative of each neuron is used to modulate the error signal in BP. Additionally, the derivative is propagated through each layer. How this derivative calculated by neurons and then propagated, is unclear.
3. Linear Feedback Problem
The BP feedback path is linear, but neurons aren't linear.
4. Spiking Problem
Neurons spike. BP is defined for rates.
5. Timing Problem
The BP error signal is expected to propagate instantaneously. Calculations and signal transmission in the brain do not happen instantaneously.
6. Target Problem
Most applications of BP rely on many examples with given labels, but where do those labels come from?
Replacements for Back-Prop
Most replacements for back-prop solve a subset of these problems, however none of them see to solve all of them. In "Spiking Deep Neural Networks: Engineered and Biological Approaches to Object Recognition", a method is constructed using the NEF that solves all problems except for the Target Problem. This means there are no replacements for BP that work for all cases where BP is applied, which is to be expected given this is what usually happens when biological plausibility is emphasized.
Beyond Back-Prop
However, it should be noted that there is more than back-propagation through fully-connected networks involved in Deep Learning. Specifically, the are a number of different architectures used, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).
- CNNs aren't biologically plausible due to their reliance on weights being exactly equal across multiple locations.
- RNNs rely on Back-Propagation Through Time (BPTT), of which there is currently no biologically plausible analogue [2].
This confusion between Deep Learning, neural network architectures and back-prop is reason that Yann LeCunn is trying to get people to adopt the term "Differentiable Computing" instead of "Deep Learning". At which point, the question must shift from "Is Deep Learning biologically plausible?" into "What aspects of Differentiable Computing are biologically plausible?" Which is much harder to answer, but at least we have a better question than when we started!
[1] Summarized from Eric Hunsberger's PhD Thesis "Spiking Deep Neural Networks: Engineered and Biological Approaches to Object Recognition"
[2] That being said, there are other means of learning (supposedly) biologically plausible recurrent neural network weights (FORCE training, Conceptors), however they don't resemble BPTT in any way. The discussion of these methods areis outside the scope of this question.