6
$\begingroup$

I was reading over some sources about ANNs, and here I found this diagram of a Recurrent Neural Network (RNN, the source calls it a feedback neural network). enter image description here

I noticed that a few neurons in this diagram lead to other neurons that are dead ends (have no connections to other neurons). For example, neuron U5 leads to no other neurons, and U9 has a connection to U4, but U4 doesn't have a connection to any other neurons.

Is there a purpose for U5 and U4 to even exist?

Or am I missing something bigger here?

$\endgroup$
  • $\begingroup$ I think it's just indicating that most often, biologically, input gets re-routed and re-used. $\endgroup$ – Seanny123 Apr 2 '16 at 0:28
  • 1
    $\begingroup$ To the best of my knowledge, dead end neurons do not appear in artificial neural networks that are actually used and they do not have a purpose since they can't contribute to the output of the network. $\endgroup$ – awakenting Sep 6 '16 at 21:12
  • 1
    $\begingroup$ In this particular architecture they don't seem to contribute to the outputs, which carry the "b" label. It is possible though to have outputs at intermediate levels, in addition to the traditional final output layer. So some neural network designer might conceivably attach outputs to U5 and U4 that are presented to the end user before the B-layer outputs. Just food for thought & something to watch out for when interpreting similar diagrams, where the dead-end neurons might not be as dead as they appear at first glance. $\endgroup$ – SQLServerSteve Sep 10 '16 at 1:51
3
$\begingroup$

As said in the comments, $U_4$ and $U_5$ are not used to compute the values of the output neurons.

However, in some cases one may allow the connections between neurons to change over time, in which case $U_4$ and $U_5$ may be later used to compute the values of the output neurons. Seanny123 gives biology as an example. For another example, you may want to look at the literature on neuro-evolution, i.e. optimization of neural architectures with evolutionary algorithms, which may allow changing the connections between neurons over generations (e.g. that's what I did for my master thesis: Franck Dernoncourt. "The medial Reticular Formation: a neural substrate for action selection? An evaluation via evolutionary computation.". Master's Thesis. École Normale Supérieure Ulm. 2011., though I may have coded the mutation and crossover operators in such a way that dead-end neurons don't appear, I don't recall).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.