6
$\begingroup$

I was reading over some sources about ANNs, and here I found this diagram of a Recurrent Neural Network (RNN, the source calls it a feedback neural network). enter image description here

I noticed that a few neurons in this diagram lead to other neurons that are dead ends (have no connections to other neurons). For example, neuron U5 leads to no other neurons, and U9 has a connection to U4, but U4 doesn't have a connection to any other neurons.

Is there a purpose for U5 and U4 to even exist?

Or am I missing something bigger here?

Improve this question
$\endgroup$
3
  • $\begingroup$ I think it's just indicating that most often, biologically, input gets re-routed and re-used. $\endgroup$
    – Seanny123
    Apr 2, 2016 at 0:28
  • 1
    $\begingroup$ To the best of my knowledge, dead end neurons do not appear in artificial neural networks that are actually used and they do not have a purpose since they can't contribute to the output of the network. $\endgroup$
    – awakenting
    Sep 6, 2016 at 21:12
  • 1
    $\begingroup$ In this particular architecture they don't seem to contribute to the outputs, which carry the "b" label. It is possible though to have outputs at intermediate levels, in addition to the traditional final output layer. So some neural network designer might conceivably attach outputs to U5 and U4 that are presented to the end user before the B-layer outputs. Just food for thought & something to watch out for when interpreting similar diagrams, where the dead-end neurons might not be as dead as they appear at first glance. $\endgroup$
    – SQLServerSteve
    Sep 10, 2016 at 1:51

1 Answer 1

Reset to default
3
$\begingroup$

As said in the comments, $U_4$ and $U_5$ are not used to compute the values of the output neurons.

However, in some cases one may allow the connections between neurons to change over time, in which case $U_4$ and $U_5$ may be later used to compute the values of the output neurons. Seanny123 gives biology as an example. For another example, you may want to look at the literature on neuro-evolution, i.e. optimization of neural architectures with evolutionary algorithms, which may allow changing the connections between neurons over generations (e.g. that's what I did for my master thesis: Franck Dernoncourt. "The medial Reticular Formation: a neural substrate for action selection? An evaluation via evolutionary computation.". Master's Thesis. École Normale Supérieure Ulm. 2011., though I may have coded the mutation and crossover operators in such a way that dead-end neurons don't appear, I don't recall).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.