Is it fair to categorize the neurons in the output layer of artificial neural networks as grandmother cells?

Question

I'm interested in how memory is distributed in the brain. That inevitably leads one to think about grandmother cells.

Under my interpretation, a neuron (or a small ensemble) would classify as a grandmother cell, if it activates, without fail, whenever a specific stimulus is presented to a subject, over an extended period of time.

To my knowledge, there is no evidence for the existence of such neurons, except some experiments that show the activation of specific neurons for a short period of time between trial sessions. The last part is important, because to my understanding grandmother cells are explicitly about long term memory.

If I look at most artificial neural networks however, the cells in the output layer usually exactly fit this definition. You can easily train a network to recognize pictures of your grandmother and consistently find the exact same cells active during evaluation whenever such a picture is presented. That's important for a computer of course, because it needs to have a consistent way of telling something is learned.

This would be an important difference to the brain though, because one could argue that any system, involving grandmother cells and that has only a limited number of cells, eventually suffers from overfitting or at least very limited generality.

Is this assessment fair? Is my definition wrong, somehow? Did I miss some evidence?

Answer 1

I would say it depends on what your goals are and what parts of the grandmother cell "story" you want to highlight.

Sure, the output layer is grandmother-like because it can represent single concepts

If you are writing a classifier to identify objects, then in some ways, yes, the outputs of an ANN reflect "grandmother" cells in that they represent a single output concept/object. The way that a hypothetical grandmother cell in a real nervous system would be activated (as sort of a final step after perceiving all the sensory information seeing/hearing another person presents) would be analogous to how the output of an ANN is chosen.

However, the outputs of a real nervous system are probably better identified as the motor plans: the goal in object recognition is to decide whether to fight or flee or do nothing at all. Similarly, if your ANN is trying to classify radiological images as "cancer" or "no cancer" these outputs don't really fit with the idea of a grandmother cell.

But no, the grandmother cell is about more than that and the output layer of an ANN doesn't really fit

A key element in the grandmother cell concept is that this cell is somehow connected to all the other subconcepts that make up the grandmother concept: the sound of her voice, the color of her hair, where she lives, etc. In real nervous systems these elements probably involve large ensembles of neurons organized into attractor networks, but even so, because the output layer of an ANN is just that, an output layer, it's really missing the major feature that makes a grandmother cell so key in the first place.

The studies you refer to that showed some evidence of grandmother-like cells are really just demonstrating a certain level of sparseness such that within a limited subset of experimental stimuli, those units were only activated by one or two of the inputs. In an ANN, these types of nodes would be better represented as units in some hidden layer. If I were to look for "grandmother cell-like behavior" to make comparisons between a real brain and an ANN, I would look for sparseness in the hidden layers rather than pointing to the output.

Summary

In summary, I would say that you could refer to grandmother cells in discussing ANN behavior, but I would be careful about taking the analogy too far or to assume that something true for a grandmother cell must therefore be true for a certain element of an ANN. However, I don't see a lot of value in doing so, and if I were teaching a course on the topic, I think I would avoid the comparison to avoid misinterpretation or confusion among the students.

---

Connor, C. E. (2005). Neuroscience: Friends and grandmothers. Nature, 435(7045), 1036.

Gross, C. G. (2002). Genealogy of the “grandmother cell”. The Neuroscientist, 8(5), 512-518.

Kreiman, G., Koch, C., & Fried, I. (2000). Category-specific visual responses of single neurons in the human medial temporal lobe. Nature neuroscience, 3(9), 946.

Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005). Invariant visual representation by single neurons in the human brain. Nature, 435(7045), 1102.

Quian Quiroga, R., & Kreiman, G. (2010). Postscript: About grandmother cells and Jennifer Aniston neurons.