2
$\begingroup$

One of the fundamental operations in image neural networks is the operation of convolution, where a filter slides across the receptive field, activating on particular features. This is incredibly useful because it allows us to recycle filters, without having to train neurons on each piece of the receptive field.

It seems like such an operation is mechanically impossible within the human visual cortex. From the paper How does the brain solve visual object recognition - DiCarlo, et. al., the brain opts to instead use a massive amount of neurons in each layer, most of which are redundant in that many neurons detect the same feature at different locations. This is clearly unscalable to apply everywhere, so only about 10% of the visual field concentrates these neurons.

On the other hand, we do have the ability to perform convolutions by moving our eyes. So in effect, we can jitter our eyes thereby applying the same neurons to different parts of the scene infront of us. I would think that this, combined with short-term memory can help us process images.

Suppose I force a subject to focus their eyes on the center of some common object (e.g. a car), such that the object is within the center 10% of our visual field. Are there situations where the brain is unable to process what the object is, without jittering our eyes? Said another way, how much visual processing are we able to do without any physical movement of our eyes or head?

I'm interested in understanding this question because I'm not convinced that even the massive amount of neurons in the visual cortex are sufficient for visually parsing the object. Whereas by (even slightly) shifting the object in our vision, the combined featurizations from each location would significantly reduce the computational complexity.

$\endgroup$
  • $\begingroup$ The massively parallel convolution is only true for the simplest visual features which are important for detecting objects of all types (like edges). Cells with more specific best-stimuli always have bigger receptive fields. $\endgroup$ – Bryan Krause Apr 30 '18 at 20:05
  • $\begingroup$ @BryanKrause: But I think the "bigger receptive field" here is still around 10 degrees of vision (for example see Figure 4C on page 33 of the above link). That particular IT neuron is responding to binoculars and an otter, but not a face. It's still not clear what features exactly that neuron is capturing, but they are surely higher order combinations of lower-level features. $\endgroup$ – Alex R. Apr 30 '18 at 21:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.