Is the visual cortex of newborn babies right off the bat capable of making sense of raw visual data, for instance, converting the constant stream of raw RGB images perceived by the eyes into a meaningful higher-level representation of objects in motion in a 3D world? Yes? No? If not, then does it mean this skill has to be developed over time by means of some learning mechanism?

If the visual cortex needs time to learn advanced visual skills, how does the actual learning mechanism work? Does the visual cortex have to optimize the connections between neurons, in a way analogous to how artificial neural networks optimize their parameters through backpropagation algorithm (machine learning)? If this is the case, then where does the visual cortex get its error signals from? To make the last question clear, in machine learning the typical approach is to compute the gradient of a loss function which compares the model's prediction with the ground truth, and the model's parameters are updated by moving the parameters in the direction of the gradient. If the visual cortex is learning advanced visual skills by virtue of a similar learning mechanism, then what kind of loss function is the visual cortex optimizing?


A newborn baby has difficulty focusing its eyes or telling the difference between two objects presented to it.

A baby learns to recognise its mothers face within the first week after birth, long before it can recognise objects that are not faces. They already show a preference for face-like visual stimuli while in the womb.

The Human Fetus Preferentially Engages with Face-like Visual Stimuli Reid et al. Current Biology, March 05, 2018 https://doi.org/10.1016/j.cub.2017.05.044

Babies start to follow moving objects with their eyes at around three months. They are capable of depth perception and have good colour vision by five months (at birth many babies are incapable of detecting some colours).

The American optometric association has a page on infant vision:


In precocial animals object recognition abilities are apparent very early on. For instance, chicks which are presented with an object immediately after they have hatched will often follow that object as if it were their mother. They show a preference for moving objects over stationary objects. For more information on this check out http://www.scholarpedia.org/article/Imprinting.

The visual cortex already starts off with areas specialised for edge detection, colour detection, motion detection etc. However, much of visual perception is learned - if an animal has its eyes patched over at birth then it will have limited perceptual abilities when the patches are removed.

As for the question of how errors are propagated through the visual cortex, Seanny123's answer to this question might be relevant: Is back-prop biologically plausible?

| improve this answer | |
  • $\begingroup$ Even if it's not clear whether backprop is used, at least do we have any hints of what the ultimate loss function the visual cortex is optimizing could be? I have the feeling that maybe the brain is training itself to predict the future. Say, given the last K frames perceived by the eyes, predict what the next K+1 frame will look like. The visual cortex would need to learn to model 3D objects and some level of intuitive physics in order to more accurately predict the future. Probably it should be more complicated than that if we mix other senses into the picture, but overall that's the idea. $\endgroup$ – Pablo Messina Apr 7 '19 at 19:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.