7
$\begingroup$

Some time ago I remember reading about how the human brain breaks down visual information into a number of individual "channels".

For example, one channel might focus on edges and lines, another might focus on being a contrast map, while another provide a color information overlay. Each "channel" being a much more simplified part of the entire visual input.

From what I understood, modern research into the visual cortex has provided some insight into what the channels might be and how they're broken down. If my understanding of current research is correct I was hoping to find some more detailed information explaining how the visual information is broken down and processed (articles at the technical level of Scientific American are fine, so long as they detail the types of "channels" present).

My desired take-away from this is to brainstorm ideas for how I might better pre-process an image into feature data for a standard neural network learning algorithm, based on the human experience.

$\endgroup$
8
+25
$\begingroup$

The human visual processing system receives input from the eyes, and then passes it through a number of areas of the brain that break it down, process it in various different ways, recombine it, and break it down again several times. I'm assuming this question is only about the visual cortex, general theories about how information might be broken down for processing there, and only concerning parallel-processing (channels), as opposed to sequential specialization (layers), otherwise the answer should fill up a textbook.

Main pathways through visual cortex:

The traditional understanding of the visual cortex divides it into 2 main pathways:

Each V1 transmits information to two primary pathways, called the ventral stream and the dorsal stream.

  • The ventral stream begins with V1, goes through visual area V2, then through visual area V4, and to the inferior temporal cortex (IP cortex). The ventral stream, sometimes called the "What Pathway", is associated with form recognition and object representation. It is also associated with storage of long-term memory.

  • The dorsal stream begins with V1, goes through Visual area V2, then to the dorsomedial area (DM/ V6) and Visual area MT (middle temporal/ V5) and to the posterior parietal cortex. The dorsal stream, sometimes called the "Where Pathway" or "How Pathway", is associated with motion, representation of object locations, and control of the eyes and arms, especially when visual information is used to guide saccades or reaching.

The "what" and "where" pathways are also sometimes referred to as the "perception" and "action" pathways, respectively.

enter image description here

Another common theory about the visual cortex identifies at least 3 modules: Motion, colour, and form. Evidence for these proposed modules comes largely from lesion studies and related disorders of the visual system. In general, an impairment in one module doesn't seem to affect the others, suggesting that these specializations are processed independently and in parallel. However, it is not clear if they form distinct pathways through the visual cortex.

There is also evidence for a long-term memory pathway (visual-perirhinal-hippocampal stream) that feeds memory information from and to the visual cortex. Many other pathways have been postulated; this field of research is very active, consensus on further detail remains elusive for now.

Complications to such models:

Such "pathway" models of the visual cortex are limited by a number of factors:

Furthermore, as mentioned, the visual cortex is only part of the visual system. Significant processing happens earlier - you mentioned about edge detection, some of that appears to already be happening by the time signals reach the visual cortex. And more processing happens later as well, in the parietal and temporal cortices for example, such as facial recognition.

Implications for artificial neural networks:

The visual cortex may not be ideal inspiration for ANNs, especially since it only handles a not-very-early part of the process, such that the signals it receives have already undergone significant modification from the raw stimuli. Additionally, its pathway structure is fairly complex, and not fully understood. There have nonetheless been attempts to simulate parts of the visual cortex and compare to non-biologically inspired networks.

The LGN - an area of the brain involved earlier in the process - may be more amenable to some inspiration and simulation, as it is a simpler structure with clearer functions and pathways. It handles some of the functions you described at a more basic level, including some edge detection, colour processing, contrast, and various other aspects of perception such as depth, brightness, and movement, albeit with many of the same caveats as mentioned for the visual cortex.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.