What does the visual feed from the retina look like before it is processed by the visual cortex?

Question

I was recently at the ACM Symposium on Applied Perception, where I saw a keynote on visual attention. The speaker (Laurent Itti) showed an animation that blew my mind. It showed a video of fish swimming in the ocean, and next to it, the same video as "seen" by the retina. The latter was upside down and distorted beyond recognition except at the fovea, which darted around to salient locations in the video. I had heard it explained before that "your eye doesn't 'see' images, your brain reconstructs them," but I had never before seen it illustrated so well.

I want to find a similar animation to show friends and students, but I'm having trouble finding anything. Searches yield mostly diagrams of light passing through the eye or something similar. The only thing I've found that shows what an image actually looks like when it reaches the retina is this picture. It's a nice start, but I'd really like an animation, like a video or GIF, to show how the brain stitches together the retinal images at each fixation into what seems like a coherent image/video.

Can you help me find such an animation? Something that shows what a raw visual feed actually looks like?

Answer 1

There is a LOT more to "your eye doesn't 'see' images, your brain reconstructs them" then what is shown in that picture. After the image is flipped and distorted, contrast information at various scales is extracted and pretty much everything else is thrown away. Here is a simulation of that:

This information is then used to build up progressively more complex and abstract representations of the visual scene.

I found a video showing a simulation of how this edge detection works, but it doesn't appear to include the distortion and receptive field sizes. I haven't been able to find a video that combined everything.

Answer 2

There's a recent article on decoding retinal signals from primates (macaque monkeys)

https://direct.mit.edu/neco/article/33/7/1719/100579/Nonlinear-Decoding-of-Natural-Images-From-Large

Answer 3

Professor Laurent Itti's videos are here: http://ilab.usc.edu/retina/

The photo in the original post can be found here, in its own context: https://techlongreads.com/photography/eye-vs-image-sensor/