Quantifying chromatic adaptations made by the brain

Question

What are the ways one can use to measure the kind of adaptations done by the brain to provide colour constancy? Has this been done before?

Answer 1

You have specifically asked about the "adaptations done by the brain" in your initial question and an interest in "what has been done towards finding the formula and what it was replaced with for the time being?" By formula, I'm assuming that you meant a perceptual algorithm that can be useful in machine learning and the attempt to reverse-engineer human's perceptual systems.

Yes, there has been extensive work done on this. I cannot give you an extensive answer on this platform, but hopefully point you towards the right direction. I have mentioned two papers and their abstracts, the former is more accessible, and the later in case you want a sneak-peak of the mathematical methods involved in making a computational model of color constancy and chromatic adaptations.

On the other hand, if you're new to this or you want to simply refresh your memory - I have attached my ex-Professor's notes from a Perception class I took a few years ago to give a clearer idea. It might just be best to first read through those first, in which case you should skip the first two papers I have mentioned.

---

Sensory, computational and cognitive components of human colour constancy.
Smithson H.
Philosophical Transactions of the Royal Society, Biological Sciences. 2005; 360(1458): 1329-1346.

Abstract

When the illumination on a scene changes, so do the visual signals elicited by that scene. In spite of these changes, the objects within a scene tend to remain constant in their apparent colour. We start this review by discussing the psychophysical procedures that have been used to quantify colour constancy. The transformation imposed on the visual signals by a change in illumination dictates what the visual system must ‘undo’ to achieve constancy. The problem is mathematically underdetermined, and can be solved only by exploiting regularities of the visual world. The last decade has seen a substantial increase in our knowledge of such regularities as technical advances have made it possible to make empirical measurements of large numbers of environmental scenes and illuminants. This review provides a taxonomy of models of human colour constancy based first on the assumptions they make about how the inverse transformation might be simplified, and second, on how the parameters of the inverse transformation might be set by elements of a complex scene. Candidate algorithms for human colour constancy are represented graphically and pictorially, and the availability and utility of an accurate estimate of the illuminant is discussed. Throughout this review, we consider both the information that is, in principle, available and empirical assessments of what information the visual system actually uses. In the final section we discuss where in our visual systems these computations might be implemented.

---

Color constancy using natural image statistics and scene semantics.
A. Gijsenij and T.Gevers.
IEEE PAMI, 33(4):687–698, 2011.

Abstract

To achieve selection and combining of color constancy algorithms, in this paper natural image statistics are used to identify the most important characteristics of color images. Then, based on these image characteristics, the proper color constancy algorithm (or best combination of algorithms) is selected for a specific image. To capture the image characteristics, the Weibull parameterization (e.g., grain size and contrast) is used. It is shown that the Weibull parameterization is related to the image attributes to which the used color constancy methods are sensitive. An MoG-classifier is used to learn the correlation and weighting between the Weibull-parameters and the image attributes (number of edges, amount of texture, and SNR). The output of the classifier is the selection of the best performing color constancy method for a certain image. Experimental results show a large improvement over state-of-the-art single algorithms. On a data set consisting of more than 11,000 images, an increase in color constancy performance up to 20 percent (median angular error) can be obtained compared to the best-performing single algorithm. Further, it is shown that for certain scene categories, one specific color constancy algorithm can be used instead of the classifier considering several algorithms.

---

Below are examples of a few experiments that can be played out from behind your computer screen:

From Jonathon Winawer's notes on Perception:

Color Constancy and Chromatic Adaption

Take a photograph under fluorescent light, and compare it to the same picture under daylight. The colors come out totally differently - greenish under the fluorescent light and reddish under daylight - unless you do some "color correction" while developing the film.

But you wouldn't see it that way if you were in the room. To you the colors would look pretty much the same under both illuminants. This phenomenon is called color constancy, analogous to brightness constancy that we discussed earlier. The eye does not act like a camera, simply recording the image. Rather, the eye adapts to compensate for the color (SPD) of the light source.

Above is another example of a pair of photographs taken under different lighting conditions without color correction. The physical characteristics of the light reaching the camera is very different depending on the color of the illuminant. This results in dramatically different photographs. But if you were there when the pictures were taken, this object would look pretty much the same to you under both illuminants.

Glance at the penguin and dragon pictures above by fixated on the dot between them. The penguin picture looks very blueish and the dragon looks very yellowish. Next, you will hold your gaze on the dot between the blue and yellow fields. Continue staring at that dot for 30 secs or so. Then look back at the penguin and dragon by fixating the dot between them. What do you see? Why? The change in percept following adaptation is due to chromatic adaptation. Chromatic adaptation is like light and dark adaptation but instead of adapting just to light and dark, it adapts to whatever the color is of the ambient illumination.

Each cone type adapts independently. For example, a given L cone adapts according to local average L cone excitation. Likewise for the M cones. Thus, the retinal image adjusts to compensate not only for the overall intensity of the light source, but also to compensate for the color of the light source.

Chromatic adaptation, like light adaptation, can give rise to dramatic aftereffects. For example, adapt to this green, black, and yellow flag for 60 secs, then look at a white field and you will see an afterimage of a red, white, and blue flag. Red/green, blue/yellow, black/white are complementary colors. Normally, when you look at a white field, L and M cones give about the same response so the red/green opponnent colors mechanism does not respond at all. If you adapt to green, the M cone sensitivity is reduced. Then, when you look at a white field, the L:M cones are out of balance; the L cones are now more sensitive than the M cones so the red/green mechanism gives a positive response and you see red instead of white. This only lasts for a couple of seconds because the M cone sensitivity starts to readjust right away. The visual system is designed to try to achieve a perceptual constancy. But, as with the various brightness illusions I showed earlier, color adaptation also results in some misperceptions. The colored afterimage is an undesirable consequence of chromatic adaptation coupled with color opponency. Usually chromatic adaptation does the right thing, it compensates for the color of the illuminant.