**Short answer**  
Contrast is hardwired in the visual system and is not directly related to adaptive processes. 

**Background**  
According to the model of [Hering][1], color contrast in trichromatic species (such as most humans) is basically established between three sets of opponent systems: yellow-blue, red-green and the achromatic channel (dark-bright), and depicted in Fig. 1. The [opponent system][1] is established in the brain by cells in the [visual cortex][2] responding to both colors of a pair, but one color (e.g. blue) excites and the other (yellow) inhibits the cell, or *vice versa*. These systems are basically established by funneling the color information of opponent colors from the primary sensory neurons (the photoreceptive rods and cones) to color sensitive cells in the visual cortex [(Gouras, 2009)][3]. In effect, we cannot perceive a yellowish blue, or a reddish green, because these colors are opponent colors in a pair. 

[![!\[Hering][4]][4]  
<sup>Fig. 1. Hering model of color opponency. source: [Mark Green][1]</sup>

Because of this, the primary color-pairs yellow-blue and red-green yield excellent contrast, because in the brain the opponent response of cells will cause the edge to be sharply defined. Likewise, the achromatic channel yields high-contrast acuity. Mixed colors will impinge on this system not in an ON/OFF way, but in a gradient. For example, in your example the blue-side of the yellow-blue channel will be activated and contrasted against the achromatic channel. The highest possible contrast would be delivered the blue chalk with a yellow opponent system (use a yellow 'white' board :-). Now contrast has to be formed using the yellow-blue and achromatic channels that are phsyiologically not opponents.   

Another reason is foveal tritanopia: the fovea has the highest resolution for perceiving fine detail. There are no short wave cones (blue cones) in the very center, the area of maximum resolution [(Williams *et al*., 1981)][5], so presumably it is impossible to see blue with the area of the retina with the highest contrast acuity.

Note that while the aforementioned Hering model does not depend on adaptation, many illusions do depend on adaptive processes in the color system, most notably [after-images][6]. 

<sub>**References**  
**-** [Gouras, Color Vision. In: *Webvision. The Organization of the Retina and Visual System* (2009)][3]  
**-** [Williams *et al*., *Vis Res* (1981) **21**(9): 1341–56][5]</sub>


  [1]: http://www.visualexpert.com/FAQ/Part1/cfaqPart1.html
  [2]: http://isle.hanover.edu/Ch06Color/Ch06OpponentCells.html
  [3]: http://webvision.med.utah.edu/book/part-vii-color-vision/color-vision/
  [4]: https://i.sstatic.net/ls6Ub.png
  [5]: http://www.sciencedirect.com/science/article/pii/0042698981902418
  [6]: http://www.colorcube.com/illusions/aftrimge.htm