On the Color Vision of Deep Neural Networks: parallels with Humans

Abstract

Seeing in color is a primordial aspect of our visual experience. Despite its importance, it is still misunderstood what exact purpose our color vision serves. A common belief is that object recognition, crucial to our survival, is a core driving force in the development of our visual system, and our color perception by extension. And indeed, color is known for improving our ability to recognize objects. In this thesis, I explored the limits to which Deep Neural Networks, optimized for object recognition or color constancy, can explain and help us understand our color vision. Using advanced feature visualization, stimuli generation and representational analysis methods, I carefully examined the color vision of these trained models, also comparing their artificial responses to biological visual systems. I find that both artificial and biological systems exhibit some striking differences, but these are outweighed by the sheer number of similarities. These similarities include (1) large computing power for the processing of color, (2) single and double opponent units in their early processing stages, (3) more sensitivity to variations in hue than saturation, and (4) color representations that follow similar perceptual dimensions. Despite their limitations, Deep Neural Networks can thus astonishingly explain many color properties of our visual system. This thesis hence provides evidence that our color vision is largely shaped for and motivated by a feedforward recognition of natural objects and their surface colors.