William T. Freeman/MIT

Abstract:  The human visual system is well-tuned to detect faces of all shapes and sizes. While this brings obvious survival advantages, such as a better chance of spotting unknown predators in the bush, it also leads to spurious face detections. "Face pareidolia" describes the perception of face-like structure among otherwise random stimuli: seeing faces in coffee stains or clouds in the sky. In this paper, we study face pareidolia from a computer vision perspective. We present an image dataset of "Faces in Things," consisting of five thousand images from the web annotated with pareidolic faces. We examine the extent to which a state-of-the-art neural network trained to detect human faces exhibits pareidolia, and find a significant behavioral gap between humans and machines. We show that the ability of humans to detect animal faces, as well as human faces, may help to explain some of this gap. Finally, we propose a statistical model of pareidolia for general objects in images. A study on human subjects and an empirical study with our pareidolic face detector confirm a prediction of our model regarding the image conditions most likely to induce pareidolia.