Natural Scene Statistics for Understanding Face Perception Mechanisms

Abstract

The human visual system is capable of rapidly translating an abundance of visual inputs into meaningful representations. For example, a single glance is sufficient for observers to recognize a typical natural scene with cars in the street, surrounded by houses, as a “town.” While much prior work has studied visual processing using simple artificial stimuli, little is known about how the visual system extracts information from significantly more complex natural scenes to form representations. In this study, we focus on the question of face representation and in what ways the visual system represents faces present in realistic naturalistic contexts. Here, we developed behavioral measurements of scene statistics that can be used to analyze and interpret the fMRI data provided by the large-scale 7T Natural Scenes Dataset (NSD) (Allen et al. 2022). Ten participants viewed 1000 natural scene images under central fixation and made a judgment on their level of face perception for each image. We asked participants to judge as quickly and accurately as they can based on whether (1) there is at least one face that they can identify (if they see it again), (2) there is at least one face with some facial features (that they can see clearly), (3) there is at least one head of an animate thing, and (4) there is no face present. The participants then viewed and made judgments on a selected set of 100 repeated images to measure their test-retest reliability. Their responses and reaction times were recorded. We found that participants take significantly longer times to recognize faces, identify faces, and identify animate objects as compared to identifying scenes without faces or animate objects. There is a high across-subject test-retest reliability (r = 0.85) even when the effect of the high agreement on control images between the participants is removed (r = 0.69). It is now possible to analyze and interpret fMRI data from NSD to elucidate mechanisms of face perception.