Low-level features predict perceived similarity for naturalistic images

Abstract

The mechanisms by which humans perceptually organize individual regions of a visual scene to generate a coherent scene representation remain largely unknown. Our perception of statistical regularities has been relatively well-studied in simple stimuli, and explicit computational mechanisms that use low-level image features (e.g., luminance, contrast energy) to explain these perceptions have been described. Here, we investigate to what extent observers can effectively use such low-level information present in isolated naturalistic scene regions to facilitate associations between said regions. Across two experiments, participants were shown an isolated reference patch, then required to select which of two subsequently presented patches came from the same scene as the reference (two-alternative forced choice method). In Experiment 1, participants made their judgments based on unaltered image patches, and were consistently above chance when performing such association judgments. Additionally, participants’ responses were well-predicted by a generalized linear multilevel model using predictors based on low-level feature similarity metrics (specifically, pixel-wise luminance and phase-invariant structure correlations). In Experiment 2, participants were presented with unaltered image regions, thresholded image regions, or regions reduced to only their edge content. Performance for thresholded and edge regions was significantly poorer than for unaltered image regions. Nonetheless, the model still correlated well with participants’ judgments. Our findings suggest that image region associations can be accounted for using low-level feature correlations, suggesting such basic features are strongly associated with those underlying judgments made for complex visual stimuli.