Perception of Material Kinematics

Abstract

Many objects that we encounter have typical material properties that are related to their specific affordances: spoons are hard, pillows are soft, and Jell-O is wobbly. Over a lifetime of experiences and interacting with these objects, strong associations between an object and its typical material properties may be formed, and these associations not only include how glossy, rough, or pink an object is but also how it behaves under force: we expect knocked over vases to shatter, popped bike tires to deflate, and gooey grilled cheese to hang between two slices of bread when pulled apart. Here we ask how such rich visual priors affect the visual perception of material qualities, and present particularly striking examples of expectation violation. In a cue conflict design, we pair computer-rendered familiar objects with surprising material behaviors (a linen curtain shattering, a porcelain teacup wrinkling, etc.) and find that material qualities are not solely estimated from the object s kinematics (i.e. its physical (atypical) motion while shattering, wrinkling, wobbling etc.); rather, material appearance is sometimes pulled towards the native motion, shape, and optical properties that are associated with this object. Our results, in addition to patterns we find in reaction time data, suggest that visual priors about materials can set up high-level expectations about complex future states of an object and show how these priors modulate material appearance. Understanding how high-level expectations are integrated with incoming sensory evidence is an essential step towards understanding how the human visual system accomplishes material perception. We take this finding a step further and ask, when people are making judgments about the material properties of an object, where on the object do they look when making such judgments? Which areas are most informative for which material judgments? Can a linear classifier predict this? Ultimately, understanding which regions of an object are critical for material perception will provide insight into how the human visual system solves the problem of material perception rapidly and efficiently.