Individual differences in gaze and neural representations

Abstract

This thesis addresses two fundamental yet distinct questions from complementary perspectives: one focuses on the average observer and the other on the individual. First, we asked whether the ability of faces to modulate gaze dynamics even pre-saccadically (i.e. before moving the eyes to a new gaze location) generalizes to natural scene viewing. Second, we investigated whether individual differences in gaze lead to individual neural representations of complex movie stimuli. Both studies are centered on examining visual behavior under naturalistic conditions. In the first study, we investigate gaze dynamics in complex scenes, revealing that saccades toward faces are of higher velocity than those directed at inanimate objects, and fixations preceding face targets are shorter, especially when the face is near the current fixation point. These findings suggest that visual processing mechanisms prioritize faces even in naturalistic settings. This is especially remarkable given that gaze dynamics are in a complex scene exposed to objects embedded in visual clutter and the processing of the current and upcoming targets happens in parallel. The second study examined how individual gaze behavior contributes to neural representations. We derived cross-brain decoding accuracy across pairs of observers using hyperalignment while participants viewed a movie under free-viewing and fixation conditions. The results showed that free-viewing led to increased neural activity compared to fixation. Moreover, the individual differences in gaze preferences, particularly fixation tendencies on faces and text, were related to variability in neural representations in the inferior temporal cortex (IT), and the individual differences in Euclidean gaze position contributed to neural divergence in both IT and the primary visual cortex (V1). Together, these studies highlight the importance of high-level factors in gaze behavior and neural representations. We provide evidence that gaze rapidly orients toward semantically relevant features in a scene and that variability in preferential looking at specific semantic categories across observers can be linked to individual differences in the IT cortex. By examining gaze behavior across static and dynamic stimuli, this thesis underscores the role of active vision in shaping perception under naturalistic viewing conditions.