Publications

Human visual cortex is organized into dorsal, lateral, and ventral streams. A long-standing hypothesis is that the functional organization into streams emerged to support distinct visual behaviors. Here, we use a neural network-based computational model and a massive fMRI dataset to test how visual streams emerge. We find that models trained for stream-specific visual behaviors poorly capture neural responses and organization. Instead, a self-supervised Topographic Deep Artificial Neural Network, which encourages nearby units to respond similarly, successfully predicts neural responses, spatial segregation, and functional differentiation across streams. These findings challenge the prevailing view that streams evolved to separately support different behaviors, and suggest instead that functional organization arises from a single principle - balancing general representation learning with local spatial constraints.

A key feature of many cortical systems is functional organization- the arrangement of neurons with specific functional properties in characteristic spatial patterns across the cortical surface. However, the principles underlying the emergence and utility of functional organization are poorly understood. Here we develop the Topographic Deep Artificial Neural Network (TDANN), the first unified model to accurately predict the functional organization of multiple cortical areas in the primate visual system. We analyze the key factors responsible for the TDANN’s success and find that it strikes a balance between two specific objectives- achieving a task-general sensory representation that is self-supervised, and maximizing the smoothness of responses across the cortical sheet according to a metric that scales relative to cortical surface area. In turn, the representations learned by the TDANN are lower dimensional and more brain-like than those in models that lack a spatial smoothness constraint. Finally, we provide evidence that the TDANN’s functional organization balances performance with inter-area connection length, and use the resulting models for a proof-of-principle optimization of cortical prosthetic design. Our results thus offer a unified principle for understanding functional organization and a novel view of the functional role of the visual system in particular.

A model trained with a spatial constraint, on a single self-supervised task, can recapitulate the functional organization of visual cortex into multiple processing streams.

While primate visual cortex has typically been divided into two processing streams, recent research suggests that there may be at least three functionally distinct streams, extending along the ventral, lateral, and parietal surfaces of the brain. Here, we leveraged the Natural Scenes Dataset (Allen et al., 2022) to compare and model responses across these proposed streams. We show that cortical responses cluster by stream and reflect the hierarchical organization of cortex. We then tested how accurately deep convolutional neural networks (DCNNs) trained on supervised object categorization and action recognition objectives could predict responses in each stream. Given the differences in responses across streams and the prevailing view that only the ventral stream serves object categorization, we were surprised to find that these models fit ventral and lateral responses equally well, though they were slightly worse at predicting parietal responses. These findings suggest that additional constraints are required for model predictivity to match the functional organization of visual cortex.

Ventral temporal cortex (VTC) consists of high-level visual regions that are arranged in consistent anatomical locations across individuals. This consistency has led to several hypotheses about the factors that constrain the functional organization of VTC. A prevailing theory is that white matter connections influence the organization of VTC, however, the nature of this constraint is unclear. Here, we test two hypotheses - (1) white matter tracts are specific for each category or (2) white matter tracts are specific to cytoarchitectonic areas of VTC. To test these hypotheses, we used diffusion magnetic resonance imaging (dMRI) to identify white matter tracts and functional magnetic resonance imaging (fMRI) to identify category-selective regions in VTC in children and adults. We find that in childhood, white matter connections are linked to cytoarchitecture rather than category-selectivity. In adulthood, however, white matter connections are linked to both cytoarchitecture and category-selectivity. These results suggest a rethinking of the view that category-selective regions in VTC have category-specific white matter connections early in development. Instead, these findings suggest that the neural hardware underlying the processing of categorical stimuli may be more domain-general than previously thought, particularly in childhood.

Spatial processing by receptive fields is a core property of the visual system. However, it is unknown how spatial processing in high-level regions contributes to recognition behavior. As face inversion is thought to disrupt typical holistic processing of information in faces, we mapped population receptive fields (pRFs) with upright and inverted faces in the human visual system. Here we show that in face-selective regions, but not primary visual cortex, pRFs and overall visual field coverage are smaller and shifted downward in response to face inversion. From these measurements, we successfully predict the relative behavioral detriment of face inversion at different positions in the visual field. This correspondence between neural measurements and behavior demonstrates how spatial processing in face-selective regions may enable holistic perception. These results not only show that spatial processing in high-level visual regions is dynamically used towards recognition, but also suggest a powerful approach for bridging neural computations by receptive fields to behavior.

Human ventral temporal cortex contains category-selective regions that respond preferentially to ecologically relevant categories such as faces, bodies, places and words and that are causally involved in the perception of these categories. How do these regions develop during childhood? We used functional magnetic resonance imaging to measure longitudinal development of category selectivity in school-age children over 1 to 5 years. We discovered that, from young childhood to the teens, face- and word-selective regions in ventral temporal cortex expand and become more category selective, but limb-selective regions shrink and lose their preference for limbs. Critically, as a child develops, increases in face and word selectivity are directly linked to decreases in limb selectivity, revealing that during childhood, limb selectivity in ventral temporal cortex is repurposed into word and face selectivity. These data provide evidence for cortical recycling during childhood development. This has important implications for understanding typical as well as atypical brain development and necessitates a rethinking of how cortical function develops during childhood.

Face-processing occurs across ventral and lateral visual streams, which are involved in static and dynamic face perception, respectively. However, the nature of spatial computations across streams is unknown. Using functional MRI and population receptive field (pRF) mapping, we measured pRFs in face-selective regions. Results reveal that spatial computations by pRFs in ventral face-selective regions are concentrated around the center of gaze (fovea), but spatial computations in lateral face-selective regions extend peripherally. Diffusion MRI reveals that these differences are mirrored by a preponderance of white matter connections between ventral face-selective regions and foveal early visual cortex (EVC), while connections with lateral regions are distributed more uniformly across EVC eccentricities. These findings suggest a rethinking of spatial computations in face-selective regions, showing that they vary across ventral and lateral streams, and further propose that spatial computations in high-level regions are scaffolded by the fine-grain pattern of white matter connections from EVC.

When rewards are available, people expend more energy, increasing their motivational vigor. In theory incentives might drive behavior for two distinct reasons- First, they increase expected reward; second, they increase the difference in subjective value between successful and unsuccessful performance, which increases contingency—the degree to which action determines outcome. Previous studies of motivational vigor have never compared these directly. Here, we indexed motivational vigor by measuring the speed of eye movements toward a target after participants heard a cue indicating how outcomes would be determined. Eye movements were faster when the cue indicated that monetary rewards would be contingent on performance than when the cue indicated that rewards would be random. But even when the cue indicated that a reward was guaranteed regardless of speed, movement was still faster than when no reward was available. Motivation by contingent and certain rewards was uncorrelated across individuals, which suggests that there are two separable, independent components of motivation. Contingent motivation generated autonomic arousal, and unlike noncontingent motivation, was effective with penalties as well as rewards.

Faces are processed more holistically than other objects, and it has been suggested that the loss of holistic face processing causes acquired prosopagnosia. Support for this hypothesis comes from several cases who failed to show holistic face effects as well as the absence of reports of prosopagnosics with unequivocally normal holistic face perception. The current study examines the relationship between holistic face processing and prosopagnosia by testing seven acquired prosopagnosics with the face composite task, a classic measure of holistic face processing. To enhance the robustness of the findings, each prosopagnosic was tested with two versions of the composite task showing upright faces. We also tested an inverted condition to exclude the possibility that more general factors account for composite effects for upright faces. Four of the seven acquired prosopagnosic participants showed consistent upright face composite effects with minimal inverted face composite effects. We conclude that severe face processing deficits can co-occur with intact holistic face processing and that factors other than a loss of holistic processing contribute to the perceptual and recognition deficits in acquired prosopagnosia.