vs

1 dot = 1 model. Only including models where all scores are computed.


Examples of relationships discovered in the literature:

Object classification performance is correlated with brain alignment.

Researchers found that the performance of computational models in classifying the object in an image (aka the popular Computer Vision ImageNet benchmark) is correlated with the alignment of the model to brain and behavioral data. This is true across different stages of the ventral stream and has been covered in several publications (e.g., Yamins & Hong et al. 2014 with a smaller image dataset, Schrimpf & Kubilius et al. 2018 with ImageNet). This connection between machine learning and neuroscience desiderata implies a striking synergy between research in these two domains, as building better-performing models might also lead to better models of the brain.

Alignment to early visual cortex V1 is correlated with model robustness.

The more similar model representations are to recordings from primary visual cortex V1, the more robust the output of the model is to perturbations in the input such as image distortions and adversarial attacks ( Dapello & Marques et al. 2020). This finding is based on Brain-Score V1 benchmarks with data from Freeman & Ziemba et al. 2013 as well as ImageNet-C, and inspired the development of the VOneNet model.

Models that exhibit more V1-like properties in early stages are more aligned to behavior.

As we make models more aligned to a diverse set of properties that have been discovered in primary visual cortex V1 -- such as receptive field sizes, response selectivity, surround and texture modulation, -- these more brain-aligned models also produce more human-like behavioral choices on a match-to-sample task. The V1 properties here have been compiled by Marques et al. 2021 from many classic neuroscience studies, and the behavioral data was collected by Rajalingham et al. 2018.