Deep Dive 1

A potentially organizing goal of the brain and cognitive sciences is to accurately explain domains of human intelligence as executable, neurally mechanistic models. Years of research have led to models that capture experimental results in individual behavioral tasks and individual brain regions. We here advocate for taking the next step: integrating experimental results from many laboratories into suites of benchmarks that, when considered together, push mechanistic models toward explaining entire domains of intelligence, such as vision, language, and motor control. Given recent successes of neurally mechanistic models and the surging availability of neural, anatomical, and behavioral data, we believe that now is the time to create integrative benchmarking platforms (such as BrainScore) that incentivize ambitious, unified models.

The Philosophy of Integrative Benchmarking:
Integrative Benchmarking to Advance Neurally Mechanistic Models of Human Intelligence

Implementations:
Brain-Score: Which Artificial Neural Network for Object Recognition is most Brain-Like?
The neural architecture of language: Integrative modeling converges on predictive processing
SyntaxGym: An Online Platform for Targeted Evaluation of Language Models

Building models that improve on benchmarks:
Performance-optimized hierarchical models predict neural responses in higher visual cortex
Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
Bag of Tricks for Training Brain-Like Deep Neural Networks

Jim DiCarlo Quest Launch Presentation
MIT Quest for Intelligence Launch: The Future of Intelligence Science

Martin Schrimpf Thesis Defense (Passcode: Bra1ns@MIT!)
Martin Schrimpf - Thesis Defense - Speaker View
Advancing System Models of Brain Processing via Integrative Benchmarking

NeurIPS'19 oral:
Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
Clubhouse: Neural Architecture of Language-Models, ML, and more!
The neural architecture of language: Integrative modeling converges on predictive processing

Documentation for Vision:
Brain-Score readthedocs
Brain-Score Model Tutorials
Brain-Score Benchmark Tutorials

Relevant Code:
neural-nlp
langbrainscore
syntaxgym-core
vision
candidate_models

In Deep Dive 2, we will cover the submission package. You can use this as a formatting guide for your own future submissions.