Brain-Score

Sample stimuli

sample 0

sample 1

sample 2

sample 3

sample 4

sample 5

sample 6

sample 7

sample 8

sample 9

How to use

from brainscore_language import load_benchmark
benchmark = load_benchmark("Pereira2018.384sentences-linear")
score = benchmark(my_model)

Model scores

Score Legend Min Alignment Max Alignment

Rank	Model	Score
1	gpt-neo-2.7B	1.0
2	gpt-neo-1.3B	1.0
3	gpt2-xl	1.0
4	gpt2-large	.974
5	gpt2-medium	.927
6	gpt2	.828
7	distilgpt2	.815
8	openai-gpt	.800
9	gpt-neo-125m	.788
10	lm1b	.568
11	randomembedding-1600	.419
12	randomembedding-100	.221
13	glove-840b	.184
14	earley-parser-minivocab	X

Benchmark bibtex

None

Ceiling

0.36.

Note that scores are relative to this ceiling.

Data: Pereira2018.384sentences

Metric: linear