Brain-Score

Sample stimuli

sample 0

sample 1

sample 2

sample 3

sample 4

sample 5

sample 6

sample 7

sample 8

sample 9

How to use

from brainscore_language import load_benchmark
benchmark = load_benchmark("Pereira2018.243sentences-linear")
score = benchmark(my_model)

Model scores

Score Legend Min Alignment Max Alignment

Rank	Model	Score
1	gpt-neo-1.3B	1.0
2	gpt-neo-2.7B	.980
3	gpt2-xl	.973
4	gpt2-large	.922
5	gpt2-medium	.908
6	gpt2	.871
7	distilgpt2	.822
8	openai-gpt	.804
9	gpt-neo-125m	.781
10	lm1b	.593
11	randomembedding-1600	.244
12	glove-840b	.242
13	randomembedding-100	.029
14	earley-parser-minivocab	X

Benchmark bibtex

None

Ceiling

0.35.

Note that scores are relative to this ceiling.

Data: Pereira2018.243sentences

Metric: linear