Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021colour-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.891
2
.885
3
.866
4
.815
5
.812
6
.811
7
.804
8
.800
9
.799
10
.794
11
.792
12
.788
13
.788
14
.787
15
.769
16
.769
17
.766
18
.762
19
.756
20
.755
21
.748
22
.746
23
.739
24
.739
25
.738
26
.731
27
.727
28
.721
29
.716
30
.715
31
.703
32
.696
33
.692
34
.691
35
.689
36
.685
37
.684
38
.675
39
.675
40
.670
41
.657
42
.653
43
.647
44
.640
45
.637
46
.631
47
.624
48
.619
49
.617
50
.591
51
.587
52
.587
53
.585
54
.574
55
.561
56
.560
57
.551
58
.546
59
.543
60
.541
61
.538
62
.534
63
.528
64
.521
65
.521
66
.519
67
.512
68
.510
69
.505
70
.504
71
.499
72
.491
73
.491
74
.489
75
.488
76
.479
77
.478
78
.474
79
.474
80
.474
81
.474
82
.474
83
.472
84
.470
85
.468
86
.468
87
.464
88
.464
89
.463
90
.462
91
.461
92
.456
93
.454
94
.450
95
.450
96
.448
97
.448
98
.448
99
.445
100
.443
101
.442
102
.441
103
.439
104
.438
105
.432
106
.430
107
.429
108
.428
109
.427
110
.422
111
.419
112
.411
113
.406
114
.406
115
.404
116
.403
117
.403
118
.400
119
.395
120
.395
121
.391
122
.390
123
.387
124
.377
125
.373
126
.370
127
.370
128
.370
129
.365
130
.363
131
.361
132
.356
133
.346
134
.344
135
.344
136
.343
137
.342
138
.341
139
.328
140
.325
141
.324
142
.322
143
.320
144
.320
145
.316
146
.314
147
.314
148
.311
149
.309
150
.300
151
.299
152
.298
153
.293
154
.290
155
.290
156
.288
157
.288
158
.288
159
.286
160
.284
161
.269
162
.268
163
.263
164
.263
165
.261
166
.260
167
.260
168
.254
169
.253
170
.252
171
.248
172
.248
173
.246
174
.239
175
.236
176
.231
177
.228
178
.218
179
.216
180
.215
181
.214
182
.214
183
.211
184
.211
185
.188
186
.182
187
.182
188
.180
189
.177
190
.173
191
.170
192
.168
193
.165
194
.163
195
.162
196
.161
197
.159
198
.152
199
.152
200
.150
201
.150
202
.146
203
.143
204
.137
205
.134
206
.134
207
.134
208
.134
209
.134
210
.134
211
.134
212
.134
213
.134
214
.134
215
.134
216
.134
217
.134
218
.131
219
.130
220
.123
221
.122
222
.120
223
.119
224
.115
225
.113
226
.111
227
.108
228
.104
229
.104
230
.104
231
.104
232
.104
233
.096
234
.090
235
.084
236
.072
237
.072
238
.071
239
.068
240
.068
241
.067
242
.065
243
.062
244
.061
245
.060
246
.060
247
.058
248
.050
249
.049
250
.049
251
.049
252
.049
253
.048
254
.047
255
.045
256
.044
257
.043
258
.043
259
.041
260
.039
261
.038
262
.037
263
.037
264
.036
265
.036
266
.035
267
.035
268
.031
269
.030
270
.030
271
.030
272
.027
273
.027
274
.027
275
.025
276
.025
277
.025
278
.022
279
.022
280
.020
281
.020
282
.020
283
.020
284
.015
285
.014
286
.014
287
.009
288
.009
289
.009
290
.009
291
.007
292
.006
293
.004
294
.004
295
.003
296
.002
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X
407
X
408
X
409
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.42.

Note that scores are relative to this ceiling.

Data: Geirhos2021colour

Metric: error_consistency