Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021contrast-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.771
2
.731
3
.718
4
.718
5
.709
6
.708
7
.696
8
.688
9
.658
10
.647
11
.635
12
.632
13
.631
14
.623
15
.621
16
.617
17
.617
18
.606
19
.598
20
.590
21
.581
22
.577
23
.574
24
.568
25
.565
26
.563
27
.562
28
.555
29
.554
30
.547
31
.544
32
.544
33
.540
34
.533
35
.511
36
.507
37
.506
38
.503
39
.503
40
.496
41
.492
42
.481
43
.478
44
.476
45
.460
46
.456
47
.452
48
.445
49
.433
50
.431
51
.430
52
.429
53
.418
54
.416
55
.413
56
.403
57
.400
58
.399
59
.397
60
.376
61
.370
62
.367
63
.351
64
.349
65
.347
66
.347
67
.345
68
.344
69
.340
70
.335
71
.332
72
.329
73
.328
74
.324
75
.322
76
.317
77
.309
78
.308
79
.308
80
.302
81
.301
82
.294
83
.283
84
.276
85
.274
86
.271
87
.269
88
.265
89
.261
90
.260
91
.254
92
.253
93
.253
94
.250
95
.246
96
.242
97
.241
98
.240
99
.237
100
.229
101
.229
102
.228
103
.221
104
.221
105
.221
106
.221
107
.220
108
.219
109
.216
110
.215
111
.206
112
.204
113
.200
114
.200
115
.193
116
.191
117
.186
118
.184
119
.182
120
.182
121
.182
122
.181
123
.179
124
.178
125
.176
126
.176
127
.171
128
.171
129
.166
130
.166
131
.166
132
.166
133
.163
134
.158
135
.158
136
.156
137
.156
138
.156
139
.155
140
.155
141
.155
142
.155
143
.154
144
.152
145
.152
146
.151
147
.148
148
.147
149
.147
150
.147
151
.145
152
.144
153
.143
154
.140
155
.140
156
.139
157
.139
158
.135
159
.133
160
.131
161
.131
162
.125
163
.123
164
.123
165
.123
166
.123
167
.123
168
.123
169
.123
170
.123
171
.123
172
.123
173
.123
174
.122
175
.121
176
.121
177
.120
178
.120
179
.120
180
.119
181
.119
182
.119
183
.118
184
.117
185
.116
186
.115
187
.114
188
.113
189
.113
190
.113
191
.113
192
.113
193
.110
194
.109
195
.108
196
.108
197
.107
198
.107
199
.107
200
.106
201
.106
202
.106
203
.106
204
.105
205
.103
206
.102
207
.102
208
.101
209
.101
210
.101
211
.099
212
.097
213
.096
214
.094
215
.094
216
.094
217
.093
218
.091
219
.088
220
.088
221
.087
222
.085
223
.083
224
.081
225
.081
226
.081
227
.080
228
.079
229
.075
230
.074
231
.072
232
.072
233
.071
234
.070
235
.070
236
.070
237
.070
238
.070
239
.069
240
.069
241
.064
242
.058
243
.056
244
.054
245
.052
246
.052
247
.050
248
.050
249
.049
250
.048
251
.047
252
.046
253
.045
254
.044
255
.044
256
.044
257
.044
258
.044
259
.043
260
.043
261
.037
262
.035
263
.033
264
.033
265
.032
266
.030
267
.025
268
.025
269
.024
270
.024
271
.022
272
.022
273
.022
274
.021
275
.020
276
.020
277
.020
278
.020
279
.019
280
.018
281
.018
282
.017
283
.017
284
.016
285
.016
286
.016
287
.015
288
.015
289
.014
290
.013
291
.013
292
.012
293
.012
294
.011
295
.008
296
.006
297
.006
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X
407
X
408
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.44.

Note that scores are relative to this ceiling.

Data: Geirhos2021contrast

Metric: error_consistency