Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021uniformnoise-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.781
2
.754
3
.743
4
.734
5
.731
6
.728
7
.723
8
.720
9
.719
10
.686
11
.685
12
.675
13
.674
14
.670
15
.661
16
.647
17
.647
18
.645
19
.639
20
.637
21
.635
22
.627
23
.627
24
.617
25
.614
26
.604
27
.601
28
.601
29
.599
30
.591
31
.590
32
.587
33
.583
34
.577
35
.576
36
.576
37
.572
38
.555
39
.537
40
.524
41
.522
42
.520
43
.520
44
.515
45
.509
46
.493
47
.493
48
.476
49
.475
50
.473
51
.462
52
.458
53
.455
54
.443
55
.441
56
.418
57
.404
58
.401
59
.386
60
.381
61
.377
62
.354
63
.348
64
.344
65
.340
66
.330
67
.328
68
.322
69
.309
70
.309
71
.309
72
.289
73
.289
74
.285
75
.284
76
.279
77
.272
78
.266
79
.265
80
.260
81
.257
82
.257
83
.254
84
.253
85
.250
86
.250
87
.249
88
.245
89
.244
90
.243
91
.231
92
.231
93
.224
94
.209
95
.202
96
.196
97
.194
98
.194
99
.194
100
.194
101
.194
102
.188
103
.188
104
.183
105
.182
106
.180
107
.179
108
.174
109
.173
110
.172
111
.172
112
.166
113
.162
114
.162
115
.157
116
.157
117
.156
118
.153
119
.149
120
.144
121
.143
122
.137
123
.136
124
.134
125
.133
126
.132
127
.131
128
.130
129
.130
130
.129
131
.127
132
.127
133
.125
134
.122
135
.121
136
.121
137
.121
138
.119
139
.119
140
.116
141
.112
142
.105
143
.105
144
.103
145
.099
146
.098
147
.097
148
.091
149
.090
150
.089
151
.089
152
.089
153
.088
154
.088
155
.088
156
.088
157
.088
158
.088
159
.088
160
.080
161
.079
162
.079
163
.078
164
.078
165
.078
166
.077
167
.076
168
.075
169
.075
170
.075
171
.073
172
.073
173
.071
174
.069
175
.068
176
.068
177
.067
178
.067
179
.067
180
.064
181
.062
182
.061
183
.059
184
.059
185
.058
186
.058
187
.057
188
.057
189
.057
190
.057
191
.057
192
.056
193
.056
194
.054
195
.054
196
.050
197
.049
198
.049
199
.049
200
.049
201
.049
202
.048
203
.048
204
.048
205
.048
206
.047
207
.047
208
.047
209
.046
210
.046
211
.046
212
.046
213
.045
214
.045
215
.045
216
.044
217
.044
218
.043
219
.043
220
.041
221
.041
222
.040
223
.039
224
.039
225
.038
226
.038
227
.038
228
.038
229
.038
230
.038
231
.038
232
.038
233
.038
234
.038
235
.038
236
.038
237
.038
238
.038
239
.038
240
.038
241
.036
242
.036
243
.036
244
.036
245
.036
246
.035
247
.034
248
.034
249
.033
250
.033
251
.031
252
.031
253
.030
254
.028
255
.028
256
.027
257
.027
258
.026
259
.024
260
.024
261
.023
262
.022
263
.022
264
.022
265
.022
266
.021
267
.021
268
.020
269
.017
270
.016
271
.016
272
.013
273
.011
274
.011
275
.011
276
.010
277
.010
278
.008
279
.006
280
.006
281
.006
282
.005
283
.005
284
.005
285
.005
286
.004
287
.004
288
.003
289
.003
290
.002
291
.001
292
.000
293
X
294
X
295
X
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.43.

Note that scores are relative to this ceiling.

Data: Geirhos2021uniformnoise

Metric: error_consistency