Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021lowpass-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.730
2
.695
3
.690
4
.667
5
.663
6
.648
7
.630
8
.625
9
.623
10
.613
11
.603
12
.602
13
.600
14
.589
15
.579
16
.578
17
.563
18
.554
19
.544
20
.544
21
.543
22
.537
23
.535
24
.535
25
.527
26
.527
27
.516
28
.499
29
.498
30
.497
31
.496
32
.492
33
.471
34
.463
35
.463
36
.460
37
.460
38
.454
39
.436
40
.435
41
.419
42
.419
43
.415
44
.410
45
.409
46
.406
47
.400
48
.397
49
.396
50
.391
51
.385
52
.382
53
.379
54
.379
55
.371
56
.368
57
.361
58
.354
59
.352
60
.338
61
.338
62
.326
63
.317
64
.314
65
.313
66
.311
67
.306
68
.305
69
.299
70
.295
71
.290
72
.285
73
.274
74
.270
75
.268
76
.268
77
.260
78
.260
79
.258
80
.258
81
.252
82
.252
83
.251
84
.244
85
.243
86
.232
87
.225
88
.221
89
.218
90
.214
91
.214
92
.213
93
.209
94
.207
95
.203
96
.203
97
.203
98
.202
99
.201
100
.197
101
.194
102
.187
103
.186
104
.186
105
.185
106
.185
107
.185
108
.185
109
.185
110
.184
111
.180
112
.175
113
.175
114
.174
115
.168
116
.167
117
.165
118
.165
119
.165
120
.163
121
.161
122
.161
123
.156
124
.156
125
.155
126
.153
127
.153
128
.152
129
.144
130
.143
131
.141
132
.140
133
.140
134
.139
135
.138
136
.135
137
.135
138
.133
139
.133
140
.132
141
.132
142
.130
143
.129
144
.129
145
.129
146
.127
147
.127
148
.127
149
.126
150
.126
151
.126
152
.126
153
.125
154
.124
155
.124
156
.123
157
.122
158
.122
159
.121
160
.119
161
.116
162
.115
163
.113
164
.113
165
.112
166
.112
167
.111
168
.111
169
.110
170
.110
171
.109
172
.109
173
.109
174
.109
175
.108
176
.107
177
.103
178
.103
179
.103
180
.100
181
.100
182
.100
183
.100
184
.097
185
.097
186
.096
187
.096
188
.096
189
.095
190
.095
191
.094
192
.091
193
.087
194
.086
195
.085
196
.085
197
.085
198
.085
199
.085
200
.084
201
.084
202
.082
203
.081
204
.081
205
.081
206
.081
207
.080
208
.079
209
.078
210
.078
211
.077
212
.077
213
.077
214
.075
215
.074
216
.074
217
.074
218
.072
219
.071
220
.070
221
.070
222
.069
223
.067
224
.066
225
.065
226
.064
227
.063
228
.061
229
.061
230
.061
231
.060
232
.060
233
.059
234
.059
235
.058
236
.057
237
.057
238
.056
239
.056
240
.056
241
.056
242
.056
243
.056
244
.056
245
.056
246
.056
247
.056
248
.056
249
.056
250
.055
251
.055
252
.054
253
.054
254
.052
255
.049
256
.049
257
.047
258
.045
259
.045
260
.044
261
.044
262
.043
263
.038
264
.038
265
.037
266
.032
267
.031
268
.031
269
.031
270
.031
271
.030
272
.029
273
.028
274
.026
275
.026
276
.025
277
.024
278
.022
279
.020
280
.019
281
.017
282
.016
283
.016
284
.014
285
.013
286
.013
287
.013
288
.013
289
.013
290
.007
291
.006
292
.006
293
.005
294
.003
295
.003
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.47.

Note that scores are relative to this ceiling.

Data: Geirhos2021lowpass

Metric: error_consistency