Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021highpass-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.875
2
.858
3
.851
4
.827
5
.812
6
.807
7
.806
8
.793
9
.722
10
.716
11
.709
12
.652
13
.592
14
.550
15
.550
16
.506
17
.493
18
.457
19
.442
20
.422
21
.419
22
.410
23
.404
24
.380
25
.367
26
.364
27
.355
28
.330
29
.305
30
.303
31
.294
32
.293
33
.291
34
.287
35
.285
36
.262
37
.248
38
.226
39
.220
40
.202
41
.200
42
.195
43
.189
44
.177
45
.176
46
.175
47
.154
48
.151
49
.150
50
.149
51
.143
52
.142
53
.139
54
.139
55
.139
56
.139
57
.135
58
.132
59
.131
60
.127
61
.123
62
.120
63
.118
64
.116
65
.116
66
.111
67
.110
68
.101
69
.100
70
.100
71
.100
72
.096
73
.096
74
.095
75
.095
76
.095
77
.095
78
.094
79
.093
80
.093
81
.092
82
.091
83
.091
84
.090
85
.090
86
.089
87
.089
88
.088
89
.088
90
.086
91
.085
92
.083
93
.083
94
.082
95
.081
96
.081
97
.080
98
.079
99
.078
100
.078
101
.077
102
.075
103
.075
104
.075
105
.075
106
.075
107
.075
108
.075
109
.075
110
.074
111
.073
112
.073
113
.073
114
.073
115
.072
116
.072
117
.072
118
.071
119
.071
120
.066
121
.066
122
.066
123
.065
124
.064
125
.064
126
.064
127
.063
128
.062
129
.061
130
.060
131
.060
132
.060
133
.059
134
.059
135
.059
136
.059
137
.056
138
.056
139
.056
140
.055
141
.055
142
.055
143
.055
144
.055
145
.055
146
.054
147
.054
148
.054
149
.054
150
.053
151
.053
152
.053
153
.053
154
.052
155
.052
156
.051
157
.051
158
.051
159
.051
160
.050
161
.050
162
.050
163
.050
164
.050
165
.049
166
.049
167
.048
168
.048
169
.047
170
.046
171
.046
172
.046
173
.045
174
.045
175
.045
176
.045
177
.045
178
.045
179
.045
180
.045
181
.045
182
.045
183
.045
184
.045
185
.045
186
.044
187
.044
188
.043
189
.043
190
.043
191
.043
192
.043
193
.043
194
.042
195
.042
196
.042
197
.041
198
.041
199
.040
200
.040
201
.040
202
.039
203
.038
204
.038
205
.038
206
.038
207
.037
208
.037
209
.037
210
.036
211
.036
212
.036
213
.036
214
.035
215
.035
216
.035
217
.034
218
.033
219
.033
220
.033
221
.033
222
.033
223
.032
224
.031
225
.031
226
.031
227
.031
228
.030
229
.030
230
.030
231
.030
232
.030
233
.030
234
.028
235
.027
236
.026
237
.026
238
.024
239
.024
240
.023
241
.023
242
.023
243
.023
244
.023
245
.022
246
.022
247
.022
248
.022
249
.022
250
.022
251
.021
252
.021
253
.020
254
.020
255
.019
256
.019
257
.019
258
.018
259
.016
260
.015
261
.015
262
.015
263
.015
264
.015
265
.015
266
.015
267
.015
268
.014
269
.014
270
.014
271
.013
272
.013
273
.013
274
.012
275
.012
276
.010
277
.010
278
.010
279
.010
280
.010
281
.010
282
.009
283
.007
284
.005
285
.004
286
.004
287
.004
288
.004
289
.003
290
.003
291
.003
292
.002
293
.002
294
.001
295
-0.030
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.44.

Note that scores are relative to this ceiling.

Data: Geirhos2021highpass

Metric: error_consistency