Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021edge-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
1.2
2
1.2
3
1.2
4
1.0
5
.995
6
.979
7
.899
8
.854
9
.807
10
.753
11
.744
12
.728
13
.719
14
.699
15
.670
16
.645
17
.635
18
.619
19
.618
20
.608
21
.543
22
.543
23
.531
24
.519
25
.516
26
.516
27
.502
28
.395
29
.285
30
.264
31
.264
32
.252
33
.243
34
.232
35
.219
36
.198
37
.195
38
.194
39
.192
40
.187
41
.184
42
.183
43
.182
44
.182
45
.175
46
.173
47
.168
48
.163
49
.163
50
.160
51
.158
52
.153
53
.151
54
.151
55
.151
56
.151
57
.148
58
.144
59
.144
60
.140
61
.139
62
.139
63
.138
64
.133
65
.132
66
.132
67
.131
68
.128
69
.124
70
.124
71
.122
72
.116
73
.116
74
.115
75
.115
76
.115
77
.114
78
.113
79
.113
80
.113
81
.109
82
.109
83
.107
84
.107
85
.107
86
.107
87
.107
88
.107
89
.106
90
.106
91
.106
92
.106
93
.105
94
.105
95
.105
96
.104
97
.104
98
.101
99
.100
100
.099
101
.099
102
.099
103
.098
104
.098
105
.098
106
.098
107
.098
108
.093
109
.092
110
.092
111
.092
112
.092
113
.092
114
.092
115
.092
116
.092
117
.092
118
.092
119
.091
120
.091
121
.091
122
.091
123
.091
124
.091
125
.091
126
.091
127
.091
128
.090
129
.090
130
.085
131
.085
132
.084
133
.084
134
.084
135
.084
136
.084
137
.084
138
.084
139
.084
140
.084
141
.084
142
.084
143
.084
144
.084
145
.084
146
.083
147
.083
148
.083
149
.079
150
.078
151
.078
152
.077
153
.077
154
.077
155
.077
156
.077
157
.077
158
.077
159
.077
160
.076
161
.076
162
.076
163
.076
164
.076
165
.076
166
.076
167
.076
168
.076
169
.076
170
.076
171
.075
172
.075
173
.075
174
.075
175
.074
176
.074
177
.074
178
.074
179
.070
180
.070
181
.070
182
.070
183
.069
184
.069
185
.069
186
.069
187
.069
188
.069
189
.069
190
.069
191
.069
192
.069
193
.069
194
.069
195
.069
196
.069
197
.069
198
.069
199
.068
200
.068
201
.068
202
.068
203
.068
204
.068
205
.068
206
.068
207
.068
208
.066
209
.064
210
.063
211
.063
212
.063
213
.063
214
.062
215
.062
216
.062
217
.062
218
.062
219
.062
220
.062
221
.062
222
.062
223
.062
224
.062
225
.062
226
.061
227
.061
228
.061
229
.060
230
.059
231
.059
232
.056
233
.056
234
.055
235
.055
236
.055
237
.055
238
.055
239
.055
240
.055
241
.055
242
.055
243
.055
244
.055
245
.055
246
.055
247
.055
248
.055
249
.055
250
.055
251
.055
252
.055
253
.055
254
.055
255
.055
256
.055
257
.055
258
.055
259
.055
260
.055
261
.053
262
.053
263
.053
264
.053
265
.050
266
.046
267
.046
268
.042
269
.042
270
.042
271
.039
272
.038
273
.037
274
.036
275
.034
276
.034
277
.032
278
.032
279
.032
280
.032
281
.032
282
.032
283
.032
284
.024
285
.024
286
.024
287
.024
288
.021
289
.021
290
.021
291
.021
292
.021
293
.014
294
.011
295
.010
296
.007
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X
407
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.32.

Note that scores are relative to this ceiling.

Data: Geirhos2021edge

Metric: error_consistency