Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021eidolonI-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.789
2
.783
3
.777
4
.776
5
.731
6
.724
7
.715
8
.698
9
.689
10
.677
11
.667
12
.663
13
.661
14
.655
15
.651
16
.647
17
.646
18
.638
19
.637
20
.637
21
.636
22
.629
23
.629
24
.621
25
.619
26
.619
27
.619
28
.615
29
.614
30
.610
31
.607
32
.603
33
.593
34
.593
35
.584
36
.583
37
.582
38
.577
39
.575
40
.574
41
.574
42
.567
43
.556
44
.553
45
.551
46
.549
47
.546
48
.541
49
.541
50
.537
51
.529
52
.527
53
.527
54
.524
55
.522
56
.520
57
.519
58
.513
59
.513
60
.512
61
.505
62
.499
63
.498
64
.497
65
.492
66
.491
67
.487
68
.482
69
.481
70
.476
71
.472
72
.471
73
.470
74
.464
75
.459
76
.459
77
.459
78
.456
79
.453
80
.448
81
.446
82
.445
83
.443
84
.443
85
.443
86
.442
87
.436
88
.435
89
.432
90
.432
91
.431
92
.430
93
.429
94
.424
95
.417
96
.417
97
.416
98
.412
99
.409
100
.408
101
.406
102
.403
103
.401
104
.400
105
.400
106
.400
107
.400
108
.394
109
.393
110
.384
111
.383
112
.378
113
.377
114
.371
115
.367
116
.363
117
.359
118
.355
119
.355
120
.354
121
.350
122
.349
123
.346
124
.344
125
.341
126
.338
127
.336
128
.336
129
.335
130
.334
131
.333
132
.331
133
.329
134
.328
135
.328
136
.327
137
.326
138
.325
139
.324
140
.324
141
.318
142
.318
143
.317
144
.309
145
.304
146
.301
147
.299
148
.297
149
.297
150
.297
151
.296
152
.295
153
.295
154
.293
155
.291
156
.290
157
.288
158
.287
159
.287
160
.286
161
.281
162
.281
163
.281
164
.281
165
.281
166
.281
167
.281
168
.281
169
.281
170
.281
171
.281
172
.281
173
.277
174
.276
175
.274
176
.272
177
.269
178
.264
179
.260
180
.256
181
.255
182
.253
183
.252
184
.251
185
.250
186
.249
187
.247
188
.245
189
.243
190
.242
191
.242
192
.242
193
.241
194
.239
195
.238
196
.235
197
.230
198
.229
199
.223
200
.223
201
.222
202
.218
203
.217
204
.213
205
.210
206
.210
207
.208
208
.208
209
.208
210
.206
211
.204
212
.198
213
.197
214
.196
215
.196
216
.194
217
.192
218
.188
219
.182
220
.181
221
.180
222
.180
223
.180
224
.179
225
.179
226
.171
227
.169
228
.168
229
.164
230
.161
231
.161
232
.160
233
.160
234
.159
235
.159
236
.156
237
.156
238
.150
239
.148
240
.143
241
.141
242
.140
243
.140
244
.138
245
.132
246
.132
247
.131
248
.131
249
.129
250
.126
251
.126
252
.125
253
.122
254
.120
255
.120
256
.113
257
.113
258
.111
259
.107
260
.097
261
.097
262
.094
263
.094
264
.094
265
.093
266
.089
267
.085
268
.085
269
.085
270
.085
271
.084
272
.084
273
.077
274
.070
275
.063
276
.059
277
.058
278
.058
279
.057
280
.056
281
.055
282
.051
283
.044
284
.042
285
.041
286
.038
287
.036
288
.029
289
.015
290
.013
291
.009
292
.008
293
.008
294
.002
295
.002
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X
407
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.39.

Note that scores are relative to this ceiling.

Data: Geirhos2021eidolonI

Metric: error_consistency