Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021eidolonIII-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.726
2
.644
3
.641
4
.638
5
.629
6
.610
7
.597
8
.597
9
.582
10
.579
11
.578
12
.574
13
.569
14
.567
15
.564
16
.561
17
.558
18
.556
19
.556
20
.554
21
.551
22
.550
23
.548
24
.546
25
.541
26
.535
27
.535
28
.535
29
.521
30
.518
31
.517
32
.511
33
.505
34
.502
35
.502
36
.501
37
.500
38
.495
39
.494
40
.490
41
.489
42
.489
43
.484
44
.480
45
.477
46
.476
47
.474
48
.473
49
.473
50
.470
51
.469
52
.468
53
.468
54
.468
55
.465
56
.464
57
.464
58
.459
59
.453
60
.453
61
.451
62
.445
63
.444
64
.443
65
.442
66
.441
67
.439
68
.438
69
.437
70
.433
71
.433
72
.432
73
.432
74
.429
75
.427
76
.424
77
.423
78
.418
79
.416
80
.415
81
.415
82
.415
83
.412
84
.412
85
.400
86
.397
87
.395
88
.393
89
.391
90
.388
91
.383
92
.380
93
.379
94
.377
95
.373
96
.369
97
.368
98
.366
99
.364
100
.360
101
.360
102
.357
103
.356
104
.353
105
.353
106
.352
107
.350
108
.346
109
.343
110
.340
111
.334
112
.333
113
.327
114
.324
115
.320
116
.320
117
.319
118
.319
119
.319
120
.318
121
.317
122
.316
123
.315
124
.314
125
.313
126
.313
127
.312
128
.304
129
.302
130
.302
131
.299
132
.299
133
.297
134
.297
135
.295
136
.294
137
.294
138
.292
139
.292
140
.291
141
.291
142
.291
143
.291
144
.290
145
.290
146
.287
147
.286
148
.286
149
.285
150
.284
151
.284
152
.283
153
.282
154
.281
155
.278
156
.276
157
.274
158
.273
159
.273
160
.273
161
.272
162
.271
163
.267
164
.263
165
.262
166
.258
167
.257
168
.256
169
.255
170
.252
171
.251
172
.250
173
.247
174
.245
175
.245
176
.244
177
.242
178
.241
179
.240
180
.238
181
.233
182
.231
183
.228
184
.226
185
.221
186
.220
187
.219
188
.219
189
.219
190
.215
191
.214
192
.213
193
.212
194
.204
195
.203
196
.200
197
.196
198
.190
199
.189
200
.174
201
.173
202
.171
203
.168
204
.168
205
.167
206
.167
207
.167
208
.167
209
.167
210
.167
211
.167
212
.167
213
.167
214
.167
215
.162
216
.161
217
.157
218
.154
219
.154
220
.151
221
.151
222
.146
223
.145
224
.142
225
.138
226
.137
227
.136
228
.132
229
.130
230
.129
231
.129
232
.129
233
.127
234
.126
235
.125
236
.124
237
.124
238
.124
239
.123
240
.121
241
.121
242
.117
243
.113
244
.112
245
.112
246
.112
247
.112
248
.108
249
.105
250
.105
251
.104
252
.103
253
.101
254
.099
255
.097
256
.096
257
.095
258
.094
259
.093
260
.093
261
.093
262
.091
263
.091
264
.090
265
.090
266
.089
267
.085
268
.081
269
.080
270
.079
271
.078
272
.076
273
.071
274
.069
275
.063
276
.056
277
.040
278
.039
279
.037
280
.037
281
.037
282
.037
283
.037
284
.036
285
.034
286
.033
287
.033
288
.026
289
.023
290
.018
291
.010
292
.007
293
.003
294
.001
295
.000
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.46.

Note that scores are relative to this ceiling.

Data: Geirhos2021eidolonIII

Metric: error_consistency