Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021powerequalisation-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.755
2
.718
3
.713
4
.709
5
.707
6
.707
7
.705
8
.680
9
.672
10
.670
11
.668
12
.651
13
.650
14
.647
15
.638
16
.633
17
.599
18
.592
19
.586
20
.579
21
.564
22
.560
23
.548
24
.540
25
.501
26
.494
27
.489
28
.486
29
.482
30
.472
31
.468
32
.467
33
.462
34
.455
35
.453
36
.451
37
.450
38
.427
39
.427
40
.427
41
.397
42
.388
43
.382
44
.380
45
.378
46
.363
47
.339
48
.335
49
.323
50
.317
51
.317
52
.315
53
.307
54
.295
55
.292
56
.289
57
.289
58
.287
59
.287
60
.280
61
.278
62
.269
63
.269
64
.260
65
.260
66
.257
67
.248
68
.246
69
.245
70
.237
71
.234
72
.228
73
.227
74
.222
75
.221
76
.217
77
.217
78
.216
79
.214
80
.209
81
.201
82
.197
83
.195
84
.193
85
.190
86
.189
87
.189
88
.187
89
.186
90
.186
91
.185
92
.184
93
.182
94
.181
95
.173
96
.169
97
.167
98
.164
99
.159
100
.155
101
.149
102
.147
103
.145
104
.145
105
.140
106
.134
107
.133
108
.132
109
.132
110
.128
111
.127
112
.127
113
.126
114
.125
115
.125
116
.125
117
.122
118
.121
119
.119
120
.119
121
.118
122
.116
123
.115
124
.114
125
.113
126
.112
127
.108
128
.107
129
.105
130
.105
131
.105
132
.100
133
.098
134
.098
135
.098
136
.098
137
.098
138
.098
139
.097
140
.097
141
.097
142
.097
143
.097
144
.097
145
.097
146
.097
147
.097
148
.097
149
.097
150
.096
151
.096
152
.095
153
.094
154
.094
155
.094
156
.093
157
.091
158
.089
159
.089
160
.089
161
.088
162
.088
163
.088
164
.086
165
.085
166
.084
167
.083
168
.083
169
.083
170
.083
171
.083
172
.082
173
.082
174
.078
175
.078
176
.078
177
.076
178
.075
179
.075
180
.073
181
.070
182
.070
183
.068
184
.065
185
.065
186
.065
187
.064
188
.064
189
.063
190
.062
191
.062
192
.062
193
.062
194
.060
195
.059
196
.059
197
.057
198
.053
199
.053
200
.053
201
.053
202
.053
203
.053
204
.052
205
.050
206
.048
207
.047
208
.047
209
.046
210
.045
211
.044
212
.043
213
.041
214
.041
215
.040
216
.040
217
.040
218
.040
219
.039
220
.039
221
.038
222
.038
223
.038
224
.038
225
.038
226
.037
227
.037
228
.037
229
.037
230
.036
231
.036
232
.036
233
.035
234
.035
235
.034
236
.034
237
.034
238
.033
239
.033
240
.033
241
.033
242
.032
243
.032
244
.031
245
.031
246
.031
247
.031
248
.031
249
.030
250
.029
251
.028
252
.027
253
.027
254
.027
255
.026
256
.024
257
.024
258
.023
259
.023
260
.023
261
.021
262
.021
263
.020
264
.020
265
.019
266
.019
267
.019
268
.018
269
.018
270
.017
271
.016
272
.015
273
.015
274
.014
275
.014
276
.014
277
.014
278
.012
279
.011
280
.010
281
.010
282
.010
283
.009
284
.008
285
.007
286
.006
287
.006
288
.005
289
.004
290
.003
291
.003
292
.003
293
.002
294
.002
295
.001
296
.000
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.51.

Note that scores are relative to this ceiling.

Data: Geirhos2021powerequalisation

Metric: error_consistency