Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021phasescrambling-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.734
2
.730
3
.677
4
.624
5
.621
6
.612
7
.606
8
.600
9
.599
10
.596
11
.584
12
.576
13
.576
14
.572
15
.567
16
.562
17
.561
18
.521
19
.512
20
.511
21
.511
22
.506
23
.503
24
.499
25
.478
26
.462
27
.459
28
.448
29
.442
30
.431
31
.419
32
.416
33
.404
34
.404
35
.383
36
.382
37
.378
38
.376
39
.372
40
.367
41
.364
42
.363
43
.363
44
.361
45
.350
46
.347
47
.344
48
.342
49
.339
50
.338
51
.331
52
.320
53
.312
54
.308
55
.307
56
.289
57
.289
58
.289
59
.286
60
.273
61
.269
62
.268
63
.267
64
.256
65
.251
66
.250
67
.250
68
.250
69
.249
70
.247
71
.246
72
.223
73
.222
74
.222
75
.219
76
.214
77
.210
78
.209
79
.207
80
.202
81
.195
82
.192
83
.189
84
.183
85
.182
86
.181
87
.179
88
.177
89
.170
90
.168
91
.165
92
.162
93
.161
94
.161
95
.159
96
.158
97
.156
98
.154
99
.153
100
.153
101
.151
102
.150
103
.148
104
.147
105
.146
106
.145
107
.145
108
.144
109
.142
110
.138
111
.137
112
.134
113
.133
114
.130
115
.129
116
.125
117
.123
118
.123
119
.120
120
.120
121
.119
122
.118
123
.118
124
.118
125
.118
126
.116
127
.114
128
.114
129
.114
130
.114
131
.114
132
.114
133
.112
134
.112
135
.111
136
.111
137
.110
138
.110
139
.109
140
.108
141
.108
142
.107
143
.107
144
.104
145
.101
146
.101
147
.100
148
.100
149
.097
150
.094
151
.094
152
.094
153
.094
154
.093
155
.091
156
.091
157
.091
158
.090
159
.090
160
.089
161
.089
162
.087
163
.084
164
.084
165
.083
166
.083
167
.081
168
.081
169
.080
170
.080
171
.080
172
.080
173
.079
174
.079
175
.079
176
.077
177
.076
178
.075
179
.074
180
.073
181
.069
182
.069
183
.069
184
.068
185
.067
186
.067
187
.066
188
.066
189
.066
190
.063
191
.060
192
.060
193
.060
194
.060
195
.060
196
.060
197
.060
198
.060
199
.060
200
.060
201
.060
202
.060
203
.060
204
.060
205
.060
206
.060
207
.060
208
.059
209
.059
210
.057
211
.057
212
.055
213
.055
214
.054
215
.052
216
.052
217
.051
218
.051
219
.051
220
.050
221
.049
222
.048
223
.048
224
.048
225
.047
226
.047
227
.045
228
.045
229
.044
230
.044
231
.042
232
.041
233
.040
234
.040
235
.037
236
.034
237
.033
238
.033
239
.033
240
.032
241
.032
242
.031
243
.031
244
.030
245
.030
246
.029
247
.029
248
.029
249
.029
250
.029
251
.028
252
.028
253
.027
254
.027
255
.025
256
.024
257
.023
258
.023
259
.022
260
.022
261
.020
262
.019
263
.019
264
.019
265
.018
266
.017
267
.017
268
.016
269
.016
270
.015
271
.014
272
.013
273
.013
274
.012
275
.012
276
.011
277
.010
278
.009
279
.008
280
.007
281
.007
282
.006
283
.006
284
.006
285
.005
286
.005
287
.005
288
.003
289
.003
290
.002
291
.001
292
.001
293
.001
294
.001
295
.000
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.45.

Note that scores are relative to this ceiling.

Data: Geirhos2021phasescrambling

Metric: error_consistency