Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021stylized-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.871
2
.859
3
.851
4
.830
5
.787
6
.758
7
.751
8
.750
9
.746
10
.737
11
.734
12
.723
13
.708
14
.706
15
.705
16
.705
17
.704
18
.702
19
.697
20
.697
21
.693
22
.680
23
.680
24
.678
25
.678
26
.666
27
.654
28
.653
29
.646
30
.646
31
.646
32
.646
33
.628
34
.627
35
.623
36
.603
37
.600
38
.590
39
.588
40
.583
41
.579
42
.570
43
.555
44
.538
45
.516
46
.515
47
.511
48
.494
49
.488
50
.467
51
.463
52
.457
53
.456
54
.450
55
.449
56
.443
57
.443
58
.439
59
.438
60
.438
61
.433
62
.429
63
.427
64
.418
65
.417
66
.417
67
.417
68
.411
69
.409
70
.405
71
.405
72
.395
73
.392
74
.391
75
.389
76
.381
77
.370
78
.369
79
.365
80
.362
81
.358
82
.358
83
.351
84
.346
85
.342
86
.340
87
.324
88
.322
89
.321
90
.320
91
.319
92
.319
93
.317
94
.315
95
.315
96
.315
97
.315
98
.315
99
.314
100
.314
101
.313
102
.311
103
.307
104
.305
105
.303
106
.302
107
.301
108
.300
109
.299
110
.299
111
.298
112
.297
113
.295
114
.295
115
.293
116
.292
117
.290
118
.289
119
.288
120
.281
121
.281
122
.278
123
.274
124
.274
125
.273
126
.272
127
.271
128
.270
129
.267
130
.266
131
.263
132
.263
133
.262
134
.262
135
.258
136
.255
137
.254
138
.254
139
.254
140
.254
141
.252
142
.251
143
.251
144
.236
145
.233
146
.232
147
.230
148
.229
149
.229
150
.224
151
.223
152
.219
153
.218
154
.216
155
.213
156
.208
157
.202
158
.202
159
.201
160
.199
161
.198
162
.198
163
.198
164
.198
165
.197
166
.195
167
.195
168
.194
169
.192
170
.192
171
.191
172
.187
173
.185
174
.185
175
.184
176
.182
177
.181
178
.181
179
.176
180
.166
181
.166
182
.163
183
.160
184
.159
185
.157
186
.155
187
.154
188
.146
189
.145
190
.141
191
.139
192
.139
193
.139
194
.139
195
.139
196
.139
197
.139
198
.139
199
.139
200
.139
201
.139
202
.134
203
.133
204
.132
205
.131
206
.131
207
.129
208
.127
209
.127
210
.123
211
.121
212
.121
213
.120
214
.119
215
.119
216
.116
217
.114
218
.113
219
.111
220
.110
221
.110
222
.107
223
.107
224
.106
225
.106
226
.103
227
.103
228
.102
229
.100
230
.098
231
.097
232
.097
233
.097
234
.097
235
.096
236
.094
237
.094
238
.093
239
.093
240
.092
241
.090
242
.089
243
.088
244
.088
245
.087
246
.087
247
.086
248
.084
249
.084
250
.084
251
.080
252
.079
253
.078
254
.078
255
.074
256
.070
257
.066
258
.065
259
.064
260
.064
261
.064
262
.062
263
.061
264
.060
265
.057
266
.057
267
.049
268
.047
269
.046
270
.045
271
.045
272
.044
273
.044
274
.044
275
.040
276
.040
277
.040
278
.038
279
.037
280
.028
281
.026
282
.026
283
.025
284
.016
285
.015
286
.015
287
.015
288
.014
289
.012
290
.012
291
.008
292
.006
293
X
294
X
295
X
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.50.

Note that scores are relative to this ceiling.

Data: Geirhos2021stylized

Metric: error_consistency