Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Geirhos2021cueconflict-error_consistency")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.943
2
.938
3
.855
4
.840
5
.804
6
.784
7
.780
8
.752
9
.748
10
.740
11
.698
12
.686
13
.656
14
.651
15
.633
16
.611
17
.596
18
.595
19
.571
20
.560
21
.560
22
.546
23
.540
24
.482
25
.477
26
.451
27
.448
28
.441
29
.405
30
.402
31
.394
32
.393
33
.390
34
.383
35
.380
36
.376
37
.372
38
.371
39
.371
40
.357
41
.355
42
.346
43
.345
44
.344
45
.343
46
.332
47
.328
48
.327
49
.325
50
.321
51
.317
52
.316
53
.316
54
.316
55
.313
56
.311
57
.309
58
.309
59
.309
60
.300
61
.299
62
.297
63
.294
64
.292
65
.292
66
.284
67
.284
68
.282
69
.278
70
.272
71
.271
72
.271
73
.266
74
.260
75
.255
76
.254
77
.254
78
.253
79
.250
80
.244
81
.242
82
.240
83
.238
84
.236
85
.236
86
.235
87
.235
88
.233
89
.233
90
.233
91
.233
92
.232
93
.232
94
.229
95
.228
96
.228
97
.228
98
.226
99
.226
100
.226
101
.226
102
.220
103
.219
104
.218
105
.213
106
.213
107
.213
108
.212
109
.211
110
.210
111
.210
112
.210
113
.208
114
.206
115
.204
116
.204
117
.202
118
.196
119
.191
120
.191
121
.190
122
.189
123
.189
124
.189
125
.189
126
.188
127
.187
128
.186
129
.184
130
.182
131
.181
132
.181
133
.181
134
.181
135
.180
136
.179
137
.179
138
.177
139
.177
140
.177
141
.175
142
.175
143
.173
144
.173
145
.173
146
.171
147
.167
148
.166
149
.166
150
.165
151
.164
152
.164
153
.163
154
.163
155
.162
156
.162
157
.160
158
.160
159
.159
160
.157
161
.157
162
.157
163
.157
164
.156
165
.156
166
.155
167
.155
168
.154
169
.153
170
.153
171
.153
172
.153
173
.153
174
.153
175
.153
176
.153
177
.152
178
.152
179
.149
180
.147
181
.146
182
.146
183
.146
184
.145
185
.144
186
.144
187
.144
188
.144
189
.144
190
.142
191
.142
192
.141
193
.140
194
.137
195
.137
196
.136
197
.136
198
.135
199
.135
200
.135
201
.134
202
.134
203
.133
204
.133
205
.133
206
.132
207
.132
208
.132
209
.132
210
.132
211
.132
212
.131
213
.131
214
.131
215
.130
216
.128
217
.128
218
.128
219
.128
220
.128
221
.128
222
.128
223
.128
224
.128
225
.128
226
.128
227
.128
228
.127
229
.127
230
.127
231
.125
232
.125
233
.125
234
.123
235
.122
236
.121
237
.121
238
.121
239
.119
240
.117
241
.114
242
.112
243
.112
244
.110
245
.110
246
.109
247
.107
248
.106
249
.102
250
.102
251
.102
252
.101
253
.101
254
.098
255
.098
256
.097
257
.096
258
.096
259
.096
260
.093
261
.091
262
.090
263
.090
264
.087
265
.083
266
.079
267
.078
268
.068
269
.068
270
.068
271
.068
272
.067
273
.066
274
.066
275
.063
276
.059
277
.055
278
.053
279
.052
280
.050
281
.049
282
.046
283
.045
284
.034
285
.032
286
.032
287
.031
288
.028
289
.013
290
.011
291
.011
292
.011
293
.003
294
.003
295
.003
296
X
297
X
298
X
299
X
300
X
301
X
302
X
303
X
304
X
305
X
306
X
307
X
308
X
309
X
310
X
311
X
312
X
313
X
314
X
315
X
316
X
317
X
318
X
319
X
320
X
321
X
322
X
323
X
324
X
325
X
326
X
327
X
328
X
329
X
330
X
331
X
332
X
333
X
334
X
335
X
336
X
337
X
338
X
339
X
340
X
341
X
342
X
343
X
344
X
345
X
346
X
347
X
348
X
349
X
350
X
351
X
352
X
353
X
354
X
355
X
356
X
357
X
358
X
359
X
360
X
361
X
362
X
363
X
364
X
365
X
366
X
367
X
368
X
369
X
370
X
371
X
372
X
373
X
374
X
375
X
376
X
377
X
378
X
379
X
380
X
381
X
382
X
383
X
384
X
385
X
386
X
387
X
388
X
389
X
390
X
391
X
392
X
393
X
394
X
395
X
396
X
397
X
398
X
399
X
400
X
401
X
402
X
403
X
404
X
405
X
406
X
407
X

Benchmark bibtex

@article{geirhos2021partial,
              title={Partial success in closing the gap between human and machine vision},
              author={Geirhos, Robert and Narayanappa, Kantharaju and Mitzkus, Benjamin and Thieringer, Tizian and Bethge, Matthias and Wichmann, Felix A and Brendel, Wieland},
              journal={Advances in Neural Information Processing Systems},
              volume={34},
              year={2021},
              url={https://openreview.net/forum?id=QkljT4mrfs}
        }

Ceiling

0.33.

Note that scores are relative to this ceiling.

Data: Geirhos2021cueconflict

Metric: error_consistency