Sample stimuli

sample 0 sample 1 sample 2 sample 3 sample 4 sample 5 sample 6 sample 7 sample 8 sample 9

How to use

from brainscore_vision import load_benchmark
benchmark = load_benchmark("Baker2022frankenstein-accuracy_delta")
score = benchmark(my_model)

Model scores

Min Alignment Max Alignment

Rank

Model

Score

1
.983
2
.974
3
.972
4
.939
5
.929
6
.929
7
.929
8
.899
9
.897
10
.889
11
.887
12
.874
13
.873
14
.867
15
.867
16
.865
17
.862
18
.858
19
.855
20
.854
21
.850
22
.844
23
.824
24
.806
25
.803
26
.801
27
.798
28
.793
29
.775
30
.770
31
.767
32
.767
33
.766
34
.766
35
.761
36
.754
37
.741
38
.731
39
.731
40
.727
41
.724
42
.717
43
.715
44
.714
45
.713
46
.712
47
.698
48
.696
49
.692
50
.681
51
.680
52
.671
53
.666
54
.662
55
.658
56
.645
57
.644
58
.641
59
.623
60
.617
61
.608
62
.607
63
.599
64
.578
65
.572
66
.568
67
.568
68
.567
69
.558
70
.545
71
.539
72
.536
73
.527
74
.523
75
.503
76
.503
77
.494
78
.487
79
.485
80
.485
81
.465
82
.464
83
.463
84
.463
85
.462
86
.457
87
.448
88
.448
89
.448
90
.435
91
.427
92
.420
93
.418
94
.410
95
.398
96
.395
97
.378
98
.374
99
.372
100
.365
101
.365
102
.362
103
.362
104
.343
105
.339
106
.335
107
.335
108
.335
109
.334
110
.314
111
.313
112
.310
113
.308
114
.301
115
.286
116
.284
117
.282
118
.278
119
.276
120
.270
121
.257
122
.232
123
.232
124
.220
125
.212
126
.204
127
.201
128
.160
129
.156
130
.155
131
.142
132
.131
133
.119
134
.111
135
.089
136
.042
137
.038
138
.007
139
.006
140
.003
141
.002
142
.000
143
.000
144
.000
145
.000
146
.000
147
.000
148
.000
149
.000
150
.000
151
.000
152
.000
153
.000
154
.000
155
.000
156
.000
157
.000
158
.000
159
.000
160
.000
161
.000
162
.000
163
.000
164
.000
165
.000
166
.000
167
.000
168
X
169
X
170
X
171
X
172
X
173
X
174
X
175
X
176
X
177
X
178
X
179
X
180
X
181
X
182
X
183
X
184
X
185
X
186
X
187
X
188
X
189
X
190
X
191
X
192
X
193
X
194
X
195
X
196
X
197
X
198
X
199
X
200
X
201
X
202
X
203
X
204
X
205
X
206
X
207
X
208
X
209
X
210
X
211
X
212
X
213
X
214
X
215
X
216
X
217
X
218
X
219
X
220
X
221
X
222
X

Benchmark bibtex

@article{BAKER2022104913,
                title = {Deep learning models fail to capture the configural nature of human shape perception},
                journal = {iScience},
                volume = {25},
                number = {9},
                pages = {104913},
                year = {2022},
                issn = {2589-0042},
                doi = {https://doi.org/10.1016/j.isci.2022.104913},
                url = {https://www.sciencedirect.com/science/article/pii/S2589004222011853},
                author = {Nicholas Baker and James H. Elder},
                keywords = {Biological sciences, Neuroscience, Sensory neuroscience},
                abstract = {Summary
                A hallmark of human object perception is sensitivity to the holistic configuration of the local shape features of an object. Deep convolutional neural networks (DCNNs) are currently the dominant models for object recognition processing in the visual cortex, but do they capture this configural sensitivity? To answer this question, we employed a dataset of animal silhouettes and created a variant of this dataset that disrupts the configuration of each object while preserving local features. While human performance was impacted by this manipulation, DCNN performance was not, indicating insensitivity to object configuration. Modifications to training and architecture to make networks more brain-like did not lead to configural processing, and none of the networks were able to accurately predict trial-by-trial human object judgements. We speculate that to match human configural sensitivity, networks must be trained to solve a broader range of object tasks beyond category recognition.}
        }

Ceiling

Not available

Data: Baker2022frankenstein

Metric: accuracy_delta