google-flan-t5-small
Find this model in the FlanT5 model summary
google-flan-t5-small Model Summary Plots
google-flan-t5-small Model Selected Details
layer_type
N
M
Q
alpha
D
alpha-hat
log_SN
% Rand
num_traps
num_fingers
rank_loss
layer_id
1
EMBEDDING
32128
512
62.75
2.28
0.07
18.77
8.22
74.06
0
4
0
8
DENSE
512
384
1.33
2.06
0.14
2.29
1.11
90.43
0
1
0
9
DENSE
512
384
1.33
1.97
0.13
5.74
2.92
91.16
0
1
0
10
DENSE
512
384
1.33
3.96
0.14
10.38
2.62
93.86
0
5
0
11
DENSE
512
384
1.33
3.15
0.03
10.98
3.48
84.97
0
0
0
17
DENSE
1024
512
2.00
3.19
0.05
10.65
3.34
88.06
0
0
0
18
DENSE
1024
512
2.00
3.58
0.08
11.66
3.26
91.92
0
0
0
19
DENSE
1024
512
2.00
3.08
0.03
11.14
3.61
82.80
0
0
0
28
DENSE
512
384
1.33
5.39
0.05
6.61
1.23
92.23
0
0
0
29
DENSE
512
384
1.33
5.20
0.05
15.46
2.97
92.62
0
0
0
30
DENSE
512
384
1.33
3.66
0.11
9.46
2.59
94.97
0
0
0
31
DENSE
512
384
1.33
3.30
0.04
12.29
3.73
81.74
0
0
0
36
DENSE
1024
512
2.00
3.53
0.03
11.74
3.33
89.18
0
0
0
37
DENSE
1024
512
2.00
4.92
0.08
16.02
3.26
94.95
0
0
0
38
DENSE
1024
512
2.00
3.19
0.02
11.26
3.53
84.84
0
0
0
47
DENSE
512
384
1.33
3.61
0.06
3.57
0.99
89.70
0
0
0
48
DENSE
512
384
1.33
5.01
0.05
14.44
2.88
91.78
0
0
0
49
DENSE
512
384
1.33
3.31
0.11
9.76
2.94
89.23
0
0
0
50
DENSE
512
384
1.33
4.03
0.04
14.45
3.58
83.18
0
0
0
55
DENSE
1024
512
2.00
3.34
0.04
11.13
3.33
87.25
0
0
0
56
DENSE
1024
512
2.00
6.37
0.06
22.15
3.48
94.34
0
0
0
57
DENSE
1024
512
2.00
3.53
0.04
13.16
3.72
86.28
0
0
0
66
DENSE
512
384
1.33
4.75
0.06
4.05
0.85
92.65
0
0
0
67
DENSE
512
384
1.33
5.44
0.06
14.78
2.72
93.45
0
0
0
68
DENSE
512
384
1.33
4.85
0.14
14.12
2.91
94.55
0
7
0
69
DENSE
512
384
1.33
4.09
0.05
13.71
3.35
91.18
0
0
0
74
DENSE
1024
512
2.00
3.41
0.04
11.20
3.29
86.15
0
0
0
75
DENSE
1024
512
2.00
5.67
0.10
20.22
3.57
93.66
0
1
0
76
DENSE
1024
512
2.00
3.69
0.04
13.47
3.65
88.10
0
0
0
85
DENSE
512
384
1.33
3.74
0.05
3.16
0.85
88.58
0
0
0
86
DENSE
512
384
1.33
3.44
0.10
9.19
2.67
88.25
0
1
0
87
DENSE
512
384
1.33
2.81
0.14
8.74
3.11
90.26
0
8
0
88
DENSE
512
384
1.33
4.20
0.05
16.10
3.83
85.70
0
0
0
93
DENSE
1024
512
2.00
3.48
0.06
11.22
3.23
85.54
0
0
0
94
DENSE
1024
512
2.00
5.16
0.07
19.03
3.69
92.96
0
0
0
95
DENSE
1024
512
2.00
3.88
0.04
14.62
3.76
87.85
0
0
0
104
DENSE
512
384
1.33
4.77
0.04
4.22
0.88
87.65
0
0
0
105
DENSE
512
384
1.33
2.16
0.14
5.67
2.62
90.76
0
8
0
106
DENSE
512
384
1.33
4.63
0.11
14.27
3.08
92.25
0
0
0
107
DENSE
512
384
1.33
4.00
0.06
15.14
3.79
86.05
0
0
0
112
DENSE
1024
512
2.00
3.24
0.10
10.25
3.16
85.37
0
6
0
113
DENSE
1024
512
2.00
5.38
0.07
20.53
3.82
91.77
0
0
0
114
DENSE
1024
512
2.00
3.91
0.04
14.64
3.74
88.24
0
0
0
123
DENSE
512
384
1.33
3.73
0.05
3.81
1.02
85.99
0
0
0
124
DENSE
512
384
1.33
3.50
0.07
9.39
2.68
88.38
0
0
0
125
DENSE
512
384
1.33
4.08
0.13
13.32
3.26
90.21
0
3
0
126
DENSE
512
384
1.33
3.09
0.10
12.08
3.91
84.91
0
1
0
131
DENSE
1024
512
2.00
2.87
0.08
8.87
3.08
85.35
0
1
0
132
DENSE
1024
512
2.00
4.54
0.05
17.83
3.93
91.15
0
0
0
133
DENSE
1024
512
2.00
3.98
0.03
14.52
3.65
89.08
0
0
0
142
DENSE
512
384
1.33
2.91
0.04
2.47
0.85
80.59
0
0
0
143
DENSE
512
384
1.33
2.57
0.07
6.36
2.48
83.88
0
0
0
144
DENSE
512
384
1.33
4.82
0.12
15.94
3.30
91.48
0
1
0
145
DENSE
512
384
1.33
4.25
0.06
17.23
4.06
83.27
0
0
0
150
DENSE
1024
512
2.00
2.80
0.09
8.71
3.12
85.16
0
2
0
151
DENSE
1024
512
2.00
4.39
0.05
18.08
4.12
91.25
1
0
0
152
DENSE
1024
512
2.00
3.46
0.04
13.14
3.80
86.15
0
0
0
165
DENSE
512
384
1.33
3.90
0.04
5.10
1.31
86.20
0
0
0
166
DENSE
512
384
1.33
4.57
0.08
13.71
3.00
88.79
0
0
0
167
DENSE
512
384
1.33
4.97
0.07
13.52
2.72
90.84
0
0
0
168
DENSE
512
384
1.33
3.20
0.06
10.59
3.31
83.44
0
0
0
174
DENSE
512
384
1.33
4.21
0.04
5.12
1.22
91.51
0
0
0
175
DENSE
512
384
1.33
3.36
0.09
10.10
3.01
84.17
0
0
0
176
DENSE
512
384
1.33
5.18
0.04
13.19
2.55
91.97
0
0
0
177
DENSE
512
384
1.33
3.02
0.04
10.58
3.51
79.19
0
0
0
182
DENSE
1024
512
2.00
2.82
0.03
10.69
3.79
82.85
0
0
0
183
DENSE
1024
512
2.00
3.56
0.04
11.43
3.21
91.56
0
0
0
184
DENSE
1024
512
2.00
3.28
0.03
12.37
3.77
82.60
0
0
0
193
DENSE
512
384
1.33
2.91
0.04
4.13
1.42
82.05
0
0
0
194
DENSE
512
384
1.33
3.11
0.04
9.10
2.93
83.17
0
0
0
195
DENSE
512
384
1.33
3.63
0.05
11.04
3.04
87.86
0
0
0
196
DENSE
512
384
1.33
2.84
0.02
11.19
3.94
74.90
0
0
0
201
DENSE
512
384
1.33
4.11
0.05
5.22
1.27
88.53
0
0
0
202
DENSE
512
384
1.33
3.62
0.04
10.71
2.95
89.60
0
0
0
203
DENSE
512
384
1.33
3.54
0.08
9.64
2.72
89.91
0
0
0
204
DENSE
512
384
1.33
2.95
0.03
11.15
3.79
75.44
0
0
0
209
DENSE
1024
512
2.00
3.90
0.03
13.05
3.35
88.94
0
0
0
210
DENSE
1024
512
2.00
4.48
0.07
15.22
3.40
92.78
0
0
0
211
DENSE
1024
512
2.00
3.76
0.05
15.29
4.06
83.09
0
0
0
220
DENSE
512
384
1.33
3.09
0.05
2.93
0.95
83.03
0
0
0
221
DENSE
512
384
1.33
3.43
0.05
9.39
2.74
85.40
0
0
0
222
DENSE
512
384
1.33
3.86
0.12
11.74
3.04
88.54
0
1
0
223
DENSE
512
384
1.33
2.85
0.04
11.88
4.17
73.12
0
0
0
228
DENSE
512
384
1.33
4.70
0.06
5.59
1.19
91.37
0
0
0
229
DENSE
512
384
1.33
3.98
0.05
11.44
2.88
90.52
0
0
0
230
DENSE
512
384
1.33
4.39
0.11
12.70
2.89
94.06
0
1
0
231
DENSE
512
384
1.33
3.11
0.05
12.43
4.00
77.90
0
0
0
236
DENSE
1024
512
2.00
3.63
0.05
11.90
3.28
87.26
0
0
0
237
DENSE
1024
512
2.00
4.67
0.04
17.81
3.81
91.92
0
0
0
238
DENSE
1024
512
2.00
3.82
0.05
17.33
4.53
78.68
1
0
0
247
DENSE
512
384
1.33
3.78
0.07
3.54
0.94
86.32
0
0
0
248
DENSE
512
384
1.33
3.36
0.05
9.07
2.70
86.97
0
0
0
249
DENSE
512
384
1.33
6.01
0.10
19.50
3.24
91.86
0
0
0
250
DENSE
512
384
1.33
3.15
0.09
13.93
4.43
74.34
0
0
0
255
DENSE
512
384
1.33
2.78
0.05
3.78
1.36
84.01
0
0
0
256
DENSE
512
384
1.33
2.73
0.05
8.96
3.28
79.52
0
0
0
257
DENSE
512
384
1.33
7.44
0.05
22.28
2.99
94.89
0
0
0
258
DENSE
512
384
1.33
3.34
0.06
13.60
4.07
75.64
0
0
0
263
DENSE
1024
512
2.00
2.92
0.05
9.31
3.19
85.06
0
0
0
264
DENSE
1024
512
2.00
4.09
0.09
15.56
3.81
91.44
0
1
0
265
DENSE
1024
512
2.00
3.65
0.02
16.55
4.54
80.08
0
0
0
274
DENSE
512
384
1.33
4.98
0.04
5.69
1.14
89.00
0
0
0
275
DENSE
512
384
1.33
3.65
0.14
9.70
2.66
92.18
0
8
0
276
DENSE
512
384
1.33
5.15
0.12
16.62
3.23
91.85
0
1
0
277
DENSE
512
384
1.33
3.05
0.04
14.09
4.63
73.89
0
0
0
282
DENSE
512
384
1.33
3.30
0.05
4.55
1.38
84.37
0
0
0
283
DENSE
512
384
1.33
3.06
0.04
10.19
3.33
83.29
0
0
0
284
DENSE
512
384
1.33
3.69
0.14
11.61
3.15
91.92
0
5
0
285
DENSE
512
384
1.33
3.93
0.06
15.80
4.02
82.21
0
0
0
290
DENSE
1024
512
2.00
3.31
0.05
10.04
3.03
86.29
0
0
0
291
DENSE
1024
512
2.00
3.58
0.11
13.75
3.84
92.23
0
4
0
292
DENSE
1024
512
2.00
3.60
0.04
16.21
4.50
80.80
0
0
0
301
DENSE
512
384
1.33
3.67
0.04
4.86
1.33
85.16
0
0
0
302
DENSE
512
384
1.33
5.73
0.07
14.51
2.53
91.01
0
0
0
303
DENSE
512
384
1.33
4.88
0.13
16.57
3.39
94.92
0
5
0
304
DENSE
512
384
1.33
1.91
0.08
8.57
4.48
78.29
0
1
0
309
DENSE
512
384
1.33
4.24
0.06
5.06
1.19
88.65
0
0
0
310
DENSE
512
384
1.33
4.75
0.06
14.04
2.96
88.77
0
0
0
311
DENSE
512
384
1.33
2.60
0.11
8.58
3.30
92.70
0
0
0
312
DENSE
512
384
1.33
3.30
0.08
14.74
4.46
77.28
0
0
0
317
DENSE
1024
512
2.00
3.14
0.05
9.01
2.87
86.04
0
0
0
318
DENSE
1024
512
2.00
4.55
0.09
18.00
3.96
92.32
0
1
0
319
DENSE
1024
512
2.00
3.94
0.04
17.16
4.36
83.85
0
0
0
328
DENSE
512
384
1.33
4.06
0.07
2.81
0.69
89.98
0
0
0
329
DENSE
512
384
1.33
4.10
0.07
10.14
2.47
89.10
0
0
0
330
DENSE
512
384
1.33
2.36
0.14
7.90
3.35
94.70
0
2
0
331
DENSE
512
384
1.33
4.09
0.10
20.10
4.91
75.87
0
0
0
336
DENSE
512
384
1.33
3.62
0.06
4.43
1.22
83.47
0
0
0
337
DENSE
512
384
1.33
2.04
0.11
6.21
3.05
83.93
0
4
0
338
DENSE
512
384
1.33
10.00
0.11
33.78
3.38
95.50
0
0
0
339
DENSE
512
384
1.33
3.83
0.11
17.76
4.63
81.64
0
0
0
344
DENSE
1024
512
2.00
3.38
0.05
9.37
2.78
85.78
0
0
0
345
DENSE
1024
512
2.00
4.20
0.07
17.81
4.24
90.86
0
0
0
346
DENSE
1024
512
2.00
3.92
0.05
17.31
4.42
85.35
0
0
0
355
DENSE
512
384
1.33
4.61
0.03
3.94
0.86
92.82
0
0
0
356
DENSE
512
384
1.33
4.69
0.05
13.03
2.78
92.94
0
0
0
357
DENSE
512
384
1.33
7.66
0.06
27.15
3.55
93.97
0
0
0
358
DENSE
512
384
1.33
3.26
0.07
14.32
4.39
77.52
0
0
0
363
DENSE
512
384
1.33
2.83
0.09
4.41
1.56
75.52
0
0
0
364
DENSE
512
384
1.33
1.94
0.09
6.44
3.32
77.55
0
1
0
365
DENSE
512
384
1.33
6.62
0.13
25.27
3.82
88.86
0
1
0
366
DENSE
512
384
1.33
3.68
0.06
17.22
4.68
85.99
0
0
0
371
DENSE
1024
512
2.00
3.24
0.04
10.59
3.27
81.62
0
0
0
372
DENSE
1024
512
2.00
3.10
0.08
15.34
4.95
85.96
0
0
0
373
DENSE
1024
512
2.00
2.86
0.05
15.16
5.30
71.02
5
0
0
google-flan-t5-small Layer Plots
Layer 1
Layer=1 | N=32128 | M=512 | Q=62.75 | alpha=2.28 | D_ks=0.07 | alpha-hat=18.77 | num traps=0
Layer 8
Layer=8 | N=512 | M=384 | Q=1.33 | alpha=2.06 | D_ks=0.14 | alpha-hat=2.29 | num traps=0
Layer 9
Layer=9 | N=512 | M=384 | Q=1.33 | alpha=1.97 | D_ks=0.13 | alpha-hat=5.74 | num traps=0
Layer 10
Layer=10 | N=512 | M=384 | Q=1.33 | alpha=3.96 | D_ks=0.14 | alpha-hat=10.38 | num traps=0
Layer 11
Layer=11 | N=512 | M=384 | Q=1.33 | alpha=3.15 | D_ks=0.03 | alpha-hat=10.98 | num traps=0
Layer 17
Layer=17 | N=1024 | M=512 | Q=2.00 | alpha=3.19 | D_ks=0.05 | alpha-hat=10.65 | num traps=0
Layer 18
Layer=18 | N=1024 | M=512 | Q=2.00 | alpha=3.58 | D_ks=0.08 | alpha-hat=11.66 | num traps=0
Layer 19
Layer=19 | N=1024 | M=512 | Q=2.00 | alpha=3.08 | D_ks=0.03 | alpha-hat=11.14 | num traps=0
Layer 28
Layer=28 | N=512 | M=384 | Q=1.33 | alpha=5.39 | D_ks=0.05 | alpha-hat=6.61 | num traps=0
Layer 29
Layer=29 | N=512 | M=384 | Q=1.33 | alpha=5.20 | D_ks=0.05 | alpha-hat=15.46 | num traps=0
Layer 30
Layer=30 | N=512 | M=384 | Q=1.33 | alpha=3.66 | D_ks=0.11 | alpha-hat=9.46 | num traps=0
Layer 31
Layer=31 | N=512 | M=384 | Q=1.33 | alpha=3.30 | D_ks=0.04 | alpha-hat=12.29 | num traps=0
Layer 36
Layer=36 | N=1024 | M=512 | Q=2.00 | alpha=3.53 | D_ks=0.03 | alpha-hat=11.74 | num traps=0
Layer 37
Layer=37 | N=1024 | M=512 | Q=2.00 | alpha=4.92 | D_ks=0.08 | alpha-hat=16.02 | num traps=0
Layer 38
Layer=38 | N=1024 | M=512 | Q=2.00 | alpha=3.19 | D_ks=0.02 | alpha-hat=11.26 | num traps=0
Layer 47
Layer=47 | N=512 | M=384 | Q=1.33 | alpha=3.61 | D_ks=0.06 | alpha-hat=3.57 | num traps=0
Layer 48
Layer=48 | N=512 | M=384 | Q=1.33 | alpha=5.01 | D_ks=0.05 | alpha-hat=14.44 | num traps=0
Layer 49
Layer=49 | N=512 | M=384 | Q=1.33 | alpha=3.31 | D_ks=0.11 | alpha-hat=9.76 | num traps=0
Layer 50
Layer=50 | N=512 | M=384 | Q=1.33 | alpha=4.03 | D_ks=0.04 | alpha-hat=14.45 | num traps=0
Layer 55
Layer=55 | N=1024 | M=512 | Q=2.00 | alpha=3.34 | D_ks=0.04 | alpha-hat=11.13 | num traps=0
Layer 56
Layer=56 | N=1024 | M=512 | Q=2.00 | alpha=6.37 | D_ks=0.06 | alpha-hat=22.15 | num traps=0
Layer 57
Layer=57 | N=1024 | M=512 | Q=2.00 | alpha=3.53 | D_ks=0.04 | alpha-hat=13.16 | num traps=0
Layer 66
Layer=66 | N=512 | M=384 | Q=1.33 | alpha=4.75 | D_ks=0.06 | alpha-hat=4.05 | num traps=0
Layer 67
Layer=67 | N=512 | M=384 | Q=1.33 | alpha=5.44 | D_ks=0.06 | alpha-hat=14.78 | num traps=0
Layer 68
Layer=68 | N=512 | M=384 | Q=1.33 | alpha=4.85 | D_ks=0.14 | alpha-hat=14.12 | num traps=0
Layer 69
Layer=69 | N=512 | M=384 | Q=1.33 | alpha=4.09 | D_ks=0.05 | alpha-hat=13.71 | num traps=0
Layer 74
Layer=74 | N=1024 | M=512 | Q=2.00 | alpha=3.41 | D_ks=0.04 | alpha-hat=11.20 | num traps=0
Layer 75
Layer=75 | N=1024 | M=512 | Q=2.00 | alpha=5.67 | D_ks=0.10 | alpha-hat=20.22 | num traps=0
Layer 76
Layer=76 | N=1024 | M=512 | Q=2.00 | alpha=3.69 | D_ks=0.04 | alpha-hat=13.47 | num traps=0
Layer 85
Layer=85 | N=512 | M=384 | Q=1.33 | alpha=3.74 | D_ks=0.05 | alpha-hat=3.16 | num traps=0
Layer 86
Layer=86 | N=512 | M=384 | Q=1.33 | alpha=3.44 | D_ks=0.10 | alpha-hat=9.19 | num traps=0
Layer 87
Layer=87 | N=512 | M=384 | Q=1.33 | alpha=2.81 | D_ks=0.14 | alpha-hat=8.74 | num traps=0
Layer 88
Layer=88 | N=512 | M=384 | Q=1.33 | alpha=4.20 | D_ks=0.05 | alpha-hat=16.10 | num traps=0
Layer 93
Layer=93 | N=1024 | M=512 | Q=2.00 | alpha=3.48 | D_ks=0.06 | alpha-hat=11.22 | num traps=0
Layer 94
Layer=94 | N=1024 | M=512 | Q=2.00 | alpha=5.16 | D_ks=0.07 | alpha-hat=19.03 | num traps=0
Layer 95
Layer=95 | N=1024 | M=512 | Q=2.00 | alpha=3.88 | D_ks=0.04 | alpha-hat=14.62 | num traps=0
Layer 104
Layer=104 | N=512 | M=384 | Q=1.33 | alpha=4.77 | D_ks=0.04 | alpha-hat=4.22 | num traps=0
Layer 105
Layer=105 | N=512 | M=384 | Q=1.33 | alpha=2.16 | D_ks=0.14 | alpha-hat=5.67 | num traps=0
Layer 106
Layer=106 | N=512 | M=384 | Q=1.33 | alpha=4.63 | D_ks=0.11 | alpha-hat=14.27 | num traps=0
Layer 107
Layer=107 | N=512 | M=384 | Q=1.33 | alpha=4.00 | D_ks=0.06 | alpha-hat=15.14 | num traps=0
Layer 112
Layer=112 | N=1024 | M=512 | Q=2.00 | alpha=3.24 | D_ks=0.10 | alpha-hat=10.25 | num traps=0
Layer 113
Layer=113 | N=1024 | M=512 | Q=2.00 | alpha=5.38 | D_ks=0.07 | alpha-hat=20.53 | num traps=0
Layer 114
Layer=114 | N=1024 | M=512 | Q=2.00 | alpha=3.91 | D_ks=0.04 | alpha-hat=14.64 | num traps=0
Layer 123
Layer=123 | N=512 | M=384 | Q=1.33 | alpha=3.73 | D_ks=0.05 | alpha-hat=3.81 | num traps=0
Layer 124
Layer=124 | N=512 | M=384 | Q=1.33 | alpha=3.50 | D_ks=0.07 | alpha-hat=9.39 | num traps=0
Layer 125
Layer=125 | N=512 | M=384 | Q=1.33 | alpha=4.08 | D_ks=0.13 | alpha-hat=13.32 | num traps=0
Layer 126
Layer=126 | N=512 | M=384 | Q=1.33 | alpha=3.09 | D_ks=0.10 | alpha-hat=12.08 | num traps=0
Layer 131
Layer=131 | N=1024 | M=512 | Q=2.00 | alpha=2.87 | D_ks=0.08 | alpha-hat=8.87 | num traps=0
Layer 132
Layer=132 | N=1024 | M=512 | Q=2.00 | alpha=4.54 | D_ks=0.05 | alpha-hat=17.83 | num traps=0
Layer 133
Layer=133 | N=1024 | M=512 | Q=2.00 | alpha=3.98 | D_ks=0.03 | alpha-hat=14.52 | num traps=0
Layer 142
Layer=142 | N=512 | M=384 | Q=1.33 | alpha=2.91 | D_ks=0.04 | alpha-hat=2.47 | num traps=0
Layer 143
Layer=143 | N=512 | M=384 | Q=1.33 | alpha=2.57 | D_ks=0.07 | alpha-hat=6.36 | num traps=0
Layer 144
Layer=144 | N=512 | M=384 | Q=1.33 | alpha=4.82 | D_ks=0.12 | alpha-hat=15.94 | num traps=0
Layer 145
Layer=145 | N=512 | M=384 | Q=1.33 | alpha=4.25 | D_ks=0.06 | alpha-hat=17.23 | num traps=0
Layer 150
Layer=150 | N=1024 | M=512 | Q=2.00 | alpha=2.80 | D_ks=0.09 | alpha-hat=8.71 | num traps=0
Layer 151
Layer=151 | N=1024 | M=512 | Q=2.00 | alpha=4.39 | D_ks=0.05 | alpha-hat=18.08 | num traps=1
Layer 152
Layer=152 | N=1024 | M=512 | Q=2.00 | alpha=3.46 | D_ks=0.04 | alpha-hat=13.14 | num traps=0
Layer 165
Layer=165 | N=512 | M=384 | Q=1.33 | alpha=3.90 | D_ks=0.04 | alpha-hat=5.10 | num traps=0
Layer 166
Layer=166 | N=512 | M=384 | Q=1.33 | alpha=4.57 | D_ks=0.08 | alpha-hat=13.71 | num traps=0
Layer 167
Layer=167 | N=512 | M=384 | Q=1.33 | alpha=4.97 | D_ks=0.07 | alpha-hat=13.52 | num traps=0
Layer 168
Layer=168 | N=512 | M=384 | Q=1.33 | alpha=3.20 | D_ks=0.06 | alpha-hat=10.59 | num traps=0
Layer 174
Layer=174 | N=512 | M=384 | Q=1.33 | alpha=4.21 | D_ks=0.04 | alpha-hat=5.12 | num traps=0
Layer 175
Layer=175 | N=512 | M=384 | Q=1.33 | alpha=3.36 | D_ks=0.09 | alpha-hat=10.10 | num traps=0
Layer 176
Layer=176 | N=512 | M=384 | Q=1.33 | alpha=5.18 | D_ks=0.04 | alpha-hat=13.19 | num traps=0
Layer 177
Layer=177 | N=512 | M=384 | Q=1.33 | alpha=3.02 | D_ks=0.04 | alpha-hat=10.58 | num traps=0
Layer 182
Layer=182 | N=1024 | M=512 | Q=2.00 | alpha=2.82 | D_ks=0.03 | alpha-hat=10.69 | num traps=0
Layer 183
Layer=183 | N=1024 | M=512 | Q=2.00 | alpha=3.56 | D_ks=0.04 | alpha-hat=11.43 | num traps=0
Layer 184
Layer=184 | N=1024 | M=512 | Q=2.00 | alpha=3.28 | D_ks=0.03 | alpha-hat=12.37 | num traps=0
Layer 193
Layer=193 | N=512 | M=384 | Q=1.33 | alpha=2.91 | D_ks=0.04 | alpha-hat=4.13 | num traps=0
Layer 194
Layer=194 | N=512 | M=384 | Q=1.33 | alpha=3.11 | D_ks=0.04 | alpha-hat=9.10 | num traps=0
Layer 195
Layer=195 | N=512 | M=384 | Q=1.33 | alpha=3.63 | D_ks=0.05 | alpha-hat=11.04 | num traps=0
Layer 196
Layer=196 | N=512 | M=384 | Q=1.33 | alpha=2.84 | D_ks=0.02 | alpha-hat=11.19 | num traps=0
Layer 201
Layer=201 | N=512 | M=384 | Q=1.33 | alpha=4.11 | D_ks=0.05 | alpha-hat=5.22 | num traps=0
Layer 202
Layer=202 | N=512 | M=384 | Q=1.33 | alpha=3.62 | D_ks=0.04 | alpha-hat=10.71 | num traps=0
Layer 203
Layer=203 | N=512 | M=384 | Q=1.33 | alpha=3.54 | D_ks=0.08 | alpha-hat=9.64 | num traps=0
Layer 204
Layer=204 | N=512 | M=384 | Q=1.33 | alpha=2.95 | D_ks=0.03 | alpha-hat=11.15 | num traps=0
Layer 209
Layer=209 | N=1024 | M=512 | Q=2.00 | alpha=3.90 | D_ks=0.03 | alpha-hat=13.05 | num traps=0
Layer 210
Layer=210 | N=1024 | M=512 | Q=2.00 | alpha=4.48 | D_ks=0.07 | alpha-hat=15.22 | num traps=0
Layer 211
Layer=211 | N=1024 | M=512 | Q=2.00 | alpha=3.76 | D_ks=0.05 | alpha-hat=15.29 | num traps=0
Layer 220
Layer=220 | N=512 | M=384 | Q=1.33 | alpha=3.09 | D_ks=0.05 | alpha-hat=2.93 | num traps=0
Layer 221
Layer=221 | N=512 | M=384 | Q=1.33 | alpha=3.43 | D_ks=0.05 | alpha-hat=9.39 | num traps=0
Layer 222
Layer=222 | N=512 | M=384 | Q=1.33 | alpha=3.86 | D_ks=0.12 | alpha-hat=11.74 | num traps=0
Layer 223
Layer=223 | N=512 | M=384 | Q=1.33 | alpha=2.85 | D_ks=0.04 | alpha-hat=11.88 | num traps=0
Layer 228
Layer=228 | N=512 | M=384 | Q=1.33 | alpha=4.70 | D_ks=0.06 | alpha-hat=5.59 | num traps=0
Layer 229
Layer=229 | N=512 | M=384 | Q=1.33 | alpha=3.98 | D_ks=0.05 | alpha-hat=11.44 | num traps=0
Layer 230
Layer=230 | N=512 | M=384 | Q=1.33 | alpha=4.39 | D_ks=0.11 | alpha-hat=12.70 | num traps=0
Layer 231
Layer=231 | N=512 | M=384 | Q=1.33 | alpha=3.11 | D_ks=0.05 | alpha-hat=12.43 | num traps=0
Layer 236
Layer=236 | N=1024 | M=512 | Q=2.00 | alpha=3.63 | D_ks=0.05 | alpha-hat=11.90 | num traps=0
Layer 237
Layer=237 | N=1024 | M=512 | Q=2.00 | alpha=4.67 | D_ks=0.04 | alpha-hat=17.81 | num traps=0
Layer 238
Layer=238 | N=1024 | M=512 | Q=2.00 | alpha=3.82 | D_ks=0.05 | alpha-hat=17.33 | num traps=1
Layer 247
Layer=247 | N=512 | M=384 | Q=1.33 | alpha=3.78 | D_ks=0.07 | alpha-hat=3.54 | num traps=0
Layer 248
Layer=248 | N=512 | M=384 | Q=1.33 | alpha=3.36 | D_ks=0.05 | alpha-hat=9.07 | num traps=0
Layer 249
Layer=249 | N=512 | M=384 | Q=1.33 | alpha=6.01 | D_ks=0.10 | alpha-hat=19.50 | num traps=0
Layer 250
Layer=250 | N=512 | M=384 | Q=1.33 | alpha=3.15 | D_ks=0.09 | alpha-hat=13.93 | num traps=0
Layer 255
Layer=255 | N=512 | M=384 | Q=1.33 | alpha=2.78 | D_ks=0.05 | alpha-hat=3.78 | num traps=0
Layer 256
Layer=256 | N=512 | M=384 | Q=1.33 | alpha=2.73 | D_ks=0.05 | alpha-hat=8.96 | num traps=0
Layer 257
Layer=257 | N=512 | M=384 | Q=1.33 | alpha=7.44 | D_ks=0.05 | alpha-hat=22.28 | num traps=0
Layer 258
Layer=258 | N=512 | M=384 | Q=1.33 | alpha=3.34 | D_ks=0.06 | alpha-hat=13.60 | num traps=0
Layer 263
Layer=263 | N=1024 | M=512 | Q=2.00 | alpha=2.92 | D_ks=0.05 | alpha-hat=9.31 | num traps=0
Layer 264
Layer=264 | N=1024 | M=512 | Q=2.00 | alpha=4.09 | D_ks=0.09 | alpha-hat=15.56 | num traps=0
Layer 265
Layer=265 | N=1024 | M=512 | Q=2.00 | alpha=3.65 | D_ks=0.02 | alpha-hat=16.55 | num traps=0
Layer 274
Layer=274 | N=512 | M=384 | Q=1.33 | alpha=4.98 | D_ks=0.04 | alpha-hat=5.69 | num traps=0
Layer 275
Layer=275 | N=512 | M=384 | Q=1.33 | alpha=3.65 | D_ks=0.14 | alpha-hat=9.70 | num traps=0
Layer 276
Layer=276 | N=512 | M=384 | Q=1.33 | alpha=5.15 | D_ks=0.12 | alpha-hat=16.62 | num traps=0
Layer 277
Layer=277 | N=512 | M=384 | Q=1.33 | alpha=3.05 | D_ks=0.04 | alpha-hat=14.09 | num traps=0
Layer 282
Layer=282 | N=512 | M=384 | Q=1.33 | alpha=3.30 | D_ks=0.05 | alpha-hat=4.55 | num traps=0
Layer 283
Layer=283 | N=512 | M=384 | Q=1.33 | alpha=3.06 | D_ks=0.04 | alpha-hat=10.19 | num traps=0
Layer 284
Layer=284 | N=512 | M=384 | Q=1.33 | alpha=3.69 | D_ks=0.14 | alpha-hat=11.61 | num traps=0
Layer 285
Layer=285 | N=512 | M=384 | Q=1.33 | alpha=3.93 | D_ks=0.06 | alpha-hat=15.80 | num traps=0
Layer 290
Layer=290 | N=1024 | M=512 | Q=2.00 | alpha=3.31 | D_ks=0.05 | alpha-hat=10.04 | num traps=0
Layer 291
Layer=291 | N=1024 | M=512 | Q=2.00 | alpha=3.58 | D_ks=0.11 | alpha-hat=13.75 | num traps=0
Layer 292
Layer=292 | N=1024 | M=512 | Q=2.00 | alpha=3.60 | D_ks=0.04 | alpha-hat=16.21 | num traps=0
Layer 301
Layer=301 | N=512 | M=384 | Q=1.33 | alpha=3.67 | D_ks=0.04 | alpha-hat=4.86 | num traps=0
Layer 302
Layer=302 | N=512 | M=384 | Q=1.33 | alpha=5.73 | D_ks=0.07 | alpha-hat=14.51 | num traps=0
Layer 303
Layer=303 | N=512 | M=384 | Q=1.33 | alpha=4.88 | D_ks=0.13 | alpha-hat=16.57 | num traps=0
Layer 304
Layer=304 | N=512 | M=384 | Q=1.33 | alpha=1.91 | D_ks=0.08 | alpha-hat=8.57 | num traps=0
Layer 309
Layer=309 | N=512 | M=384 | Q=1.33 | alpha=4.24 | D_ks=0.06 | alpha-hat=5.06 | num traps=0
Layer 310
Layer=310 | N=512 | M=384 | Q=1.33 | alpha=4.75 | D_ks=0.06 | alpha-hat=14.04 | num traps=0
Layer 311
Layer=311 | N=512 | M=384 | Q=1.33 | alpha=2.60 | D_ks=0.11 | alpha-hat=8.58 | num traps=0
Layer 312
Layer=312 | N=512 | M=384 | Q=1.33 | alpha=3.30 | D_ks=0.08 | alpha-hat=14.74 | num traps=0
Layer 317
Layer=317 | N=1024 | M=512 | Q=2.00 | alpha=3.14 | D_ks=0.05 | alpha-hat=9.01 | num traps=0
Layer 318
Layer=318 | N=1024 | M=512 | Q=2.00 | alpha=4.55 | D_ks=0.09 | alpha-hat=18.00 | num traps=0
Layer 319
Layer=319 | N=1024 | M=512 | Q=2.00 | alpha=3.94 | D_ks=0.04 | alpha-hat=17.16 | num traps=0
Layer 328
Layer=328 | N=512 | M=384 | Q=1.33 | alpha=4.06 | D_ks=0.07 | alpha-hat=2.81 | num traps=0
Layer 329
Layer=329 | N=512 | M=384 | Q=1.33 | alpha=4.10 | D_ks=0.07 | alpha-hat=10.14 | num traps=0
Layer 330
Layer=330 | N=512 | M=384 | Q=1.33 | alpha=2.36 | D_ks=0.14 | alpha-hat=7.90 | num traps=0
Layer 331
Layer=331 | N=512 | M=384 | Q=1.33 | alpha=4.09 | D_ks=0.10 | alpha-hat=20.10 | num traps=0
Layer 336
Layer=336 | N=512 | M=384 | Q=1.33 | alpha=3.62 | D_ks=0.06 | alpha-hat=4.43 | num traps=0
Layer 337
Layer=337 | N=512 | M=384 | Q=1.33 | alpha=2.04 | D_ks=0.11 | alpha-hat=6.21 | num traps=0
Layer 338
Layer=338 | N=512 | M=384 | Q=1.33 | alpha=10.00 | D_ks=0.11 | alpha-hat=33.78 | num traps=0
Layer 339
Layer=339 | N=512 | M=384 | Q=1.33 | alpha=3.83 | D_ks=0.11 | alpha-hat=17.76 | num traps=0
Layer 344
Layer=344 | N=1024 | M=512 | Q=2.00 | alpha=3.38 | D_ks=0.05 | alpha-hat=9.37 | num traps=0
Layer 345
Layer=345 | N=1024 | M=512 | Q=2.00 | alpha=4.20 | D_ks=0.07 | alpha-hat=17.81 | num traps=0
Layer 346
Layer=346 | N=1024 | M=512 | Q=2.00 | alpha=3.92 | D_ks=0.05 | alpha-hat=17.31 | num traps=0
Layer 355
Layer=355 | N=512 | M=384 | Q=1.33 | alpha=4.61 | D_ks=0.03 | alpha-hat=3.94 | num traps=0
Layer 356
Layer=356 | N=512 | M=384 | Q=1.33 | alpha=4.69 | D_ks=0.05 | alpha-hat=13.03 | num traps=0
Layer 357
Layer=357 | N=512 | M=384 | Q=1.33 | alpha=7.66 | D_ks=0.06 | alpha-hat=27.15 | num traps=0
Layer 358
Layer=358 | N=512 | M=384 | Q=1.33 | alpha=3.26 | D_ks=0.07 | alpha-hat=14.32 | num traps=0
Layer 363
Layer=363 | N=512 | M=384 | Q=1.33 | alpha=2.83 | D_ks=0.09 | alpha-hat=4.41 | num traps=0
Layer 364
Layer=364 | N=512 | M=384 | Q=1.33 | alpha=1.94 | D_ks=0.09 | alpha-hat=6.44 | num traps=0
Layer 365
Layer=365 | N=512 | M=384 | Q=1.33 | alpha=6.62 | D_ks=0.13 | alpha-hat=25.27 | num traps=0
Layer 366
Layer=366 | N=512 | M=384 | Q=1.33 | alpha=3.68 | D_ks=0.06 | alpha-hat=17.22 | num traps=0
Layer 371
Layer=371 | N=1024 | M=512 | Q=2.00 | alpha=3.24 | D_ks=0.04 | alpha-hat=10.59 | num traps=0
Layer 372
Layer=372 | N=1024 | M=512 | Q=2.00 | alpha=3.10 | D_ks=0.08 | alpha-hat=15.34 | num traps=0
Layer 373
Layer=373 | N=1024 | M=512 | Q=2.00 | alpha=2.86 | D_ks=0.05 | alpha-hat=15.16 | num traps=5
google-flan-t5-small Model Summary Plots
google-flan-t5-small Model Selected Details
| layer_type | N | M | Q | alpha | D | alpha-hat | log_SN | % Rand | num_traps | num_fingers | rank_loss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| layer_id | ||||||||||||
| 1 | EMBEDDING | 32128 | 512 | 62.75 | 2.28 | 0.07 | 18.77 | 8.22 | 74.06 | 0 | 4 | 0 |
| 8 | DENSE | 512 | 384 | 1.33 | 2.06 | 0.14 | 2.29 | 1.11 | 90.43 | 0 | 1 | 0 |
| 9 | DENSE | 512 | 384 | 1.33 | 1.97 | 0.13 | 5.74 | 2.92 | 91.16 | 0 | 1 | 0 |
| 10 | DENSE | 512 | 384 | 1.33 | 3.96 | 0.14 | 10.38 | 2.62 | 93.86 | 0 | 5 | 0 |
| 11 | DENSE | 512 | 384 | 1.33 | 3.15 | 0.03 | 10.98 | 3.48 | 84.97 | 0 | 0 | 0 |
| 17 | DENSE | 1024 | 512 | 2.00 | 3.19 | 0.05 | 10.65 | 3.34 | 88.06 | 0 | 0 | 0 |
| 18 | DENSE | 1024 | 512 | 2.00 | 3.58 | 0.08 | 11.66 | 3.26 | 91.92 | 0 | 0 | 0 |
| 19 | DENSE | 1024 | 512 | 2.00 | 3.08 | 0.03 | 11.14 | 3.61 | 82.80 | 0 | 0 | 0 |
| 28 | DENSE | 512 | 384 | 1.33 | 5.39 | 0.05 | 6.61 | 1.23 | 92.23 | 0 | 0 | 0 |
| 29 | DENSE | 512 | 384 | 1.33 | 5.20 | 0.05 | 15.46 | 2.97 | 92.62 | 0 | 0 | 0 |
| 30 | DENSE | 512 | 384 | 1.33 | 3.66 | 0.11 | 9.46 | 2.59 | 94.97 | 0 | 0 | 0 |
| 31 | DENSE | 512 | 384 | 1.33 | 3.30 | 0.04 | 12.29 | 3.73 | 81.74 | 0 | 0 | 0 |
| 36 | DENSE | 1024 | 512 | 2.00 | 3.53 | 0.03 | 11.74 | 3.33 | 89.18 | 0 | 0 | 0 |
| 37 | DENSE | 1024 | 512 | 2.00 | 4.92 | 0.08 | 16.02 | 3.26 | 94.95 | 0 | 0 | 0 |
| 38 | DENSE | 1024 | 512 | 2.00 | 3.19 | 0.02 | 11.26 | 3.53 | 84.84 | 0 | 0 | 0 |
| 47 | DENSE | 512 | 384 | 1.33 | 3.61 | 0.06 | 3.57 | 0.99 | 89.70 | 0 | 0 | 0 |
| 48 | DENSE | 512 | 384 | 1.33 | 5.01 | 0.05 | 14.44 | 2.88 | 91.78 | 0 | 0 | 0 |
| 49 | DENSE | 512 | 384 | 1.33 | 3.31 | 0.11 | 9.76 | 2.94 | 89.23 | 0 | 0 | 0 |
| 50 | DENSE | 512 | 384 | 1.33 | 4.03 | 0.04 | 14.45 | 3.58 | 83.18 | 0 | 0 | 0 |
| 55 | DENSE | 1024 | 512 | 2.00 | 3.34 | 0.04 | 11.13 | 3.33 | 87.25 | 0 | 0 | 0 |
| 56 | DENSE | 1024 | 512 | 2.00 | 6.37 | 0.06 | 22.15 | 3.48 | 94.34 | 0 | 0 | 0 |
| 57 | DENSE | 1024 | 512 | 2.00 | 3.53 | 0.04 | 13.16 | 3.72 | 86.28 | 0 | 0 | 0 |
| 66 | DENSE | 512 | 384 | 1.33 | 4.75 | 0.06 | 4.05 | 0.85 | 92.65 | 0 | 0 | 0 |
| 67 | DENSE | 512 | 384 | 1.33 | 5.44 | 0.06 | 14.78 | 2.72 | 93.45 | 0 | 0 | 0 |
| 68 | DENSE | 512 | 384 | 1.33 | 4.85 | 0.14 | 14.12 | 2.91 | 94.55 | 0 | 7 | 0 |
| 69 | DENSE | 512 | 384 | 1.33 | 4.09 | 0.05 | 13.71 | 3.35 | 91.18 | 0 | 0 | 0 |
| 74 | DENSE | 1024 | 512 | 2.00 | 3.41 | 0.04 | 11.20 | 3.29 | 86.15 | 0 | 0 | 0 |
| 75 | DENSE | 1024 | 512 | 2.00 | 5.67 | 0.10 | 20.22 | 3.57 | 93.66 | 0 | 1 | 0 |
| 76 | DENSE | 1024 | 512 | 2.00 | 3.69 | 0.04 | 13.47 | 3.65 | 88.10 | 0 | 0 | 0 |
| 85 | DENSE | 512 | 384 | 1.33 | 3.74 | 0.05 | 3.16 | 0.85 | 88.58 | 0 | 0 | 0 |
| 86 | DENSE | 512 | 384 | 1.33 | 3.44 | 0.10 | 9.19 | 2.67 | 88.25 | 0 | 1 | 0 |
| 87 | DENSE | 512 | 384 | 1.33 | 2.81 | 0.14 | 8.74 | 3.11 | 90.26 | 0 | 8 | 0 |
| 88 | DENSE | 512 | 384 | 1.33 | 4.20 | 0.05 | 16.10 | 3.83 | 85.70 | 0 | 0 | 0 |
| 93 | DENSE | 1024 | 512 | 2.00 | 3.48 | 0.06 | 11.22 | 3.23 | 85.54 | 0 | 0 | 0 |
| 94 | DENSE | 1024 | 512 | 2.00 | 5.16 | 0.07 | 19.03 | 3.69 | 92.96 | 0 | 0 | 0 |
| 95 | DENSE | 1024 | 512 | 2.00 | 3.88 | 0.04 | 14.62 | 3.76 | 87.85 | 0 | 0 | 0 |
| 104 | DENSE | 512 | 384 | 1.33 | 4.77 | 0.04 | 4.22 | 0.88 | 87.65 | 0 | 0 | 0 |
| 105 | DENSE | 512 | 384 | 1.33 | 2.16 | 0.14 | 5.67 | 2.62 | 90.76 | 0 | 8 | 0 |
| 106 | DENSE | 512 | 384 | 1.33 | 4.63 | 0.11 | 14.27 | 3.08 | 92.25 | 0 | 0 | 0 |
| 107 | DENSE | 512 | 384 | 1.33 | 4.00 | 0.06 | 15.14 | 3.79 | 86.05 | 0 | 0 | 0 |
| 112 | DENSE | 1024 | 512 | 2.00 | 3.24 | 0.10 | 10.25 | 3.16 | 85.37 | 0 | 6 | 0 |
| 113 | DENSE | 1024 | 512 | 2.00 | 5.38 | 0.07 | 20.53 | 3.82 | 91.77 | 0 | 0 | 0 |
| 114 | DENSE | 1024 | 512 | 2.00 | 3.91 | 0.04 | 14.64 | 3.74 | 88.24 | 0 | 0 | 0 |
| 123 | DENSE | 512 | 384 | 1.33 | 3.73 | 0.05 | 3.81 | 1.02 | 85.99 | 0 | 0 | 0 |
| 124 | DENSE | 512 | 384 | 1.33 | 3.50 | 0.07 | 9.39 | 2.68 | 88.38 | 0 | 0 | 0 |
| 125 | DENSE | 512 | 384 | 1.33 | 4.08 | 0.13 | 13.32 | 3.26 | 90.21 | 0 | 3 | 0 |
| 126 | DENSE | 512 | 384 | 1.33 | 3.09 | 0.10 | 12.08 | 3.91 | 84.91 | 0 | 1 | 0 |
| 131 | DENSE | 1024 | 512 | 2.00 | 2.87 | 0.08 | 8.87 | 3.08 | 85.35 | 0 | 1 | 0 |
| 132 | DENSE | 1024 | 512 | 2.00 | 4.54 | 0.05 | 17.83 | 3.93 | 91.15 | 0 | 0 | 0 |
| 133 | DENSE | 1024 | 512 | 2.00 | 3.98 | 0.03 | 14.52 | 3.65 | 89.08 | 0 | 0 | 0 |
| 142 | DENSE | 512 | 384 | 1.33 | 2.91 | 0.04 | 2.47 | 0.85 | 80.59 | 0 | 0 | 0 |
| 143 | DENSE | 512 | 384 | 1.33 | 2.57 | 0.07 | 6.36 | 2.48 | 83.88 | 0 | 0 | 0 |
| 144 | DENSE | 512 | 384 | 1.33 | 4.82 | 0.12 | 15.94 | 3.30 | 91.48 | 0 | 1 | 0 |
| 145 | DENSE | 512 | 384 | 1.33 | 4.25 | 0.06 | 17.23 | 4.06 | 83.27 | 0 | 0 | 0 |
| 150 | DENSE | 1024 | 512 | 2.00 | 2.80 | 0.09 | 8.71 | 3.12 | 85.16 | 0 | 2 | 0 |
| 151 | DENSE | 1024 | 512 | 2.00 | 4.39 | 0.05 | 18.08 | 4.12 | 91.25 | 1 | 0 | 0 |
| 152 | DENSE | 1024 | 512 | 2.00 | 3.46 | 0.04 | 13.14 | 3.80 | 86.15 | 0 | 0 | 0 |
| 165 | DENSE | 512 | 384 | 1.33 | 3.90 | 0.04 | 5.10 | 1.31 | 86.20 | 0 | 0 | 0 |
| 166 | DENSE | 512 | 384 | 1.33 | 4.57 | 0.08 | 13.71 | 3.00 | 88.79 | 0 | 0 | 0 |
| 167 | DENSE | 512 | 384 | 1.33 | 4.97 | 0.07 | 13.52 | 2.72 | 90.84 | 0 | 0 | 0 |
| 168 | DENSE | 512 | 384 | 1.33 | 3.20 | 0.06 | 10.59 | 3.31 | 83.44 | 0 | 0 | 0 |
| 174 | DENSE | 512 | 384 | 1.33 | 4.21 | 0.04 | 5.12 | 1.22 | 91.51 | 0 | 0 | 0 |
| 175 | DENSE | 512 | 384 | 1.33 | 3.36 | 0.09 | 10.10 | 3.01 | 84.17 | 0 | 0 | 0 |
| 176 | DENSE | 512 | 384 | 1.33 | 5.18 | 0.04 | 13.19 | 2.55 | 91.97 | 0 | 0 | 0 |
| 177 | DENSE | 512 | 384 | 1.33 | 3.02 | 0.04 | 10.58 | 3.51 | 79.19 | 0 | 0 | 0 |
| 182 | DENSE | 1024 | 512 | 2.00 | 2.82 | 0.03 | 10.69 | 3.79 | 82.85 | 0 | 0 | 0 |
| 183 | DENSE | 1024 | 512 | 2.00 | 3.56 | 0.04 | 11.43 | 3.21 | 91.56 | 0 | 0 | 0 |
| 184 | DENSE | 1024 | 512 | 2.00 | 3.28 | 0.03 | 12.37 | 3.77 | 82.60 | 0 | 0 | 0 |
| 193 | DENSE | 512 | 384 | 1.33 | 2.91 | 0.04 | 4.13 | 1.42 | 82.05 | 0 | 0 | 0 |
| 194 | DENSE | 512 | 384 | 1.33 | 3.11 | 0.04 | 9.10 | 2.93 | 83.17 | 0 | 0 | 0 |
| 195 | DENSE | 512 | 384 | 1.33 | 3.63 | 0.05 | 11.04 | 3.04 | 87.86 | 0 | 0 | 0 |
| 196 | DENSE | 512 | 384 | 1.33 | 2.84 | 0.02 | 11.19 | 3.94 | 74.90 | 0 | 0 | 0 |
| 201 | DENSE | 512 | 384 | 1.33 | 4.11 | 0.05 | 5.22 | 1.27 | 88.53 | 0 | 0 | 0 |
| 202 | DENSE | 512 | 384 | 1.33 | 3.62 | 0.04 | 10.71 | 2.95 | 89.60 | 0 | 0 | 0 |
| 203 | DENSE | 512 | 384 | 1.33 | 3.54 | 0.08 | 9.64 | 2.72 | 89.91 | 0 | 0 | 0 |
| 204 | DENSE | 512 | 384 | 1.33 | 2.95 | 0.03 | 11.15 | 3.79 | 75.44 | 0 | 0 | 0 |
| 209 | DENSE | 1024 | 512 | 2.00 | 3.90 | 0.03 | 13.05 | 3.35 | 88.94 | 0 | 0 | 0 |
| 210 | DENSE | 1024 | 512 | 2.00 | 4.48 | 0.07 | 15.22 | 3.40 | 92.78 | 0 | 0 | 0 |
| 211 | DENSE | 1024 | 512 | 2.00 | 3.76 | 0.05 | 15.29 | 4.06 | 83.09 | 0 | 0 | 0 |
| 220 | DENSE | 512 | 384 | 1.33 | 3.09 | 0.05 | 2.93 | 0.95 | 83.03 | 0 | 0 | 0 |
| 221 | DENSE | 512 | 384 | 1.33 | 3.43 | 0.05 | 9.39 | 2.74 | 85.40 | 0 | 0 | 0 |
| 222 | DENSE | 512 | 384 | 1.33 | 3.86 | 0.12 | 11.74 | 3.04 | 88.54 | 0 | 1 | 0 |
| 223 | DENSE | 512 | 384 | 1.33 | 2.85 | 0.04 | 11.88 | 4.17 | 73.12 | 0 | 0 | 0 |
| 228 | DENSE | 512 | 384 | 1.33 | 4.70 | 0.06 | 5.59 | 1.19 | 91.37 | 0 | 0 | 0 |
| 229 | DENSE | 512 | 384 | 1.33 | 3.98 | 0.05 | 11.44 | 2.88 | 90.52 | 0 | 0 | 0 |
| 230 | DENSE | 512 | 384 | 1.33 | 4.39 | 0.11 | 12.70 | 2.89 | 94.06 | 0 | 1 | 0 |
| 231 | DENSE | 512 | 384 | 1.33 | 3.11 | 0.05 | 12.43 | 4.00 | 77.90 | 0 | 0 | 0 |
| 236 | DENSE | 1024 | 512 | 2.00 | 3.63 | 0.05 | 11.90 | 3.28 | 87.26 | 0 | 0 | 0 |
| 237 | DENSE | 1024 | 512 | 2.00 | 4.67 | 0.04 | 17.81 | 3.81 | 91.92 | 0 | 0 | 0 |
| 238 | DENSE | 1024 | 512 | 2.00 | 3.82 | 0.05 | 17.33 | 4.53 | 78.68 | 1 | 0 | 0 |
| 247 | DENSE | 512 | 384 | 1.33 | 3.78 | 0.07 | 3.54 | 0.94 | 86.32 | 0 | 0 | 0 |
| 248 | DENSE | 512 | 384 | 1.33 | 3.36 | 0.05 | 9.07 | 2.70 | 86.97 | 0 | 0 | 0 |
| 249 | DENSE | 512 | 384 | 1.33 | 6.01 | 0.10 | 19.50 | 3.24 | 91.86 | 0 | 0 | 0 |
| 250 | DENSE | 512 | 384 | 1.33 | 3.15 | 0.09 | 13.93 | 4.43 | 74.34 | 0 | 0 | 0 |
| 255 | DENSE | 512 | 384 | 1.33 | 2.78 | 0.05 | 3.78 | 1.36 | 84.01 | 0 | 0 | 0 |
| 256 | DENSE | 512 | 384 | 1.33 | 2.73 | 0.05 | 8.96 | 3.28 | 79.52 | 0 | 0 | 0 |
| 257 | DENSE | 512 | 384 | 1.33 | 7.44 | 0.05 | 22.28 | 2.99 | 94.89 | 0 | 0 | 0 |
| 258 | DENSE | 512 | 384 | 1.33 | 3.34 | 0.06 | 13.60 | 4.07 | 75.64 | 0 | 0 | 0 |
| 263 | DENSE | 1024 | 512 | 2.00 | 2.92 | 0.05 | 9.31 | 3.19 | 85.06 | 0 | 0 | 0 |
| 264 | DENSE | 1024 | 512 | 2.00 | 4.09 | 0.09 | 15.56 | 3.81 | 91.44 | 0 | 1 | 0 |
| 265 | DENSE | 1024 | 512 | 2.00 | 3.65 | 0.02 | 16.55 | 4.54 | 80.08 | 0 | 0 | 0 |
| 274 | DENSE | 512 | 384 | 1.33 | 4.98 | 0.04 | 5.69 | 1.14 | 89.00 | 0 | 0 | 0 |
| 275 | DENSE | 512 | 384 | 1.33 | 3.65 | 0.14 | 9.70 | 2.66 | 92.18 | 0 | 8 | 0 |
| 276 | DENSE | 512 | 384 | 1.33 | 5.15 | 0.12 | 16.62 | 3.23 | 91.85 | 0 | 1 | 0 |
| 277 | DENSE | 512 | 384 | 1.33 | 3.05 | 0.04 | 14.09 | 4.63 | 73.89 | 0 | 0 | 0 |
| 282 | DENSE | 512 | 384 | 1.33 | 3.30 | 0.05 | 4.55 | 1.38 | 84.37 | 0 | 0 | 0 |
| 283 | DENSE | 512 | 384 | 1.33 | 3.06 | 0.04 | 10.19 | 3.33 | 83.29 | 0 | 0 | 0 |
| 284 | DENSE | 512 | 384 | 1.33 | 3.69 | 0.14 | 11.61 | 3.15 | 91.92 | 0 | 5 | 0 |
| 285 | DENSE | 512 | 384 | 1.33 | 3.93 | 0.06 | 15.80 | 4.02 | 82.21 | 0 | 0 | 0 |
| 290 | DENSE | 1024 | 512 | 2.00 | 3.31 | 0.05 | 10.04 | 3.03 | 86.29 | 0 | 0 | 0 |
| 291 | DENSE | 1024 | 512 | 2.00 | 3.58 | 0.11 | 13.75 | 3.84 | 92.23 | 0 | 4 | 0 |
| 292 | DENSE | 1024 | 512 | 2.00 | 3.60 | 0.04 | 16.21 | 4.50 | 80.80 | 0 | 0 | 0 |
| 301 | DENSE | 512 | 384 | 1.33 | 3.67 | 0.04 | 4.86 | 1.33 | 85.16 | 0 | 0 | 0 |
| 302 | DENSE | 512 | 384 | 1.33 | 5.73 | 0.07 | 14.51 | 2.53 | 91.01 | 0 | 0 | 0 |
| 303 | DENSE | 512 | 384 | 1.33 | 4.88 | 0.13 | 16.57 | 3.39 | 94.92 | 0 | 5 | 0 |
| 304 | DENSE | 512 | 384 | 1.33 | 1.91 | 0.08 | 8.57 | 4.48 | 78.29 | 0 | 1 | 0 |
| 309 | DENSE | 512 | 384 | 1.33 | 4.24 | 0.06 | 5.06 | 1.19 | 88.65 | 0 | 0 | 0 |
| 310 | DENSE | 512 | 384 | 1.33 | 4.75 | 0.06 | 14.04 | 2.96 | 88.77 | 0 | 0 | 0 |
| 311 | DENSE | 512 | 384 | 1.33 | 2.60 | 0.11 | 8.58 | 3.30 | 92.70 | 0 | 0 | 0 |
| 312 | DENSE | 512 | 384 | 1.33 | 3.30 | 0.08 | 14.74 | 4.46 | 77.28 | 0 | 0 | 0 |
| 317 | DENSE | 1024 | 512 | 2.00 | 3.14 | 0.05 | 9.01 | 2.87 | 86.04 | 0 | 0 | 0 |
| 318 | DENSE | 1024 | 512 | 2.00 | 4.55 | 0.09 | 18.00 | 3.96 | 92.32 | 0 | 1 | 0 |
| 319 | DENSE | 1024 | 512 | 2.00 | 3.94 | 0.04 | 17.16 | 4.36 | 83.85 | 0 | 0 | 0 |
| 328 | DENSE | 512 | 384 | 1.33 | 4.06 | 0.07 | 2.81 | 0.69 | 89.98 | 0 | 0 | 0 |
| 329 | DENSE | 512 | 384 | 1.33 | 4.10 | 0.07 | 10.14 | 2.47 | 89.10 | 0 | 0 | 0 |
| 330 | DENSE | 512 | 384 | 1.33 | 2.36 | 0.14 | 7.90 | 3.35 | 94.70 | 0 | 2 | 0 |
| 331 | DENSE | 512 | 384 | 1.33 | 4.09 | 0.10 | 20.10 | 4.91 | 75.87 | 0 | 0 | 0 |
| 336 | DENSE | 512 | 384 | 1.33 | 3.62 | 0.06 | 4.43 | 1.22 | 83.47 | 0 | 0 | 0 |
| 337 | DENSE | 512 | 384 | 1.33 | 2.04 | 0.11 | 6.21 | 3.05 | 83.93 | 0 | 4 | 0 |
| 338 | DENSE | 512 | 384 | 1.33 | 10.00 | 0.11 | 33.78 | 3.38 | 95.50 | 0 | 0 | 0 |
| 339 | DENSE | 512 | 384 | 1.33 | 3.83 | 0.11 | 17.76 | 4.63 | 81.64 | 0 | 0 | 0 |
| 344 | DENSE | 1024 | 512 | 2.00 | 3.38 | 0.05 | 9.37 | 2.78 | 85.78 | 0 | 0 | 0 |
| 345 | DENSE | 1024 | 512 | 2.00 | 4.20 | 0.07 | 17.81 | 4.24 | 90.86 | 0 | 0 | 0 |
| 346 | DENSE | 1024 | 512 | 2.00 | 3.92 | 0.05 | 17.31 | 4.42 | 85.35 | 0 | 0 | 0 |
| 355 | DENSE | 512 | 384 | 1.33 | 4.61 | 0.03 | 3.94 | 0.86 | 92.82 | 0 | 0 | 0 |
| 356 | DENSE | 512 | 384 | 1.33 | 4.69 | 0.05 | 13.03 | 2.78 | 92.94 | 0 | 0 | 0 |
| 357 | DENSE | 512 | 384 | 1.33 | 7.66 | 0.06 | 27.15 | 3.55 | 93.97 | 0 | 0 | 0 |
| 358 | DENSE | 512 | 384 | 1.33 | 3.26 | 0.07 | 14.32 | 4.39 | 77.52 | 0 | 0 | 0 |
| 363 | DENSE | 512 | 384 | 1.33 | 2.83 | 0.09 | 4.41 | 1.56 | 75.52 | 0 | 0 | 0 |
| 364 | DENSE | 512 | 384 | 1.33 | 1.94 | 0.09 | 6.44 | 3.32 | 77.55 | 0 | 1 | 0 |
| 365 | DENSE | 512 | 384 | 1.33 | 6.62 | 0.13 | 25.27 | 3.82 | 88.86 | 0 | 1 | 0 |
| 366 | DENSE | 512 | 384 | 1.33 | 3.68 | 0.06 | 17.22 | 4.68 | 85.99 | 0 | 0 | 0 |
| 371 | DENSE | 1024 | 512 | 2.00 | 3.24 | 0.04 | 10.59 | 3.27 | 81.62 | 0 | 0 | 0 |
| 372 | DENSE | 1024 | 512 | 2.00 | 3.10 | 0.08 | 15.34 | 4.95 | 85.96 | 0 | 0 | 0 |
| 373 | DENSE | 1024 | 512 | 2.00 | 2.86 | 0.05 | 15.16 | 5.30 | 71.02 | 5 | 0 | 0 |
google-flan-t5-small Layer Plots
Layer 1
Layer=1 | N=32128 | M=512 | Q=62.75 | alpha=2.28 | D_ks=0.07 | alpha-hat=18.77 | num traps=0
Layer 8
Layer=8 | N=512 | M=384 | Q=1.33 | alpha=2.06 | D_ks=0.14 | alpha-hat=2.29 | num traps=0
Layer 9
Layer=9 | N=512 | M=384 | Q=1.33 | alpha=1.97 | D_ks=0.13 | alpha-hat=5.74 | num traps=0
Layer 10
Layer=10 | N=512 | M=384 | Q=1.33 | alpha=3.96 | D_ks=0.14 | alpha-hat=10.38 | num traps=0
Layer 11
Layer=11 | N=512 | M=384 | Q=1.33 | alpha=3.15 | D_ks=0.03 | alpha-hat=10.98 | num traps=0
Layer 17
Layer=17 | N=1024 | M=512 | Q=2.00 | alpha=3.19 | D_ks=0.05 | alpha-hat=10.65 | num traps=0
Layer 18
Layer=18 | N=1024 | M=512 | Q=2.00 | alpha=3.58 | D_ks=0.08 | alpha-hat=11.66 | num traps=0
Layer 19
Layer=19 | N=1024 | M=512 | Q=2.00 | alpha=3.08 | D_ks=0.03 | alpha-hat=11.14 | num traps=0
Layer 28
Layer=28 | N=512 | M=384 | Q=1.33 | alpha=5.39 | D_ks=0.05 | alpha-hat=6.61 | num traps=0
Layer 29
Layer=29 | N=512 | M=384 | Q=1.33 | alpha=5.20 | D_ks=0.05 | alpha-hat=15.46 | num traps=0
Layer 30
Layer=30 | N=512 | M=384 | Q=1.33 | alpha=3.66 | D_ks=0.11 | alpha-hat=9.46 | num traps=0
Layer 31
Layer=31 | N=512 | M=384 | Q=1.33 | alpha=3.30 | D_ks=0.04 | alpha-hat=12.29 | num traps=0
Layer 36
Layer=36 | N=1024 | M=512 | Q=2.00 | alpha=3.53 | D_ks=0.03 | alpha-hat=11.74 | num traps=0
Layer 37
Layer=37 | N=1024 | M=512 | Q=2.00 | alpha=4.92 | D_ks=0.08 | alpha-hat=16.02 | num traps=0
Layer 38
Layer=38 | N=1024 | M=512 | Q=2.00 | alpha=3.19 | D_ks=0.02 | alpha-hat=11.26 | num traps=0
Layer 47
Layer=47 | N=512 | M=384 | Q=1.33 | alpha=3.61 | D_ks=0.06 | alpha-hat=3.57 | num traps=0
Layer 48
Layer=48 | N=512 | M=384 | Q=1.33 | alpha=5.01 | D_ks=0.05 | alpha-hat=14.44 | num traps=0
Layer 49
Layer=49 | N=512 | M=384 | Q=1.33 | alpha=3.31 | D_ks=0.11 | alpha-hat=9.76 | num traps=0
Layer 50
Layer=50 | N=512 | M=384 | Q=1.33 | alpha=4.03 | D_ks=0.04 | alpha-hat=14.45 | num traps=0
Layer 55
Layer=55 | N=1024 | M=512 | Q=2.00 | alpha=3.34 | D_ks=0.04 | alpha-hat=11.13 | num traps=0
Layer 56
Layer=56 | N=1024 | M=512 | Q=2.00 | alpha=6.37 | D_ks=0.06 | alpha-hat=22.15 | num traps=0
Layer 57
Layer=57 | N=1024 | M=512 | Q=2.00 | alpha=3.53 | D_ks=0.04 | alpha-hat=13.16 | num traps=0
Layer 66
Layer=66 | N=512 | M=384 | Q=1.33 | alpha=4.75 | D_ks=0.06 | alpha-hat=4.05 | num traps=0
Layer 67
Layer=67 | N=512 | M=384 | Q=1.33 | alpha=5.44 | D_ks=0.06 | alpha-hat=14.78 | num traps=0
Layer 68
Layer=68 | N=512 | M=384 | Q=1.33 | alpha=4.85 | D_ks=0.14 | alpha-hat=14.12 | num traps=0
Layer 69
Layer=69 | N=512 | M=384 | Q=1.33 | alpha=4.09 | D_ks=0.05 | alpha-hat=13.71 | num traps=0
Layer 74
Layer=74 | N=1024 | M=512 | Q=2.00 | alpha=3.41 | D_ks=0.04 | alpha-hat=11.20 | num traps=0
Layer 75
Layer=75 | N=1024 | M=512 | Q=2.00 | alpha=5.67 | D_ks=0.10 | alpha-hat=20.22 | num traps=0
Layer 76
Layer=76 | N=1024 | M=512 | Q=2.00 | alpha=3.69 | D_ks=0.04 | alpha-hat=13.47 | num traps=0
Layer 85
Layer=85 | N=512 | M=384 | Q=1.33 | alpha=3.74 | D_ks=0.05 | alpha-hat=3.16 | num traps=0
Layer 86
Layer=86 | N=512 | M=384 | Q=1.33 | alpha=3.44 | D_ks=0.10 | alpha-hat=9.19 | num traps=0
Layer 87
Layer=87 | N=512 | M=384 | Q=1.33 | alpha=2.81 | D_ks=0.14 | alpha-hat=8.74 | num traps=0
Layer 88
Layer=88 | N=512 | M=384 | Q=1.33 | alpha=4.20 | D_ks=0.05 | alpha-hat=16.10 | num traps=0
Layer 93
Layer=93 | N=1024 | M=512 | Q=2.00 | alpha=3.48 | D_ks=0.06 | alpha-hat=11.22 | num traps=0
Layer 94
Layer=94 | N=1024 | M=512 | Q=2.00 | alpha=5.16 | D_ks=0.07 | alpha-hat=19.03 | num traps=0
Layer 95
Layer=95 | N=1024 | M=512 | Q=2.00 | alpha=3.88 | D_ks=0.04 | alpha-hat=14.62 | num traps=0
Layer 104
Layer=104 | N=512 | M=384 | Q=1.33 | alpha=4.77 | D_ks=0.04 | alpha-hat=4.22 | num traps=0
Layer 105
Layer=105 | N=512 | M=384 | Q=1.33 | alpha=2.16 | D_ks=0.14 | alpha-hat=5.67 | num traps=0
Layer 106
Layer=106 | N=512 | M=384 | Q=1.33 | alpha=4.63 | D_ks=0.11 | alpha-hat=14.27 | num traps=0
Layer 107
Layer=107 | N=512 | M=384 | Q=1.33 | alpha=4.00 | D_ks=0.06 | alpha-hat=15.14 | num traps=0
Layer 112
Layer=112 | N=1024 | M=512 | Q=2.00 | alpha=3.24 | D_ks=0.10 | alpha-hat=10.25 | num traps=0
Layer 113
Layer=113 | N=1024 | M=512 | Q=2.00 | alpha=5.38 | D_ks=0.07 | alpha-hat=20.53 | num traps=0
Layer 114
Layer=114 | N=1024 | M=512 | Q=2.00 | alpha=3.91 | D_ks=0.04 | alpha-hat=14.64 | num traps=0
Layer 123
Layer=123 | N=512 | M=384 | Q=1.33 | alpha=3.73 | D_ks=0.05 | alpha-hat=3.81 | num traps=0
Layer 124
Layer=124 | N=512 | M=384 | Q=1.33 | alpha=3.50 | D_ks=0.07 | alpha-hat=9.39 | num traps=0
Layer 125
Layer=125 | N=512 | M=384 | Q=1.33 | alpha=4.08 | D_ks=0.13 | alpha-hat=13.32 | num traps=0
Layer 126
Layer=126 | N=512 | M=384 | Q=1.33 | alpha=3.09 | D_ks=0.10 | alpha-hat=12.08 | num traps=0
Layer 131
Layer=131 | N=1024 | M=512 | Q=2.00 | alpha=2.87 | D_ks=0.08 | alpha-hat=8.87 | num traps=0
Layer 132
Layer=132 | N=1024 | M=512 | Q=2.00 | alpha=4.54 | D_ks=0.05 | alpha-hat=17.83 | num traps=0
Layer 133
Layer=133 | N=1024 | M=512 | Q=2.00 | alpha=3.98 | D_ks=0.03 | alpha-hat=14.52 | num traps=0
Layer 142
Layer=142 | N=512 | M=384 | Q=1.33 | alpha=2.91 | D_ks=0.04 | alpha-hat=2.47 | num traps=0
Layer 143
Layer=143 | N=512 | M=384 | Q=1.33 | alpha=2.57 | D_ks=0.07 | alpha-hat=6.36 | num traps=0
Layer 144
Layer=144 | N=512 | M=384 | Q=1.33 | alpha=4.82 | D_ks=0.12 | alpha-hat=15.94 | num traps=0
Layer 145
Layer=145 | N=512 | M=384 | Q=1.33 | alpha=4.25 | D_ks=0.06 | alpha-hat=17.23 | num traps=0
Layer 150
Layer=150 | N=1024 | M=512 | Q=2.00 | alpha=2.80 | D_ks=0.09 | alpha-hat=8.71 | num traps=0
Layer 151
Layer=151 | N=1024 | M=512 | Q=2.00 | alpha=4.39 | D_ks=0.05 | alpha-hat=18.08 | num traps=1
Layer 152
Layer=152 | N=1024 | M=512 | Q=2.00 | alpha=3.46 | D_ks=0.04 | alpha-hat=13.14 | num traps=0
Layer 165
Layer=165 | N=512 | M=384 | Q=1.33 | alpha=3.90 | D_ks=0.04 | alpha-hat=5.10 | num traps=0
Layer 166
Layer=166 | N=512 | M=384 | Q=1.33 | alpha=4.57 | D_ks=0.08 | alpha-hat=13.71 | num traps=0
Layer 167
Layer=167 | N=512 | M=384 | Q=1.33 | alpha=4.97 | D_ks=0.07 | alpha-hat=13.52 | num traps=0
Layer 168
Layer=168 | N=512 | M=384 | Q=1.33 | alpha=3.20 | D_ks=0.06 | alpha-hat=10.59 | num traps=0
Layer 174
Layer=174 | N=512 | M=384 | Q=1.33 | alpha=4.21 | D_ks=0.04 | alpha-hat=5.12 | num traps=0
Layer 175
Layer=175 | N=512 | M=384 | Q=1.33 | alpha=3.36 | D_ks=0.09 | alpha-hat=10.10 | num traps=0
Layer 176
Layer=176 | N=512 | M=384 | Q=1.33 | alpha=5.18 | D_ks=0.04 | alpha-hat=13.19 | num traps=0
Layer 177
Layer=177 | N=512 | M=384 | Q=1.33 | alpha=3.02 | D_ks=0.04 | alpha-hat=10.58 | num traps=0
Layer 182
Layer=182 | N=1024 | M=512 | Q=2.00 | alpha=2.82 | D_ks=0.03 | alpha-hat=10.69 | num traps=0
Layer 183
Layer=183 | N=1024 | M=512 | Q=2.00 | alpha=3.56 | D_ks=0.04 | alpha-hat=11.43 | num traps=0
Layer 184
Layer=184 | N=1024 | M=512 | Q=2.00 | alpha=3.28 | D_ks=0.03 | alpha-hat=12.37 | num traps=0
Layer 193
Layer=193 | N=512 | M=384 | Q=1.33 | alpha=2.91 | D_ks=0.04 | alpha-hat=4.13 | num traps=0
Layer 194
Layer=194 | N=512 | M=384 | Q=1.33 | alpha=3.11 | D_ks=0.04 | alpha-hat=9.10 | num traps=0
Layer 195
Layer=195 | N=512 | M=384 | Q=1.33 | alpha=3.63 | D_ks=0.05 | alpha-hat=11.04 | num traps=0
Layer 196
Layer=196 | N=512 | M=384 | Q=1.33 | alpha=2.84 | D_ks=0.02 | alpha-hat=11.19 | num traps=0
Layer 201
Layer=201 | N=512 | M=384 | Q=1.33 | alpha=4.11 | D_ks=0.05 | alpha-hat=5.22 | num traps=0
Layer 202
Layer=202 | N=512 | M=384 | Q=1.33 | alpha=3.62 | D_ks=0.04 | alpha-hat=10.71 | num traps=0
Layer 203
Layer=203 | N=512 | M=384 | Q=1.33 | alpha=3.54 | D_ks=0.08 | alpha-hat=9.64 | num traps=0
Layer 204
Layer=204 | N=512 | M=384 | Q=1.33 | alpha=2.95 | D_ks=0.03 | alpha-hat=11.15 | num traps=0
Layer 209
Layer=209 | N=1024 | M=512 | Q=2.00 | alpha=3.90 | D_ks=0.03 | alpha-hat=13.05 | num traps=0
Layer 210
Layer=210 | N=1024 | M=512 | Q=2.00 | alpha=4.48 | D_ks=0.07 | alpha-hat=15.22 | num traps=0
Layer 211
Layer=211 | N=1024 | M=512 | Q=2.00 | alpha=3.76 | D_ks=0.05 | alpha-hat=15.29 | num traps=0
Layer 220
Layer=220 | N=512 | M=384 | Q=1.33 | alpha=3.09 | D_ks=0.05 | alpha-hat=2.93 | num traps=0
Layer 221
Layer=221 | N=512 | M=384 | Q=1.33 | alpha=3.43 | D_ks=0.05 | alpha-hat=9.39 | num traps=0
Layer 222
Layer=222 | N=512 | M=384 | Q=1.33 | alpha=3.86 | D_ks=0.12 | alpha-hat=11.74 | num traps=0
Layer 223
Layer=223 | N=512 | M=384 | Q=1.33 | alpha=2.85 | D_ks=0.04 | alpha-hat=11.88 | num traps=0
Layer 228
Layer=228 | N=512 | M=384 | Q=1.33 | alpha=4.70 | D_ks=0.06 | alpha-hat=5.59 | num traps=0
Layer 229
Layer=229 | N=512 | M=384 | Q=1.33 | alpha=3.98 | D_ks=0.05 | alpha-hat=11.44 | num traps=0
Layer 230
Layer=230 | N=512 | M=384 | Q=1.33 | alpha=4.39 | D_ks=0.11 | alpha-hat=12.70 | num traps=0
Layer 231
Layer=231 | N=512 | M=384 | Q=1.33 | alpha=3.11 | D_ks=0.05 | alpha-hat=12.43 | num traps=0
Layer 236
Layer=236 | N=1024 | M=512 | Q=2.00 | alpha=3.63 | D_ks=0.05 | alpha-hat=11.90 | num traps=0
Layer 237
Layer=237 | N=1024 | M=512 | Q=2.00 | alpha=4.67 | D_ks=0.04 | alpha-hat=17.81 | num traps=0
Layer 238
Layer=238 | N=1024 | M=512 | Q=2.00 | alpha=3.82 | D_ks=0.05 | alpha-hat=17.33 | num traps=1
Layer 247
Layer=247 | N=512 | M=384 | Q=1.33 | alpha=3.78 | D_ks=0.07 | alpha-hat=3.54 | num traps=0
Layer 248
Layer=248 | N=512 | M=384 | Q=1.33 | alpha=3.36 | D_ks=0.05 | alpha-hat=9.07 | num traps=0
Layer 249
Layer=249 | N=512 | M=384 | Q=1.33 | alpha=6.01 | D_ks=0.10 | alpha-hat=19.50 | num traps=0
Layer 250
Layer=250 | N=512 | M=384 | Q=1.33 | alpha=3.15 | D_ks=0.09 | alpha-hat=13.93 | num traps=0
Layer 255
Layer=255 | N=512 | M=384 | Q=1.33 | alpha=2.78 | D_ks=0.05 | alpha-hat=3.78 | num traps=0
Layer 256
Layer=256 | N=512 | M=384 | Q=1.33 | alpha=2.73 | D_ks=0.05 | alpha-hat=8.96 | num traps=0
Layer 257
Layer=257 | N=512 | M=384 | Q=1.33 | alpha=7.44 | D_ks=0.05 | alpha-hat=22.28 | num traps=0
Layer 258
Layer=258 | N=512 | M=384 | Q=1.33 | alpha=3.34 | D_ks=0.06 | alpha-hat=13.60 | num traps=0
Layer 263
Layer=263 | N=1024 | M=512 | Q=2.00 | alpha=2.92 | D_ks=0.05 | alpha-hat=9.31 | num traps=0
Layer 264
Layer=264 | N=1024 | M=512 | Q=2.00 | alpha=4.09 | D_ks=0.09 | alpha-hat=15.56 | num traps=0
Layer 265
Layer=265 | N=1024 | M=512 | Q=2.00 | alpha=3.65 | D_ks=0.02 | alpha-hat=16.55 | num traps=0
Layer 274
Layer=274 | N=512 | M=384 | Q=1.33 | alpha=4.98 | D_ks=0.04 | alpha-hat=5.69 | num traps=0
Layer 275
Layer=275 | N=512 | M=384 | Q=1.33 | alpha=3.65 | D_ks=0.14 | alpha-hat=9.70 | num traps=0
Layer 276
Layer=276 | N=512 | M=384 | Q=1.33 | alpha=5.15 | D_ks=0.12 | alpha-hat=16.62 | num traps=0
Layer 277
Layer=277 | N=512 | M=384 | Q=1.33 | alpha=3.05 | D_ks=0.04 | alpha-hat=14.09 | num traps=0
Layer 282
Layer=282 | N=512 | M=384 | Q=1.33 | alpha=3.30 | D_ks=0.05 | alpha-hat=4.55 | num traps=0
Layer 283
Layer=283 | N=512 | M=384 | Q=1.33 | alpha=3.06 | D_ks=0.04 | alpha-hat=10.19 | num traps=0
Layer 284
Layer=284 | N=512 | M=384 | Q=1.33 | alpha=3.69 | D_ks=0.14 | alpha-hat=11.61 | num traps=0
Layer 285
Layer=285 | N=512 | M=384 | Q=1.33 | alpha=3.93 | D_ks=0.06 | alpha-hat=15.80 | num traps=0
Layer 290
Layer=290 | N=1024 | M=512 | Q=2.00 | alpha=3.31 | D_ks=0.05 | alpha-hat=10.04 | num traps=0
Layer 291
Layer=291 | N=1024 | M=512 | Q=2.00 | alpha=3.58 | D_ks=0.11 | alpha-hat=13.75 | num traps=0
Layer 292
Layer=292 | N=1024 | M=512 | Q=2.00 | alpha=3.60 | D_ks=0.04 | alpha-hat=16.21 | num traps=0
Layer 301
Layer=301 | N=512 | M=384 | Q=1.33 | alpha=3.67 | D_ks=0.04 | alpha-hat=4.86 | num traps=0
Layer 302
Layer=302 | N=512 | M=384 | Q=1.33 | alpha=5.73 | D_ks=0.07 | alpha-hat=14.51 | num traps=0
Layer 303
Layer=303 | N=512 | M=384 | Q=1.33 | alpha=4.88 | D_ks=0.13 | alpha-hat=16.57 | num traps=0
Layer 304
Layer=304 | N=512 | M=384 | Q=1.33 | alpha=1.91 | D_ks=0.08 | alpha-hat=8.57 | num traps=0
Layer 309
Layer=309 | N=512 | M=384 | Q=1.33 | alpha=4.24 | D_ks=0.06 | alpha-hat=5.06 | num traps=0
Layer 310
Layer=310 | N=512 | M=384 | Q=1.33 | alpha=4.75 | D_ks=0.06 | alpha-hat=14.04 | num traps=0
Layer 311
Layer=311 | N=512 | M=384 | Q=1.33 | alpha=2.60 | D_ks=0.11 | alpha-hat=8.58 | num traps=0
Layer 312
Layer=312 | N=512 | M=384 | Q=1.33 | alpha=3.30 | D_ks=0.08 | alpha-hat=14.74 | num traps=0
Layer 317
Layer=317 | N=1024 | M=512 | Q=2.00 | alpha=3.14 | D_ks=0.05 | alpha-hat=9.01 | num traps=0
Layer 318
Layer=318 | N=1024 | M=512 | Q=2.00 | alpha=4.55 | D_ks=0.09 | alpha-hat=18.00 | num traps=0
Layer 319
Layer=319 | N=1024 | M=512 | Q=2.00 | alpha=3.94 | D_ks=0.04 | alpha-hat=17.16 | num traps=0
Layer 328
Layer=328 | N=512 | M=384 | Q=1.33 | alpha=4.06 | D_ks=0.07 | alpha-hat=2.81 | num traps=0
Layer 329
Layer=329 | N=512 | M=384 | Q=1.33 | alpha=4.10 | D_ks=0.07 | alpha-hat=10.14 | num traps=0
Layer 330
Layer=330 | N=512 | M=384 | Q=1.33 | alpha=2.36 | D_ks=0.14 | alpha-hat=7.90 | num traps=0
Layer 331
Layer=331 | N=512 | M=384 | Q=1.33 | alpha=4.09 | D_ks=0.10 | alpha-hat=20.10 | num traps=0
Layer 336
Layer=336 | N=512 | M=384 | Q=1.33 | alpha=3.62 | D_ks=0.06 | alpha-hat=4.43 | num traps=0
Layer 337
Layer=337 | N=512 | M=384 | Q=1.33 | alpha=2.04 | D_ks=0.11 | alpha-hat=6.21 | num traps=0
Layer 338
Layer=338 | N=512 | M=384 | Q=1.33 | alpha=10.00 | D_ks=0.11 | alpha-hat=33.78 | num traps=0
Layer 339
Layer=339 | N=512 | M=384 | Q=1.33 | alpha=3.83 | D_ks=0.11 | alpha-hat=17.76 | num traps=0
Layer 344
Layer=344 | N=1024 | M=512 | Q=2.00 | alpha=3.38 | D_ks=0.05 | alpha-hat=9.37 | num traps=0
Layer 345
Layer=345 | N=1024 | M=512 | Q=2.00 | alpha=4.20 | D_ks=0.07 | alpha-hat=17.81 | num traps=0
Layer 346
Layer=346 | N=1024 | M=512 | Q=2.00 | alpha=3.92 | D_ks=0.05 | alpha-hat=17.31 | num traps=0
Layer 355
Layer=355 | N=512 | M=384 | Q=1.33 | alpha=4.61 | D_ks=0.03 | alpha-hat=3.94 | num traps=0
Layer 356
Layer=356 | N=512 | M=384 | Q=1.33 | alpha=4.69 | D_ks=0.05 | alpha-hat=13.03 | num traps=0
Layer 357
Layer=357 | N=512 | M=384 | Q=1.33 | alpha=7.66 | D_ks=0.06 | alpha-hat=27.15 | num traps=0
Layer 358
Layer=358 | N=512 | M=384 | Q=1.33 | alpha=3.26 | D_ks=0.07 | alpha-hat=14.32 | num traps=0
Layer 363
Layer=363 | N=512 | M=384 | Q=1.33 | alpha=2.83 | D_ks=0.09 | alpha-hat=4.41 | num traps=0
Layer 364
Layer=364 | N=512 | M=384 | Q=1.33 | alpha=1.94 | D_ks=0.09 | alpha-hat=6.44 | num traps=0
Layer 365
Layer=365 | N=512 | M=384 | Q=1.33 | alpha=6.62 | D_ks=0.13 | alpha-hat=25.27 | num traps=0
Layer 366
Layer=366 | N=512 | M=384 | Q=1.33 | alpha=3.68 | D_ks=0.06 | alpha-hat=17.22 | num traps=0
Layer 371
Layer=371 | N=1024 | M=512 | Q=2.00 | alpha=3.24 | D_ks=0.04 | alpha-hat=10.59 | num traps=0
Layer 372
Layer=372 | N=1024 | M=512 | Q=2.00 | alpha=3.10 | D_ks=0.08 | alpha-hat=15.34 | num traps=0
Layer 373
Layer=373 | N=1024 | M=512 | Q=2.00 | alpha=2.86 | D_ks=0.05 | alpha-hat=15.16 | num traps=5