bigscience-bloom-1b7
Find this model in the BLOOM model summary
bigscience-bloom-1b7 Model Summary Plots
bigscience-bloom-1b7 Model Selected Details
layer_type
N
M
Q
alpha
D
alpha-hat
log_SN
% Rand
num_traps
num_fingers
rank_loss
layer_id
7
DENSE
6144
2048
3.00
3.77
0.02
5.82
1.55
86.32
0
0
0
8
DENSE
2048
2048
1.00
3.48
0.04
5.44
1.56
81.65
7
0
4
12
DENSE
8192
2048
4.00
4.64
0.04
9.51
2.05
86.11
1
0
0
14
DENSE
8192
2048
4.00
3.84
0.02
7.49
1.95
83.45
8
0
0
18
DENSE
6144
2048
3.00
4.09
0.02
6.29
1.54
87.85
0
0
0
19
DENSE
2048
2048
1.00
3.87
0.04
5.66
1.46
85.01
9
0
3
23
DENSE
8192
2048
4.00
3.37
0.06
7.23
2.14
85.07
1
0
0
25
DENSE
8192
2048
4.00
4.50
0.01
8.35
1.85
90.06
0
0
0
29
DENSE
6144
2048
3.00
3.94
0.02
6.04
1.53
87.98
0
0
0
30
DENSE
2048
2048
1.00
3.74
0.03
5.16
1.38
85.90
3
0
3
34
DENSE
8192
2048
4.00
3.31
0.04
6.82
2.06
85.38
1
0
0
36
DENSE
8192
2048
4.00
4.21
0.04
7.42
1.76
88.06
0
0
0
40
DENSE
6144
2048
3.00
4.26
0.03
6.46
1.52
87.97
0
0
0
41
DENSE
2048
2048
1.00
4.22
0.03
6.08
1.44
87.53
2
0
3
45
DENSE
8192
2048
4.00
3.48
0.01
6.58
1.89
84.71
0
0
0
47
DENSE
8192
2048
4.00
4.17
0.03
7.31
1.75
86.26
0
0
0
51
DENSE
6144
2048
3.00
4.11
0.02
6.44
1.57
86.84
0
0
0
52
DENSE
2048
2048
1.00
4.01
0.03
5.69
1.42
86.59
2
0
3
56
DENSE
8192
2048
4.00
3.28
0.01
7.11
2.17
80.85
0
0
0
58
DENSE
8192
2048
4.00
3.76
0.02
8.20
2.18
84.02
18
0
0
62
DENSE
6144
2048
3.00
4.12
0.03
6.29
1.53
87.85
0
0
0
63
DENSE
2048
2048
1.00
4.42
0.05
5.90
1.34
87.21
0
0
2
67
DENSE
8192
2048
4.00
3.27
0.01
7.16
2.19
80.46
0
0
0
69
DENSE
8192
2048
4.00
3.51
0.02
7.24
2.06
84.35
10
0
0
73
DENSE
6144
2048
3.00
3.82
0.04
5.50
1.44
87.30
0
0
0
74
DENSE
2048
2048
1.00
3.69
0.09
4.30
1.16
87.70
0
2
4
78
DENSE
8192
2048
4.00
3.30
0.01
6.29
1.91
83.15
0
0
0
80
DENSE
8192
2048
4.00
3.58
0.02
6.11
1.71
86.73
4
0
0
84
DENSE
6144
2048
3.00
2.94
0.05
4.32
1.47
85.47
0
0
0
85
DENSE
2048
2048
1.00
5.28
0.02
3.81
0.72
92.69
0
0
4
89
DENSE
8192
2048
4.00
3.24
0.02
6.23
1.92
84.03
0
0
0
91
DENSE
8192
2048
4.00
3.75
0.02
5.45
1.46
88.66
4
0
0
95
DENSE
6144
2048
3.00
4.42
0.06
6.73
1.52
82.17
0
0
0
96
DENSE
2048
2048
1.00
3.17
0.11
1.98
0.62
90.09
0
6
4
100
DENSE
8192
2048
4.00
3.38
0.02
6.57
1.95
86.15
0
0
0
102
DENSE
8192
2048
4.00
4.18
0.01
6.74
1.61
88.44
1
0
0
106
DENSE
6144
2048
3.00
2.10
0.09
3.06
1.46
78.92
0
3
0
107
DENSE
2048
2048
1.00
4.04
0.06
2.59
0.64
86.32
0
0
3
111
DENSE
8192
2048
4.00
4.07
0.03
7.84
1.93
86.65
0
0
0
113
DENSE
8192
2048
4.00
4.20
0.04
5.60
1.33
89.73
0
0
0
117
DENSE
6144
2048
3.00
4.06
0.08
5.99
1.48
82.46
0
1
0
118
DENSE
2048
2048
1.00
3.90
0.10
2.36
0.60
91.73
0
1
3
122
DENSE
8192
2048
4.00
3.65
0.04
6.72
1.84
84.02
0
0
0
124
DENSE
8192
2048
4.00
4.12
0.04
5.08
1.23
89.65
0
0
0
128
DENSE
6144
2048
3.00
2.96
0.07
4.27
1.44
80.79
0
0
0
129
DENSE
2048
2048
1.00
3.80
0.07
2.32
0.61
87.91
0
0
4
133
DENSE
8192
2048
4.00
3.41
0.03
6.74
1.98
84.36
0
0
0
135
DENSE
8192
2048
4.00
3.93
0.05
4.89
1.24
90.21
0
0
0
139
DENSE
6144
2048
3.00
3.44
0.06
4.83
1.41
84.36
0
0
0
140
DENSE
2048
2048
1.00
3.50
0.06
2.53
0.72
88.35
0
0
3
144
DENSE
8192
2048
4.00
3.57
0.03
7.03
1.97
84.53
0
0
0
146
DENSE
8192
2048
4.00
3.89
0.05
4.65
1.20
89.68
0
0
0
150
DENSE
6144
2048
3.00
3.15
0.06
4.76
1.51
83.24
0
0
0
151
DENSE
2048
2048
1.00
3.13
0.09
2.30
0.74
87.19
0
4
4
155
DENSE
8192
2048
4.00
3.30
0.04
6.62
2.00
84.21
0
0
0
157
DENSE
8192
2048
4.00
4.04
0.05
4.95
1.22
89.37
0
0
0
161
DENSE
6144
2048
3.00
3.37
0.06
4.92
1.46
84.50
0
0
0
162
DENSE
2048
2048
1.00
3.26
0.10
2.08
0.64
89.25
0
2
4
166
DENSE
8192
2048
4.00
3.24
0.03
6.53
2.02
83.76
0
0
0
168
DENSE
8192
2048
4.00
3.96
0.05
4.95
1.25
89.06
0
0
0
172
DENSE
6144
2048
3.00
3.18
0.06
4.72
1.49
84.18
0
1
0
173
DENSE
2048
2048
1.00
2.89
0.10
1.96
0.68
85.42
0
1
4
177
DENSE
8192
2048
4.00
3.32
0.03
6.72
2.02
84.52
0
0
0
179
DENSE
8192
2048
4.00
3.82
0.04
5.57
1.46
88.87
0
0
0
183
DENSE
6144
2048
3.00
4.06
0.07
5.63
1.39
86.70
0
1
0
184
DENSE
2048
2048
1.00
3.66
0.06
2.89
0.79
84.91
0
0
4
188
DENSE
8192
2048
4.00
3.41
0.02
6.91
2.03
85.69
0
0
0
190
DENSE
8192
2048
4.00
3.97
0.03
5.85
1.47
89.64
0
0
0
194
DENSE
6144
2048
3.00
3.54
0.06
5.46
1.54
88.45
0
1
0
195
DENSE
2048
2048
1.00
6.58
0.03
6.90
1.05
92.60
0
0
2
199
DENSE
8192
2048
4.00
3.50
0.02
7.24
2.07
85.87
0
0
0
201
DENSE
8192
2048
4.00
4.65
0.03
7.09
1.53
90.04
0
0
0
205
DENSE
6144
2048
3.00
4.02
0.06
5.82
1.45
90.83
0
2
0
206
DENSE
2048
2048
1.00
4.70
0.07
5.28
1.12
90.59
0
0
4
210
DENSE
8192
2048
4.00
3.50
0.02
7.33
2.09
85.70
0
0
0
212
DENSE
8192
2048
4.00
4.72
0.03
7.64
1.62
90.67
0
0
0
216
DENSE
6144
2048
3.00
4.91
0.03
7.17
1.46
91.11
0
0
0
217
DENSE
2048
2048
1.00
2.73
0.07
4.09
1.50
84.07
0
0
3
221
DENSE
8192
2048
4.00
3.49
0.01
7.45
2.14
85.50
0
0
0
223
DENSE
8192
2048
4.00
4.30
0.02
7.32
1.70
90.85
0
0
0
227
DENSE
6144
2048
3.00
4.46
0.05
6.19
1.39
89.88
0
0
0
228
DENSE
2048
2048
1.00
3.50
0.08
5.61
1.60
85.08
0
0
2
232
DENSE
8192
2048
4.00
3.25
0.01
6.97
2.15
84.47
0
0
0
234
DENSE
8192
2048
4.00
4.36
0.02
7.42
1.70
91.85
1
0
0
238
DENSE
6144
2048
3.00
4.00
0.02
5.77
1.44
90.66
0
0
0
239
DENSE
2048
2048
1.00
1.97
0.07
3.72
1.89
77.61
16
0
2
243
DENSE
8192
2048
4.00
3.08
0.01
6.86
2.23
83.29
0
0
0
245
DENSE
8192
2048
4.00
4.09
0.03
8.17
2.00
89.52
2
0
0
249
DENSE
6144
2048
3.00
3.73
0.02
6.11
1.64
88.25
0
0
0
250
DENSE
2048
2048
1.00
1.90
0.09
4.68
2.46
69.27
42
0
3
254
DENSE
8192
2048
4.00
3.12
0.01
7.10
2.28
83.42
0
0
0
256
DENSE
8192
2048
4.00
3.91
0.05
10.66
2.73
84.79
12
0
0
260
DENSE
6144
2048
3.00
3.42
0.02
6.01
1.76
87.15
0
0
0
261
DENSE
2048
2048
1.00
2.76
0.06
6.04
2.19
66.29
25
0
3
265
DENSE
8192
2048
4.00
3.35
0.01
7.25
2.16
84.78
0
0
0
267
DENSE
8192
2048
4.00
4.41
0.04
9.43
2.14
89.63
5
0
0
bigscience-bloom-1b7 Layer Plots
Layer 7
Layer=7 | N=6144 | M=2048 | Q=3.00 | alpha=3.77 | D_ks=0.02 | alpha-hat=5.82 | num traps=0
Layer 8
Layer=8 | N=2048 | M=2048 | Q=1.00 | alpha=3.48 | D_ks=0.04 | alpha-hat=5.44 | num traps=7
Layer 12
Layer=12 | N=8192 | M=2048 | Q=4.00 | alpha=4.64 | D_ks=0.04 | alpha-hat=9.51 | num traps=1
Layer 14
Layer=14 | N=8192 | M=2048 | Q=4.00 | alpha=3.84 | D_ks=0.02 | alpha-hat=7.49 | num traps=8
Layer 18
Layer=18 | N=6144 | M=2048 | Q=3.00 | alpha=4.09 | D_ks=0.02 | alpha-hat=6.29 | num traps=0
Layer 19
Layer=19 | N=2048 | M=2048 | Q=1.00 | alpha=3.87 | D_ks=0.04 | alpha-hat=5.66 | num traps=9
Layer 23
Layer=23 | N=8192 | M=2048 | Q=4.00 | alpha=3.37 | D_ks=0.06 | alpha-hat=7.23 | num traps=1
Layer 25
Layer=25 | N=8192 | M=2048 | Q=4.00 | alpha=4.50 | D_ks=0.01 | alpha-hat=8.35 | num traps=0
Layer 29
Layer=29 | N=6144 | M=2048 | Q=3.00 | alpha=3.94 | D_ks=0.02 | alpha-hat=6.04 | num traps=0
Layer 30
Layer=30 | N=2048 | M=2048 | Q=1.00 | alpha=3.74 | D_ks=0.03 | alpha-hat=5.16 | num traps=3
Layer 34
Layer=34 | N=8192 | M=2048 | Q=4.00 | alpha=3.31 | D_ks=0.04 | alpha-hat=6.82 | num traps=1
Layer 36
Layer=36 | N=8192 | M=2048 | Q=4.00 | alpha=4.21 | D_ks=0.04 | alpha-hat=7.42 | num traps=0
Layer 40
Layer=40 | N=6144 | M=2048 | Q=3.00 | alpha=4.26 | D_ks=0.03 | alpha-hat=6.46 | num traps=0
Layer 41
Layer=41 | N=2048 | M=2048 | Q=1.00 | alpha=4.22 | D_ks=0.03 | alpha-hat=6.08 | num traps=2
Layer 45
Layer=45 | N=8192 | M=2048 | Q=4.00 | alpha=3.48 | D_ks=0.01 | alpha-hat=6.58 | num traps=0
Layer 47
Layer=47 | N=8192 | M=2048 | Q=4.00 | alpha=4.17 | D_ks=0.03 | alpha-hat=7.31 | num traps=0
Layer 51
Layer=51 | N=6144 | M=2048 | Q=3.00 | alpha=4.11 | D_ks=0.02 | alpha-hat=6.44 | num traps=0
Layer 52
Layer=52 | N=2048 | M=2048 | Q=1.00 | alpha=4.01 | D_ks=0.03 | alpha-hat=5.69 | num traps=2
Layer 56
Layer=56 | N=8192 | M=2048 | Q=4.00 | alpha=3.28 | D_ks=0.01 | alpha-hat=7.11 | num traps=0
Layer 58
Layer=58 | N=8192 | M=2048 | Q=4.00 | alpha=3.76 | D_ks=0.02 | alpha-hat=8.20 | num traps=18
Layer 62
Layer=62 | N=6144 | M=2048 | Q=3.00 | alpha=4.12 | D_ks=0.03 | alpha-hat=6.29 | num traps=0
Layer 63
Layer=63 | N=2048 | M=2048 | Q=1.00 | alpha=4.42 | D_ks=0.05 | alpha-hat=5.90 | num traps=0
Layer 67
Layer=67 | N=8192 | M=2048 | Q=4.00 | alpha=3.27 | D_ks=0.01 | alpha-hat=7.16 | num traps=0
Layer 69
Layer=69 | N=8192 | M=2048 | Q=4.00 | alpha=3.51 | D_ks=0.02 | alpha-hat=7.24 | num traps=10
Layer 73
Layer=73 | N=6144 | M=2048 | Q=3.00 | alpha=3.82 | D_ks=0.04 | alpha-hat=5.50 | num traps=0
Layer 74
Layer=74 | N=2048 | M=2048 | Q=1.00 | alpha=3.69 | D_ks=0.09 | alpha-hat=4.30 | num traps=0
Layer 78
Layer=78 | N=8192 | M=2048 | Q=4.00 | alpha=3.30 | D_ks=0.01 | alpha-hat=6.29 | num traps=0
Layer 80
Layer=80 | N=8192 | M=2048 | Q=4.00 | alpha=3.58 | D_ks=0.02 | alpha-hat=6.11 | num traps=4
Layer 84
Layer=84 | N=6144 | M=2048 | Q=3.00 | alpha=2.94 | D_ks=0.05 | alpha-hat=4.32 | num traps=0
Layer 85
Layer=85 | N=2048 | M=2048 | Q=1.00 | alpha=5.28 | D_ks=0.02 | alpha-hat=3.81 | num traps=0
Layer 89
Layer=89 | N=8192 | M=2048 | Q=4.00 | alpha=3.24 | D_ks=0.02 | alpha-hat=6.23 | num traps=0
Layer 91
Layer=91 | N=8192 | M=2048 | Q=4.00 | alpha=3.75 | D_ks=0.02 | alpha-hat=5.45 | num traps=4
Layer 95
Layer=95 | N=6144 | M=2048 | Q=3.00 | alpha=4.42 | D_ks=0.06 | alpha-hat=6.73 | num traps=0
Layer 96
Layer=96 | N=2048 | M=2048 | Q=1.00 | alpha=3.17 | D_ks=0.11 | alpha-hat=1.98 | num traps=0
Layer 100
Layer=100 | N=8192 | M=2048 | Q=4.00 | alpha=3.38 | D_ks=0.02 | alpha-hat=6.57 | num traps=0
Layer 102
Layer=102 | N=8192 | M=2048 | Q=4.00 | alpha=4.18 | D_ks=0.01 | alpha-hat=6.74 | num traps=1
Layer 106
Layer=106 | N=6144 | M=2048 | Q=3.00 | alpha=2.10 | D_ks=0.09 | alpha-hat=3.06 | num traps=0
Layer 107
Layer=107 | N=2048 | M=2048 | Q=1.00 | alpha=4.04 | D_ks=0.06 | alpha-hat=2.59 | num traps=0
Layer 111
Layer=111 | N=8192 | M=2048 | Q=4.00 | alpha=4.07 | D_ks=0.03 | alpha-hat=7.84 | num traps=0
Layer 113
Layer=113 | N=8192 | M=2048 | Q=4.00 | alpha=4.20 | D_ks=0.04 | alpha-hat=5.60 | num traps=0
Layer 117
Layer=117 | N=6144 | M=2048 | Q=3.00 | alpha=4.06 | D_ks=0.08 | alpha-hat=5.99 | num traps=0
Layer 118
Layer=118 | N=2048 | M=2048 | Q=1.00 | alpha=3.90 | D_ks=0.10 | alpha-hat=2.36 | num traps=0
Layer 122
Layer=122 | N=8192 | M=2048 | Q=4.00 | alpha=3.65 | D_ks=0.04 | alpha-hat=6.72 | num traps=0
Layer 124
Layer=124 | N=8192 | M=2048 | Q=4.00 | alpha=4.12 | D_ks=0.04 | alpha-hat=5.08 | num traps=0
Layer 128
Layer=128 | N=6144 | M=2048 | Q=3.00 | alpha=2.96 | D_ks=0.07 | alpha-hat=4.27 | num traps=0
Layer 129
Layer=129 | N=2048 | M=2048 | Q=1.00 | alpha=3.80 | D_ks=0.07 | alpha-hat=2.32 | num traps=0
Layer 133
Layer=133 | N=8192 | M=2048 | Q=4.00 | alpha=3.41 | D_ks=0.03 | alpha-hat=6.74 | num traps=0
Layer 135
Layer=135 | N=8192 | M=2048 | Q=4.00 | alpha=3.93 | D_ks=0.05 | alpha-hat=4.89 | num traps=0
Layer 139
Layer=139 | N=6144 | M=2048 | Q=3.00 | alpha=3.44 | D_ks=0.06 | alpha-hat=4.83 | num traps=0
Layer 140
Layer=140 | N=2048 | M=2048 | Q=1.00 | alpha=3.50 | D_ks=0.06 | alpha-hat=2.53 | num traps=0
Layer 144
Layer=144 | N=8192 | M=2048 | Q=4.00 | alpha=3.57 | D_ks=0.03 | alpha-hat=7.03 | num traps=0
Layer 146
Layer=146 | N=8192 | M=2048 | Q=4.00 | alpha=3.89 | D_ks=0.05 | alpha-hat=4.65 | num traps=0
Layer 150
Layer=150 | N=6144 | M=2048 | Q=3.00 | alpha=3.15 | D_ks=0.06 | alpha-hat=4.76 | num traps=0
Layer 151
Layer=151 | N=2048 | M=2048 | Q=1.00 | alpha=3.13 | D_ks=0.09 | alpha-hat=2.30 | num traps=0
Layer 155
Layer=155 | N=8192 | M=2048 | Q=4.00 | alpha=3.30 | D_ks=0.04 | alpha-hat=6.62 | num traps=0
Layer 157
Layer=157 | N=8192 | M=2048 | Q=4.00 | alpha=4.04 | D_ks=0.05 | alpha-hat=4.95 | num traps=0
Layer 161
Layer=161 | N=6144 | M=2048 | Q=3.00 | alpha=3.37 | D_ks=0.06 | alpha-hat=4.92 | num traps=0
Layer 162
Layer=162 | N=2048 | M=2048 | Q=1.00 | alpha=3.26 | D_ks=0.10 | alpha-hat=2.08 | num traps=0
Layer 166
Layer=166 | N=8192 | M=2048 | Q=4.00 | alpha=3.24 | D_ks=0.03 | alpha-hat=6.53 | num traps=0
Layer 168
Layer=168 | N=8192 | M=2048 | Q=4.00 | alpha=3.96 | D_ks=0.05 | alpha-hat=4.95 | num traps=0
Layer 172
Layer=172 | N=6144 | M=2048 | Q=3.00 | alpha=3.18 | D_ks=0.06 | alpha-hat=4.72 | num traps=0
Layer 173
Layer=173 | N=2048 | M=2048 | Q=1.00 | alpha=2.89 | D_ks=0.10 | alpha-hat=1.96 | num traps=0
Layer 177
Layer=177 | N=8192 | M=2048 | Q=4.00 | alpha=3.32 | D_ks=0.03 | alpha-hat=6.72 | num traps=0
Layer 179
Layer=179 | N=8192 | M=2048 | Q=4.00 | alpha=3.82 | D_ks=0.04 | alpha-hat=5.57 | num traps=0
Layer 183
Layer=183 | N=6144 | M=2048 | Q=3.00 | alpha=4.06 | D_ks=0.07 | alpha-hat=5.63 | num traps=0
Layer 184
Layer=184 | N=2048 | M=2048 | Q=1.00 | alpha=3.66 | D_ks=0.06 | alpha-hat=2.89 | num traps=0
Layer 188
Layer=188 | N=8192 | M=2048 | Q=4.00 | alpha=3.41 | D_ks=0.02 | alpha-hat=6.91 | num traps=0
Layer 190
Layer=190 | N=8192 | M=2048 | Q=4.00 | alpha=3.97 | D_ks=0.03 | alpha-hat=5.85 | num traps=0
Layer 194
Layer=194 | N=6144 | M=2048 | Q=3.00 | alpha=3.54 | D_ks=0.06 | alpha-hat=5.46 | num traps=0
Layer 195
Layer=195 | N=2048 | M=2048 | Q=1.00 | alpha=6.58 | D_ks=0.03 | alpha-hat=6.90 | num traps=0
Layer 199
Layer=199 | N=8192 | M=2048 | Q=4.00 | alpha=3.50 | D_ks=0.02 | alpha-hat=7.24 | num traps=0
Layer 201
Layer=201 | N=8192 | M=2048 | Q=4.00 | alpha=4.65 | D_ks=0.03 | alpha-hat=7.09 | num traps=0
Layer 205
Layer=205 | N=6144 | M=2048 | Q=3.00 | alpha=4.02 | D_ks=0.06 | alpha-hat=5.82 | num traps=0
Layer 206
Layer=206 | N=2048 | M=2048 | Q=1.00 | alpha=4.70 | D_ks=0.07 | alpha-hat=5.28 | num traps=0
Layer 210
Layer=210 | N=8192 | M=2048 | Q=4.00 | alpha=3.50 | D_ks=0.02 | alpha-hat=7.33 | num traps=0
Layer 212
Layer=212 | N=8192 | M=2048 | Q=4.00 | alpha=4.72 | D_ks=0.03 | alpha-hat=7.64 | num traps=0
Layer 216
Layer=216 | N=6144 | M=2048 | Q=3.00 | alpha=4.91 | D_ks=0.03 | alpha-hat=7.17 | num traps=0
Layer 217
Layer=217 | N=2048 | M=2048 | Q=1.00 | alpha=2.73 | D_ks=0.07 | alpha-hat=4.09 | num traps=0
Layer 221
Layer=221 | N=8192 | M=2048 | Q=4.00 | alpha=3.49 | D_ks=0.01 | alpha-hat=7.45 | num traps=0
Layer 223
Layer=223 | N=8192 | M=2048 | Q=4.00 | alpha=4.30 | D_ks=0.02 | alpha-hat=7.32 | num traps=0
Layer 227
Layer=227 | N=6144 | M=2048 | Q=3.00 | alpha=4.46 | D_ks=0.05 | alpha-hat=6.19 | num traps=0
Layer 228
Layer=228 | N=2048 | M=2048 | Q=1.00 | alpha=3.50 | D_ks=0.08 | alpha-hat=5.61 | num traps=0
Layer 232
Layer=232 | N=8192 | M=2048 | Q=4.00 | alpha=3.25 | D_ks=0.01 | alpha-hat=6.97 | num traps=0
Layer 234
Layer=234 | N=8192 | M=2048 | Q=4.00 | alpha=4.36 | D_ks=0.02 | alpha-hat=7.42 | num traps=1
Layer 238
Layer=238 | N=6144 | M=2048 | Q=3.00 | alpha=4.00 | D_ks=0.02 | alpha-hat=5.77 | num traps=0
Layer 239
Layer=239 | N=2048 | M=2048 | Q=1.00 | alpha=1.97 | D_ks=0.07 | alpha-hat=3.72 | num traps=16
Layer 243
Layer=243 | N=8192 | M=2048 | Q=4.00 | alpha=3.08 | D_ks=0.01 | alpha-hat=6.86 | num traps=0
Layer 245
Layer=245 | N=8192 | M=2048 | Q=4.00 | alpha=4.09 | D_ks=0.03 | alpha-hat=8.17 | num traps=2
Layer 249
Layer=249 | N=6144 | M=2048 | Q=3.00 | alpha=3.73 | D_ks=0.02 | alpha-hat=6.11 | num traps=0
Layer 250
Layer=250 | N=2048 | M=2048 | Q=1.00 | alpha=1.90 | D_ks=0.09 | alpha-hat=4.68 | num traps=42
Layer 254
Layer=254 | N=8192 | M=2048 | Q=4.00 | alpha=3.12 | D_ks=0.01 | alpha-hat=7.10 | num traps=0
Layer 256
Layer=256 | N=8192 | M=2048 | Q=4.00 | alpha=3.91 | D_ks=0.05 | alpha-hat=10.66 | num traps=12
Layer 260
Layer=260 | N=6144 | M=2048 | Q=3.00 | alpha=3.42 | D_ks=0.02 | alpha-hat=6.01 | num traps=0
Layer 261
Layer=261 | N=2048 | M=2048 | Q=1.00 | alpha=2.76 | D_ks=0.06 | alpha-hat=6.04 | num traps=25
Layer 265
Layer=265 | N=8192 | M=2048 | Q=4.00 | alpha=3.35 | D_ks=0.01 | alpha-hat=7.25 | num traps=0
Layer 267
Layer=267 | N=8192 | M=2048 | Q=4.00 | alpha=4.41 | D_ks=0.04 | alpha-hat=9.43 | num traps=5
bigscience-bloom-1b7 Model Summary Plots
bigscience-bloom-1b7 Model Selected Details
| layer_type | N | M | Q | alpha | D | alpha-hat | log_SN | % Rand | num_traps | num_fingers | rank_loss | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| layer_id | ||||||||||||
| 7 | DENSE | 6144 | 2048 | 3.00 | 3.77 | 0.02 | 5.82 | 1.55 | 86.32 | 0 | 0 | 0 |
| 8 | DENSE | 2048 | 2048 | 1.00 | 3.48 | 0.04 | 5.44 | 1.56 | 81.65 | 7 | 0 | 4 |
| 12 | DENSE | 8192 | 2048 | 4.00 | 4.64 | 0.04 | 9.51 | 2.05 | 86.11 | 1 | 0 | 0 |
| 14 | DENSE | 8192 | 2048 | 4.00 | 3.84 | 0.02 | 7.49 | 1.95 | 83.45 | 8 | 0 | 0 |
| 18 | DENSE | 6144 | 2048 | 3.00 | 4.09 | 0.02 | 6.29 | 1.54 | 87.85 | 0 | 0 | 0 |
| 19 | DENSE | 2048 | 2048 | 1.00 | 3.87 | 0.04 | 5.66 | 1.46 | 85.01 | 9 | 0 | 3 |
| 23 | DENSE | 8192 | 2048 | 4.00 | 3.37 | 0.06 | 7.23 | 2.14 | 85.07 | 1 | 0 | 0 |
| 25 | DENSE | 8192 | 2048 | 4.00 | 4.50 | 0.01 | 8.35 | 1.85 | 90.06 | 0 | 0 | 0 |
| 29 | DENSE | 6144 | 2048 | 3.00 | 3.94 | 0.02 | 6.04 | 1.53 | 87.98 | 0 | 0 | 0 |
| 30 | DENSE | 2048 | 2048 | 1.00 | 3.74 | 0.03 | 5.16 | 1.38 | 85.90 | 3 | 0 | 3 |
| 34 | DENSE | 8192 | 2048 | 4.00 | 3.31 | 0.04 | 6.82 | 2.06 | 85.38 | 1 | 0 | 0 |
| 36 | DENSE | 8192 | 2048 | 4.00 | 4.21 | 0.04 | 7.42 | 1.76 | 88.06 | 0 | 0 | 0 |
| 40 | DENSE | 6144 | 2048 | 3.00 | 4.26 | 0.03 | 6.46 | 1.52 | 87.97 | 0 | 0 | 0 |
| 41 | DENSE | 2048 | 2048 | 1.00 | 4.22 | 0.03 | 6.08 | 1.44 | 87.53 | 2 | 0 | 3 |
| 45 | DENSE | 8192 | 2048 | 4.00 | 3.48 | 0.01 | 6.58 | 1.89 | 84.71 | 0 | 0 | 0 |
| 47 | DENSE | 8192 | 2048 | 4.00 | 4.17 | 0.03 | 7.31 | 1.75 | 86.26 | 0 | 0 | 0 |
| 51 | DENSE | 6144 | 2048 | 3.00 | 4.11 | 0.02 | 6.44 | 1.57 | 86.84 | 0 | 0 | 0 |
| 52 | DENSE | 2048 | 2048 | 1.00 | 4.01 | 0.03 | 5.69 | 1.42 | 86.59 | 2 | 0 | 3 |
| 56 | DENSE | 8192 | 2048 | 4.00 | 3.28 | 0.01 | 7.11 | 2.17 | 80.85 | 0 | 0 | 0 |
| 58 | DENSE | 8192 | 2048 | 4.00 | 3.76 | 0.02 | 8.20 | 2.18 | 84.02 | 18 | 0 | 0 |
| 62 | DENSE | 6144 | 2048 | 3.00 | 4.12 | 0.03 | 6.29 | 1.53 | 87.85 | 0 | 0 | 0 |
| 63 | DENSE | 2048 | 2048 | 1.00 | 4.42 | 0.05 | 5.90 | 1.34 | 87.21 | 0 | 0 | 2 |
| 67 | DENSE | 8192 | 2048 | 4.00 | 3.27 | 0.01 | 7.16 | 2.19 | 80.46 | 0 | 0 | 0 |
| 69 | DENSE | 8192 | 2048 | 4.00 | 3.51 | 0.02 | 7.24 | 2.06 | 84.35 | 10 | 0 | 0 |
| 73 | DENSE | 6144 | 2048 | 3.00 | 3.82 | 0.04 | 5.50 | 1.44 | 87.30 | 0 | 0 | 0 |
| 74 | DENSE | 2048 | 2048 | 1.00 | 3.69 | 0.09 | 4.30 | 1.16 | 87.70 | 0 | 2 | 4 |
| 78 | DENSE | 8192 | 2048 | 4.00 | 3.30 | 0.01 | 6.29 | 1.91 | 83.15 | 0 | 0 | 0 |
| 80 | DENSE | 8192 | 2048 | 4.00 | 3.58 | 0.02 | 6.11 | 1.71 | 86.73 | 4 | 0 | 0 |
| 84 | DENSE | 6144 | 2048 | 3.00 | 2.94 | 0.05 | 4.32 | 1.47 | 85.47 | 0 | 0 | 0 |
| 85 | DENSE | 2048 | 2048 | 1.00 | 5.28 | 0.02 | 3.81 | 0.72 | 92.69 | 0 | 0 | 4 |
| 89 | DENSE | 8192 | 2048 | 4.00 | 3.24 | 0.02 | 6.23 | 1.92 | 84.03 | 0 | 0 | 0 |
| 91 | DENSE | 8192 | 2048 | 4.00 | 3.75 | 0.02 | 5.45 | 1.46 | 88.66 | 4 | 0 | 0 |
| 95 | DENSE | 6144 | 2048 | 3.00 | 4.42 | 0.06 | 6.73 | 1.52 | 82.17 | 0 | 0 | 0 |
| 96 | DENSE | 2048 | 2048 | 1.00 | 3.17 | 0.11 | 1.98 | 0.62 | 90.09 | 0 | 6 | 4 |
| 100 | DENSE | 8192 | 2048 | 4.00 | 3.38 | 0.02 | 6.57 | 1.95 | 86.15 | 0 | 0 | 0 |
| 102 | DENSE | 8192 | 2048 | 4.00 | 4.18 | 0.01 | 6.74 | 1.61 | 88.44 | 1 | 0 | 0 |
| 106 | DENSE | 6144 | 2048 | 3.00 | 2.10 | 0.09 | 3.06 | 1.46 | 78.92 | 0 | 3 | 0 |
| 107 | DENSE | 2048 | 2048 | 1.00 | 4.04 | 0.06 | 2.59 | 0.64 | 86.32 | 0 | 0 | 3 |
| 111 | DENSE | 8192 | 2048 | 4.00 | 4.07 | 0.03 | 7.84 | 1.93 | 86.65 | 0 | 0 | 0 |
| 113 | DENSE | 8192 | 2048 | 4.00 | 4.20 | 0.04 | 5.60 | 1.33 | 89.73 | 0 | 0 | 0 |
| 117 | DENSE | 6144 | 2048 | 3.00 | 4.06 | 0.08 | 5.99 | 1.48 | 82.46 | 0 | 1 | 0 |
| 118 | DENSE | 2048 | 2048 | 1.00 | 3.90 | 0.10 | 2.36 | 0.60 | 91.73 | 0 | 1 | 3 |
| 122 | DENSE | 8192 | 2048 | 4.00 | 3.65 | 0.04 | 6.72 | 1.84 | 84.02 | 0 | 0 | 0 |
| 124 | DENSE | 8192 | 2048 | 4.00 | 4.12 | 0.04 | 5.08 | 1.23 | 89.65 | 0 | 0 | 0 |
| 128 | DENSE | 6144 | 2048 | 3.00 | 2.96 | 0.07 | 4.27 | 1.44 | 80.79 | 0 | 0 | 0 |
| 129 | DENSE | 2048 | 2048 | 1.00 | 3.80 | 0.07 | 2.32 | 0.61 | 87.91 | 0 | 0 | 4 |
| 133 | DENSE | 8192 | 2048 | 4.00 | 3.41 | 0.03 | 6.74 | 1.98 | 84.36 | 0 | 0 | 0 |
| 135 | DENSE | 8192 | 2048 | 4.00 | 3.93 | 0.05 | 4.89 | 1.24 | 90.21 | 0 | 0 | 0 |
| 139 | DENSE | 6144 | 2048 | 3.00 | 3.44 | 0.06 | 4.83 | 1.41 | 84.36 | 0 | 0 | 0 |
| 140 | DENSE | 2048 | 2048 | 1.00 | 3.50 | 0.06 | 2.53 | 0.72 | 88.35 | 0 | 0 | 3 |
| 144 | DENSE | 8192 | 2048 | 4.00 | 3.57 | 0.03 | 7.03 | 1.97 | 84.53 | 0 | 0 | 0 |
| 146 | DENSE | 8192 | 2048 | 4.00 | 3.89 | 0.05 | 4.65 | 1.20 | 89.68 | 0 | 0 | 0 |
| 150 | DENSE | 6144 | 2048 | 3.00 | 3.15 | 0.06 | 4.76 | 1.51 | 83.24 | 0 | 0 | 0 |
| 151 | DENSE | 2048 | 2048 | 1.00 | 3.13 | 0.09 | 2.30 | 0.74 | 87.19 | 0 | 4 | 4 |
| 155 | DENSE | 8192 | 2048 | 4.00 | 3.30 | 0.04 | 6.62 | 2.00 | 84.21 | 0 | 0 | 0 |
| 157 | DENSE | 8192 | 2048 | 4.00 | 4.04 | 0.05 | 4.95 | 1.22 | 89.37 | 0 | 0 | 0 |
| 161 | DENSE | 6144 | 2048 | 3.00 | 3.37 | 0.06 | 4.92 | 1.46 | 84.50 | 0 | 0 | 0 |
| 162 | DENSE | 2048 | 2048 | 1.00 | 3.26 | 0.10 | 2.08 | 0.64 | 89.25 | 0 | 2 | 4 |
| 166 | DENSE | 8192 | 2048 | 4.00 | 3.24 | 0.03 | 6.53 | 2.02 | 83.76 | 0 | 0 | 0 |
| 168 | DENSE | 8192 | 2048 | 4.00 | 3.96 | 0.05 | 4.95 | 1.25 | 89.06 | 0 | 0 | 0 |
| 172 | DENSE | 6144 | 2048 | 3.00 | 3.18 | 0.06 | 4.72 | 1.49 | 84.18 | 0 | 1 | 0 |
| 173 | DENSE | 2048 | 2048 | 1.00 | 2.89 | 0.10 | 1.96 | 0.68 | 85.42 | 0 | 1 | 4 |
| 177 | DENSE | 8192 | 2048 | 4.00 | 3.32 | 0.03 | 6.72 | 2.02 | 84.52 | 0 | 0 | 0 |
| 179 | DENSE | 8192 | 2048 | 4.00 | 3.82 | 0.04 | 5.57 | 1.46 | 88.87 | 0 | 0 | 0 |
| 183 | DENSE | 6144 | 2048 | 3.00 | 4.06 | 0.07 | 5.63 | 1.39 | 86.70 | 0 | 1 | 0 |
| 184 | DENSE | 2048 | 2048 | 1.00 | 3.66 | 0.06 | 2.89 | 0.79 | 84.91 | 0 | 0 | 4 |
| 188 | DENSE | 8192 | 2048 | 4.00 | 3.41 | 0.02 | 6.91 | 2.03 | 85.69 | 0 | 0 | 0 |
| 190 | DENSE | 8192 | 2048 | 4.00 | 3.97 | 0.03 | 5.85 | 1.47 | 89.64 | 0 | 0 | 0 |
| 194 | DENSE | 6144 | 2048 | 3.00 | 3.54 | 0.06 | 5.46 | 1.54 | 88.45 | 0 | 1 | 0 |
| 195 | DENSE | 2048 | 2048 | 1.00 | 6.58 | 0.03 | 6.90 | 1.05 | 92.60 | 0 | 0 | 2 |
| 199 | DENSE | 8192 | 2048 | 4.00 | 3.50 | 0.02 | 7.24 | 2.07 | 85.87 | 0 | 0 | 0 |
| 201 | DENSE | 8192 | 2048 | 4.00 | 4.65 | 0.03 | 7.09 | 1.53 | 90.04 | 0 | 0 | 0 |
| 205 | DENSE | 6144 | 2048 | 3.00 | 4.02 | 0.06 | 5.82 | 1.45 | 90.83 | 0 | 2 | 0 |
| 206 | DENSE | 2048 | 2048 | 1.00 | 4.70 | 0.07 | 5.28 | 1.12 | 90.59 | 0 | 0 | 4 |
| 210 | DENSE | 8192 | 2048 | 4.00 | 3.50 | 0.02 | 7.33 | 2.09 | 85.70 | 0 | 0 | 0 |
| 212 | DENSE | 8192 | 2048 | 4.00 | 4.72 | 0.03 | 7.64 | 1.62 | 90.67 | 0 | 0 | 0 |
| 216 | DENSE | 6144 | 2048 | 3.00 | 4.91 | 0.03 | 7.17 | 1.46 | 91.11 | 0 | 0 | 0 |
| 217 | DENSE | 2048 | 2048 | 1.00 | 2.73 | 0.07 | 4.09 | 1.50 | 84.07 | 0 | 0 | 3 |
| 221 | DENSE | 8192 | 2048 | 4.00 | 3.49 | 0.01 | 7.45 | 2.14 | 85.50 | 0 | 0 | 0 |
| 223 | DENSE | 8192 | 2048 | 4.00 | 4.30 | 0.02 | 7.32 | 1.70 | 90.85 | 0 | 0 | 0 |
| 227 | DENSE | 6144 | 2048 | 3.00 | 4.46 | 0.05 | 6.19 | 1.39 | 89.88 | 0 | 0 | 0 |
| 228 | DENSE | 2048 | 2048 | 1.00 | 3.50 | 0.08 | 5.61 | 1.60 | 85.08 | 0 | 0 | 2 |
| 232 | DENSE | 8192 | 2048 | 4.00 | 3.25 | 0.01 | 6.97 | 2.15 | 84.47 | 0 | 0 | 0 |
| 234 | DENSE | 8192 | 2048 | 4.00 | 4.36 | 0.02 | 7.42 | 1.70 | 91.85 | 1 | 0 | 0 |
| 238 | DENSE | 6144 | 2048 | 3.00 | 4.00 | 0.02 | 5.77 | 1.44 | 90.66 | 0 | 0 | 0 |
| 239 | DENSE | 2048 | 2048 | 1.00 | 1.97 | 0.07 | 3.72 | 1.89 | 77.61 | 16 | 0 | 2 |
| 243 | DENSE | 8192 | 2048 | 4.00 | 3.08 | 0.01 | 6.86 | 2.23 | 83.29 | 0 | 0 | 0 |
| 245 | DENSE | 8192 | 2048 | 4.00 | 4.09 | 0.03 | 8.17 | 2.00 | 89.52 | 2 | 0 | 0 |
| 249 | DENSE | 6144 | 2048 | 3.00 | 3.73 | 0.02 | 6.11 | 1.64 | 88.25 | 0 | 0 | 0 |
| 250 | DENSE | 2048 | 2048 | 1.00 | 1.90 | 0.09 | 4.68 | 2.46 | 69.27 | 42 | 0 | 3 |
| 254 | DENSE | 8192 | 2048 | 4.00 | 3.12 | 0.01 | 7.10 | 2.28 | 83.42 | 0 | 0 | 0 |
| 256 | DENSE | 8192 | 2048 | 4.00 | 3.91 | 0.05 | 10.66 | 2.73 | 84.79 | 12 | 0 | 0 |
| 260 | DENSE | 6144 | 2048 | 3.00 | 3.42 | 0.02 | 6.01 | 1.76 | 87.15 | 0 | 0 | 0 |
| 261 | DENSE | 2048 | 2048 | 1.00 | 2.76 | 0.06 | 6.04 | 2.19 | 66.29 | 25 | 0 | 3 |
| 265 | DENSE | 8192 | 2048 | 4.00 | 3.35 | 0.01 | 7.25 | 2.16 | 84.78 | 0 | 0 | 0 |
| 267 | DENSE | 8192 | 2048 | 4.00 | 4.41 | 0.04 | 9.43 | 2.14 | 89.63 | 5 | 0 | 0 |
bigscience-bloom-1b7 Layer Plots
Layer 7
Layer=7 | N=6144 | M=2048 | Q=3.00 | alpha=3.77 | D_ks=0.02 | alpha-hat=5.82 | num traps=0
Layer 8
Layer=8 | N=2048 | M=2048 | Q=1.00 | alpha=3.48 | D_ks=0.04 | alpha-hat=5.44 | num traps=7
Layer 12
Layer=12 | N=8192 | M=2048 | Q=4.00 | alpha=4.64 | D_ks=0.04 | alpha-hat=9.51 | num traps=1
Layer 14
Layer=14 | N=8192 | M=2048 | Q=4.00 | alpha=3.84 | D_ks=0.02 | alpha-hat=7.49 | num traps=8
Layer 18
Layer=18 | N=6144 | M=2048 | Q=3.00 | alpha=4.09 | D_ks=0.02 | alpha-hat=6.29 | num traps=0
Layer 19
Layer=19 | N=2048 | M=2048 | Q=1.00 | alpha=3.87 | D_ks=0.04 | alpha-hat=5.66 | num traps=9
Layer 23
Layer=23 | N=8192 | M=2048 | Q=4.00 | alpha=3.37 | D_ks=0.06 | alpha-hat=7.23 | num traps=1
Layer 25
Layer=25 | N=8192 | M=2048 | Q=4.00 | alpha=4.50 | D_ks=0.01 | alpha-hat=8.35 | num traps=0
Layer 29
Layer=29 | N=6144 | M=2048 | Q=3.00 | alpha=3.94 | D_ks=0.02 | alpha-hat=6.04 | num traps=0
Layer 30
Layer=30 | N=2048 | M=2048 | Q=1.00 | alpha=3.74 | D_ks=0.03 | alpha-hat=5.16 | num traps=3
Layer 34
Layer=34 | N=8192 | M=2048 | Q=4.00 | alpha=3.31 | D_ks=0.04 | alpha-hat=6.82 | num traps=1
Layer 36
Layer=36 | N=8192 | M=2048 | Q=4.00 | alpha=4.21 | D_ks=0.04 | alpha-hat=7.42 | num traps=0
Layer 40
Layer=40 | N=6144 | M=2048 | Q=3.00 | alpha=4.26 | D_ks=0.03 | alpha-hat=6.46 | num traps=0
Layer 41
Layer=41 | N=2048 | M=2048 | Q=1.00 | alpha=4.22 | D_ks=0.03 | alpha-hat=6.08 | num traps=2
Layer 45
Layer=45 | N=8192 | M=2048 | Q=4.00 | alpha=3.48 | D_ks=0.01 | alpha-hat=6.58 | num traps=0
Layer 47
Layer=47 | N=8192 | M=2048 | Q=4.00 | alpha=4.17 | D_ks=0.03 | alpha-hat=7.31 | num traps=0
Layer 51
Layer=51 | N=6144 | M=2048 | Q=3.00 | alpha=4.11 | D_ks=0.02 | alpha-hat=6.44 | num traps=0
Layer 52
Layer=52 | N=2048 | M=2048 | Q=1.00 | alpha=4.01 | D_ks=0.03 | alpha-hat=5.69 | num traps=2
Layer 56
Layer=56 | N=8192 | M=2048 | Q=4.00 | alpha=3.28 | D_ks=0.01 | alpha-hat=7.11 | num traps=0
Layer 58
Layer=58 | N=8192 | M=2048 | Q=4.00 | alpha=3.76 | D_ks=0.02 | alpha-hat=8.20 | num traps=18
Layer 62
Layer=62 | N=6144 | M=2048 | Q=3.00 | alpha=4.12 | D_ks=0.03 | alpha-hat=6.29 | num traps=0
Layer 63
Layer=63 | N=2048 | M=2048 | Q=1.00 | alpha=4.42 | D_ks=0.05 | alpha-hat=5.90 | num traps=0
Layer 67
Layer=67 | N=8192 | M=2048 | Q=4.00 | alpha=3.27 | D_ks=0.01 | alpha-hat=7.16 | num traps=0
Layer 69
Layer=69 | N=8192 | M=2048 | Q=4.00 | alpha=3.51 | D_ks=0.02 | alpha-hat=7.24 | num traps=10
Layer 73
Layer=73 | N=6144 | M=2048 | Q=3.00 | alpha=3.82 | D_ks=0.04 | alpha-hat=5.50 | num traps=0
Layer 74
Layer=74 | N=2048 | M=2048 | Q=1.00 | alpha=3.69 | D_ks=0.09 | alpha-hat=4.30 | num traps=0
Layer 78
Layer=78 | N=8192 | M=2048 | Q=4.00 | alpha=3.30 | D_ks=0.01 | alpha-hat=6.29 | num traps=0
Layer 80
Layer=80 | N=8192 | M=2048 | Q=4.00 | alpha=3.58 | D_ks=0.02 | alpha-hat=6.11 | num traps=4
Layer 84
Layer=84 | N=6144 | M=2048 | Q=3.00 | alpha=2.94 | D_ks=0.05 | alpha-hat=4.32 | num traps=0
Layer 85
Layer=85 | N=2048 | M=2048 | Q=1.00 | alpha=5.28 | D_ks=0.02 | alpha-hat=3.81 | num traps=0
Layer 89
Layer=89 | N=8192 | M=2048 | Q=4.00 | alpha=3.24 | D_ks=0.02 | alpha-hat=6.23 | num traps=0
Layer 91
Layer=91 | N=8192 | M=2048 | Q=4.00 | alpha=3.75 | D_ks=0.02 | alpha-hat=5.45 | num traps=4
Layer 95
Layer=95 | N=6144 | M=2048 | Q=3.00 | alpha=4.42 | D_ks=0.06 | alpha-hat=6.73 | num traps=0
Layer 96
Layer=96 | N=2048 | M=2048 | Q=1.00 | alpha=3.17 | D_ks=0.11 | alpha-hat=1.98 | num traps=0
Layer 100
Layer=100 | N=8192 | M=2048 | Q=4.00 | alpha=3.38 | D_ks=0.02 | alpha-hat=6.57 | num traps=0
Layer 102
Layer=102 | N=8192 | M=2048 | Q=4.00 | alpha=4.18 | D_ks=0.01 | alpha-hat=6.74 | num traps=1
Layer 106
Layer=106 | N=6144 | M=2048 | Q=3.00 | alpha=2.10 | D_ks=0.09 | alpha-hat=3.06 | num traps=0
Layer 107
Layer=107 | N=2048 | M=2048 | Q=1.00 | alpha=4.04 | D_ks=0.06 | alpha-hat=2.59 | num traps=0
Layer 111
Layer=111 | N=8192 | M=2048 | Q=4.00 | alpha=4.07 | D_ks=0.03 | alpha-hat=7.84 | num traps=0
Layer 113
Layer=113 | N=8192 | M=2048 | Q=4.00 | alpha=4.20 | D_ks=0.04 | alpha-hat=5.60 | num traps=0
Layer 117
Layer=117 | N=6144 | M=2048 | Q=3.00 | alpha=4.06 | D_ks=0.08 | alpha-hat=5.99 | num traps=0
Layer 118
Layer=118 | N=2048 | M=2048 | Q=1.00 | alpha=3.90 | D_ks=0.10 | alpha-hat=2.36 | num traps=0
Layer 122
Layer=122 | N=8192 | M=2048 | Q=4.00 | alpha=3.65 | D_ks=0.04 | alpha-hat=6.72 | num traps=0
Layer 124
Layer=124 | N=8192 | M=2048 | Q=4.00 | alpha=4.12 | D_ks=0.04 | alpha-hat=5.08 | num traps=0
Layer 128
Layer=128 | N=6144 | M=2048 | Q=3.00 | alpha=2.96 | D_ks=0.07 | alpha-hat=4.27 | num traps=0
Layer 129
Layer=129 | N=2048 | M=2048 | Q=1.00 | alpha=3.80 | D_ks=0.07 | alpha-hat=2.32 | num traps=0
Layer 133
Layer=133 | N=8192 | M=2048 | Q=4.00 | alpha=3.41 | D_ks=0.03 | alpha-hat=6.74 | num traps=0
Layer 135
Layer=135 | N=8192 | M=2048 | Q=4.00 | alpha=3.93 | D_ks=0.05 | alpha-hat=4.89 | num traps=0
Layer 139
Layer=139 | N=6144 | M=2048 | Q=3.00 | alpha=3.44 | D_ks=0.06 | alpha-hat=4.83 | num traps=0
Layer 140
Layer=140 | N=2048 | M=2048 | Q=1.00 | alpha=3.50 | D_ks=0.06 | alpha-hat=2.53 | num traps=0
Layer 144
Layer=144 | N=8192 | M=2048 | Q=4.00 | alpha=3.57 | D_ks=0.03 | alpha-hat=7.03 | num traps=0
Layer 146
Layer=146 | N=8192 | M=2048 | Q=4.00 | alpha=3.89 | D_ks=0.05 | alpha-hat=4.65 | num traps=0
Layer 150
Layer=150 | N=6144 | M=2048 | Q=3.00 | alpha=3.15 | D_ks=0.06 | alpha-hat=4.76 | num traps=0
Layer 151
Layer=151 | N=2048 | M=2048 | Q=1.00 | alpha=3.13 | D_ks=0.09 | alpha-hat=2.30 | num traps=0
Layer 155
Layer=155 | N=8192 | M=2048 | Q=4.00 | alpha=3.30 | D_ks=0.04 | alpha-hat=6.62 | num traps=0
Layer 157
Layer=157 | N=8192 | M=2048 | Q=4.00 | alpha=4.04 | D_ks=0.05 | alpha-hat=4.95 | num traps=0
Layer 161
Layer=161 | N=6144 | M=2048 | Q=3.00 | alpha=3.37 | D_ks=0.06 | alpha-hat=4.92 | num traps=0
Layer 162
Layer=162 | N=2048 | M=2048 | Q=1.00 | alpha=3.26 | D_ks=0.10 | alpha-hat=2.08 | num traps=0
Layer 166
Layer=166 | N=8192 | M=2048 | Q=4.00 | alpha=3.24 | D_ks=0.03 | alpha-hat=6.53 | num traps=0
Layer 168
Layer=168 | N=8192 | M=2048 | Q=4.00 | alpha=3.96 | D_ks=0.05 | alpha-hat=4.95 | num traps=0
Layer 172
Layer=172 | N=6144 | M=2048 | Q=3.00 | alpha=3.18 | D_ks=0.06 | alpha-hat=4.72 | num traps=0
Layer 173
Layer=173 | N=2048 | M=2048 | Q=1.00 | alpha=2.89 | D_ks=0.10 | alpha-hat=1.96 | num traps=0
Layer 177
Layer=177 | N=8192 | M=2048 | Q=4.00 | alpha=3.32 | D_ks=0.03 | alpha-hat=6.72 | num traps=0
Layer 179
Layer=179 | N=8192 | M=2048 | Q=4.00 | alpha=3.82 | D_ks=0.04 | alpha-hat=5.57 | num traps=0
Layer 183
Layer=183 | N=6144 | M=2048 | Q=3.00 | alpha=4.06 | D_ks=0.07 | alpha-hat=5.63 | num traps=0
Layer 184
Layer=184 | N=2048 | M=2048 | Q=1.00 | alpha=3.66 | D_ks=0.06 | alpha-hat=2.89 | num traps=0
Layer 188
Layer=188 | N=8192 | M=2048 | Q=4.00 | alpha=3.41 | D_ks=0.02 | alpha-hat=6.91 | num traps=0
Layer 190
Layer=190 | N=8192 | M=2048 | Q=4.00 | alpha=3.97 | D_ks=0.03 | alpha-hat=5.85 | num traps=0
Layer 194
Layer=194 | N=6144 | M=2048 | Q=3.00 | alpha=3.54 | D_ks=0.06 | alpha-hat=5.46 | num traps=0
Layer 195
Layer=195 | N=2048 | M=2048 | Q=1.00 | alpha=6.58 | D_ks=0.03 | alpha-hat=6.90 | num traps=0
Layer 199
Layer=199 | N=8192 | M=2048 | Q=4.00 | alpha=3.50 | D_ks=0.02 | alpha-hat=7.24 | num traps=0
Layer 201
Layer=201 | N=8192 | M=2048 | Q=4.00 | alpha=4.65 | D_ks=0.03 | alpha-hat=7.09 | num traps=0
Layer 205
Layer=205 | N=6144 | M=2048 | Q=3.00 | alpha=4.02 | D_ks=0.06 | alpha-hat=5.82 | num traps=0
Layer 206
Layer=206 | N=2048 | M=2048 | Q=1.00 | alpha=4.70 | D_ks=0.07 | alpha-hat=5.28 | num traps=0
Layer 210
Layer=210 | N=8192 | M=2048 | Q=4.00 | alpha=3.50 | D_ks=0.02 | alpha-hat=7.33 | num traps=0
Layer 212
Layer=212 | N=8192 | M=2048 | Q=4.00 | alpha=4.72 | D_ks=0.03 | alpha-hat=7.64 | num traps=0
Layer 216
Layer=216 | N=6144 | M=2048 | Q=3.00 | alpha=4.91 | D_ks=0.03 | alpha-hat=7.17 | num traps=0
Layer 217
Layer=217 | N=2048 | M=2048 | Q=1.00 | alpha=2.73 | D_ks=0.07 | alpha-hat=4.09 | num traps=0
Layer 221
Layer=221 | N=8192 | M=2048 | Q=4.00 | alpha=3.49 | D_ks=0.01 | alpha-hat=7.45 | num traps=0
Layer 223
Layer=223 | N=8192 | M=2048 | Q=4.00 | alpha=4.30 | D_ks=0.02 | alpha-hat=7.32 | num traps=0
Layer 227
Layer=227 | N=6144 | M=2048 | Q=3.00 | alpha=4.46 | D_ks=0.05 | alpha-hat=6.19 | num traps=0
Layer 228
Layer=228 | N=2048 | M=2048 | Q=1.00 | alpha=3.50 | D_ks=0.08 | alpha-hat=5.61 | num traps=0
Layer 232
Layer=232 | N=8192 | M=2048 | Q=4.00 | alpha=3.25 | D_ks=0.01 | alpha-hat=6.97 | num traps=0
Layer 234
Layer=234 | N=8192 | M=2048 | Q=4.00 | alpha=4.36 | D_ks=0.02 | alpha-hat=7.42 | num traps=1
Layer 238
Layer=238 | N=6144 | M=2048 | Q=3.00 | alpha=4.00 | D_ks=0.02 | alpha-hat=5.77 | num traps=0
Layer 239
Layer=239 | N=2048 | M=2048 | Q=1.00 | alpha=1.97 | D_ks=0.07 | alpha-hat=3.72 | num traps=16
Layer 243
Layer=243 | N=8192 | M=2048 | Q=4.00 | alpha=3.08 | D_ks=0.01 | alpha-hat=6.86 | num traps=0
Layer 245
Layer=245 | N=8192 | M=2048 | Q=4.00 | alpha=4.09 | D_ks=0.03 | alpha-hat=8.17 | num traps=2
Layer 249
Layer=249 | N=6144 | M=2048 | Q=3.00 | alpha=3.73 | D_ks=0.02 | alpha-hat=6.11 | num traps=0
Layer 250
Layer=250 | N=2048 | M=2048 | Q=1.00 | alpha=1.90 | D_ks=0.09 | alpha-hat=4.68 | num traps=42
Layer 254
Layer=254 | N=8192 | M=2048 | Q=4.00 | alpha=3.12 | D_ks=0.01 | alpha-hat=7.10 | num traps=0
Layer 256
Layer=256 | N=8192 | M=2048 | Q=4.00 | alpha=3.91 | D_ks=0.05 | alpha-hat=10.66 | num traps=12
Layer 260
Layer=260 | N=6144 | M=2048 | Q=3.00 | alpha=3.42 | D_ks=0.02 | alpha-hat=6.01 | num traps=0
Layer 261
Layer=261 | N=2048 | M=2048 | Q=1.00 | alpha=2.76 | D_ks=0.06 | alpha-hat=6.04 | num traps=25
Layer 265
Layer=265 | N=8192 | M=2048 | Q=4.00 | alpha=3.35 | D_ks=0.01 | alpha-hat=7.25 | num traps=0
Layer 267
Layer=267 | N=8192 | M=2048 | Q=4.00 | alpha=4.41 | D_ks=0.04 | alpha-hat=9.43 | num traps=5