1 2020-09-10T00:00:02  *** maurits1 has quit IRC
  2 2020-09-10T00:02:04  *** melande1 has quit IRC
  3 2020-09-10T00:02:19  *** melande1 has joined #bitcoin-core-dev
  4 2020-09-10T00:21:20  *** mrostecki has quit IRC
  5 2020-09-10T00:28:01  *** melande1 has quit IRC
  6 2020-09-10T00:28:21  *** melande1 has joined #bitcoin-core-dev
  7 2020-09-10T00:33:05  *** arowser has quit IRC
  8 2020-09-10T00:33:25  *** arowser has joined #bitcoin-core-dev
  9 2020-09-10T00:48:40  *** braydonf has quit IRC
 10 2020-09-10T00:49:05  *** braydonf has joined #bitcoin-core-dev
 11 2020-09-10T00:55:09  *** zepheiryan has joined #bitcoin-core-dev
 12 2020-09-10T01:00:51  *** EagleTM has joined #bitcoin-core-dev
 13 2020-09-10T01:04:49  *** alko89 has quit IRC
 14 2020-09-10T01:05:53  *** troygiorshev has joined #bitcoin-core-dev
 15 2020-09-10T01:12:55  *** melande1 has quit IRC
 16 2020-09-10T01:13:20  *** melande1 has joined #bitcoin-core-dev
 17 2020-09-10T01:22:55  *** melande1 has quit IRC
 18 2020-09-10T01:23:19  *** melande1 has joined #bitcoin-core-dev
 19 2020-09-10T01:25:34  *** troygiorshev has quit IRC
 20 2020-09-10T01:25:51  *** alko89 has joined #bitcoin-core-dev
 21 2020-09-10T01:38:41  *** EagleTM has quit IRC
 22 2020-09-10T01:41:20  *** justanotheruser has joined #bitcoin-core-dev
 23 2020-09-10T01:43:49  *** mdunnio has joined #bitcoin-core-dev
 24 2020-09-10T01:48:34  *** mdunnio has quit IRC
 25 2020-09-10T01:49:23  *** cato_ has quit IRC
 26 2020-09-10T01:54:46  *** cato_ has joined #bitcoin-core-dev
 27 2020-09-10T01:58:05  *** arowser has quit IRC
 28 2020-09-10T01:58:25  *** arowser has joined #bitcoin-core-dev
 29 2020-09-10T02:24:56  *** melande1 has quit IRC
 30 2020-09-10T02:25:18  *** melande1 has joined #bitcoin-core-dev
 31 2020-09-10T02:26:21  *** melande1 has quit IRC
 32 2020-09-10T02:26:21  *** melande2 has joined #bitcoin-core-dev
 33 2020-09-10T02:33:59  *** proofofkeags has joined #bitcoin-core-dev
 34 2020-09-10T02:47:05  *** arowser has quit IRC
 35 2020-09-10T02:49:00  *** arowser has joined #bitcoin-core-dev
 36 2020-09-10T02:54:47  *** brianhoffman has joined #bitcoin-core-dev
 37 2020-09-10T02:58:21  *** Highway61 has quit IRC
 38 2020-09-10T03:00:01  *** zepheiryan has quit IRC
 39 2020-09-10T03:06:56  *** tralfaz has joined #bitcoin-core-dev
 40 2020-09-10T03:07:09  *** davterra has quit IRC
 41 2020-09-10T03:19:57  *** melande2 has quit IRC
 42 2020-09-10T03:20:19  *** melande2 has joined #bitcoin-core-dev
 43 2020-09-10T03:21:18  *** melande11 has joined #bitcoin-core-dev
 44 2020-09-10T03:21:58  *** shoman94 has joined #bitcoin-core-dev
 45 2020-09-10T03:23:22  *** wullon587 has joined #bitcoin-core-dev
 46 2020-09-10T03:25:59  *** proofofkeags has quit IRC
 47 2020-09-10T03:27:56  *** andreacab has joined #bitcoin-core-dev
 48 2020-09-10T03:28:43  *** melande11 has quit IRC
 49 2020-09-10T03:32:07  *** jonatack has quit IRC
 50 2020-09-10T03:32:21  *** andreacab has quit IRC
 51 2020-09-10T03:34:29  *** jonatack has joined #bitcoin-core-dev
 52 2020-09-10T03:35:30  *** melande11 has joined #bitcoin-core-dev
 53 2020-09-10T03:57:50  *** Highway61 has joined #bitcoin-core-dev
 54 2020-09-10T04:07:56  *** melande11 has quit IRC
 55 2020-09-10T04:08:21  *** melande11 has joined #bitcoin-core-dev
 56 2020-09-10T04:17:05  *** arowser has quit IRC
 57 2020-09-10T04:17:29  *** arowser has joined #bitcoin-core-dev
 58 2020-09-10T04:21:00  *** melande11 has quit IRC
 59 2020-09-10T04:21:25  *** melande11 has joined #bitcoin-core-dev
 60 2020-09-10T04:28:56  <kallewoof> Signet review beg -- no more changes planned, probably RFM: https://github.com/bitcoin/bitcoin/pull/18267
 61 2020-09-10T04:31:06  *** arowser has quit IRC
 62 2020-09-10T04:31:30  *** arowser has joined #bitcoin-core-dev
 63 2020-09-10T04:39:54  <phantomcircuit> sipa, 563,289,128
 64 2020-09-10T04:40:21  <sipa> phantomcircuit: that's unpossible
 65 2020-09-10T04:40:25  <sipa> txouts?
 66 2020-09-10T04:40:50  <phantomcircuit> wait did i count transactions
 67 2020-09-10T04:41:09  <sipa> i count 566823026 transactions
 68 2020-09-10T04:41:25  <phantomcircuit> damn it
 69 2020-09-10T04:42:06  *** arowser has quit IRC
 70 2020-09-10T04:42:23  <phantomcircuit> sipa, yeha i counted short at 99.4% done
 71 2020-09-10T04:42:28  <phantomcircuit> lets try that again
 72 2020-09-10T04:42:47  *** arowser has joined #bitcoin-core-dev
 73 2020-09-10T04:51:46  <phantomcircuit> 2020-09-10T04:49:27Z FlushStateToDisk: write coins cache to disk (72011604 coins, 9823747kB) started
 74 2020-09-10T04:51:52  <phantomcircuit> ok lets try that in 10-20 minutes
 75 2020-09-10T04:54:08  *** brianhoffman_ has joined #bitcoin-core-dev
 76 2020-09-10T04:55:02  *** brianhoffman has quit IRC
 77 2020-09-10T04:55:02  *** brianhoffman_ is now known as brianhoffman
 78 2020-09-10T04:57:25  <phantomcircuit> only took 411.70s
 79 2020-09-10T05:01:58  <sipa> i hope my numbers are accurated, i have a script that maintains a database with some per-block statistics
 80 2020-09-10T05:02:03  <sipa> so i just added all the numbers in it
 81 2020-09-10T05:08:59  *** melande11 has quit IRC
 82 2020-09-10T05:09:21  *** melande11 has joined #bitcoin-core-dev
 83 2020-09-10T05:13:50  *** shesek has quit IRC
 84 2020-09-10T05:16:07  *** Mercury_Vapor has quit IRC
 85 2020-09-10T05:24:05  *** arowser has quit IRC
 86 2020-09-10T05:24:25  *** arowser has joined #bitcoin-core-dev
 87 2020-09-10T05:29:40  <phantomcircuit> sipa, 1,506,642,276
 88 2020-09-10T05:32:06  <sipa> that's more like it
 89 2020-09-10T05:34:19  *** MrPaz has quit IRC
 90 2020-09-10T05:38:48  <phantomcircuit> sipa, yeah so this guy has no hope https://github.com/bitcoin/bitcoin/issues/19921
 91 2020-09-10T05:39:47  <sipa> phantomcircuit: it's not O(m*n)
 92 2020-09-10T05:40:17  <sipa> IsMine is O(log n) in the number of keys you have, i think
 93 2020-09-10T05:43:49  <phantomcircuit> sipa, there's a mapWallet.count, IsMine, IsFromMe, all of which will run if the transaction doesn't involve your wallet
 94 2020-09-10T05:44:29  <sipa> phantomcircuit: yes, but they run once per transaction, not per transactions*keycount
 95 2020-09-10T05:45:13  <sipa> only the lookups in the keystore have a time that scales with the number of keys, but it's not linear
 96 2020-09-10T05:51:59  <phantomcircuit> sipa, right so it's O(m log n) for m transactions and n keys
 97 2020-09-10T05:52:04  <phantomcircuit> or something close to that
 98 2020-09-10T05:52:13  <phantomcircuit> transaction outputs not transactions
 99 2020-09-10T05:54:01  <sipa> yes
100 2020-09-10T05:54:20  <sipa> O(outputs+inputs)
101 2020-09-10T05:54:30  <sipa> O((outputs+inputs)*log(keys))
102 2020-09-10T06:00:01  *** shoman94 has quit IRC
103 2020-09-10T06:09:37  <achow101> it'll just take a long time
104 2020-09-10T06:10:07  <achow101> probably a bit faster if mapKeys was unordered map instead of map
105 2020-09-10T06:11:51  <sipa> yeah
106 2020-09-10T06:12:03  <sipa> and way faster if it was a map of scriptPubKeys ;)
107 2020-09-10T06:12:24  <achow101> I suppose he could try #16910
108 2020-09-10T06:12:33  <gribble> https://github.com/bitcoin/bitcoin/issues/16910 | wallet: reduce loading time by using unordered maps by achow101 · Pull Request #16910 · bitcoin/bitcoin · GitHub
109 2020-09-10T06:12:35  <achow101> changes a bunch of things to unordered_map/unordered_set
110 2020-09-10T06:17:55  *** mdunnio has joined #bitcoin-core-dev
111 2020-09-10T06:21:56  *** steven1 has joined #bitcoin-core-dev
112 2020-09-10T06:22:43  *** mdunnio has quit IRC
113 2020-09-10T06:26:37  *** promag has joined #bitcoin-core-dev
114 2020-09-10T06:36:05  *** arowser has quit IRC
115 2020-09-10T06:37:27  *** promag has quit IRC
116 2020-09-10T06:38:25  *** arowser has joined #bitcoin-core-dev
117 2020-09-10T06:43:46  *** tralfaz has quit IRC
118 2020-09-10T06:43:50  *** davterra has joined #bitcoin-core-dev
119 2020-09-10T06:50:05  *** arowser has quit IRC
120 2020-09-10T06:50:25  *** arowser has joined #bitcoin-core-dev
121 2020-09-10T06:51:39  *** vasild has joined #bitcoin-core-dev
122 2020-09-10T06:59:29  *** sdaftuar has quit IRC
123 2020-09-10T06:59:58  *** sdaftuar has joined #bitcoin-core-dev
124 2020-09-10T07:00:06  *** andreacab has joined #bitcoin-core-dev
125 2020-09-10T07:09:03  *** tralfaz has joined #bitcoin-core-dev
126 2020-09-10T07:10:01  <hebasto> who is interested in today's meeting topic suggested by jnewbery mind considering the overview gist https://gist.github.com/hebasto/072ad3a9370641b035a36d08607a3d34
127 2020-09-10T07:10:29  *** davterra has quit IRC
128 2020-09-10T07:11:15  *** marcoagner has joined #bitcoin-core-dev
129 2020-09-10T07:21:06  *** arowser has quit IRC
130 2020-09-10T07:21:47  *** arowser has joined #bitcoin-core-dev
131 2020-09-10T07:23:05  *** arowser has quit IRC
132 2020-09-10T07:23:27  *** arowser has joined #bitcoin-core-dev
133 2020-09-10T07:28:02  *** jonatack has quit IRC
134 2020-09-10T07:28:11  *** reallll has joined #bitcoin-core-dev
135 2020-09-10T07:31:28  *** bitcoin-git has joined #bitcoin-core-dev
136 2020-09-10T07:31:28  <bitcoin-git> [bitcoin] sipa opened pull request #19931: Change CSipHasher's count variable to uint8_t (master...202009_siphash_sillyness) https://github.com/bitcoin/bitcoin/pull/19931
137 2020-09-10T07:31:29  *** bitcoin-git has left #bitcoin-core-dev
138 2020-09-10T07:32:03  *** belcher_ has quit IRC
139 2020-09-10T07:34:01  *** Guyver2 has joined #bitcoin-core-dev
140 2020-09-10T07:47:23  *** Madars has quit IRC
141 2020-09-10T07:47:30  *** promag has joined #bitcoin-core-dev
142 2020-09-10T07:48:03  *** jb55 has quit IRC
143 2020-09-10T07:48:25  *** kristapsk has quit IRC
144 2020-09-10T07:48:57  *** kristapsk has joined #bitcoin-core-dev
145 2020-09-10T07:51:15  *** andreacab has quit IRC
146 2020-09-10T07:51:21  *** andreacab has joined #bitcoin-core-dev
147 2020-09-10T07:54:35  *** Cory has quit IRC
148 2020-09-10T08:00:54  *** Madars has joined #bitcoin-core-dev
149 2020-09-10T08:04:05  *** arowser has quit IRC
150 2020-09-10T08:04:10  *** promag_ has joined #bitcoin-core-dev
151 2020-09-10T08:04:29  *** arowser has joined #bitcoin-core-dev
152 2020-09-10T08:07:09  *** Pavlenex has joined #bitcoin-core-dev
153 2020-09-10T08:08:53  *** promag_ has quit IRC
154 2020-09-10T08:12:29  *** Cory has joined #bitcoin-core-dev
155 2020-09-10T08:17:43  *** promag has quit IRC
156 2020-09-10T08:19:54  *** Pavlenex has quit IRC
157 2020-09-10T08:20:52  *** jb55 has joined #bitcoin-core-dev
158 2020-09-10T08:21:24  *** promag has joined #bitcoin-core-dev
159 2020-09-10T08:23:11  *** promag has quit IRC
160 2020-09-10T08:23:37  *** andreacab has quit IRC
161 2020-09-10T08:24:11  *** Madars has quit IRC
162 2020-09-10T08:24:29  *** promag has joined #bitcoin-core-dev
163 2020-09-10T08:24:34  *** andreacab has joined #bitcoin-core-dev
164 2020-09-10T08:28:47  *** andreacab has quit IRC
165 2020-09-10T08:29:48  *** andreacab has joined #bitcoin-core-dev
166 2020-09-10T08:31:53  *** reallll is now known as belcher
167 2020-09-10T08:33:31  *** bitcoin-git has joined #bitcoin-core-dev
168 2020-09-10T08:33:31  <bitcoin-git> [bitcoin] hebasto closed pull request #19926: gui: Add Tor icon (master...200909-tor) https://github.com/bitcoin/bitcoin/pull/19926
169 2020-09-10T08:33:32  *** bitcoin-git has left #bitcoin-core-dev
170 2020-09-10T08:45:41  *** promag has quit IRC
171 2020-09-10T08:47:10  *** Pavlenex has joined #bitcoin-core-dev
172 2020-09-10T08:52:05  *** promag has joined #bitcoin-core-dev
173 2020-09-10T08:52:12  *** Madars has joined #bitcoin-core-dev
174 2020-09-10T08:57:20  *** jonatack has joined #bitcoin-core-dev
175 2020-09-10T09:00:01  *** vincenzopalazzo has joined #bitcoin-core-dev
176 2020-09-10T09:00:01  *** steven1 has quit IRC
177 2020-09-10T09:04:10  *** promag has quit IRC
178 2020-09-10T09:05:50  *** promag has joined #bitcoin-core-dev
179 2020-09-10T09:15:02  *** promag has quit IRC
180 2020-09-10T09:15:11  *** Madars has quit IRC
181 2020-09-10T09:21:55  *** Avelino has joined #bitcoin-core-dev
182 2020-09-10T09:27:50  *** andreacab has quit IRC
183 2020-09-10T09:28:16  *** andreacab has joined #bitcoin-core-dev
184 2020-09-10T09:29:30  *** andreacab has quit IRC
185 2020-09-10T09:29:37  *** andreacab has joined #bitcoin-core-dev
186 2020-09-10T09:34:05  *** jb55 has quit IRC
187 2020-09-10T09:34:37  *** jb55 has joined #bitcoin-core-dev
188 2020-09-10T09:36:24  *** Pavlenex has quit IRC
189 2020-09-10T09:36:54  *** promag has joined #bitcoin-core-dev
190 2020-09-10T09:37:24  *** justanotheruser has quit IRC
191 2020-09-10T09:38:25  *** Pavlenex has joined #bitcoin-core-dev
192 2020-09-10T09:38:31  *** Madars has joined #bitcoin-core-dev
193 2020-09-10T09:42:09  *** promag has quit IRC
194 2020-09-10T09:48:54  *** EagleTM has joined #bitcoin-core-dev
195 2020-09-10T09:53:26  *** EagleTM has quit IRC
196 2020-09-10T09:53:45  *** EagleTM has joined #bitcoin-core-dev
197 2020-09-10T09:58:09  *** jonatack has quit IRC
198 2020-09-10T09:58:51  *** promag has joined #bitcoin-core-dev
199 2020-09-10T10:03:20  *** Madars has quit IRC
200 2020-09-10T10:03:23  *** promag has quit IRC
201 2020-09-10T10:05:24  *** andreacab has quit IRC
202 2020-09-10T10:05:51  *** andreacab has joined #bitcoin-core-dev
203 2020-09-10T10:09:17  *** mrostecki has joined #bitcoin-core-dev
204 2020-09-10T10:10:20  *** andreacab has quit IRC
205 2020-09-10T10:13:01  *** tralfaz has quit IRC
206 2020-09-10T10:13:40  *** tralfaz has joined #bitcoin-core-dev
207 2020-09-10T10:17:38  *** Bullit has quit IRC
208 2020-09-10T10:18:05  *** arowser has quit IRC
209 2020-09-10T10:18:11  *** Bullit has joined #bitcoin-core-dev
210 2020-09-10T10:18:23  *** Tyrell41Kris has joined #bitcoin-core-dev
211 2020-09-10T10:18:26  *** arowser has joined #bitcoin-core-dev
212 2020-09-10T10:20:25  *** Evel-Knievel has quit IRC
213 2020-09-10T10:21:01  *** Evel-Knievel has joined #bitcoin-core-dev
214 2020-09-10T10:24:55  *** promag has joined #bitcoin-core-dev
215 2020-09-10T10:28:05  *** arowser has quit IRC
216 2020-09-10T10:28:24  *** arowser has joined #bitcoin-core-dev
217 2020-09-10T10:29:35  *** promag has quit IRC
218 2020-09-10T10:29:52  *** promag has joined #bitcoin-core-dev
219 2020-09-10T10:32:31  *** Madars has joined #bitcoin-core-dev
220 2020-09-10T10:36:26  *** promag has quit IRC
221 2020-09-10T10:40:04  *** Tyrell41Kris has quit IRC
222 2020-09-10T10:41:55  <jonasschnelli> #proposedmeetingtopic review the experience of 3 month bitcoin-core/gui repository
223 2020-09-10T10:49:53  *** andreacab has joined #bitcoin-core-dev
224 2020-09-10T10:52:35  *** Madars has quit IRC
225 2020-09-10T10:54:17  *** andreacab has quit IRC
226 2020-09-10T11:00:03  *** vasild has quit IRC
227 2020-09-10T11:01:34  *** vasild has joined #bitcoin-core-dev
228 2020-09-10T11:01:50  *** andreacab has joined #bitcoin-core-dev
229 2020-09-10T11:02:25  *** Jackielove4u has quit IRC
230 2020-09-10T11:11:31  *** ExEric3 has quit IRC
231 2020-09-10T11:12:18  *** ExEric3 has joined #bitcoin-core-dev
232 2020-09-10T11:14:50  *** Madars has joined #bitcoin-core-dev
233 2020-09-10T11:31:26  *** paz_ has joined #bitcoin-core-dev
234 2020-09-10T11:31:49  *** paz_ is now known as Guest48339
235 2020-09-10T11:35:05  *** Madars has quit IRC
236 2020-09-10T11:38:20  *** promag has joined #bitcoin-core-dev
237 2020-09-10T11:42:04  *** andreacab has quit IRC
238 2020-09-10T11:42:48  *** promag has quit IRC
239 2020-09-10T11:46:46  *** andreacab has joined #bitcoin-core-dev
240 2020-09-10T11:57:23  *** promag has joined #bitcoin-core-dev
241 2020-09-10T11:57:27  *** Avelino has quit IRC
242 2020-09-10T12:01:43  *** promag has quit IRC
243 2020-09-10T12:02:09  *** Madars has joined #bitcoin-core-dev
244 2020-09-10T12:03:37  *** Mercury_Vapor has joined #bitcoin-core-dev
245 2020-09-10T12:05:47  *** alko89 has quit IRC
246 2020-09-10T12:05:58  *** alko89 has joined #bitcoin-core-dev
247 2020-09-10T12:15:47  *** andreacab has quit IRC
248 2020-09-10T12:16:14  *** andreacab has joined #bitcoin-core-dev
249 2020-09-10T12:18:16  *** provoostenator has joined #bitcoin-core-dev
250 2020-09-10T12:20:30  *** andreacab has quit IRC
251 2020-09-10T12:20:44  *** wolfy13391 has joined #bitcoin-core-dev
252 2020-09-10T12:24:11  *** Madars has quit IRC
253 2020-09-10T12:27:31  *** Aaronvan_ has joined #bitcoin-core-dev
254 2020-09-10T12:29:33  *** AaronvanW has quit IRC
255 2020-09-10T12:33:10  *** EagleTM has quit IRC
256 2020-09-10T12:39:28  *** Guyver2 has quit IRC
257 2020-09-10T12:43:02  *** jonatack has joined #bitcoin-core-dev
258 2020-09-10T12:47:46  *** Emcy_ has joined #bitcoin-core-dev
259 2020-09-10T12:47:47  *** gzhao408 has joined #bitcoin-core-dev
260 2020-09-10T12:47:48  *** Emcy has quit IRC
261 2020-09-10T13:02:30  *** EagleTM has joined #bitcoin-core-dev
262 2020-09-10T13:06:05  *** arowser has quit IRC
263 2020-09-10T13:06:46  *** arowser has joined #bitcoin-core-dev
264 2020-09-10T13:08:06  *** arowser has quit IRC
265 2020-09-10T13:08:26  *** arowser has joined #bitcoin-core-dev
266 2020-09-10T13:09:06  *** arowser has quit IRC
267 2020-09-10T13:09:26  *** arowser has joined #bitcoin-core-dev
268 2020-09-10T13:10:05  *** arowser has quit IRC
269 2020-09-10T13:10:48  *** arowser has joined #bitcoin-core-dev
270 2020-09-10T13:11:02  *** Madars has joined #bitcoin-core-dev
271 2020-09-10T13:11:05  *** arowser has quit IRC
272 2020-09-10T13:11:26  *** arowser has joined #bitcoin-core-dev
273 2020-09-10T13:12:05  *** arowser has quit IRC
274 2020-09-10T13:12:25  *** arowser has joined #bitcoin-core-dev
275 2020-09-10T13:24:04  *** arowser has quit IRC
276 2020-09-10T13:24:25  *** arowser has joined #bitcoin-core-dev
277 2020-09-10T13:36:50  *** Madars has quit IRC
278 2020-09-10T13:37:22  *** jonatack has quit IRC
279 2020-09-10T13:41:45  *** promag has joined #bitcoin-core-dev
280 2020-09-10T13:58:21  *** promag has quit IRC
281 2020-09-10T14:00:43  *** mdunnio has joined #bitcoin-core-dev
282 2020-09-10T14:02:15  *** jonatack has joined #bitcoin-core-dev
283 2020-09-10T14:06:05  *** arowser has quit IRC
284 2020-09-10T14:06:27  *** arowser has joined #bitcoin-core-dev
285 2020-09-10T14:06:47  *** proofofkeags has joined #bitcoin-core-dev
286 2020-09-10T14:13:21  *** mdunnio has quit IRC
287 2020-09-10T14:13:36  *** mdunnio has joined #bitcoin-core-dev
288 2020-09-10T14:20:54  *** shaunsun has joined #bitcoin-core-dev
289 2020-09-10T14:26:14  *** opsec_x122 has joined #bitcoin-core-dev
290 2020-09-10T14:29:05  *** jeremyrubin has quit IRC
291 2020-09-10T14:29:18  *** jeremyrubin has joined #bitcoin-core-dev
292 2020-09-10T14:29:38  *** opsec_x12 has quit IRC
293 2020-09-10T14:30:00  *** bitcoin-git has joined #bitcoin-core-dev
294 2020-09-10T14:30:00  <bitcoin-git> [bitcoin] Saibato opened pull request #19933: wallet: bugfix; if datadir has a trailing '/'  listwalletdir would strip lead char of walletname (master...wallet-fix-missing-chars-boost-1.47) https://github.com/bitcoin/bitcoin/pull/19933
295 2020-09-10T14:30:02  *** bitcoin-git has left #bitcoin-core-dev
296 2020-09-10T14:31:23  *** sdaftuar has quit IRC
297 2020-09-10T14:31:24  *** afk11 has quit IRC
298 2020-09-10T14:33:46  *** sdaftuar has joined #bitcoin-core-dev
299 2020-09-10T14:33:53  *** afk11 has joined #bitcoin-core-dev
300 2020-09-10T14:35:06  *** Madars has joined #bitcoin-core-dev
301 2020-09-10T14:36:34  *** promag has joined #bitcoin-core-dev
302 2020-09-10T14:38:49  *** bitcoin-git has joined #bitcoin-core-dev
303 2020-09-10T14:38:49  <bitcoin-git> [bitcoin] laanwj pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/564e1ab0f3dc...a47e5964861d
304 2020-09-10T14:38:50  <bitcoin-git> bitcoin/master 2ac8bf9 Pieter Wuille: Implement keccak-f[1600] and SHA3-256
305 2020-09-10T14:38:50  <bitcoin-git> bitcoin/master 3f01ddb Pieter Wuille: Add SHA3 benchmark
306 2020-09-10T14:38:51  <bitcoin-git> bitcoin/master ab654c7 Pieter Wuille: Unroll Keccak-f implementation
307 2020-09-10T14:38:59  *** bitcoin-git has left #bitcoin-core-dev
308 2020-09-10T14:39:14  *** bitcoin-git has joined #bitcoin-core-dev
309 2020-09-10T14:39:14  <bitcoin-git> [bitcoin] laanwj merged pull request #19841: Implement Keccak and SHA3_256 (master...202008_sha3) https://github.com/bitcoin/bitcoin/pull/19841
310 2020-09-10T14:39:15  *** bitcoin-git has left #bitcoin-core-dev
311 2020-09-10T14:41:00  *** promag has quit IRC
312 2020-09-10T14:43:43  *** opsec_x122 is now known as opsec_x12
313 2020-09-10T14:46:30  *** luke-jr has quit IRC
314 2020-09-10T14:49:34  *** luke-jr has joined #bitcoin-core-dev
315 2020-09-10T14:51:19  *** promag has joined #bitcoin-core-dev
316 2020-09-10T14:51:37  *** promag has joined #bitcoin-core-dev
317 2020-09-10T14:53:01  *** Dean_Guss has quit IRC
318 2020-09-10T14:53:29  *** Dean_Guss has joined #bitcoin-core-dev
319 2020-09-10T14:57:14  *** gleb has quit IRC
320 2020-09-10T14:57:28  *** bitcoin-git has joined #bitcoin-core-dev
321 2020-09-10T14:57:28  <bitcoin-git> [bitcoin] practicalswift opened pull request #19934: tests: Add fuzzing harness for Keccak and SHA3_256 (master...fuzzers-keccak-and-sha3_256) https://github.com/bitcoin/bitcoin/pull/19934
322 2020-09-10T14:57:29  *** bitcoin-git has left #bitcoin-core-dev
323 2020-09-10T15:00:02  *** wolfy13391 has quit IRC
324 2020-09-10T15:02:44  *** Guyver2 has joined #bitcoin-core-dev
325 2020-09-10T15:04:27  *** andreacab has joined #bitcoin-core-dev
326 2020-09-10T15:06:12  *** promag has quit IRC
327 2020-09-10T15:07:56  *** promag has joined #bitcoin-core-dev
328 2020-09-10T15:11:49  *** Madars has quit IRC
329 2020-09-10T15:16:59  *** satoshi has joined #bitcoin-core-dev
330 2020-09-10T15:20:02  *** andreacab has quit IRC
331 2020-09-10T15:22:23  *** chrippa has joined #bitcoin-core-dev
332 2020-09-10T15:30:43  *** Guyver2_ has joined #bitcoin-core-dev
333 2020-09-10T15:31:48  *** andreacab has joined #bitcoin-core-dev
334 2020-09-10T15:32:44  *** Guyver2 has quit IRC
335 2020-09-10T15:33:58  *** bitcoin-git has joined #bitcoin-core-dev
336 2020-09-10T15:33:58  <bitcoin-git> [bitcoin] achow101 opened pull request #19935: Move SaltedHashers to separate file and add some new ones (master...mv-saltedhash) https://github.com/bitcoin/bitcoin/pull/19935
337 2020-09-10T15:33:59  *** bitcoin-git has left #bitcoin-core-dev
338 2020-09-10T15:37:06  *** troygiorshev has joined #bitcoin-core-dev
339 2020-09-10T15:38:27  *** troygiorshev has quit IRC
340 2020-09-10T15:39:05  *** IGHOR has quit IRC
341 2020-09-10T15:43:22  *** IGHOR has joined #bitcoin-core-dev
342 2020-09-10T15:46:20  *** afk11 has quit IRC
343 2020-09-10T15:46:43  *** afk11 has joined #bitcoin-core-dev
344 2020-09-10T15:50:33  *** Talkless has joined #bitcoin-core-dev
345 2020-09-10T16:10:21  *** Madars has joined #bitcoin-core-dev
346 2020-09-10T16:12:50  *** promag has quit IRC
347 2020-09-10T16:32:05  *** arowser has quit IRC
348 2020-09-10T16:32:26  *** arowser has joined #bitcoin-core-dev
349 2020-09-10T16:34:41  *** Madars has quit IRC
350 2020-09-10T16:37:02  *** Emcy_ has quit IRC
351 2020-09-10T16:38:26  *** Emcy has joined #bitcoin-core-dev
352 2020-09-10T16:42:43  *** afk11 has quit IRC
353 2020-09-10T16:43:57  *** afk11 has joined #bitcoin-core-dev
354 2020-09-10T16:47:01  *** MrPaz has joined #bitcoin-core-dev
355 2020-09-10T16:47:16  *** Guest48339 has quit IRC
356 2020-09-10T16:48:29  *** promag has joined #bitcoin-core-dev
357 2020-09-10T16:49:09  *** MrPaz has quit IRC
358 2020-09-10T16:49:32  *** MrPaz has joined #bitcoin-core-dev
359 2020-09-10T16:50:05  *** kristapsk has joined #bitcoin-core-dev
360 2020-09-10T16:53:08  *** justanotheruser has joined #bitcoin-core-dev
361 2020-09-10T16:53:23  *** mrostecki has quit IRC
362 2020-09-10T16:53:40  *** justtestingthiso has joined #bitcoin-core-dev
363 2020-09-10T16:54:06  *** mrostecki has joined #bitcoin-core-dev
364 2020-09-10T16:56:02  *** promag has quit IRC
365 2020-09-10T16:59:17  *** promag_ has joined #bitcoin-core-dev
366 2020-09-10T17:03:32  *** Jackielove4u has joined #bitcoin-core-dev
367 2020-09-10T17:09:16  *** justtestingthiso has left #bitcoin-core-dev
368 2020-09-10T17:10:19  *** robertnnms has joined #bitcoin-core-dev
369 2020-09-10T17:12:12  *** robertnnms has left #bitcoin-core-dev
370 2020-09-10T17:13:10  *** robertnnms has joined #bitcoin-core-dev
371 2020-09-10T17:13:47  <robertnnms> Hi,I need to do advanced testing on testnet, but I would need between 10-15 BTC, could someone send me to the following address: tb1qs25dr4e8k29rwclxvnxedfdtprh5e89jtp5wleMany thanks
372 2020-09-10T17:14:06  *** satoshi has joined #bitcoin-core-dev
373 2020-09-10T17:14:33  *** satoshi has quit IRC
374 2020-09-10T17:14:39  <dongcarl> Anyone know what the `UnloadBlockIndex` call here does in the context of initialization? https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L1562
375 2020-09-10T17:14:51  <dongcarl> First introduced in: https://github.com/bitcoin/bitcoin/commit/f7f3a96b74bb795d6e184a628adce21c744d234f#diff-c865a8939105e6350a50af02766291b7R805
376 2020-09-10T17:14:56  *** Madars has joined #bitcoin-core-dev
377 2020-09-10T17:17:19  *** Talkless has quit IRC
378 2020-09-10T17:17:40  *** Talkless has joined #bitcoin-core-dev
379 2020-09-10T17:20:05  *** arowser has quit IRC
380 2020-09-10T17:20:47  *** arowser has joined #bitcoin-core-dev
381 2020-09-10T17:24:50  *** promag has joined #bitcoin-core-dev
382 2020-09-10T17:25:00  *** promag_ has quit IRC
383 2020-09-10T17:26:23  *** IGHOR has quit IRC
384 2020-09-10T17:27:33  *** IGHOR has joined #bitcoin-core-dev
385 2020-09-10T17:35:43  *** robertnnms has quit IRC
386 2020-09-10T17:36:27  *** promag has quit IRC
387 2020-09-10T17:36:47  *** promag has joined #bitcoin-core-dev
388 2020-09-10T17:36:59  *** Madars has quit IRC
389 2020-09-10T17:37:03  *** IGHOR has quit IRC
390 2020-09-10T17:38:12  *** IGHOR has joined #bitcoin-core-dev
391 2020-09-10T17:40:01  *** Pavlenex has quit IRC
392 2020-09-10T17:41:59  *** Pavlenex has joined #bitcoin-core-dev
393 2020-09-10T17:52:10  *** promag has quit IRC
394 2020-09-10T17:53:29  *** Cassidy67Schambe has joined #bitcoin-core-dev
395 2020-09-10T17:53:31  *** promag has joined #bitcoin-core-dev
396 2020-09-10T17:58:28  *** Cassidy67Schambe has quit IRC
397 2020-09-10T18:00:02  *** chrippa has quit IRC
398 2020-09-10T18:02:42  <wumpus> dongcarl: it's in a retry loop, I think it's there in case a previous block index load only succeeds partially?
399 2020-09-10T18:03:04  <sipa> yep
400 2020-09-10T18:05:33  <dongcarl> I see, that makes sense...
401 2020-09-10T18:10:21  <phantomcircuit> achow101, the only real solution for rescanning with a huge number of keys is to use the block filter indexes
402 2020-09-10T18:10:23  <phantomcircuit> it takes absolutely ages otherwise
403 2020-09-10T18:11:10  *** Madars has joined #bitcoin-core-dev
404 2020-09-10T18:15:28  *** promag has quit IRC
405 2020-09-10T18:17:30  *** Guyver2_ has quit IRC
406 2020-09-10T18:18:57  *** Guyver2 has joined #bitcoin-core-dev
407 2020-09-10T18:21:44  *** popey1 has joined #bitcoin-core-dev
408 2020-09-10T18:23:32  <phantomcircuit> sipa, lol he closed the issue, what a child
409 2020-09-10T18:23:41  <phantomcircuit> still wondering how he ended up with 150m keys
410 2020-09-10T18:24:54  <sipa> i assume he generated them using a dictionary
411 2020-09-10T18:30:57  <jonasschnelli> a cracker!
412 2020-09-10T18:31:27  <jonasschnelli> Why he is not using a full indexer like electrs or so is unclear to me.
413 2020-09-10T18:31:51  <wumpus> if it's a criminal even more reason to not give them ideas
414 2020-09-10T18:32:32  <jonasschnelli> maybe he lost some of his mnemonic words...
415 2020-09-10T18:33:14  <jonasschnelli> also... i would rather scan agains the UTXO set in that case (and not against all blocks)
416 2020-09-10T18:33:35  <yanmaani> It's not a very competent attack that's for suer
417 2020-09-10T18:37:01  *** Highway62 has joined #bitcoin-core-dev
418 2020-09-10T18:37:36  *** Highway61 has quit IRC
419 2020-09-10T18:37:36  *** Highway62 is now known as Highway61
420 2020-09-10T18:38:07  *** justanotheruser has quit IRC
421 2020-09-10T18:38:34  *** Madars has quit IRC
422 2020-09-10T18:47:08  *** gzhao408 has quit IRC
423 2020-09-10T18:58:50  *** lightlike has joined #bitcoin-core-dev
424 2020-09-10T19:00:25  <wumpus> #startmeeting
425 2020-09-10T19:00:25  <lightningbot> Meeting started Thu Sep 10 19:00:25 2020 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
426 2020-09-10T19:00:25  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
427 2020-09-10T19:00:30  <kanzure> hi
428 2020-09-10T19:00:31  <achow101> hi
429 2020-09-10T19:00:33  <jonasschnelli> hi
430 2020-09-10T19:00:35  <hebasto> hi
431 2020-09-10T19:00:36  <fjahr> hi
432 2020-09-10T19:00:44  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral ariard digi_james
433 2020-09-10T19:00:47  <wumpus> amiti fjahr jeremyrubin lightlike emilengler jonatack hebasto jb55 elichai2
434 2020-09-10T19:01:01  <wumpus> two proposed meeting topics for today
435 2020-09-10T19:01:02  <vasild> hi
436 2020-09-10T19:01:06  <wumpus> strategies for removing recursive locking in the mempool (jnewbery), review the experience of 3 month bitcoin-core/gui repository (jonasschnelli)
437 2020-09-10T19:01:06  <meshcollider> hi
438 2020-09-10T19:01:23  <wumpus> any last minute topic proposals?
439 2020-09-10T19:01:23  <jonatack> hi
440 2020-09-10T19:02:36  <wumpus> #topic High priority for review
441 2020-09-10T19:03:07  <wumpus> https://github.com/bitcoin/bitcoin/projects/8  11 blockers, 1 bugfix, 2 chasins concept ACK
442 2020-09-10T19:03:16  *** rorp has joined #bitcoin-core-dev
443 2020-09-10T19:03:24  <jonatack> #19845?
444 2020-09-10T19:03:37  <gribble> https://github.com/bitcoin/bitcoin/issues/19845 | net: CNetAddr: add support to (un)serialize as ADDRv2 by vasild · Pull Request #19845 · bitcoin/bitcoin · GitHub
445 2020-09-10T19:04:17  <meshcollider> Actually 10 now :) one was merged and hadn't been removed
446 2020-09-10T19:04:27  <jnewbery> hi
447 2020-09-10T19:04:48  <wumpus> jonatack: added
448 2020-09-10T19:04:53  *** promag has joined #bitcoin-core-dev
449 2020-09-10T19:05:06  <meshcollider>  #16378 pls
450 2020-09-10T19:05:10  <gribble> https://github.com/bitcoin/bitcoin/issues/16378 | The ultimate send RPC by Sjors · Pull Request #16378 · bitcoin/bitcoin · GitHub
451 2020-09-10T19:05:52  <achow101> #19077 please
452 2020-09-10T19:06:00  <gribble> https://github.com/bitcoin/bitcoin/issues/19077 | wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets by achow101 · Pull Request #19077 · bitcoin/bitcoin · GitHub
453 2020-09-10T19:06:12  <promag> #19033 for hp, ty
454 2020-09-10T19:06:15  <gribble> https://github.com/bitcoin/bitcoin/issues/19033 | http: Release work queue after event base finish by promag · Pull Request #19033 · bitcoin/bitcoin · GitHub
455 2020-09-10T19:06:21  <phantomcircuit> hi
456 2020-09-10T19:06:52  * luke-jr pokes gribble
457 2020-09-10T19:07:22  <wumpus> meshcollider, achow101  added
458 2020-09-10T19:07:27  <wumpus> promag: already on there, right?
459 2020-09-10T19:08:01  <jeremyrubin> i think so
460 2020-09-10T19:08:58  <promag> indeed, I guess I have to name it "http: blocklist work queue..."
461 2020-09-10T19:09:01  *** luke-jr has quit IRC
462 2020-09-10T19:09:04  <wumpus> unless someone else just added it, but I vaguely remember it was already there
463 2020-09-10T19:09:43  <wumpus> #topic Strategies for removing recursive locking in the mempool (jnewbery)
464 2020-09-10T19:09:51  <wumpus> (https://github.com/bitcoin/bitcoin/pull/19872#issuecomment-688852261)
465 2020-09-10T19:09:56  *** luke-jr has joined #bitcoin-core-dev
466 2020-09-10T19:09:58  <jnewbery> Thanks wumpus
467 2020-09-10T19:10:16  <jnewbery> gist here: https://gist.github.com/hebasto/072ad3a9370641b035a36d08607a3d34
468 2020-09-10T19:10:53  <jnewbery> summary: RecusiveMutex is generally considered bad, and avoided if possible. It'd be nice to change the mempool's mutex from recursive to non-recursive
469 2020-09-10T19:10:58  *** Talkless has quit IRC
470 2020-09-10T19:11:00  <wumpus> I think getting rid of the recursive locks is good, though, it seems changes are becoming increasingly risky
471 2020-09-10T19:11:07  <wumpus> and involved
472 2020-09-10T19:11:22  <jnewbery> There are different ways to do this, and hebasto would like some input on which way is preferred
473 2020-09-10T19:11:23  <wumpus> we had all the low-hanging fruit I guess
474 2020-09-10T19:12:00  <jnewbery> wumpus: i don't think so. I think there's a lot more low-hanging fruit (although cs_mempool isn't very low hanging)
475 2020-09-10T19:12:33  <hebasto> tracking issue for all recursive mutexes #19303
476 2020-09-10T19:12:35  <gribble> https://github.com/bitcoin/bitcoin/issues/19303 | Replace all of the RecursiveMutex instances with the Mutex ones · Issue #19303 · bitcoin/bitcoin · GitHub
477 2020-09-10T19:12:40  <jeremyrubin> Probably makes more sense to have the locking external to the functions
478 2020-09-10T19:12:55  <jeremyrubin> More general to e.g., calling a function in a for loop
479 2020-09-10T19:13:01  <jnewbery> Two possible ways are making a lot more of the mempool functions require the lock, and make locking external to the mempool
480 2020-09-10T19:13:03  <wumpus> "... two functions could be used - one that locks and another that does not..." yes, that's the common solution
481 2020-09-10T19:13:26  <jnewbery> and the other way is having two versions of the function - one which requires the lock and one which takes the lock
482 2020-09-10T19:13:31  <jnewbery> wumpus: right
483 2020-09-10T19:13:32  <wumpus> moving locking external has its own risks
484 2020-09-10T19:13:51  <hebasto> what risk?
485 2020-09-10T19:13:52  <jeremyrubin> wumpus: if we use the clang safety annotations it's lesser, right?
486 2020-09-10T19:13:55  <wumpus> having separate private "without lock" avoids that
487 2020-09-10T19:13:57  <jnewbery> wumpus: I agree. Best to keep locking internal wherever possible
488 2020-09-10T19:14:24  <vasild> "the locking mechanism used by a thread-safe class is part of its internal implementation" (from https://gist.github.com/hebasto/072ad3a9370641b035a36d08607a3d34#gistcomment-3448023)
489 2020-09-10T19:14:32  * luke-jr stabs ISP
490 2020-09-10T19:14:36  <sipa> ideally, locks are internal
491 2020-09-10T19:14:37  <wumpus> hebasto: well, it goes against, the principle of encapsulation, knowledge of how to remove locks moves to outside the code
492 2020-09-10T19:15:06  <wumpus> hebasto: at the least it makes reasoning more difficult
493 2020-09-10T19:15:14  <hebasto> wumpus: agree
494 2020-09-10T19:15:25  <sipa> if a lock is internal to a class, and the class never invokes a callback passed down from elsewhere while holding that lock, it even guarantees no deadlocks
495 2020-09-10T19:15:35  <wumpus> lock annotations are not a replacement for that
496 2020-09-10T19:16:05  *** rorp has quit IRC
497 2020-09-10T19:16:30  <jnewbery> if locks can't always be internal, then making two versions of the public interface function makes it explicit where the function is called with the lock held and without.
498 2020-09-10T19:16:49  <sipa> jnewbery: that's what CAddrMan does
499 2020-09-10T19:16:59  <sipa> but i don't see how one conflicts with the other
500 2020-09-10T19:17:04  <jeremyrubin> What about a type-safe (using newtype) pass lock guard in as arugment?
501 2020-09-10T19:17:14  <jeremyrubin> That would have both safety properties
502 2020-09-10T19:17:20  <sipa> jeremyrubin: it works, but imho it still reeks of bad design
503 2020-09-10T19:17:28  <jeremyrubin> as there would only be one way to get a satisfying lock
504 2020-09-10T19:17:33  <wumpus> I don't really like that, I hope that can be avoided
505 2020-09-10T19:17:34  <promag> and I'd like to discuss for() { LOCK(cs); } VS { LOCK() for() {} }
506 2020-09-10T19:18:11  <hebasto> how to name this two functions? some kind of suffix to distinguish them?
507 2020-09-10T19:18:12  <jnewbery> sipa: I'm not saying they do. Ideally, all locks are internal. If they can't be and sometimes need to be held by the caller, make it explicit where that is by using Mutex and having two versions of public functions where needed.
508 2020-09-10T19:18:20  <aj> promag: with or without performance benchmarks?
509 2020-09-10T19:18:29  <wumpus> promag: well that depends on the granularity of the work, so locking overhad versus the loop body, and whether it's undesirable to hold the lock for long
510 2020-09-10T19:18:31  <sipa> promag: grabbing an uncontended lock is ~100 cpu cycles
511 2020-09-10T19:18:46  <jeremyrubin> hebasto: I'd do a type tag argument maybe; like atmoic primitives.
512 2020-09-10T19:19:09  <jeremyrubin> but maybe that's overkill
513 2020-09-10T19:19:21  *** molz_ has joined #bitcoin-core-dev
514 2020-09-10T19:19:25  <sipa> jnewbery: sorry, i misread what you suggested
515 2020-09-10T19:20:00  <aj> jeremyrubin: void foo(int x); void foo(int x, LockAlreadyHeld h);  foo(3, LockAlreadyHeld()); // ?
516 2020-09-10T19:20:21  <wumpus> I'm starting to understand Marcofalke's point now--this is a lot of work, conflicts with other changes, while we don't have a direct problem with RecursiveMutex
517 2020-09-10T19:20:23  <aj> jeremyrubin: LockAlreadyHeld(cs_main); ?
518 2020-09-10T19:20:26  <jeremyrubin> aj: Uh that can work yeah
519 2020-09-10T19:20:28  <vasild> promag: I would say if the current code does one of that, we better have some proof that changing it is for the better.
520 2020-09-10T19:20:39  <sdaftuar> wumpus: that is my gut reaction as well
521 2020-09-10T19:21:11  <promag> aj: in simple cases not really
522 2020-09-10T19:21:26  <sdaftuar> it would be nice to motivate this change with some new behavior we wish to implement, that woudl benefit from it?
523 2020-09-10T19:21:30  <wumpus> if it helps to have a non-recursive mutex for future mempool changes it makes sense to do it, of course, but this seems a lot of design work for a pure refactor
524 2020-09-10T19:21:34  <wumpus> sdaftuar: +1
525 2020-09-10T19:21:41  <jeremyrubin> overall, same. It works as is.
526 2020-09-10T19:22:14  *** mol_ has quit IRC
527 2020-09-10T19:22:20  <luke-jr> I suspect it actually might make future changes harder :x
528 2020-09-10T19:22:39  <sipa> luke-jr: depends on how it's done, i think
529 2020-09-10T19:22:48  <jeremyrubin> I think that's the motivation; if future changes are leaning more on the recursive mutexing we don't want to make them that way
530 2020-09-10T19:22:51  <sipa> really usually cleaning up locking means cleaning up the boundary between interfaces
531 2020-09-10T19:23:01  <sipa> and if that's a side effect, it may definitely make things easier
532 2020-09-10T19:23:04  <promag> sipa: I'm taking about cs_main
533 2020-09-10T19:23:05  <jnewbery> sipa: +1
534 2020-09-10T19:23:10  <wumpus> sipa: yes, if it can make that clearer it's a good thing
535 2020-09-10T19:23:10  <promag> wumpus: agree
536 2020-09-10T19:23:14  <jnewbery> cs_main is something else entirely
537 2020-09-10T19:23:30  <sipa> just doing a dumb "whatever necessary to avoid recursive locking" is unlikely to be beneficial
538 2020-09-10T19:23:37  <wumpus> agree
539 2020-09-10T19:23:53  *** Pavlenex has quit IRC
540 2020-09-10T19:24:14  <promag> vasild: agree
541 2020-09-10T19:24:26  <sipa> but it's probably hard to speak very generically here
542 2020-09-10T19:24:43  <jnewbery> I think adding a lock-not-held version of all public functions which may or may not hold a lock is a pretty trivial change
543 2020-09-10T19:25:16  <promag> i'm meh with that approach
544 2020-09-10T19:25:19  <jeremyrubin> It's kinda annoying for other patches if you have to change 2 func signatures
545 2020-09-10T19:25:35  <wumpus> jnewbery: at least that can be done virtually mechanically which makes it easier to review
546 2020-09-10T19:25:36  <jnewbery> I think changing the function to assert the lock is held and then adding locks outside the class _isn't_ a good change
547 2020-09-10T19:25:53  <vasild> funcUnprotected() { do stuff; }; func() { LOCK(); funcUnprotected(); }
548 2020-09-10T19:25:54  <sipa> jnewbery: that does not seem crazy, but it's easier to judge with code
549 2020-09-10T19:26:11  <wumpus> jnewbery: +1
550 2020-09-10T19:26:18  <sdaftuar> i'm a bit confused -- would some of those functions not be used?
551 2020-09-10T19:26:30  <promag> jnewbery: IMO it's a good change if it comes with a follow up refactor
552 2020-09-10T19:26:40  <luke-jr> I don't see what's wrong with func() that locks if necessary, but also works inside an external lock :x
553 2020-09-10T19:26:55  <jnewbery> sdaftuar: you'd only add new functions where it may or may not already hold the lock
554 2020-09-10T19:27:20  <jeremyrubin> luke-jr: I think that's basically a recursive lock
555 2020-09-10T19:27:25  <sipa> luke-jr: it's a very abstract objection, but to me it is: code should be written to work inside our outside the private parts of a class that need locking
556 2020-09-10T19:27:34  <vasild> luke-jr: that's like re-implementing recursive mutex
557 2020-09-10T19:27:35  <jeremyrubin> the difference being maybe you can assert if you know you have lock already or not
558 2020-09-10T19:27:53  <luke-jr> sipa: this does..?
559 2020-09-10T19:27:55  <sipa> luke-jr: it's just hard to reason about the interface if you have code that works in both
560 2020-09-10T19:28:10  <wumpus> hand rolling a recursive-ish mutex sounds even worse than simply using one
561 2020-09-10T19:28:17  <sipa> please no
562 2020-09-10T19:28:22  <nehan> the argument in https://gist.github.com/hebasto/072ad3a9370641b035a36d08607a3d34 for why RecursiveMutex is bad indicates why it's bad to use RecursiveMutex in the first place (it is a symptom of an underlying problem). i do not see how it explains the benefit of removing RecursiveMutex without addressing the underlying problem.
563 2020-09-10T19:28:38  *** Highway61 has quit IRC
564 2020-09-10T19:28:38  <sipa> nehan: +1
565 2020-09-10T19:28:41  <nehan> which it sounds like what this dual-function thing is doing
566 2020-09-10T19:28:45  *** Highway62 has joined #bitcoin-core-dev
567 2020-09-10T19:28:58  <sipa> nehan: i think the dual function makes the interface layer explicit
568 2020-09-10T19:29:44  <nehan> so the benefit of it is making a small (perhaps important) step towards making the interface more clear?
569 2020-09-10T19:29:48  <luke-jr> my point wasn't to reinvent RM, but that RM has a use :p
570 2020-09-10T19:29:52  <jnewbery> and by making it explicit, brings to attention what the underlying problem may be
571 2020-09-10T19:29:58  <sipa> nehan: right, it makes it obvious where the problem is
572 2020-09-10T19:30:02  <sipa> it doesn't fix it
573 2020-09-10T19:30:06  <wumpus> nehan: yes, the general arguments against recursive mutex described there definitely make sense
574 2020-09-10T19:30:08  <sdaftuar> so i think the right next step would be to precisely define what the interface ought ot be
575 2020-09-10T19:30:17  <sdaftuar> and then we can work towards that
576 2020-09-10T19:30:18  <jeremyrubin> It sounds like it might be worth to prepare this PR and not merge it
577 2020-09-10T19:30:34  <jeremyrubin> Because if it indentifies what the problem is, then we can see if a fix can be built on it
578 2020-09-10T19:30:39  <sdaftuar> refactoring without a design in mind seems bad to me
579 2020-09-10T19:31:02  <jnewbery> sometimes it's not a problem to have a function that can be called with or without a lock. Sometimes you want to carry out multiple operations on an object atomically. That's not a problem, but it's always better to be explicit about what you're doing
580 2020-09-10T19:31:06  *** Highway62 is now known as Highway61
581 2020-09-10T19:31:37  *** justanotheruser has joined #bitcoin-core-dev
582 2020-09-10T19:32:18  *** bitcoin-git has joined #bitcoin-core-dev
583 2020-09-10T19:32:18  <bitcoin-git> [bitcoin] instagibbs opened pull request #19936: Test: batch rpc with params (master...batch_param) https://github.com/bitcoin/bitcoin/pull/19936
584 2020-09-10T19:32:19  *** bitcoin-git has left #bitcoin-core-dev
585 2020-09-10T19:33:14  <promag> sipa: WITH_LOCK(..., ...) is already an indicator
586 2020-09-10T19:33:25  <jnewbery> hebasto: did you have anything else that you wanted input on?
587 2020-09-10T19:33:33  <vasild> I think we can/should make mempool.cs mutex private
588 2020-09-10T19:33:51  <jnewbery> vasild: that's a lot of work
589 2020-09-10T19:34:00  <jnewbery> and might not even be desirable
590 2020-09-10T19:34:02  <hebasto> jnewbery: it seems all discussed. what is consensus?
591 2020-09-10T19:34:21  <promag> vasild: lots of refactors right?
592 2020-09-10T19:34:21  *** Highway62 has joined #bitcoin-core-dev
593 2020-09-10T19:34:24  <vasild> IMO amount of work would be comparable to making it non-recursive
594 2020-09-10T19:35:10  <jnewbery> vasild: not even close, I don't think. There are lots of places where we make multiple calls to the mempool atomically. Those would all need to be changed into higher-level interfaces to the mempool
595 2020-09-10T19:35:12  <wumpus> vasild: at least that'd remove the problem of public functions needing the lock held
596 2020-09-10T19:35:25  *** Highway61 has quit IRC
597 2020-09-10T19:35:25  *** Highway62 is now known as Highway61
598 2020-09-10T19:35:28  <wumpus> but yes, it's also much more work
599 2020-09-10T19:35:54  <vasild> ok, my assessment could be wrong
600 2020-09-10T19:35:54  *** sr_gi has quit IRC
601 2020-09-10T19:36:10  <jeremyrubin> wild idea: mempool via message passing only?
602 2020-09-10T19:36:12  *** sr_gi has joined #bitcoin-core-dev
603 2020-09-10T19:36:24  <jnewbery> vasild: eg all of this would need to become a single call to the mempool: https://github.com/bitcoin/bitcoin/blob/564e1ab0f3dc573bd3ea60a80f6649c361243df9/src/rpc/blockchain.cpp#L1406-L1421
604 2020-09-10T19:36:25  <jeremyrubin> Might be good as there are some proposals to make the mempool a separate entity
605 2020-09-10T19:36:44  <jeremyrubin> and message passing based would cleanly separate internal details
606 2020-09-10T19:37:01  <hebasto> vasild: that is possible
607 2020-09-10T19:37:16  <vasild> jnewbery: yes, I looked into that, it would be one call that returns a struct holding all the data, not a big deal
608 2020-09-10T19:38:02  *** Madars has joined #bitcoin-core-dev
609 2020-09-10T19:38:14  <jonatack> i like sdaftuar's suggestion that it be motivated by new behavior or a clear idea of the desired design/interface
610 2020-09-10T19:38:27  <jnewbery> vasild: maybe. It just feels like a much bigger change. If you had a branch that moved the lock to be totally internal, I'd be very interested to see it
611 2020-09-10T19:38:40  <vasild> mempool.AtomicallyGiveMeYourStats(&pool_stats); ;-)
612 2020-09-10T19:39:00  <vasild> jnewbery: no, I don't have
613 2020-09-10T19:39:15  <aj> time for the gui repo topic?
614 2020-09-10T19:39:24  <jeremyrubin> vasild: a mempool interior thread/workqueue with condvar for wake on return could implement that API :p
615 2020-09-10T19:39:25  <wumpus> I guess it would be good to see an example of the various proposals and maybe re-discuss next week
616 2020-09-10T19:39:46  <hebasto> wumpus: +1
617 2020-09-10T19:39:51  <wumpus> if this is all worth it at all
618 2020-09-10T19:40:03  <wumpus> maybe we're creaeting a problem where there's none
619 2020-09-10T19:40:27  <wumpus> #topic Review the experience of 3 month bitcoin-core/gui repository (jonasschnelli)
620 2020-09-10T19:40:39  <sdaftuar> i would reiterate that a design document for how the mempool would ideally interact with the rest of our software would be very helpful for evaulating any proposals
621 2020-09-10T19:40:48  <jonasschnelli> I'd like to collect feedback on how well the separation of the GUI repository has been received.
622 2020-09-10T19:41:16  <jonasschnelli> my feedback: it's seems still unclear how we merge things back to the main repository and if that has been documented.
623 2020-09-10T19:41:23  <wumpus> yes, what's the plan for merging back the GUI changes?
624 2020-09-10T19:41:32  <jonasschnelli> on top, the mix of PRs on both sides feels confusing to me.
625 2020-09-10T19:41:54  <jnewbery> I think Marco has a script that merges them into both repositories simultaneously, no?
626 2020-09-10T19:41:55  <jonasschnelli> Overall,... i would have prefered to split of the repository with a split off through a clean interface (process seperation in some ways)
627 2020-09-10T19:42:08  <wumpus> I think it's nice to have GUI design discussions etc in the other repo
628 2020-09-10T19:42:19  <jonasschnelli> jnewbery: could be. I don't know.
629 2020-09-10T19:42:35  <jonasschnelli> Yes. I also like the split of the discussions and some GUI only PRs.
630 2020-09-10T19:42:47  <jonasschnelli> I just don't know what burden we have when merging things back
631 2020-09-10T19:42:58  <wumpus> it's sufficiently distinct from what happens in the main repo that that makes sense
632 2020-09-10T19:43:05  <jonasschnelli> Also,... things that touch both "sides": still unclear how to handle that
633 2020-09-10T19:43:16  <wumpus> I don't know either, looks like Marco Falke is missing for this discussion
634 2020-09-10T19:43:22  <jonasschnelli> looks like
635 2020-09-10T19:43:27  <promag> wumpus: most people just have to go to both repos.. it was a good change for those that want stay away from gui stuff
636 2020-09-10T19:43:40  <jonasschnelli> However we go further, I think it would be nice to document the process more clear
637 2020-09-10T19:44:29  <jnewbery> they seem to be pretty synchronized. The GUI repository is just one merge commit behind bitcoin/bitcoin
638 2020-09-10T19:44:31  <jonasschnelli> Also unclear to me how we handle merge conflicts if we merge node stuff into the GUI repository
639 2020-09-10T19:44:39  <jeremyrubin> jonasschnelli: +1 on process isolation being important for making the separation clearer
640 2020-09-10T19:44:41  <luke-jr> did anyone ever get in touch with GitHub about the PR issues?
641 2020-09-10T19:44:53  <jnewbery> jonasschnelli: they're the same commits
642 2020-09-10T19:45:26  <jnewbery> there's no conflicts because it's always fast-forwards
643 2020-09-10T19:45:28  <wumpus> jeremyrubin: IIRC there's a PR that does process isolation
644 2020-09-10T19:45:29  <jonasschnelli> jnewbery: Right. I tihink its a non-issue as long as they stay in sync. Which I guess is a manual script
645 2020-09-10T19:46:02  <jeremyrubin> wumpus: ryanofsky's right?
646 2020-09-10T19:46:02  <michaelfolkson> I think the major concern is if review decreases with it being separate. I think people on the GUI repo need to regularly prod (cc) GUI reviewers who spend most of their time on the main repo.
647 2020-09-10T19:46:22  <wumpus> ryanofsky is working on that #19461
648 2020-09-10T19:46:25  <gribble> https://github.com/bitcoin/bitcoin/issues/19461 | multiprocess: Add bitcoin-gui -ipcconnect option by ryanofsky · Pull Request #19461 · bitcoin/bitcoin · GitHub
649 2020-09-10T19:46:25  <wumpus> yes
650 2020-09-10T19:46:33  <jonasschnelli> michaelfolkson: maybe only an initial issue. I think there are now also contributors stepping forward _because_ its GUI only
651 2020-09-10T19:47:06  <hebasto> ^ and designers
652 2020-09-10T19:47:07  <wumpus> jonasschnelli: exactly, though there's some overlap, it's also meant to get a different group of people involved
653 2020-09-10T19:47:22  <michaelfolkson> But mainly designers right jonasschnelli? It needs normal Core reviewers too
654 2020-09-10T19:48:03  <michaelfolkson> I think for design related review it is a material success so far
655 2020-09-10T19:48:08  <wumpus> ideally we want more GUI contributors, improving the GUI, they don't necessarily need to get involved with the rest of the code
656 2020-09-10T19:48:18  <jonasschnelli> michaelfolkson: Yes. Thats correct.
657 2020-09-10T19:48:29  <jonasschnelli> It's a different complexity level (mostly). Also the risks are different.
658 2020-09-10T19:48:36  <vasild> I think a separate GUI repository only makes sense as long as both repositories contain different chunks of software. Right now both contain the same, e.g. the gui repo contains bitcoind.cpp, net_processing.cpp, etc. Would it be possible to make only src/qt in the gui repo (and then make it a git submodule in the main one)?
659 2020-09-10T19:48:45  <wumpus> a different kind of complexity to navigate at least
660 2020-09-10T19:48:55  <jeremyrubin> wumpus: is there a framework for moving forward ryanofsky's work? last I checked it was still nack on capnproto?
661 2020-09-10T19:49:15  <wumpus> I definitely found out I can't do it, I can't handle the subjectivity and bikeshedding involved :)
662 2020-09-10T19:49:37  <jonasschnelli> vasild: this is a valid point
663 2020-09-10T19:49:52  <jonasschnelli> wumpus : that. yes.
664 2020-09-10T19:50:08  <wumpus> vasild: maybe, though it can't be built independently anymore then, it's harder for contributors
665 2020-09-10T19:50:28  <hebasto> bikeshedding is a work for designers, no?
666 2020-09-10T19:50:31  <yanmaani> Can't you refactor it to make the GUI an RPC client?
667 2020-09-10T19:50:33  <vasild> hm, right, src/qt can't be build separately
668 2020-09-10T19:50:50  <yanmaani> (and then optionally have it spawn a bitcoind)
669 2020-09-10T19:50:55  <vasild> make make it compilable, libsrcqt.so :)
670 2020-09-10T19:51:03  <jonasschnelli> yanmaani: I would also like to see that
671 2020-09-10T19:51:16  <yanmaani> Then the GUI could indeed be moved to a separate repo
672 2020-09-10T19:51:23  <jonasschnelli> but I guess we are drifting off-topic
673 2020-09-10T19:51:24  <yanmaani> and you could have other people making other GUIs in other frameworks
674 2020-09-10T19:51:26  <wumpus> yanmaani: that's what the multiprocess PR does, basically
675 2020-09-10T19:51:28  <sipa> people tried that in 2012 or so, but it's terrible as RPC is query-response based, not asynchronous
676 2020-09-10T19:51:46  <sipa> but the multiprocess work does this the right way
677 2020-09-10T19:51:46  <jonasschnelli> sipa: we could use the long polling to make it faster though
678 2020-09-10T19:51:47  <yanmaani> sipa: is RPC single-threaded?
679 2020-09-10T19:51:48  <wumpus> yanmaani: (except it doesn't use JSON-RPC but its own RPC mechanism)
680 2020-09-10T19:52:00  <promag> yanmaani: no
681 2020-09-10T19:52:03  <wumpus> in any case ryanofsky's work already exists
682 2020-09-10T19:52:10  <sipa> yanmaani: it would be more useful if you'd comment after you've spent some time looking at the code
683 2020-09-10T19:52:21  <yanmaani> ok :)
684 2020-09-10T19:52:34  <wumpus> no need to brainstorm the 2012 idea of doing it again now
685 2020-09-10T19:52:37  <provoostenator> I suggest waiting a while before opening that can of worms though :-)
686 2020-09-10T19:52:40  <jnewbery> yanmaani: even more useful if you review/test ryanofsky's work!
687 2020-09-10T19:53:02  <jonasschnelli> I think I suggest then that we pimp up the documentation on how things are managed between the two repositories, avoid questions, etc.
688 2020-09-10T19:53:17  <wumpus> jonasschnelli: +1
689 2020-09-10T19:53:19  <jeremyrubin> wumpus: if some set of maintainers can establish a bit more of a framework on the validity of ryanofsky's approach i would spend some cycles on it. but AFAIU some contribs are nack on the approach so I'm not sure how likely it will proceed if that remains the case or what can be done to un-nack
690 2020-09-10T19:53:21  <jonasschnelli> Let's hope MarcoFalke has time and willingness to do this
691 2020-09-10T19:53:22  <wumpus> just documenting things would make sense
692 2020-09-10T19:53:40  <wumpus> jeremyrubin: who is completely against it?
693 2020-09-10T19:53:43  <jonasschnelli> people deserve to know what happens with PRs they write on the GUI repo
694 2020-09-10T19:53:45  <jeremyrubin> gmax iirc?
695 2020-09-10T19:54:15  <jeremyrubin> iirc he was against capnproto at all and wanted a custom IPC lib
696 2020-09-10T19:54:16  <hebasto> +1 for documenting
697 2020-09-10T19:54:24  <wumpus> for using capnproto for the GUI? I don't think so, he hates using it for internet protocols, but for internal communication between processes it doesn't matter too much
698 2020-09-10T19:54:41  <sipa> no reason to speculate here
699 2020-09-10T19:54:48  *** opsec_x122 has joined #bitcoin-core-dev
700 2020-09-10T19:54:56  <sipa> afaik ryanofsky said that capnproto could easily be swapped out for something else if needed
701 2020-09-10T19:55:06  <jonasschnelli> just the glue
702 2020-09-10T19:55:11  <provoostenator> Indeed
703 2020-09-10T19:55:12  <wumpus> I'd really dislike rolling our own IPC mechanism tbh
704 2020-09-10T19:55:19  <jeremyrubin> do we already depend on it?
705 2020-09-10T19:55:25  <jeremyrubin> It's in depends :)
706 2020-09-10T19:55:38  <wumpus> there are so many one already, not everything needs to be invented here
707 2020-09-10T19:55:44  <provoostenator> We already merged the build stuff for it.
708 2020-09-10T19:55:54  <jonasschnelli> I'm still in favour or RPC
709 2020-09-10T19:55:55  <jeremyrubin> https://github.com/bitcoin/bitcoin/pull/10102#issuecomment-289842646
710 2020-09-10T19:55:56  <provoostenator> But it's an opt-in config flag
711 2020-09-10T19:55:58  <wumpus> sipa: yes, it should be easy to swap out
712 2020-09-10T19:56:14  <wumpus> he designed it so that the specific mechanism used is abstracted
713 2020-09-10T19:56:15  <jnewbery> jonasschnelli: I think it's even easier than I said. Marco just merges the gui PR branch into the bitcoin/bitcoin repo
714 2020-09-10T19:56:18  <wumpus> better just review the code...
715 2020-09-10T19:56:29  <jonasschnelli> RPC can also work async. But yes. Not the topic now.
716 2020-09-10T19:56:46  <wumpus> jonasschnelli: I mean we can do something wildly different but *only* if someone is willing to do the work
717 2020-09-10T19:56:47  <jnewbery> and then I guess there's a bot or something that is syncing bitcoin-core/gui to the latest master
718 2020-09-10T19:56:52  <jonatack> there is a good recent writeup by ryanofsky, one month ago, here: https://bitcoincore.reviews/19160
719 2020-09-10T19:56:57  <wumpus> I see no reason to second-guess ryanofsky now after all this time
720 2020-09-10T19:57:00  <jonasschnelli> wumpus: that's a point.
721 2020-09-10T19:57:02  <jnewbery> https://github.com/bitcoin/bitcoin/pull/19071
722 2020-09-10T19:57:14  <jeremyrubin> I have no issue with capnproto I think it's fine :)
723 2020-09-10T19:57:18  <provoostenator> Last time I dug into it I found the approach pretty sane.
724 2020-09-10T19:57:33  <wumpus> at least it's a step forward
725 2020-09-10T19:57:34  <jeremyrubin> I spent a bunch of time reviewing it in '17 and think it's a good approach
726 2020-09-10T19:57:44  <wumpus> when RPC one day converges on what we need for the GUI, then we can switch to that
727 2020-09-10T19:58:16  *** opsec_x12 has quit IRC
728 2020-09-10T19:58:18  <jonasschnelli> Haven't looked into it. But separating the GUI from the node via the internet/wireguard would be a requirement,.. right?
729 2020-09-10T19:58:22  <wumpus> I think it's *very important* to take a step by step approach with things
730 2020-09-10T19:58:23  <provoostenator> Conversely, when the IPC interface simplifies and needs fewer locks...
731 2020-09-10T19:59:07  <wumpus> if not, we'll never go anywhere, this kind of thing has been proposed since at least 2012
732 2020-09-10T19:59:16  <jonasschnelli> indeed
733 2020-09-10T19:59:20  <wumpus> because everyone wants something else
734 2020-09-10T19:59:29  <jonasschnelli> the power of open source
735 2020-09-10T19:59:30  <wumpus> and then someone does it and peopel want yet something else
736 2020-09-10T19:59:59  <jeremyrubin> I think that's all I'm asking. If ryanofsky's pr is reasonably reviewed
737 2020-09-10T20:00:03  <wumpus> so, anyhow, if you're interested in process separation, please review ryanofsky's PRs
738 2020-09-10T20:00:06  <jeremyrubin> there is no 'hard reason' it can't be merged
739 2020-09-10T20:00:30  <jeremyrubin> which is great news for anyone willing to spend review cycles on it, it derisks that time spent
740 2020-09-10T20:00:33  <wumpus> dont' come up with new wild ideas that would have to start from scratch :)
741 2020-09-10T20:00:47  <sdaftuar> jeremyrubin: i would hope not, my understanding is that ryanofsky has been making progress towards this goal for quite some time now
742 2020-09-10T20:00:51  *** arowser has quit IRC
743 2020-09-10T20:01:18  *** arowser has joined #bitcoin-core-dev
744 2020-09-10T20:01:28  <wumpus> of course after reviewing the PR you could give more targeted suggestions
745 2020-09-10T20:01:38  <wumpus> oh, it's time
746 2020-09-10T20:01:42  <wumpus> #endmeeting
747 2020-09-10T20:01:42  <lightningbot> Meeting ended Thu Sep 10 20:01:42 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
748 2020-09-10T20:01:42  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2020/bitcoin-core-dev.2020-09-10-19.00.html
749 2020-09-10T20:01:42  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2020/bitcoin-core-dev.2020-09-10-19.00.txt
750 2020-09-10T20:01:42  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2020/bitcoin-core-dev.2020-09-10-19.00.log.html
751 2020-09-10T20:01:48  <jonasschnelli> \o
752 2020-09-10T20:01:57  <vasild> o/
753 2020-09-10T20:02:01  <jonatack> psa, the signet PR seems to be in the final stretch if anyone wants to have a look
754 2020-09-10T20:02:09  <wumpus> yay!
755 2020-09-10T20:02:30  <jonatack> \o
756 2020-09-10T20:02:59  <michaelfolkson> Final stretch as in we can start a barrage of follow up smaller signet PRs
757 2020-09-10T20:03:03  *** Madars has quit IRC
758 2020-09-10T20:03:16  <wumpus> but FWIW I had the idea that the capnproto approach for IPC communication was widely concept-ACKed a long time ago, that's why the build system part was merged
759 2020-09-10T20:03:37  <jeremyrubin> I hadn't tracked that development :)
760 2020-09-10T20:03:58  <luke-jr> I don't see why we need two RPC interfaces ;)
761 2020-09-10T20:04:10  <wumpus> it's not a RPC interface it's internal interface only
762 2020-09-10T20:04:21  <wumpus> it will not be documented for external clients so it has much more flexibility
763 2020-09-10T20:04:37  <wumpus> it will also not be compatible between versions
764 2020-09-10T20:04:43  <luke-jr> no matter how we use it, it's an RPC interface..
765 2020-09-10T20:04:49  <jeremyrubin> I reviewed the process isolation stuff relatively early on, but it wasn't clear that it was going to move forward and didn't think i could do anything to move the needle on making an IPC interface available.
766 2020-09-10T20:04:59  <luke-jr> we could just as well add an undocumented and unstable JSON-RPC interface
767 2020-09-10T20:04:59  <jeremyrubin> now that I know it is, I'll re-review :)
768 2020-09-10T20:05:10  <wumpus> there's a very different set of requirements for it anyhow
769 2020-09-10T20:05:18  <wumpus> but as said maybe they can converge in the future
770 2020-09-10T20:05:21  <jeremyrubin> luke-jr: the difference is that you can fork a process and explicitly set up the IPC channel
771 2020-09-10T20:05:24  <wumpus> the idea is to allow for some experimentation here
772 2020-09-10T20:05:39  <jeremyrubin> so that the connection is internal to one parent process
773 2020-09-10T20:06:15  <jeremyrubin> You could maybe do this for RPC, but the point of IPC is to allow deeper integration & work with e.g. shared memory.
774 2020-09-10T20:06:39  <luke-jr> so it won't be possible to run the GUI on another machine? :/
775 2020-09-10T20:06:39  <wumpus> it's simply a pipe, it could send every protocol, but in any case, capnproto is ok for now imo
776 2020-09-10T20:07:11  <sipa> jeremyrubin: first thing i hear about shared memory
777 2020-09-10T20:07:21  <wumpus> I doubt there are any plans about shared memory
778 2020-09-10T20:08:09  <wumpus> if gmaxwell is queasy about capnproto I'm sure he'll go insane when you mention shared memory :-)
779 2020-09-10T20:08:31  <wumpus> it's also really not needed for this
780 2020-09-10T20:10:35  <yanmaani> It's two different approaches. With well-defined RPC and all that, you can run it on separate machines, and you can have other projects writing GUIs. With shared memory and more "usual" IPC, you're more tightly coupled. So you can iterate faster, but you lose some of the benefits of keeping -gui separate
781 2020-09-10T20:10:40  <yanmaani> if it always has to be the same version, for instance
782 2020-09-10T20:11:07  <wumpus> yes
783 2020-09-10T20:12:51  <luke-jr> fwiw someone mentioned to me the other day a desire to revive wxBitcoin - better separation helps that too
784 2020-09-10T20:13:20  <jeremyrubin> iirc capnproto had some shared memory stuff but i may be wrong
785 2020-09-10T20:14:02  <yanmaani> If you do the separation very cleanly you could have hundreds of random bozos making GUIs. Whether this is great or horrifying is up to your imagination.
786 2020-09-10T20:15:24  <wumpus> revive wxbitcoin?!? whyy
787 2020-09-10T20:15:30  <sipa> lol
788 2020-09-10T20:15:39  <sipa> has 3.9 been released already?
789 2020-09-10T20:15:42  <yanmaani> It was I. I don't like Qt.
790 2020-09-10T20:16:01  <wumpus> sigh, I don't even want to know, I want to have 0 to do with the GUI anymore
791 2020-09-10T20:16:07  <sipa> eh, 2.9 - apparently yes, they're at 3.1.4
792 2020-09-10T20:16:48  <wumpus> sipa: progressss
793 2020-09-10T20:17:07  <sipa> yanmaani: use RPC :p
794 2020-09-10T20:17:08  <yanmaani> well, with these changes, you can just jettison the unloved components of the codebase
795 2020-09-10T20:17:49  <wumpus> I just dislike all the bikeshedding, everyone wants something else, I think "I don't like library X" is an awful reason
796 2020-09-10T20:17:55  <yanmaani> Or I can procrastinate on it and wait for these changes to be made, and use whatever you eventually bikeshed on.
797 2020-09-10T20:18:52  <wumpus> imo ideally the GUI design should really be made by someone that gets GUIs at a psychological level and designs something that is usable by end-users, not by developers bikeshedding over libraries and widget types
798 2020-09-10T20:19:11  <yanmaani> imo you should have two groups
799 2020-09-10T20:19:26  <yanmaani> one group of designers who know design who design the psychologically optimal GUI in photoshop
800 2020-09-10T20:19:30  <wumpus> developes come with ever sillier concerns
801 2020-09-10T20:19:33  <yanmaani> one group of developers who bikeshed over widget types
802 2020-09-10T20:19:39  <sipa> yanmaani: will you pay them all?
803 2020-09-10T20:19:43  <yanmaani> (and implement the mockup)
804 2020-09-10T20:20:23  <wumpus> it's interaction design, you can't really do that in photoshop
805 2020-09-10T20:20:41  <sipa> UX is far more than graphical design
806 2020-09-10T20:20:47  <wumpus> ^^ very much this
807 2020-09-10T20:20:55  <yanmaani> I actually like the way the current GUI looks, and the fact that Electrum has converged on something extremely similar to Core is reassuring
808 2020-09-10T20:22:00  <phantomcircuit> what's missing from the current gui is mostly debug related things, but im guessing the intersection between people who need debug stuff and people who won't use the cli is pretty small
809 2020-09-10T20:22:18  <sipa> i haven't used the GUI in years
810 2020-09-10T20:23:01  <luke-jr> ♥ current GUI
811 2020-09-10T20:23:14  <yanmaani> ^
812 2020-09-10T20:23:31  <sipa> then what's the problem with Qt?
813 2020-09-10T20:23:46  <wumpus> phantomcircuit: what debug information are you missing?
814 2020-09-10T20:24:32  * luke-jr ♥ Qt too
815 2020-09-10T20:24:40  <achow101> the current gui kinda sucks with multiwallet
816 2020-09-10T20:24:46  <achow101> it's really easy to accidentally use the wrong wallet
817 2020-09-10T20:25:00  <luke-jr> admittedly, I don't use multiwallet much yet
818 2020-09-10T20:25:08  <luke-jr> too much in the habit of closing one and opening another
819 2020-09-10T20:25:20  <wumpus> I only use multiwallet for watch-only things
820 2020-09-10T20:25:30  <gwillen> I mostly use multiwallet for manual testing tbh, but it's fantastic for that
821 2020-09-10T20:26:09  <achow101> I run multiple wallets watching the same thing lol
822 2020-09-10T20:26:22  <achow101> tbf, one is legacy, another descriptor, and the last sqlite descriptor
823 2020-09-10T20:26:30  <wumpus> hehe
824 2020-09-10T20:27:01  <phantomcircuit> wumpus, "is this transaction in the mempool", "how big is the mempool" ie things that aren't actually relevant to most users
825 2020-09-10T20:27:24  <wumpus> phantomcircuit: ah yes, mempool information is a bit sparse
826 2020-09-10T20:27:35  <achow101> phantomcircuit: how big is the mempool is reported
827 2020-09-10T20:28:00  <sipa> i'd like a debug tool with a secp256k1 DL solver
828 2020-09-10T20:28:02  <luke-jr> phantomcircuit: Knots has some mempool graph stuff jonasschnelli wrote years ago
829 2020-09-10T20:28:04  <sipa> would come in really.handy
830 2020-09-10T20:28:21  <wumpus> did we never merge the mempool graph stuff?
831 2020-09-10T20:28:32  <wumpus> I thought that was pretty nice
832 2020-09-10T20:28:48  <luke-jr> wumpus: I have rebased versions of an older version of the PR
833 2020-09-10T20:28:59  <jonasschnelli> I have picked this up recently
834 2020-09-10T20:29:03  <luke-jr> jonasschnelli changed the RPC side of it all around, but never updated the GUI for it
835 2020-09-10T20:29:06  <luke-jr> ah, nice
836 2020-09-10T20:29:12  <wumpus> that's the kind of things I like about the GUI tbh, debug and data visualization
837 2020-09-10T20:29:18  <sipa> indeed
838 2020-09-10T20:29:32  <jonasschnelli> Made it more like Jochen Hoenick ones
839 2020-09-10T20:29:40  <jonasschnelli> With few ranges
840 2020-09-10T20:29:57  <jonasschnelli> I’ll PR that soon.
841 2020-09-10T20:30:01  <wumpus> jonasschnelli: awesome!
842 2020-09-10T20:30:22  <wumpus> I think it would be very nice if users can see those things locally, without having to go to a site
843 2020-09-10T20:30:46  *** Aracely71Hammes has joined #bitcoin-core-dev
844 2020-09-10T20:31:52  <jeremyrubin> [9/10/20 13:28] <sipa> i'd like a debug tool with a secp256k1 DL solver
845 2020-09-10T20:32:07  <jeremyrubin> I'd think you'd see this before a good QT GUI debugger
846 2020-09-10T20:32:51  <jeremyrubin> The only question is if you can make a good trapdoor function out of bugs in QT
847 2020-09-10T20:33:24  <jeremyrubin> BIP QT-root
848 2020-09-10T20:35:41  *** Aracely71Hammes has quit IRC
849 2020-09-10T20:36:01  <wumpus> a DL solver would definitely be useful :')
850 2020-09-10T20:36:40  <wumpus> at least if it doesn't hold up the GUI loop for a million billion years
851 2020-09-10T20:37:37  <sipa> haha
852 2020-09-10T20:37:45  *** Highway61 has quit IRC
853 2020-09-10T20:39:22  <promag> <sipa> i haven't used the GUI in years
854 2020-09-10T20:39:25  <promag> because GUI uses sipa
855 2020-09-10T20:39:27  <jonasschnelli> for the ones interested in the mempool size/fee chart: https://github.com/bitcoin/bitcoin/compare/master...jonasschnelli:2020/03/mempool_graph?expand=1
856 2020-09-10T20:39:32  <jonasschnelli> (it's WIP)
857 2020-09-10T20:40:19  <jonasschnelli> It's more or less a clone of https://jochen-hoenicke.de/queue/#0,24h
858 2020-09-10T20:40:20  <jonasschnelli> (but obviously internal and QT native)
859 2020-09-10T20:40:20  <gribble> https://github.com/bitcoin/bitcoin/issues/0 | HTTP Error 404: Not Found
860 2020-09-10T20:47:00  <phantomcircuit> sipa, btw going from 1000 to 100k keys only added 50% to the runtime of a rescan
861 2020-09-10T20:47:33  <sipa> phantomcircuit: wait until you hit enough keys that the mapKeys structure doesn't fit in RAM anymore
862 2020-09-10T20:47:42  <sipa> ;)
863 2020-09-10T20:47:49  <luke-jr> could always use a bloom filter for the wallet keys, to speed it up ;)
864 2020-09-10T20:48:04  <phantomcircuit> sipa, that would be a very very large number of keys
865 2020-09-10T20:48:17  <sipa> phantomcircuit: you just have too much RAM
866 2020-09-10T20:48:19  <sipa> ;)
867 2020-09-10T20:49:04  <sipa> phantomcircuit: more seriously, i think the actual impact of number of keys on rescan speed isn't so much the O(log n) scaling, but whether or not the mapKeys fits in various levels of CPU cache or RAM
868 2020-09-10T20:49:42  <phantomcircuit> sipa, indeed
869 2020-09-10T20:50:30  <wumpus> jonasschnelli: that's great!
870 2020-09-10T20:50:36  <phantomcircuit> luke-jr, the gcs filters that 'neutrino' use are what i was using, but to actually use them requires changing violating some assumptions the current logic has (namely that every block derives the filter key from the block hash)
871 2020-09-10T20:54:50  *** promag has quit IRC
872 2020-09-10T20:54:59  *** promag has joined #bitcoin-core-dev
873 2020-09-10T20:55:47  *** promag has joined #bitcoin-core-dev
874 2020-09-10T20:55:47  *** tryphe has joined #bitcoin-core-dev
875 2020-09-10T20:57:06  *** Madars has joined #bitcoin-core-dev
876 2020-09-10T20:57:17  *** vincenzopalazzo has quit IRC
877 2020-09-10T20:57:45  *** tryphe_ has quit IRC
878 2020-09-10T21:00:01  *** popey1 has quit IRC
879 2020-09-10T21:00:16  *** promag has quit IRC
880 2020-09-10T21:00:48  *** promag has joined #bitcoin-core-dev
881 2020-09-10T21:01:11  *** Guyver2 has quit IRC
882 2020-09-10T21:01:58  <luke-jr> phantomcircuit: I meant the other direction
883 2020-09-10T21:02:07  *** promag has quit IRC
884 2020-09-10T21:02:30  *** promag has joined #bitcoin-core-dev
885 2020-09-10T21:04:47  *** Highway61 has joined #bitcoin-core-dev
886 2020-09-10T21:05:31  <phantomcircuit> luke-jr, ?
887 2020-09-10T21:05:41  <phantomcircuit> oh put the wallet keys into a filter
888 2020-09-10T21:05:44  <luke-jr> yes
889 2020-09-10T21:06:10  <luke-jr> if you do it right, you might even be able to do a union of the neutrino filters with the wallet filter..
890 2020-09-10T21:06:11  <phantomcircuit> iirc most of the time for the current rescan is actually deserializing the block and verifying the merkle root
891 2020-09-10T21:06:24  <achow101> sipa: interestingly on my branch where i've changed ~all of the wallet maps and sets to unordered_map/set, the rescan actually takes 10x longer. On both newly created legacy and descriptor wallets
892 2020-09-10T21:06:26  <luke-jr> you don't need to verify the merkle root
893 2020-09-10T21:06:52  <achow101> seems like siphash is a lot slower than one map comparison?
894 2020-09-10T21:07:08  *** Highway61 has quit IRC
895 2020-09-10T21:07:15  <sipa> achow101: interesting
896 2020-09-10T21:07:48  <achow101> actually, siphash probably isn't being used there, the hashers for CKeyID and CScriptID just use uint's GetUint64
897 2020-09-10T21:08:44  <phantomcircuit> luke-jr, ok but it does verify the merkle tree because you're loading the block from disk
898 2020-09-10T21:09:39  <yanmaani> would PRs be accepted where you replace std::unordered_map with something else for CCoinsViewCache, or is that not worth looking into?
899 2020-09-10T21:10:46  <sipa> if it's measurably faster, possibly
900 2020-09-10T21:10:48  <wumpus> jonasschnelli: will try it out a bit tomorrow
901 2020-09-10T21:11:46  <phantomcircuit> luke-jr, anyways if you setup the gcs filter index things using a local static key then you can calculate the hashes that you need to check against and save them, which makes the rescan basically just finding the union of two sets
902 2020-09-10T21:12:17  <luke-jr> phantomcircuit: but make both sets a filter
903 2020-09-10T21:12:48  <phantomcircuit> luke-jr, i guess you could take the gcs set and make a bloom filter on top of that?
904 2020-09-10T21:12:51  <wumpus> besides being demostratively faster, it also depends on 'something else', e.g. probably not if it introduces any new dependencies on boost, or anything else external
905 2020-09-10T21:13:07  <phantomcircuit> iono i suspect that would actually be slower than just doing the giant union of the sets calculation
906 2020-09-10T21:13:19  <wumpus> also as it's a consensus critical data structure it's something to be very careful with
907 2020-09-10T21:13:38  <luke-jr> bdb!
908 2020-09-10T21:13:43  * luke-jr hides
909 2020-09-10T21:14:12  <jb55> achow101: agreed on multiwallet gui. this is something I still use cli for over qt. would be great to have an "all wallets" transaction view.
910 2020-09-10T21:19:00  <sipa> jb55: if you have the same key/address/descriptor imported in multiple simultaneously-loaded wallets, an "all wallets" overview may be confusing
911 2020-09-10T21:19:39  * sipa predicts people creating 1M wallets with the same 21 BTC UTXO on it, and screenshottijg their 21M BTC combined balance
912 2020-09-10T21:20:26  <achow101> sipa: it feels like importing the same thing into multiple wallets might not be something people tend to do
913 2020-09-10T21:20:27  <luke-jr> sipa: then don't use it? :P
914 2020-09-10T21:20:29  <jb55> sipa: maybe they shouldn't do that :P
915 2020-09-10T21:20:55  <sipa> jb55: seems achow101 just did!
916 2020-09-10T21:21:43  <achow101> sipa: I suspect that you won't be able to load 1M wallets
917 2020-09-10T21:22:14  *** Theopolisme has joined #bitcoin-core-dev
918 2020-09-10T21:22:28  <sipa> achow101: software should be optimized for all use cases!
919 2020-09-10T21:22:36  *** luke-jr has quit IRC
920 2020-09-10T21:22:50  <jb55> 150m keys should be instant imo
921 2020-09-10T21:23:27  *** luke-jr has joined #bitcoin-core-dev
922 2020-09-10T21:23:50  *** Madars has quit IRC
923 2020-09-10T21:31:26  *** tryphe has quit IRC
924 2020-09-10T21:32:11  *** tryphe has joined #bitcoin-core-dev
925 2020-09-10T22:03:58  *** Madars has joined #bitcoin-core-dev
926 2020-09-10T22:07:12  *** kristapsk has quit IRC
927 2020-09-10T22:08:21  *** kristapsk has joined #bitcoin-core-dev
928 2020-09-10T22:34:06  *** Eagle[TM] has joined #bitcoin-core-dev
929 2020-09-10T22:35:37  *** EagleTM has quit IRC
930 2020-09-10T22:38:41  *** bitcoin-git has joined #bitcoin-core-dev
931 2020-09-10T22:38:41  <bitcoin-git> [bitcoin] ajtowns opened pull request #19937: signet mining utility (master...202009-signet-generate) https://github.com/bitcoin/bitcoin/pull/19937
932 2020-09-10T22:38:43  *** bitcoin-git has left #bitcoin-core-dev
933 2020-09-10T22:59:23  *** vasild has quit IRC
934 2020-09-10T23:01:24  *** vasild has joined #bitcoin-core-dev
935 2020-09-10T23:02:06  *** Aaronvan_ has quit IRC
936 2020-09-10T23:16:57  *** promag has quit IRC
937 2020-09-10T23:21:59  *** mdunnio has quit IRC
938 2020-09-10T23:26:29  *** shaunsun has quit IRC
939 2020-09-10T23:34:58  *** AaronvanW has joined #bitcoin-core-dev
940 2020-09-10T23:39:38  *** AaronvanW has quit IRC
941 2020-09-10T23:42:59  *** lightlike has quit IRC
942 2020-09-10T23:45:52  *** sr_gi has quit IRC
943 2020-09-10T23:45:52  *** mdunnio has joined #bitcoin-core-dev
944 2020-09-10T23:46:25  *** sr_gi has joined #bitcoin-core-dev
945 2020-09-10T23:50:19  *** mdunnio has quit IRC
946 2020-09-10T23:53:48  *** Sambij has joined #bitcoin-core-dev