1 2017-11-27T00:01:45  *** daedal_ has joined #bitcoin-core-dev
  2 2017-11-27T00:01:52  *** qrest has quit IRC
  3 2017-11-27T00:05:35  *** daedal__ has quit IRC
  4 2017-11-27T00:06:33  *** AaronvanW has joined #bitcoin-core-dev
  5 2017-11-27T00:08:45  *** intcat has quit IRC
  6 2017-11-27T00:08:54  *** Aaronvan_ has joined #bitcoin-core-dev
  7 2017-11-27T00:09:37  *** intcat has joined #bitcoin-core-dev
  8 2017-11-27T00:12:27  *** AaronvanW has quit IRC
  9 2017-11-27T00:17:56  *** wxss has quit IRC
 10 2017-11-27T00:20:42  *** qrest has joined #bitcoin-core-dev
 11 2017-11-27T00:26:50  *** promag has quit IRC
 12 2017-11-27T00:27:59  *** intcat has quit IRC
 13 2017-11-27T00:28:59  *** intcat has joined #bitcoin-core-dev
 14 2017-11-27T00:30:40  *** pandabull has joined #bitcoin-core-dev
 15 2017-11-27T00:31:22  *** pandabull has quit IRC
 16 2017-11-27T00:33:24  *** JamesAU has joined #bitcoin-core-dev
 17 2017-11-27T00:35:04  *** JamesAU has quit IRC
 18 2017-11-27T00:35:27  *** JamesAU has joined #bitcoin-core-dev
 19 2017-11-27T00:35:29  *** Chris_Stewart_5 has quit IRC
 20 2017-11-27T00:35:43  *** intcat has quit IRC
 21 2017-11-27T00:36:26  *** intcat has joined #bitcoin-core-dev
 22 2017-11-27T00:43:46  *** cyber55 has joined #bitcoin-core-dev
 23 2017-11-27T00:48:23  *** karelb has quit IRC
 24 2017-11-27T00:51:29  *** jtimon has quit IRC
 25 2017-11-27T00:53:32  *** jb55 has joined #bitcoin-core-dev
 26 2017-11-27T00:59:21  *** dabura667 has joined #bitcoin-core-dev
 27 2017-11-27T01:03:26  *** jb55 has quit IRC
 28 2017-11-27T01:05:45  *** Randolf has joined #bitcoin-core-dev
 29 2017-11-27T01:09:57  *** Randolf has quit IRC
 30 2017-11-27T01:10:32  *** Randolf has joined #bitcoin-core-dev
 31 2017-11-27T01:17:06  *** Victorsueca has quit IRC
 32 2017-11-27T01:25:05  *** Victorsueca has joined #bitcoin-core-dev
 33 2017-11-27T01:29:35  *** Randolf has quit IRC
 34 2017-11-27T01:43:47  *** satwo has quit IRC
 35 2017-11-27T01:53:01  *** Emcy_ has joined #bitcoin-core-dev
 36 2017-11-27T01:54:53  *** Emcy has quit IRC
 37 2017-11-27T01:55:44  *** wunpunch has quit IRC
 38 2017-11-27T01:56:30  *** quantbot has joined #bitcoin-core-dev
 39 2017-11-27T01:57:05  *** quantbot has joined #bitcoin-core-dev
 40 2017-11-27T01:57:25  *** quantbot has joined #bitcoin-core-dev
 41 2017-11-27T01:58:05  *** quantbot has joined #bitcoin-core-dev
 42 2017-11-27T02:01:45  *** l3gion has quit IRC
 43 2017-11-27T02:01:58  *** Emcy has joined #bitcoin-core-dev
 44 2017-11-27T02:02:39  *** quantbot has quit IRC
 45 2017-11-27T02:03:11  *** Emcy_ has quit IRC
 46 2017-11-27T02:04:46  *** justanotheruser has joined #bitcoin-core-dev
 47 2017-11-27T02:06:47  *** daedal__ has joined #bitcoin-core-dev
 48 2017-11-27T02:09:41  *** daedal_ has quit IRC
 49 2017-11-27T02:13:50  *** justanotheruser has quit IRC
 50 2017-11-27T02:14:07  *** justanotheruser has joined #bitcoin-core-dev
 51 2017-11-27T02:15:52  *** satwo has joined #bitcoin-core-dev
 52 2017-11-27T02:15:57  *** Ylbam has quit IRC
 53 2017-11-27T02:21:51  *** quantbot has joined #bitcoin-core-dev
 54 2017-11-27T02:22:35  *** quantbot has quit IRC
 55 2017-11-27T02:26:05  *** StopAndDecrypt has quit IRC
 56 2017-11-27T02:26:38  *** StopAndDecrypt has joined #bitcoin-core-dev
 57 2017-11-27T02:26:38  *** StopAndDecrypt has quit IRC
 58 2017-11-27T02:26:38  *** StopAndDecrypt has joined #bitcoin-core-dev
 59 2017-11-27T02:46:23  *** go1111111 has quit IRC
 60 2017-11-27T02:46:27  *** go11111111111 has quit IRC
 61 2017-11-27T02:49:47  *** meshcollider has quit IRC
 62 2017-11-27T02:59:21  *** jcorgan has quit IRC
 63 2017-11-27T02:59:51  *** Aaronvan_ has quit IRC
 64 2017-11-27T03:00:30  *** AaronvanW has joined #bitcoin-core-dev
 65 2017-11-27T03:03:13  *** jcorgan has joined #bitcoin-core-dev
 66 2017-11-27T03:04:41  *** AaronvanW has quit IRC
 67 2017-11-27T03:05:03  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 68 2017-11-27T03:06:41  *** Emcy_ has joined #bitcoin-core-dev
 69 2017-11-27T03:08:50  *** pbase has joined #bitcoin-core-dev
 70 2017-11-27T03:08:53  *** Emcy has quit IRC
 71 2017-11-27T03:17:03  *** meshcollider has joined #bitcoin-core-dev
 72 2017-11-27T03:22:02  *** dabura667 has quit IRC
 73 2017-11-27T03:32:27  *** Chris_Stewart_5 has quit IRC
 74 2017-11-27T03:38:52  *** Randolf has joined #bitcoin-core-dev
 75 2017-11-27T03:51:26  *** quantbot has joined #bitcoin-core-dev
 76 2017-11-27T03:58:54  *** daedal__ has quit IRC
 77 2017-11-27T03:59:18  *** AaronvanW has joined #bitcoin-core-dev
 78 2017-11-27T04:04:15  *** qrest has quit IRC
 79 2017-11-27T04:05:26  *** AaronvanW has quit IRC
 80 2017-11-27T04:09:13  *** qrestlove has joined #bitcoin-core-dev
 81 2017-11-27T04:37:15  *** goatpig has quit IRC
 82 2017-11-27T04:38:09  *** digifis has joined #bitcoin-core-dev
 83 2017-11-27T04:41:03  *** sin_ has quit IRC
 84 2017-11-27T04:43:45  *** pbase has quit IRC
 85 2017-11-27T04:55:35  *** jack__ has joined #bitcoin-core-dev
 86 2017-11-27T05:03:49  *** hoge_ has joined #bitcoin-core-dev
 87 2017-11-27T05:12:15  *** qrestlove has quit IRC
 88 2017-11-27T05:15:09  *** musalbas has quit IRC
 89 2017-11-27T05:20:01  *** plazar has joined #bitcoin-core-dev
 90 2017-11-27T05:23:58  *** qrestlove has joined #bitcoin-core-dev
 91 2017-11-27T05:24:17  *** plazar has quit IRC
 92 2017-11-27T05:25:26  *** plazar has joined #bitcoin-core-dev
 93 2017-11-27T05:28:39  *** plazar has joined #bitcoin-core-dev
 94 2017-11-27T05:29:48  *** meshcollider has quit IRC
 95 2017-11-27T05:33:37  *** plazar has quit IRC
 96 2017-11-27T05:37:31  *** plazar has joined #bitcoin-core-dev
 97 2017-11-27T05:40:25  *** plazar has quit IRC
 98 2017-11-27T05:40:50  *** plazar has joined #bitcoin-core-dev
 99 2017-11-27T05:43:21  *** plazar has quit IRC
100 2017-11-27T05:43:40  *** plazar has joined #bitcoin-core-dev
101 2017-11-27T05:46:57  *** plazar has quit IRC
102 2017-11-27T05:47:11  *** plazar has joined #bitcoin-core-dev
103 2017-11-27T05:48:59  *** plazar has quit IRC
104 2017-11-27T05:49:16  *** hellxcode has joined #bitcoin-core-dev
105 2017-11-27T06:08:55  *** jack__ has quit IRC
106 2017-11-27T06:13:01  *** hoge_ has quit IRC
107 2017-11-27T06:21:17  *** hoge_ has joined #bitcoin-core-dev
108 2017-11-27T06:21:33  *** hoge_ has joined #bitcoin-core-dev
109 2017-11-27T06:23:54  *** jon888 has joined #bitcoin-core-dev
110 2017-11-27T06:26:09  *** hellxcode has quit IRC
111 2017-11-27T06:26:42  *** AaronvanW has joined #bitcoin-core-dev
112 2017-11-27T06:28:43  *** hellxcode has joined #bitcoin-core-dev
113 2017-11-27T06:29:01  *** sanjeev has quit IRC
114 2017-11-27T06:33:54  *** esotericnonsense has quit IRC
115 2017-11-27T06:41:23  *** Ylbam has joined #bitcoin-core-dev
116 2017-11-27T06:45:06  *** hellxcode has quit IRC
117 2017-11-27T07:03:15  *** sanjeev has joined #bitcoin-core-dev
118 2017-11-27T07:04:28  *** sanjeev_ has joined #bitcoin-core-dev
119 2017-11-27T07:05:12  *** sanjeev has quit IRC
120 2017-11-27T07:05:32  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
121 2017-11-27T07:05:32  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
122 2017-11-27T07:09:56  *** hoge_ has quit IRC
123 2017-11-27T07:12:20  *** hoge_ has joined #bitcoin-core-dev
124 2017-11-27T07:12:38  *** StopAndDecrypt has quit IRC
125 2017-11-27T07:13:03  *** StopAndDecrypt has joined #bitcoin-core-dev
126 2017-11-27T07:13:03  *** StopAndDecrypt has quit IRC
127 2017-11-27T07:13:03  *** StopAndDecrypt has joined #bitcoin-core-dev
128 2017-11-27T07:16:35  *** hoge_ has quit IRC
129 2017-11-27T07:18:54  *** JamesAU has quit IRC
130 2017-11-27T07:22:58  *** hoge_ has joined #bitcoin-core-dev
131 2017-11-27T07:24:19  *** JamesAU has joined #bitcoin-core-dev
132 2017-11-27T07:29:10  *** JamesAU has quit IRC
133 2017-11-27T07:34:52  *** dabura667 has joined #bitcoin-core-dev
134 2017-11-27T07:50:37  *** jon888 has left #bitcoin-core-dev
135 2017-11-27T07:51:01  *** BashCo has quit IRC
136 2017-11-27T08:05:42  *** JamesAU has joined #bitcoin-core-dev
137 2017-11-27T08:11:12  *** digifis has quit IRC
138 2017-11-27T08:12:29  *** BashCo has joined #bitcoin-core-dev
139 2017-11-27T08:16:02  *** nickler has quit IRC
140 2017-11-27T08:16:31  *** promag has joined #bitcoin-core-dev
141 2017-11-27T08:17:06  *** nickler has joined #bitcoin-core-dev
142 2017-11-27T08:19:21  *** promag has quit IRC
143 2017-11-27T08:43:46  *** Cogito_Ergo_Sum has quit IRC
144 2017-11-27T08:45:06  *** sanjeev_ has quit IRC
145 2017-11-27T08:52:27  *** sanjeev has joined #bitcoin-core-dev
146 2017-11-27T09:09:48  *** atroxes has quit IRC
147 2017-11-27T09:14:45  *** timothy has joined #bitcoin-core-dev
148 2017-11-27T09:21:11  *** laurentmt has joined #bitcoin-core-dev
149 2017-11-27T09:23:39  *** laurentmt has quit IRC
150 2017-11-27T09:25:31  *** hoge_ has quit IRC
151 2017-11-27T09:29:21  *** promag has joined #bitcoin-core-dev
152 2017-11-27T09:41:45  *** promag has quit IRC
153 2017-11-27T09:43:00  *** atroxes has joined #bitcoin-core-dev
154 2017-11-27T09:57:58  *** Guyver2 has joined #bitcoin-core-dev
155 2017-11-27T10:05:34  *** Ylbam has quit IRC
156 2017-11-27T10:11:21  *** Ayana59Smitham has quit IRC
157 2017-11-27T10:24:27  *** promag has joined #bitcoin-core-dev
158 2017-11-27T10:27:10  *** promag has quit IRC
159 2017-11-27T10:35:44  *** jtimon has joined #bitcoin-core-dev
160 2017-11-27T10:45:52  <wallet42> What are the counter arguments to have a switch where the miner can set a couple of vbytes (10-50K) aside in a getblocktemplate that will include lowfee, oldest first transactions instead of highest paying?
161 2017-11-27T10:59:35  *** AndyS2 has joined #bitcoin-core-dev
162 2017-11-27T11:01:33  *** AaronvanW has quit IRC
163 2017-11-27T11:01:54  *** AaronvanW has joined #bitcoin-core-dev
164 2017-11-27T11:08:40  <sipa> wallet42: complicates the block construction code significantly, would likely not be used by a significant portion of the hashrate
165 2017-11-27T11:09:46  <sipa> further, having it available even if it was used wouldn't really benefit anyone; it's essentially introducing a lottery in the mining construction from the perspective of wallets
166 2017-11-27T11:13:20  <Provoostenator> I was thinking about creating an RPC method to add a new destination + amount to an existing RBF transaction. Or to add a "append-tx" argument to sendtoaddress. Has anyone worked on that in past?
167 2017-11-27T11:13:51  <Provoostenator> (or explained why that's a terrible idea)
168 2017-11-27T11:14:13  <sipa> Provoostenator: i've thought a bit about a more invasive idea that's related
169 2017-11-27T11:14:51  <sipa> i think it would be nice to have in the wallet, independent from the sendtoaddress/sendmany/... RPCs, a list of "outputs i would like to pay to" (with address/amount)
170 2017-11-27T11:15:21  <sipa> and any time that list is changed, or a transaction confirms, a new single transaction is constructed that pays out to all the entries in that list that haven't been paid to already
171 2017-11-27T11:15:42  <Provoostenator> And then have the wallet figure out if a new transaction is cheaper than amending one?
172 2017-11-27T11:15:57  <sipa> no, there is always just a single outstanding unconfirmed transaction
173 2017-11-27T11:16:07  <sipa> for any change, that transaction is modified
174 2017-11-27T11:16:16  <Provoostenator> That makes sense.
175 2017-11-27T11:16:33  <sipa> the cost of bumping for RBF reasons is usually trivial compared to its actual fee
176 2017-11-27T11:16:47  <sipa> (just add 1sat/vbyte)
177 2017-11-27T11:17:14  <Provoostenator> Right, unless it's a very large transaction. But that would be trivial to check.
178 2017-11-27T11:17:18  <sipa> there are complications to this... for example you need coin selection that guarantees conflicts with all previously attempted unconfirmed transaction
179 2017-11-27T11:17:47  <sipa> or you may need to pre-sign multiple candidate follow-up transactions based on which of your earlier attempts confirms
180 2017-11-27T11:17:59  <sipa> (as you can't automatically get access to encrypted keys later)
181 2017-11-27T11:18:27  <sipa> and it likely conflicts with how people actually use bitcoin in the real world now - where they expect a txid to be something you can track
182 2017-11-27T11:18:28  <Provoostenator> Oh, so in that case you'd use SIGHASH flags?
183 2017-11-27T11:18:58  <sipa> just predict the possible sceniarios, and sign a full follow-up transaction for each (or at least for the few most likely ones)
184 2017-11-27T11:22:48  *** qrestlove has quit IRC
185 2017-11-27T11:22:48  <aj> sipa, wallet42: can't you just use prioritisetransaction rpc on the 50 oldest/whatever transactions to get that behaviour anyway?
186 2017-11-27T11:23:25  <Provoostenator> Although it sounds more elegant, I don't think I can pull that off, yet :-)
187 2017-11-27T11:24:14  <sipa> aj: yes
188 2017-11-27T11:24:37  <aj> sipa: predict the possible scenarios? should be called fund-xanatos-transaction...
189 2017-11-27T11:24:48  <sipa> xanatos?
190 2017-11-27T11:25:04  <aj> http://tvtropes.org/pmwiki/pmwiki.php/Main/XanatosGambit
191 2017-11-27T11:27:44  <sipa> Provoostenator: well i think this would be pretty much independent from all existing transaction logic
192 2017-11-27T11:27:56  <sipa> it'd reuse coin selection, but that's it :)
193 2017-11-27T11:29:25  *** mumux has joined #bitcoin-core-dev
194 2017-11-27T11:30:26  *** mumux has joined #bitcoin-core-dev
195 2017-11-27T11:30:48  <wallet42> sipa: are the wallet RPC capabilities currently enough to pull of something like that with an external python app?
196 2017-11-27T11:31:12  <wallet42> As a prototype
197 2017-11-27T11:31:54  <sipa> yes, just use prioritisetransaction
198 2017-11-27T11:32:02  <sipa> and getrawmempool
199 2017-11-27T11:35:34  *** meshcollider has joined #bitcoin-core-dev
200 2017-11-27T11:40:26  *** StopAndDecrypt has quit IRC
201 2017-11-27T11:41:10  *** StopAndDecrypt has joined #bitcoin-core-dev
202 2017-11-27T11:41:10  *** StopAndDecrypt has quit IRC
203 2017-11-27T11:41:10  *** StopAndDecrypt has joined #bitcoin-core-dev
204 2017-11-27T11:41:18  *** qrestlove has joined #bitcoin-core-dev
205 2017-11-27T11:42:19  <sipa> ugh, master allocates enormous amounts of memory for me while syncing from scratch from network
206 2017-11-27T11:42:38  <sipa> 0.15.1 with the same configuration works fine
207 2017-11-27T11:43:02  <sipa> (it made my system with 32G RAM OOM)
208 2017-11-27T11:44:28  <Provoostenator> sipa: any way Travis could catch something like that?
209 2017-11-27T11:44:46  <Provoostenator> Or does it take too long before memory usage gets out of hand?
210 2017-11-27T11:45:06  <sipa> it OOMed when my debug.log said the dbcache was 1.7G
211 2017-11-27T11:45:17  <sipa> which is substantial but not excessive
212 2017-11-27T11:45:31  <sipa> and the overhead on top of what the dbcache predicts should only be a few 100 MB
213 2017-11-27T11:51:45  *** JamesAU has quit IRC
214 2017-11-27T11:52:17  *** JamesAU has joined #bitcoin-core-dev
215 2017-11-27T11:53:53  *** aspect_ has quit IRC
216 2017-11-27T11:53:53  *** Muis has quit IRC
217 2017-11-27T11:54:22  *** michagogo has quit IRC
218 2017-11-27T11:54:32  *** hsmiths has quit IRC
219 2017-11-27T11:54:33  *** aspect_ has joined #bitcoin-core-dev
220 2017-11-27T11:54:44  *** jl2012 has quit IRC
221 2017-11-27T11:54:58  *** nejon has quit IRC
222 2017-11-27T11:55:38  *** Muis has joined #bitcoin-core-dev
223 2017-11-27T11:55:38  *** nejon has joined #bitcoin-core-dev
224 2017-11-27T11:55:55  *** michagogo has joined #bitcoin-core-dev
225 2017-11-27T11:56:49  *** JamesAU has quit IRC
226 2017-11-27T11:56:57  *** lifeofguenter has quit IRC
227 2017-11-27T11:58:48  *** cyber55 has quit IRC
228 2017-11-27T11:59:06  *** cyber55 has joined #bitcoin-core-dev
229 2017-11-27T11:59:20  *** cyber55 has quit IRC
230 2017-11-27T11:59:40  *** lifeofguenter has joined #bitcoin-core-dev
231 2017-11-27T12:00:33  *** go1111111 has joined #bitcoin-core-dev
232 2017-11-27T12:09:00  *** wxss has joined #bitcoin-core-dev
233 2017-11-27T12:09:23  *** JamesAU has joined #bitcoin-core-dev
234 2017-11-27T12:10:08  *** AaronvanW has quit IRC
235 2017-11-27T12:11:05  *** isis has quit IRC
236 2017-11-27T12:11:50  *** isis has joined #bitcoin-core-dev
237 2017-11-27T12:13:29  *** nickler has quit IRC
238 2017-11-27T12:14:00  *** nickler has joined #bitcoin-core-dev
239 2017-11-27T12:14:10  *** JamesAU has quit IRC
240 2017-11-27T12:18:28  *** dabura667 has quit IRC
241 2017-11-27T12:19:22  *** JamesAU has joined #bitcoin-core-dev
242 2017-11-27T12:24:04  <Provoostenator> sipa: the second log link in the channel topic seems a bit stale...
243 2017-11-27T12:24:33  <aj> Provoostenator: yeah, the cron job broke and i haven't updated it since botbot seems better
244 2017-11-27T12:28:19  *** JamesAU has quit IRC
245 2017-11-27T12:32:53  <Provoostenator> Any idea why "test/functional/test_runner.py --loglevel=INFO bumpfee" doesn't result in "Mining blocks..." appearing my console?
246 2017-11-27T12:33:09  <Provoostenator> It should according to: https://github.com/bitcoin/bitcoin/blob/master/test/functional/bumpfee.py#L52
247 2017-11-27T12:34:54  *** quantbot has quit IRC
248 2017-11-27T12:35:30  *** quantbot has joined #bitcoin-core-dev
249 2017-11-27T12:38:07  <aj> Provoostenator: okay cron job fixed, that wasn't so hard
250 2017-11-27T12:38:32  *** AaronvanW has joined #bitcoin-core-dev
251 2017-11-27T12:39:17  <Provoostenator> aj: nice, that seems to work
252 2017-11-27T12:39:59  *** quantbot has quit IRC
253 2017-11-27T12:41:38  *** Chris_Stewart_5 has joined #bitcoin-core-dev
254 2017-11-27T12:42:13  <Provoostenator> Is anyone timestamping them?
255 2017-11-27T12:48:58  <aj> Provoostenator: each line has a timestamp :) but if you mean cryptographically, no
256 2017-11-27T12:49:02  *** JamesAU has joined #bitcoin-core-dev
257 2017-11-27T12:53:27  *** JamesAU has quit IRC
258 2017-11-27T13:09:37  *** JamesAU has joined #bitcoin-core-dev
259 2017-11-27T13:14:07  *** JamesAU has quit IRC
260 2017-11-27T13:16:54  <morcos> sipa: that sounds bad
261 2017-11-27T13:17:10  <morcos> re: wallet replacement, i think privacy concerns would also be substantial
262 2017-11-27T13:17:56  <morcos> you can assume chain analysis companies would start tracking unconfirmed txs, and use that to further group outputs as related
263 2017-11-27T13:19:20  *** JamesAU has joined #bitcoin-core-dev
264 2017-11-27T13:20:59  <Provoostenator> That's one reason I would make this RPC-only initially.
265 2017-11-27T13:22:25  <Provoostenator> morcos: but does it really matter much privacy wise if you spend the next transaction from your change or by amending the original transaction?
266 2017-11-27T13:23:52  *** JamesAU has quit IRC
267 2017-11-27T13:24:27  *** JamesAU has joined #bitcoin-core-dev
268 2017-11-27T13:26:29  <morcos> Provoostenator: well if you spend from your change, it MAY be unclear that it was your change or the person you paid spending (agreed that right now change is probably pretty easily identifiable though)
269 2017-11-27T13:26:55  <morcos> also if you spend unconfirmed, then you have a good point
270 2017-11-27T13:28:50  *** JamesAU has quit IRC
271 2017-11-27T13:28:57  <Provoostenator> I can see a use case for low priority and smaller transactions, e.g. paying back friends, where you intentionally set the fee too low, so you can keep amending it. A poor mans payment channel. Obviously there's a privacy cost.
272 2017-11-27T13:29:27  <morcos> or rich mans payment channel as the case may be.  :)
273 2017-11-27T13:29:42  *** Cheeseo has joined #bitcoin-core-dev
274 2017-11-27T13:31:01  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
275 2017-11-27T13:31:01  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
276 2017-11-27T13:33:06  <Provoostenator> "TestFramework (INFO): Mining blocks..." does show up in the test log file.
277 2017-11-27T13:35:38  *** hoge_ has joined #bitcoin-core-dev
278 2017-11-27T13:35:59  <Provoostenator> Even stranger given that the default is INFO.
279 2017-11-27T13:43:43  <Provoostenator> Ah, so I need to call "test/functional/bumpfee.py --loglevel=INFO" for this to work.
280 2017-11-27T13:49:41  *** Chris_Stewart_5 has quit IRC
281 2017-11-27T13:50:56  *** jl2012 has joined #bitcoin-core-dev
282 2017-11-27T13:51:29  *** StopAndDecrypt has quit IRC
283 2017-11-27T13:51:47  *** hsmiths has joined #bitcoin-core-dev
284 2017-11-27T13:52:28  *** StopAndDecrypt has joined #bitcoin-core-dev
285 2017-11-27T13:52:29  *** StopAndDecrypt has quit IRC
286 2017-11-27T13:52:29  *** StopAndDecrypt has joined #bitcoin-core-dev
287 2017-11-27T13:55:22  *** meshcollider has quit IRC
288 2017-11-27T14:00:48  *** kbgg_ has joined #bitcoin-core-dev
289 2017-11-27T14:03:38  *** Chris_Stewart_5 has joined #bitcoin-core-dev
290 2017-11-27T14:06:34  *** goatpig has joined #bitcoin-core-dev
291 2017-11-27T14:12:22  *** Chris_Stewart_5 has quit IRC
292 2017-11-27T14:12:47  *** SopaXorzTaker has joined #bitcoin-core-dev
293 2017-11-27T14:13:09  *** hoge_ has quit IRC
294 2017-11-27T14:19:03  *** quantbot has joined #bitcoin-core-dev
295 2017-11-27T14:19:10  *** JamesAU_ has joined #bitcoin-core-dev
296 2017-11-27T14:20:10  *** quantbot has quit IRC
297 2017-11-27T14:20:34  *** Cogito_Ergo_Sum has quit IRC
298 2017-11-27T14:20:36  *** quantbot has joined #bitcoin-core-dev
299 2017-11-27T14:23:27  *** JamesAU_ has quit IRC
300 2017-11-27T14:24:16  *** JamesAU_ has joined #bitcoin-core-dev
301 2017-11-27T14:25:54  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
302 2017-11-27T14:25:54  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
303 2017-11-27T14:29:26  *** JamesAU_ has quit IRC
304 2017-11-27T14:32:02  *** SopaXorzTaker has quit IRC
305 2017-11-27T14:32:38  *** SopaXorzTaker has joined #bitcoin-core-dev
306 2017-11-27T14:35:13  *** sanjeev has quit IRC
307 2017-11-27T14:36:13  *** promag has joined #bitcoin-core-dev
308 2017-11-27T14:48:30  <bitcoin-git> [bitcoin] joemphilips opened pull request #11770: [REST] add a rest endpoint for estimatesmartfee, docs, and test (master...rest_fee) https://github.com/bitcoin/bitcoin/pull/11770
309 2017-11-27T14:55:36  *** Chris_Stewart_5 has joined #bitcoin-core-dev
310 2017-11-27T15:00:58  *** Chris_Stewart_5 has quit IRC
311 2017-11-27T15:01:54  *** Chris_Stewart_5 has joined #bitcoin-core-dev
312 2017-11-27T15:27:27  *** Randolf has quit IRC
313 2017-11-27T15:35:50  *** StopAndDecrypt_ has joined #bitcoin-core-dev
314 2017-11-27T15:36:01  *** StopAndDecrypt has quit IRC
315 2017-11-27T15:52:40  *** archaeal_ has joined #bitcoin-core-dev
316 2017-11-27T15:53:07  *** tim has joined #bitcoin-core-dev
317 2017-11-27T15:53:55  *** tim has quit IRC
318 2017-11-27T15:53:57  *** dcousens has quit IRC
319 2017-11-27T15:54:58  *** dcousens has joined #bitcoin-core-dev
320 2017-11-27T15:55:03  *** StopAndDecrypt_ has quit IRC
321 2017-11-27T15:55:23  *** StopAndDecrypt has joined #bitcoin-core-dev
322 2017-11-27T15:55:23  *** StopAndDecrypt has quit IRC
323 2017-11-27T15:55:23  *** StopAndDecrypt has joined #bitcoin-core-dev
324 2017-11-27T16:00:50  *** cheese_ has joined #bitcoin-core-dev
325 2017-11-27T16:04:51  <promag> archaeal_: it it was me, I would have the PR title: Add REST endpoint blockhash
326 2017-11-27T16:05:11  *** RoyceX has joined #bitcoin-core-dev
327 2017-11-27T16:05:31  <promag> then one commit for the implementation, one for the test and other for the doc
328 2017-11-27T16:05:41  *** satwo has quit IRC
329 2017-11-27T16:06:29  <promag> ^ 11765
330 2017-11-27T16:07:06  <archaeal_> i see...yeah that makes sense
331 2017-11-27T16:07:57  *** cheese_ has quit IRC
332 2017-11-27T16:09:05  <Lauda> sipa I can confirm very high memory usage. I had a fresh sync (albeit a fork of 0.15.x) and it was using over 6 GB with 4GB dbcache and swap was 100% utilized.
333 2017-11-27T16:18:41  *** RoyceX has quit IRC
334 2017-11-27T16:25:16  *** zaxar has joined #bitcoin-core-dev
335 2017-11-27T16:25:48  *** zaxar has quit IRC
336 2017-11-27T16:31:31  *** zaxaar has joined #bitcoin-core-dev
337 2017-11-27T16:33:51  *** zaxaar has quit IRC
338 2017-11-27T16:33:58  *** RoyceX has joined #bitcoin-core-dev
339 2017-11-27T16:33:59  *** DvdKhl has joined #bitcoin-core-dev
340 2017-11-27T16:35:32  <bitcoin-git> [bitcoin] jnewbery opened pull request #11771:  [tests] Change invalidtxrequest to use BitcoinTestFramework (master...refactor_invalidtxrequest) https://github.com/bitcoin/bitcoin/pull/11771
341 2017-11-27T16:35:57  *** Randolf has joined #bitcoin-core-dev
342 2017-11-27T16:36:42  *** BashCo has quit IRC
343 2017-11-27T16:36:50  *** shesek has quit IRC
344 2017-11-27T16:37:18  *** BashCo has joined #bitcoin-core-dev
345 2017-11-27T16:37:30  *** cheese_ has joined #bitcoin-core-dev
346 2017-11-27T16:37:37  *** promag has quit IRC
347 2017-11-27T16:40:12  *** RoyceX has quit IRC
348 2017-11-27T16:40:50  *** Ylbam has joined #bitcoin-core-dev
349 2017-11-27T16:41:32  *** BashCo has quit IRC
350 2017-11-27T16:43:24  *** satwo has joined #bitcoin-core-dev
351 2017-11-27T16:50:09  *** cheese_ has quit IRC
352 2017-11-27T16:54:27  *** Randolf has quit IRC
353 2017-11-27T17:01:22  *** Emcy has joined #bitcoin-core-dev
354 2017-11-27T17:02:58  *** Emcy_ has quit IRC
355 2017-11-27T17:05:18  *** Khunbish has joined #bitcoin-core-dev
356 2017-11-27T17:05:58  *** Khunbish has quit IRC
357 2017-11-27T17:09:49  *** JackH has joined #bitcoin-core-dev
358 2017-11-27T17:11:31  *** cheese_ has joined #bitcoin-core-dev
359 2017-11-27T17:11:36  *** musalbas has joined #bitcoin-core-dev
360 2017-11-27T17:18:49  *** Guyver2 has quit IRC
361 2017-11-27T17:21:09  *** Randolf has joined #bitcoin-core-dev
362 2017-11-27T17:22:41  *** cheese_ has quit IRC
363 2017-11-27T17:24:39  *** BashCo has joined #bitcoin-core-dev
364 2017-11-27T17:25:27  *** Randolf has quit IRC
365 2017-11-27T17:26:31  *** jb55 has joined #bitcoin-core-dev
366 2017-11-27T17:30:19  *** dqx_ has joined #bitcoin-core-dev
367 2017-11-27T17:31:12  *** JackH has quit IRC
368 2017-11-27T17:33:15  *** Chasity76Ortiz has joined #bitcoin-core-dev
369 2017-11-27T17:37:31  <bitcoin-git> [bitcoin] jnewbery closed pull request #11047: [tests] rename functional tests (master...rename_functional_tests) https://github.com/bitcoin/bitcoin/pull/11047
370 2017-11-27T17:40:20  *** Emcy_ has joined #bitcoin-core-dev
371 2017-11-27T17:41:35  *** Emcy has quit IRC
372 2017-11-27T17:41:50  *** dqx_ has quit IRC
373 2017-11-27T17:42:40  *** archaeal_ has quit IRC
374 2017-11-27T17:42:46  <morcos> sipa: have a second to talk about SW wallet?
375 2017-11-27T17:42:52  *** dqx_ has joined #bitcoin-core-dev
376 2017-11-27T17:43:37  <morcos> i thought one of the cases we wanted to cover (which is mostly a version of 2 i think) is that you have a backup of a 0.15.0 wallet now, and you addwitnessaddress, but then later you have to restore from backup
377 2017-11-27T17:43:49  <morcos> this is what the 2(a) case was meant to handle
378 2017-11-27T17:43:59  *** Giszmo has quit IRC
379 2017-11-27T17:44:32  *** dqx_ has quit IRC
380 2017-11-27T17:44:43  *** dqx_ has joined #bitcoin-core-dev
381 2017-11-27T17:44:46  <morcos> i dont think i particularly care if we don't handle that in 0.16, but it does seem like at some point we should handle that...  with SW out, people are going to be doing that work flow.
382 2017-11-27T17:45:19  *** Giszmo has joined #bitcoin-core-dev
383 2017-11-27T17:46:31  <morcos> never mind, i didn't read carefully enough..  the 1(a) case handles that if you are running 0.16, its only the downgrade case that isn't handled
384 2017-11-27T17:46:35  <morcos> i think that is good enough
385 2017-11-27T17:47:02  *** arubi has quit IRC
386 2017-11-27T17:48:28  <sipa> morcos: ok
387 2017-11-27T17:48:28  <morcos> i'm fine with 1(a), 2(c), 3(a).  But I wonder if there is some way we can warn the user about Problem 2, like could the backup have a higher walelt version (ick)
388 2017-11-27T17:48:53  <sipa> right, that would be one solution
389 2017-11-27T17:49:13  <sipa> but it's hard to prevent backups by just file copying
390 2017-11-27T17:50:09  *** dqx_ has quit IRC
391 2017-11-27T17:50:43  *** dqx_ has joined #bitcoin-core-dev
392 2017-11-27T17:51:05  *** JackH has joined #bitcoin-core-dev
393 2017-11-27T17:51:23  <sipa> you could have some system where a wallet is 'loaded' into a node, and the wallet file itself becomes marked as a backup, but something else in the node's configuration contains knowledge to avoid treating it as a backup
394 2017-11-27T17:51:28  <sipa> but... yuck
395 2017-11-27T17:51:33  *** dqx_ has quit IRC
396 2017-11-27T17:51:44  *** dqx_ has joined #bitcoin-core-dev
397 2017-11-27T17:54:21  *** arubi has joined #bitcoin-core-dev
398 2017-11-27T17:54:39  <morcos> well the good thing is there is a workaround right.. so maybe  warnings are enough, saying that if you want to downgrade and you only have a backup, you first need to run 0.16 with the backup and make a fresh backup, and then you can downgrade
399 2017-11-27T17:55:13  <morcos> and that is true regardless of whether you backup is from 0.15 or 0.16
400 2017-11-27T18:04:35  *** Giszmo has quit IRC
401 2017-11-27T18:07:21  *** Randolf has joined #bitcoin-core-dev
402 2017-11-27T18:07:37  *** Emcy has joined #bitcoin-core-dev
403 2017-11-27T18:09:05  *** Emcy_ has quit IRC
404 2017-11-27T18:09:16  *** timothy has quit IRC
405 2017-11-27T18:11:59  <BlueMatt> sipa: I really cant say I'm a fan of 2c
406 2017-11-27T18:14:56  <sipa> the alternatives aren't great either though
407 2017-11-27T18:15:42  *** Jochem has joined #bitcoin-core-dev
408 2017-11-27T18:17:23  <BlueMatt> sipa: what did you think of my suggestion of a segwit-wallet-bestblockhash field
409 2017-11-27T18:17:27  <BlueMatt> in addition to wallet-bestblockhash
410 2017-11-27T18:17:39  *** JackH has quit IRC
411 2017-11-27T18:17:52  <BlueMatt> which allows you to rescan for segwit addresses if you downgraded to 15 for a while (and havent fully pruned those blocks)
412 2017-11-27T18:19:59  <sipa> BlueMatt: sounds doable, but it's not really something we can continue
413 2017-11-27T18:20:35  <sipa> it's a bit of an accident that segwit was introduced in 0.13.1, but will only be actually supported 3 major versions later
414 2017-11-27T18:20:47  <BlueMatt> hmm?
415 2017-11-27T18:21:09  <sipa> i don't think we should continue to expect that it's at all feasible that a new feature introduced will actually work at all after downgrading
416 2017-11-27T18:21:23  <BlueMatt> I didnt claim it would/should?
417 2017-11-27T18:21:38  <sipa> right, but the approach you suggest really only works for segwit
418 2017-11-27T18:21:46  <sipa> not for anything later
419 2017-11-27T18:21:49  <sipa> (likely)
420 2017-11-27T18:21:59  <BlueMatt> yes, this is specifically a segwit thing in large part cause we're not doing a full -walletupgrade for it
421 2017-11-27T18:22:21  <BlueMatt> if we were doing a -walletupgrade (which I still think is the "right" way to do this, but obviously with a ton more complication) then I dont think we'd need to do this
422 2017-11-27T18:22:49  *** Giszmo has joined #bitcoin-core-dev
423 2017-11-27T18:22:49  <BlueMatt> a non-default clear statement of "I'm moving my wallet to 0.16" should be sufficient, but absent that, we should be more careful to handle some downgrade cases
424 2017-11-27T18:24:14  <gmaxwell> how is walletupgrade right when it leaves current wallets people are using today unrecoverable?
425 2017-11-27T18:24:21  <BlueMatt> hmm?
426 2017-11-27T18:24:44  <gmaxwell> people use addwitnessaddress, then their backups are instantly no good-- no lookahead, no nothing.
427 2017-11-27T18:25:07  <gmaxwell> that screup is fixed by 'upgrading' by default
428 2017-11-27T18:25:21  <sipa> gmaxwell: i don't see how that has anything to do with explicit upgrade or not
429 2017-11-27T18:25:24  <BlueMatt> I mean addwitnessaddress is barely supported and largely unsupportable, at least without something like upgradewallet
430 2017-11-27T18:25:35  <BlueMatt> I mean you can auto-upgrade a user if they've previously used addwitnessaddress
431 2017-11-27T18:25:45  <BlueMatt> cause an addwitnessaddress wallet is already a mess (at least wrt backups)
432 2017-11-27T18:26:04  <sipa> explicit upgrade just guarantees that old versions will give you an error rather than silently missing things _for addresses generated in the future_
433 2017-11-27T18:26:16  <sipa> obviously any addresses generated in the must should always continue to work
434 2017-11-27T18:26:52  <BlueMatt> of course, but explicit upgrade also has a clear definition for the user: you cannot open this wallet anymore in old versions
435 2017-11-27T18:27:12  <BlueMatt> which if we *dont* do an explicit upgrade, we end up in this quagmire of backward compatibility/downgrade hell
436 2017-11-27T18:27:14  <gmaxwell> BlueMatt: if you don't autoupgrade someone then you won't make their backups good again.
437 2017-11-27T18:27:34  <BlueMatt> "make their backups good again"?
438 2017-11-27T18:28:06  <BlueMatt> I do not think we should *ever* auto-upgrade someone, with the possible exception of folks who've previously addwitnessaddress'd, cause they're kinda fucked if we dont, or at least we should nag them
439 2017-11-27T18:28:30  <sipa> i think at some point we should do a required uograde
440 2017-11-27T18:28:35  <gmaxwell> Right now, joe blow believes our claims of supporting segwit, runs the software, follows instructions to use it... unknown to him, his backup will miss coins if recovered.  Post upgrading to a release that autoupgrades his backup will no longer miss coins when recovered.
441 2017-11-27T18:28:55  <gmaxwell> _You cannot tell if they've used addwitnessaddress because they could be loading a backup which hasn't seen it_
442 2017-11-27T18:29:27  <sipa> well 1a solves that
443 2017-11-27T18:30:03  <gmaxwell> if you try, you get even more awesome effects like one backup works for address X and another doesn't for reasons entirely unrelated to address X.
444 2017-11-27T18:30:07  <BlueMatt> yes, I think for that reason 1a is probably good
445 2017-11-27T18:30:48  <sipa> the only case that is not covered is downgrading + restoring a backup simuktaneously
446 2017-11-27T18:31:15  <sipa> which is inherently impossible to solve completely
447 2017-11-27T18:31:31  <BlueMatt> yes, and hence my suggestion to do a segwit-wallet-bestblockhash, cause then we at least fix it for non-pruned or only-briefly-run-as-downgraded nodes
448 2017-11-27T18:31:32  <sipa> but some solutions are possible for the case where the backup was made after uograding
449 2017-11-27T18:31:37  <BlueMatt> sidestepping the version rabbithole
450 2017-11-27T18:32:08  *** dqx_ has quit IRC
451 2017-11-27T18:32:21  * BlueMatt would also like to further discuss 2a
452 2017-11-27T18:32:32  <BlueMatt> whats default keypool size? its not *that* bad
453 2017-11-27T18:32:39  <sipa> 2000 keys
454 2017-11-27T18:32:51  <BlueMatt> and, when we go to do things "the right way", we can clean those out, maybe
455 2017-11-27T18:32:55  <sipa> + more logic and edge cases
456 2017-11-27T18:33:02  <BlueMatt> with a -walletupgrade can clean out those entries
457 2017-11-27T18:33:24  <sipa> = something that still only works as long as you don't actually use the wallet too much after downgrading
458 2017-11-27T18:34:44  <BlueMatt> hmm? no?
459 2017-11-27T18:34:57  <BlueMatt> your backups will still work cause they have the witnesses added
460 2017-11-27T18:35:08  <BlueMatt> and the upgraded wallet obviously wont cause you set the version to be un-openable with the old version
461 2017-11-27T18:35:36  <sipa> right
462 2017-11-27T18:36:11  <sipa> but if the wallet was used a lot in the new version after the backup was made, you'll hit problems when the backup's keypool runs out
463 2017-11-27T18:36:29  <BlueMatt> yes, but that is always true
464 2017-11-27T18:36:38  <BlueMatt> if your keypool runs out you cant run from a backup or you lose funds
465 2017-11-27T18:36:39  <BlueMatt> period.
466 2017-11-27T18:37:03  <BlueMatt> (I suppose with the exception of HD, but....ugh)
467 2017-11-27T18:37:27  <sipa> no, not period
468 2017-11-27T18:37:43  <sipa> this is exactly the assumption we don't need to make with HD wallets anymore
469 2017-11-27T18:38:03  <sipa> relying on the keypool to deal with downgraded backups reintroduces that assumption
470 2017-11-27T18:38:32  <BlueMatt> yes, with the exception of HD
471 2017-11-27T18:38:40  <sipa> that's not a small exception
472 2017-11-27T18:38:48  <sipa> that's the norm
473 2017-11-27T18:39:13  <BlueMatt> ok, so now we're back to segwit-wallet-bestblockhash
474 2017-11-27T18:39:17  <BlueMatt> thats the only way to fix that issue
475 2017-11-27T18:39:52  <wallet42> sipa: ah yes. prioritisetransaction is exactly what im looking for.
476 2017-11-27T18:40:09  <sipa> BlueMatt: right, but only in the case the user actually uogrades again
477 2017-11-27T18:40:21  <BlueMatt> well if they dont upgrade again there is literally nothing we can do
478 2017-11-27T18:40:27  <sipa> that's my point
479 2017-11-27T18:40:32  <wallet42> spia: my python prototype question was in the context of the output list wallet amend tx amend idea you talked about with provoost
480 2017-11-27T18:40:41  <BlueMatt> (and I think thats reasonable, if you use segwit in a version with a headline of "SegWit Wallet" and then downgrade, you'd kinda expect to not have your segwit funds)
481 2017-11-27T18:41:04  <sipa> BlueMatt: so why do we need to support the downgrade + restore case again?
482 2017-11-27T18:41:04  <BlueMatt> well the current proposal does *not* fully work in the case you upgrade again, without a rescan
483 2017-11-27T18:41:13  <sipa> it's without guarantees at best
484 2017-11-27T18:41:23  <BlueMatt> if you are non-pruned its guaranteed
485 2017-11-27T18:41:23  <sipa> in whatever case
486 2017-11-27T18:41:50  <sipa> it's without guarantees at best as long as you don't upgrade
487 2017-11-27T18:41:57  <BlueMatt> huh?
488 2017-11-27T18:42:18  <sipa> don't upgrade after downgrading, i mean
489 2017-11-27T18:42:46  <BlueMatt> yes, if you dont ever switch back to 0.16+, then you are maybe not safe in this case
490 2017-11-27T18:42:56  <sipa> not maybe
491 2017-11-27T18:42:58  <BlueMatt> but so what? we can still improve things
492 2017-11-27T18:43:03  <sipa> in that case nothing can be guaranteed
493 2017-11-27T18:43:15  <BlueMatt> doing a partial-rescan is a very, very big improvement for many of the cases we're talking about
494 2017-11-27T18:43:19  <BlueMatt> even if its not a guarantee
495 2017-11-27T18:43:34  <BlueMatt> assuming you may never upgrade to 0.16+ and trying to support that seems like nonsense
496 2017-11-27T18:43:38  <BlueMatt> but thats not what I'm arguing here
497 2017-11-27T18:43:41  <sipa> ok, so let's walk through yhis
498 2017-11-27T18:43:48  <sipa> you upgrade to 0.16
499 2017-11-27T18:43:56  <sipa> there is no segwitwallet record
500 2017-11-27T18:44:29  <sipa> you do what? assume that the existing non-segwit wallet bestblock record is accurate?
501 2017-11-27T18:45:13  <BlueMatt> a) ignore, b) rescan from segwit activation date if you have P2SH-P2WPKH entries c) always rescan from segwit activation date
502 2017-11-27T18:45:57  <sipa> b assumes the file is recent enough to have those records
503 2017-11-27T18:46:00  <BlueMatt> d) rescan in case -findmissingsegwitoutputs=true is set
504 2017-11-27T18:46:09  <BlueMatt> indeed, b kinda sucks
505 2017-11-27T18:46:17  *** promag has joined #bitcoin-core-dev
506 2017-11-27T18:46:32  <sipa> i think the best solution is just telling people to run with 0.16 and -rescan in that case
507 2017-11-27T18:46:42  <BlueMatt> do we have a rescan from date rpc?
508 2017-11-27T18:46:45  <BlueMatt> i think we do now, right?
509 2017-11-27T18:47:06  <sipa> unsure
510 2017-11-27T18:47:22  <BlueMatt> I mean in that case we also need 2a, I think
511 2017-11-27T18:48:00  <sipa> how so?
512 2017-11-27T18:48:13  <sipa> 1a solves everything during a rescan
513 2017-11-27T18:48:24  *** windsok has quit IRC
514 2017-11-27T18:48:44  *** arubi has quit IRC
515 2017-11-27T18:49:00  <BlueMatt> yea, sorry, 1a obviously fixes it for rescan, maybe I'm just more worried about problem 2
516 2017-11-27T18:49:39  <gmaxwell> unrelated to this, through some combination of multiwallet, -rescan, restarting my node.. I ended up in a situation with a bunch of wallets that lost all their transactions but think they're in sync.
517 2017-11-27T18:49:39  <BlueMatt> more like "oops, da server with wallet blew up, bring in the one we upgraded from last week, running 0.15.1, will have to do for now"
518 2017-11-27T18:50:02  <BlueMatt> gmaxwell: thats....frightening....is it 0.15.1 with the copied-db issue?
519 2017-11-27T18:50:07  <morcos> i basically agree with sipa
520 2017-11-27T18:50:14  *** dqx_ has joined #bitcoin-core-dev
521 2017-11-27T18:50:16  <sipa> my solution to problem 2 is tell people to avoid downgrading and restoring, and if in exceptional cases that misses something, rescan
522 2017-11-27T18:50:18  <gmaxwell> w/ recent-ish master.
523 2017-11-27T18:50:39  <BlueMatt> sipa: but that doesnt solve the problem of not upgrading right away? at least 2a will help significantly
524 2017-11-27T18:50:45  <morcos> i think 1a, 3a and a guide for acceptable workflows and how to recover is the way to go..  we should not make things too difficult.
525 2017-11-27T18:51:31  <morcos> however i think we will need to do something like 2a when we have the upgrade wallet option for 0.17.  but then it will i think be fully optional as to whether you upgrade or not.
526 2017-11-27T18:51:45  <gmaxwell> I _think_ what I did was start with two dozen -wallets and -rescan then realized it would sequentially rescan each, killed the process, then went on to try using them directly.
527 2017-11-27T18:52:09  *** digifis has joined #bitcoin-core-dev
528 2017-11-27T18:52:12  <sipa> BlueMatt: but even if we do 2a, there is no solution for 4a
529 2017-11-27T18:52:16  *** windsok has joined #bitcoin-core-dev
530 2017-11-27T18:52:25  *** promag has quit IRC
531 2017-11-27T18:52:27  <sipa> recovery options are great, but they exist
532 2017-11-27T18:52:45  <BlueMatt> yea, issue 4 is unsolveable, I agree
533 2017-11-27T18:52:52  <morcos> sorry, not actually 2a for 0.17, but something that takes all your keys in your keypool and stores them as things that implicitly might have any style address
534 2017-11-27T18:52:56  <BlueMatt> with the exception of 1a + rescan
535 2017-11-27T18:53:14  <sipa> 2a is essentially trying to avoid a recovery procedure in a situation which already is something that should never be part of normal workflow
536 2017-11-27T18:54:15  *** arubi has joined #bitcoin-core-dev
537 2017-11-27T18:54:20  <gmaxwell> if we make rescan faster we could be more liberal in expecting its use.
538 2017-11-27T18:55:01  <gmaxwell> not tooo liberal, since pruned nodes can't rescan...
539 2017-11-27T18:55:48  <BlueMatt> gmaxwell: do you have an intuition in why rescan is slow? I havent looked into it
540 2017-11-27T18:56:58  <sipa> BlueMatt: we should do a benchmark for reindex with and without wallet
541 2017-11-27T18:58:16  *** promag has joined #bitcoin-core-dev
542 2017-11-27T18:58:43  <sipa> if there is a significant different between the two, there is a totally obvious improvement from not calling Solver on each output in order to match it with the wallet
543 2017-11-27T18:58:53  <sipa> (which construct a vector of vectors ffs)
544 2017-11-27T18:58:54  <gmaxwell> well we fully read and deseralize each block and each transaction then pass them through the wallet code, so that implhes 1001 memory allocations plus sha256ing each transaction (and reserializing to do so)... looking at IO we also seem to create a lot of excess  IO with opening the file, seeking to the block closing it.
545 2017-11-27T18:59:29  <sipa> then indeed there is the cost of just loading blocks, which could be improved by having a block deserializer that doesn't build/verify the merkle tree
546 2017-11-27T18:59:57  <gmaxwell> at least from rough looking at it, we do both a bunch of dump computation and IO and all of it serialized.
547 2017-11-27T19:00:21  <gmaxwell> it's worth noting that it's not remarkably faster with the blockchain all in cache.
548 2017-11-27T19:01:18  <BlueMatt> sipa: ok, so I guess I feel a lot more comfortable about things if we do #11281 and recommend rescanblockchain segwit_activation_height
549 2017-11-27T19:01:20  <gribble> https://github.com/bitcoin/bitcoin/issues/11281 | Avoid permanent cs_main/cs_wallet lock during RescanFromTime by jonasschnelli · Pull Request #11281 · bitcoin/bitcoin · GitHub
550 2017-11-27T19:02:32  <BlueMatt> sipa: I'd still be more comfortable with 2a...its far from a hard guarantee of anything, but I really could see someone ending up in the state where that'd save them
551 2017-11-27T19:02:47  <BlueMatt> if for no other reason than someone restoring a backup may load in 0.15, realize they need 0.16, then open in 0.16 right away
552 2017-11-27T19:02:51  <BlueMatt> and end up screwing themselves
553 2017-11-27T19:04:06  <BlueMatt> the segwit-wallet-best-block-hash would also similarly solve the same issue even if you do my option a where you just ignore it if it isnt there
554 2017-11-27T19:04:19  *** promag has quit IRC
555 2017-11-27T19:04:36  <BlueMatt> though I agree that kinda feels gross given its solving such a specific edge case without any hard guarantees
556 2017-11-27T19:04:41  *** windsok has quit IRC
557 2017-11-27T19:06:32  *** JamesAU has joined #bitcoin-core-dev
558 2017-11-27T19:08:12  *** windsok has joined #bitcoin-core-dev
559 2017-11-27T19:08:12  *** windsok has joined #bitcoin-core-dev
560 2017-11-27T19:08:43  <BlueMatt> can we at least high-priority #11281 for 0.16?
561 2017-11-27T19:08:45  <gribble> https://github.com/bitcoin/bitcoin/issues/11281 | Avoid permanent cs_main/cs_wallet lock during RescanFromTime by jonasschnelli · Pull Request #11281 · bitcoin/bitcoin · GitHub
562 2017-11-27T19:08:52  <BlueMatt> so that rescan is at least slightly less painful from the qt command window
563 2017-11-27T19:08:56  *** Emcy_ has joined #bitcoin-core-dev
564 2017-11-27T19:09:00  <sipa> sgtm
565 2017-11-27T19:09:28  <BlueMatt> and add a "I lost my segwit coins" button in the debug window
566 2017-11-27T19:09:30  * BlueMatt *ducks*
567 2017-11-27T19:10:34  *** Emcy has quit IRC
568 2017-11-27T19:10:57  *** JamesAU has quit IRC
569 2017-11-27T19:13:19  <gmaxwell> any kind of scanning uncertanty will cause untold amounts of prophylactic scanning.
570 2017-11-27T19:13:21  *** bobl has joined #bitcoin-core-dev
571 2017-11-27T19:13:40  <BlueMatt> yea, but I dont think we can really solve that......
572 2017-11-27T19:13:51  *** digifis has quit IRC
573 2017-11-27T19:14:06  <BlueMatt> in part cause 0.16 docs are going to have to have a bunch of "in case you did X, do this rescan"
574 2017-11-27T19:14:47  *** bobl has quit IRC
575 2017-11-27T19:15:11  <gmaxwell> thats partly why I was commenting about making rescanning faster.
576 2017-11-27T19:15:37  <BlueMatt> yes, though dunno about making that a requirement for 0.16 :p
577 2017-11-27T19:15:44  <gmaxwell> oh another point on speed-- I opened an issue for this-- right now if you load up a dozen wallets and -rescan. ... it rescans a dozen times, one for each wallet by itself...
578 2017-11-27T19:15:58  <BlueMatt> yes, that should be fixed as well
579 2017-11-27T19:16:14  <gmaxwell> my first idea about not spending the rest of my life rescanning was thwarted by that.
580 2017-11-27T19:22:41  *** jb55 has quit IRC
581 2017-11-27T19:28:09  *** jb55 has joined #bitcoin-core-dev
582 2017-11-27T19:30:51  <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #11775: Move fee estimator into validationinterface/cscheduler thread (master...2017-09-background-feeest) https://github.com/bitcoin/bitcoin/pull/11775
583 2017-11-27T19:33:35  *** Emcy_ has quit IRC
584 2017-11-27T19:34:22  *** Emcy has joined #bitcoin-core-dev
585 2017-11-27T19:47:24  *** Emcy_ has joined #bitcoin-core-dev
586 2017-11-27T19:48:15  *** meshcollider has joined #bitcoin-core-dev
587 2017-11-27T19:48:35  *** Emcy has quit IRC
588 2017-11-27T20:03:01  *** laurentmt has joined #bitcoin-core-dev
589 2017-11-27T20:06:08  *** Emcy has joined #bitcoin-core-dev
590 2017-11-27T20:07:32  *** Emcy_ has quit IRC
591 2017-11-27T20:11:56  <jonasschnelli> BlueMatt: added 11281 to the project 7
592 2017-11-27T20:11:59  *** Chasity76Ortiz has quit IRC
593 2017-11-27T20:13:02  <BlueMatt> jonasschnelli: thanks!
594 2017-11-27T20:15:07  *** ekrok has quit IRC
595 2017-11-27T20:15:28  *** ekrok has joined #bitcoin-core-dev
596 2017-11-27T20:17:16  <cluelessperson> BlueMatt: are you updating the fee estimator?
597 2017-11-27T20:17:19  *** ekrok_ has joined #bitcoin-core-dev
598 2017-11-27T20:17:40  *** ekrok has quit IRC
599 2017-11-27T20:18:35  *** Emcy has quit IRC
600 2017-11-27T20:20:27  <BlueMatt> cluelessperson: it doesnt change the fee estimator logic, except to handle a few edge cases better
601 2017-11-27T20:33:46  *** Sinclair6 has joined #bitcoin-core-dev
602 2017-11-27T20:37:33  *** laurentmt has quit IRC
603 2017-11-27T20:42:54  <sipa> Lauda: using 6GB with 4GB dbcache sounds bad but not nearly the same thing i was seeing
604 2017-11-27T20:43:20  <sipa> i was seeing 32GB usage with 1.7G dbcache
605 2017-11-27T20:43:52  *** Randolf has quit IRC
606 2017-11-27T21:07:13  *** Cheeseo has quit IRC
607 2017-11-27T21:07:21  <Lauda> Well, add 4 gb of SWAP to that so probably ~10ish (I think I allocated 7 maximum for the VM). However, yes. That is much worse :<
608 2017-11-27T21:07:48  <Lauda> Any ideas as to what is causing that?
609 2017-11-27T21:10:01  <sipa> by "using" i mean RSS size, not physical usage
610 2017-11-27T21:10:32  <sipa> it's pretty hard to map VM usage to physical usage, as processes can share memory etc
611 2017-11-27T21:11:49  <sipa> (by VM i mean virtual memory, not virtual machine - it's probably confusing in this context)
612 2017-11-27T21:12:17  <Lauda> Oh, then 6 GB yeah.
613 2017-11-27T21:13:55  *** JackH has joined #bitcoin-core-dev
614 2017-11-27T21:14:13  <Lauda> Your behavior is only in current master, not in 0.15.1?
615 2017-11-27T21:14:45  <sipa> indeed
616 2017-11-27T21:15:28  <sipa> in 0.15.1 memory usage stayed nicely within a few 100 MB of the cache size
617 2017-11-27T21:15:39  <sipa> though i now fail to reproduce with master
618 2017-11-27T21:16:55  <Lauda> That's strange.
619 2017-11-27T21:27:33  *** Randolf has joined #bitcoin-core-dev
620 2017-11-27T21:35:50  *** Randolf has quit IRC
621 2017-11-27T21:36:36  *** cryptapus_ has joined #bitcoin-core-dev
622 2017-11-27T21:36:37  *** cryptapus_ has joined #bitcoin-core-dev
623 2017-11-27T21:36:40  *** cryptapus has quit IRC
624 2017-11-27T21:36:54  *** cryptapus_ is now known as cryptapus
625 2017-11-27T21:39:05  *** Dizzle has joined #bitcoin-core-dev
626 2017-11-27T21:45:56  *** Emcy has joined #bitcoin-core-dev
627 2017-11-27T21:55:01  *** JamesAU has joined #bitcoin-core-dev
628 2017-11-27T21:57:58  *** archaeal_ has joined #bitcoin-core-dev
629 2017-11-27T21:59:32  <cluelessperson> BlueMatt: do you know who might be best to redo the logic?
630 2017-11-27T21:59:41  <cluelessperson> reason being that fees seem to jump up faster than volume
631 2017-11-27T21:59:53  <cluelessperson> https://jochen-hoenicke.de/queue/#2h
632 2017-11-27T21:59:56  <gribble> https://github.com/bitcoin/bitcoin/issues/2 | Long-term, safe, store-of-value · Issue #2 · bitcoin/bitcoin · GitHub
633 2017-11-27T22:00:31  <BlueMatt> cluelessperson: if there were a clear and easy fix it would have been done already :/
634 2017-11-27T22:00:39  <BlueMatt> sadly good fee estimation is super non-trivial
635 2017-11-27T22:01:05  <BlueMatt> *especially* for the non-economical (eg doing non-rbf so want high confidence) case
636 2017-11-27T22:01:30  <cluelessperson> BlueMatt: agreed.  I could only recommend experimenting with "lower estimates slightly" and see how the mempool responds.
637 2017-11-27T22:01:35  *** pgupta has joined #bitcoin-core-dev
638 2017-11-27T22:01:38  <cluelessperson> that would take >years to tune.
639 2017-11-27T22:01:44  <BlueMatt> doing it with rbf in economical mode could probably get improvements and go down faster, etc
640 2017-11-27T22:02:05  <BlueMatt> "lower estimates slightly"? what does that mean?
641 2017-11-27T22:02:26  <BlueMatt> in steady-state the fee estimator does incredibly well, it also happens to do incredibly well for very-high-confidence-of-inclusion
642 2017-11-27T22:02:40  <cluelessperson> BlueMatt:   (existing_fee_estimate) - 1 sat,    -5 sat,   -10 sat
643 2017-11-27T22:02:48  * cluelessperson chuckles at the absurdity
644 2017-11-27T22:02:53  <BlueMatt> though fee estimation also ends up being a bit of a self-fuffilling prophecy
645 2017-11-27T22:03:05  <BlueMatt> yea, I dont think that is a good approach :p
646 2017-11-27T22:05:32  *** Chris_Stewart_5 has quit IRC
647 2017-11-27T22:06:19  *** Cheeseo has joined #bitcoin-core-dev
648 2017-11-27T22:06:20  <cluelessperson> BlueMatt: I'm taking classes.  Hopefully I can help in the future, but right now I'm a moronw. :/
649 2017-11-27T22:06:55  *** ekrok_ has quit IRC
650 2017-11-27T22:08:01  *** ekrok has joined #bitcoin-core-dev
651 2017-11-27T22:16:28  *** ekrok has quit IRC
652 2017-11-27T22:16:35  *** ekrok_ has joined #bitcoin-core-dev
653 2017-11-27T22:18:14  <mlz> lol
654 2017-11-27T22:28:01  *** meshcollider has quit IRC
655 2017-11-27T22:48:57  *** Cheeseo has quit IRC
656 2017-11-27T22:49:10  *** AaronvanW has quit IRC
657 2017-11-27T22:52:46  *** bule has joined #bitcoin-core-dev
658 2017-11-27T22:58:57  *** JamesAU has quit IRC
659 2017-11-27T22:59:26  *** btcdrak has joined #bitcoin-core-dev
660 2017-11-27T23:02:49  *** cyber55 has joined #bitcoin-core-dev
661 2017-11-27T23:07:49  *** meshcollider has joined #bitcoin-core-dev
662 2017-11-27T23:10:29  *** pgupta has quit IRC
663 2017-11-27T23:11:11  *** js___ has joined #bitcoin-core-dev
664 2017-11-27T23:11:11  *** pgupta has joined #bitcoin-core-dev
665 2017-11-27T23:11:43  *** js___ has quit IRC
666 2017-11-27T23:12:10  *** dqx_ has quit IRC
667 2017-11-27T23:14:29  *** dqx_ has joined #bitcoin-core-dev
668 2017-11-27T23:14:39  *** quantbot has quit IRC
669 2017-11-27T23:15:56  *** pgupta has quit IRC
670 2017-11-27T23:19:11  *** dqx_ has quit IRC
671 2017-11-27T23:19:46  *** shesek has joined #bitcoin-core-dev
672 2017-11-27T23:19:46  *** shesek has joined #bitcoin-core-dev
673 2017-11-27T23:19:50  *** Randolf has joined #bitcoin-core-dev
674 2017-11-27T23:22:26  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #11779: qa: Combine logs on travis by default (master...Mf1711-travisQaLogs) https://github.com/bitcoin/bitcoin/pull/11779
675 2017-11-27T23:23:23  *** pgupta has joined #bitcoin-core-dev
676 2017-11-27T23:26:55  *** Randolf has quit IRC
677 2017-11-27T23:28:09  *** pgupta has quit IRC
678 2017-11-27T23:29:25  *** JamesAU has joined #bitcoin-core-dev
679 2017-11-27T23:35:17  *** Cogito_Ergo_Sum has quit IRC
680 2017-11-27T23:35:47  *** jb55 has quit IRC
681 2017-11-27T23:38:46  *** JamesAU has quit IRC
682 2017-11-27T23:40:39  *** jb55 has joined #bitcoin-core-dev
683 2017-11-27T23:46:06  *** esotericnonsense has joined #bitcoin-core-dev
684 2017-11-27T23:48:09  *** brianhoffman_ has joined #bitcoin-core-dev
685 2017-11-27T23:50:06  *** Randolf has joined #bitcoin-core-dev
686 2017-11-27T23:50:59  *** brianhoffman has quit IRC
687 2017-11-27T23:50:59  *** brianhoffman_ is now known as brianhoffman
688 2017-11-27T23:56:37  *** dqx_ has joined #bitcoin-core-dev
689 2017-11-27T23:56:50  *** dqx_ has joined #bitcoin-core-dev
690 2017-11-27T23:58:12  *** bule2 has joined #bitcoin-core-dev
691 2017-11-27T23:59:30  *** Dizzle has quit IRC