1 2017-11-16T00:02:56  *** meshcollider has quit IRC
  2 2017-11-16T00:11:55  *** grio_ has joined #bitcoin-core-dev
  3 2017-11-16T00:12:07  *** grio has quit IRC
  4 2017-11-16T00:12:19  *** Victor_sueca has joined #bitcoin-core-dev
  5 2017-11-16T00:13:18  *** Khunbish has quit IRC
  6 2017-11-16T00:13:38  *** Victorsueca has quit IRC
  7 2017-11-16T00:22:13  *** meshcollider has joined #bitcoin-core-dev
  8 2017-11-16T00:23:45  *** goatpig has quit IRC
  9 2017-11-16T00:31:35  <bitcoin-git> [bitcoin] willyko opened pull request #11700: Add gitian PGP key: willyko (master...master) https://github.com/bitcoin/bitcoin/pull/11700
 10 2017-11-16T00:40:38  *** jsfour has quit IRC
 11 2017-11-16T00:45:52  *** jsfour has joined #bitcoin-core-dev
 12 2017-11-16T00:48:17  *** shtirlic has quit IRC
 13 2017-11-16T00:49:05  *** shtirlic has joined #bitcoin-core-dev
 14 2017-11-16T00:53:56  *** shtirlic has quit IRC
 15 2017-11-16T00:54:23  *** jb55 has quit IRC
 16 2017-11-16T00:54:45  *** shtirlic has joined #bitcoin-core-dev
 17 2017-11-16T00:58:11  *** dabura667 has joined #bitcoin-core-dev
 18 2017-11-16T01:04:28  *** shtirlic has quit IRC
 19 2017-11-16T01:05:19  *** shtirlic has joined #bitcoin-core-dev
 20 2017-11-16T01:14:40  *** shtirlic has quit IRC
 21 2017-11-16T01:21:47  *** Antoinette31Kuhl has joined #bitcoin-core-dev
 22 2017-11-16T01:21:48  *** intcat has quit IRC
 23 2017-11-16T01:22:29  *** shtirlic has joined #bitcoin-core-dev
 24 2017-11-16T01:22:46  *** spinza has quit IRC
 25 2017-11-16T01:22:46  *** Kirstin82Ledner has quit IRC
 26 2017-11-16T01:22:46  *** Cory has quit IRC
 27 2017-11-16T01:22:46  *** BGL has quit IRC
 28 2017-11-16T01:22:46  *** davec has quit IRC
 29 2017-11-16T01:22:46  *** d9b4bef9 has quit IRC
 30 2017-11-16T01:22:46  *** puff has quit IRC
 31 2017-11-16T01:22:46  *** Lauda has quit IRC
 32 2017-11-16T01:22:46  *** victorSN has quit IRC
 33 2017-11-16T01:22:46  *** newbie-- has quit IRC
 34 2017-11-16T01:22:46  *** Apocalyptic has quit IRC
 35 2017-11-16T01:22:46  *** neha has quit IRC
 36 2017-11-16T01:22:46  *** gmaxwell has quit IRC
 37 2017-11-16T01:22:46  *** CryptAxe has quit IRC
 38 2017-11-16T01:22:46  *** isis has quit IRC
 39 2017-11-16T01:22:46  *** newbold_ has quit IRC
 40 2017-11-16T01:22:46  *** MarcoFalke has quit IRC
 41 2017-11-16T01:22:46  *** waxwing has quit IRC
 42 2017-11-16T01:22:46  *** chainhead has quit IRC
 43 2017-11-16T01:22:47  *** nanotube has quit IRC
 44 2017-11-16T01:22:47  *** kanzure has quit IRC
 45 2017-11-16T01:24:36  *** intcat has joined #bitcoin-core-dev
 46 2017-11-16T01:25:05  *** LumberCartel has quit IRC
 47 2017-11-16T01:28:16  *** spinza has joined #bitcoin-core-dev
 48 2017-11-16T01:28:17  *** Cory has joined #bitcoin-core-dev
 49 2017-11-16T01:28:17  *** BGL has joined #bitcoin-core-dev
 50 2017-11-16T01:28:17  *** davec has joined #bitcoin-core-dev
 51 2017-11-16T01:28:17  *** d9b4bef9 has joined #bitcoin-core-dev
 52 2017-11-16T01:28:17  *** puff has joined #bitcoin-core-dev
 53 2017-11-16T01:28:17  *** Lauda has joined #bitcoin-core-dev
 54 2017-11-16T01:28:17  *** victorSN has joined #bitcoin-core-dev
 55 2017-11-16T01:28:17  *** newbie-- has joined #bitcoin-core-dev
 56 2017-11-16T01:28:17  *** Apocalyptic has joined #bitcoin-core-dev
 57 2017-11-16T01:28:17  *** neha has joined #bitcoin-core-dev
 58 2017-11-16T01:28:17  *** gmaxwell has joined #bitcoin-core-dev
 59 2017-11-16T01:28:17  *** CryptAxe has joined #bitcoin-core-dev
 60 2017-11-16T01:28:17  *** isis has joined #bitcoin-core-dev
 61 2017-11-16T01:28:17  *** newbold_ has joined #bitcoin-core-dev
 62 2017-11-16T01:28:17  *** MarcoFalke has joined #bitcoin-core-dev
 63 2017-11-16T01:28:17  *** waxwing has joined #bitcoin-core-dev
 64 2017-11-16T01:28:17  *** chainhead has joined #bitcoin-core-dev
 65 2017-11-16T01:28:17  *** nanotube has joined #bitcoin-core-dev
 66 2017-11-16T01:28:17  *** kanzure has joined #bitcoin-core-dev
 67 2017-11-16T01:28:26  *** Cory has quit IRC
 68 2017-11-16T01:28:26  *** BGL has quit IRC
 69 2017-11-16T01:30:33  *** shtirlic has quit IRC
 70 2017-11-16T01:31:20  *** shtirlic has joined #bitcoin-core-dev
 71 2017-11-16T01:35:07  *** Cory has joined #bitcoin-core-dev
 72 2017-11-16T01:39:41  *** Aaronvan_ has joined #bitcoin-core-dev
 73 2017-11-16T01:40:04  *** StopAndDecrypt_ has joined #bitcoin-core-dev
 74 2017-11-16T01:40:22  *** Aaronvan_ has quit IRC
 75 2017-11-16T01:40:52  *** StopAndDecrypt has quit IRC
 76 2017-11-16T01:42:17  *** AaronvanW has quit IRC
 77 2017-11-16T01:45:05  *** sanada has joined #bitcoin-core-dev
 78 2017-11-16T01:49:08  *** Ylbam has quit IRC
 79 2017-11-16T01:58:53  *** chartractegg has joined #bitcoin-core-dev
 80 2017-11-16T01:59:39  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 81 2017-11-16T02:03:46  *** jb55 has joined #bitcoin-core-dev
 82 2017-11-16T02:06:32  *** chartractegg has quit IRC
 83 2017-11-16T02:08:20  *** chartractegg has joined #bitcoin-core-dev
 84 2017-11-16T02:14:27  *** jtimon has quit IRC
 85 2017-11-16T02:15:37  <meshcollider> sipa: re #11693 do you know if there is any way to currently signrawtransaction for P2SH-P2WSH ? Or should I open a PR to add a witness program key to that call
 86 2017-11-16T02:15:38  <gribble> https://github.com/bitcoin/bitcoin/issues/11693 | Signing raw transaction that has p2sh-p2wsh input · Issue #11693 · bitcoin/bitcoin · GitHub
 87 2017-11-16T02:15:49  *** chartractegg has left #bitcoin-core-dev
 88 2017-11-16T02:15:53  <meshcollider> Or is there something else planned
 89 2017-11-16T02:18:38  <sipa> meshcollider: hmm, that is surprising
 90 2017-11-16T02:19:08  <sipa> meshcollider: maybe we should just automatically add the p2sh version of each script passed to signrawtransaction
 91 2017-11-16T02:19:17  <sipa> that should work without api break
 92 2017-11-16T02:19:53  <meshcollider> Mmm so you would just pass the internal WSH script in the scriptSig?
 93 2017-11-16T02:21:54  *** jsfour has quit IRC
 94 2017-11-16T02:22:57  *** owowo has quit IRC
 95 2017-11-16T02:24:42  *** Murch has quit IRC
 96 2017-11-16T02:27:35  *** BGL has joined #bitcoin-core-dev
 97 2017-11-16T02:27:51  *** owowo has joined #bitcoin-core-dev
 98 2017-11-16T02:27:52  *** owowo has joined #bitcoin-core-dev
 99 2017-11-16T02:27:52  *** owowo has joined #bitcoin-core-dev
100 2017-11-16T02:37:39  <meshcollider> JSON*
101 2017-11-16T02:38:15  <sipa> right
102 2017-11-16T02:38:38  <sipa> hmm, but it would need to guess the witness version
103 2017-11-16T02:38:49  <meshcollider> sipa: it's already a JASON object so adding an extra key wouldn't break it anyway would it
104 2017-11-16T02:38:55  <meshcollider> Oops did that resend
105 2017-11-16T02:42:15  <sipa> okay
106 2017-11-16T02:42:17  <sipa> good
107 2017-11-16T02:42:23  <gmaxwell> from a usability perspective, having to have some extra key isn't super friendly, if it can be reasonably avoided.
108 2017-11-16T02:42:48  <gmaxwell> (these interfaces do exist for thigns other than driving test harnesses, you know)
109 2017-11-16T02:46:15  <meshcollider> I don't know how it could be avoided without instead passing in the witness version, like sipa mentioned
110 2017-11-16T02:47:46  *** LumberCartel has joined #bitcoin-core-dev
111 2017-11-16T02:49:49  <sipa> or alternatively, doing it automatically for all known witness types and versions
112 2017-11-16T02:50:31  <meshcollider> Why does the signrawtransaction help text already say "redeemScript": "hex",   (string, required for P2SH or P2WSH) redeem script
113 2017-11-16T02:50:47  <meshcollider> Oh don't worry, that's P2WSH not P2SH-P2WSH
114 2017-11-16T02:51:02  *** go1111111 has quit IRC
115 2017-11-16T02:52:40  <gmaxwell> whatever we do, it should be the case the you can form a valid signraw line using nothing other than a simple reformatting of listunspent's output.
116 2017-11-16T02:56:20  <meshcollider> What is the redeemScript output from listunspent for P2SH-P2WSH
117 2017-11-16T02:57:12  <meshcollider> I assume it doesn't currently output the witness script at all, which will need to be modified too
118 2017-11-16T02:57:19  <sipa> gmaxwell: that's not really possible, and already not true for P2SH
119 2017-11-16T02:58:47  <sipa> at least not for watch-only outputs
120 2017-11-16T02:59:12  <gmaxwell> sipa: if you've imported the script it should be possible (I have no watch only p2sh so I don't know if it does)
121 2017-11-16T02:59:30  <sipa> gmaxwell: it won't work
122 2017-11-16T02:59:38  <gmaxwell> We should fix that then.
123 2017-11-16T03:00:06  <sipa> i mean it won't work when passing in explicit private keys
124 2017-11-16T03:00:27  <sipa> because then it uses the keystore consteucted from the rpc arguments rather than your wallet's keystore
125 2017-11-16T03:00:58  <sipa> part of this confusion is solved by finally splitting up signrawtransaction into a wallet version and a utility version (achow has a pr i think)
126 2017-11-16T03:01:06  <gmaxwell> the flaw here is that our private key encodings don't represent (or imply) the redeemscript.
127 2017-11-16T03:01:30  <sipa> in the wallet version, things already work fine and nothing is needed for p2sh-p2wsk
128 2017-11-16T03:01:53  <sipa> in the non-wallet version you inherently need to pass in the solving information somehow
129 2017-11-16T03:02:03  <sipa> indeed one way is to encode it in the private key
130 2017-11-16T03:02:30  <gmaxwell> or just declare that it's already encoded in the private key, and expand each private key all known ways; but that doesn't work e.g. for multisig.
131 2017-11-16T03:02:49  <sipa> but that's not generally possible for all output types (and specifically won't work for.any multisig kinda thing, which is exactly where p2wsh is used), so other ways must exist as well
132 2017-11-16T03:02:58  <sipa> right
133 2017-11-16T03:03:32  <gmaxwell> e.g. current private keys = {p2pk,p2pkh,p2sh-one-key,p2wpkhv0,p2wpkhv0-p2sh}
134 2017-11-16T03:04:27  <gmaxwell> we have the redeemscript for each input we're going to sign as an argument, so it could meet in the middle with the private keys. ugh.
135 2017-11-16T03:05:49  <sipa> yeah
136 2017-11-16T03:06:22  <sipa> whatwver encoding is used for the solving information, listunspent should probably be made to report it
137 2017-11-16T03:06:28  <sipa> if known
138 2017-11-16T03:08:44  <Chris_Stewart_5> gmaxwell: So basically this would look like attaching an extra byte of data to the current format to indicate the script it corresponds to>
139 2017-11-16T03:08:47  <Chris_Stewart_5> ?
140 2017-11-16T03:09:42  <Chris_Stewart_5> and some sort of standardized scheme for standard script types I guess
141 2017-11-16T03:17:16  *** m8tion01 has joined #bitcoin-core-dev
142 2017-11-16T03:20:31  *** m8tion03 has quit IRC
143 2017-11-16T03:25:05  *** checksauce has joined #bitcoin-core-dev
144 2017-11-16T03:25:21  <meshcollider> So if I just made listunspent just report witness script for P2SH-P2WSH outputs and then signrawtransaction accept it that would be simple right
145 2017-11-16T03:26:33  <meshcollider> That's the cleanest way i can see to ensure P2SH-P2WSH multisig works for example, which is what brought on this discussion
146 2017-11-16T03:26:39  <sipa> while you're at it, also add redeemscript for P2SH?
147 2017-11-16T03:30:05  *** bule has joined #bitcoin-core-dev
148 2017-11-16T03:32:47  *** Chris_Stewart_5 has quit IRC
149 2017-11-16T03:38:57  <meshcollider> To listunspent?
150 2017-11-16T03:39:24  <sipa> yes, it's also needed in signrawtransaction when giving private keys manually
151 2017-11-16T03:39:53  <meshcollider> sipa: I thought listunspent already had it
152 2017-11-16T03:42:05  <sipa> oh, indeed!
153 2017-11-16T04:01:30  *** checksauce has quit IRC
154 2017-11-16T04:08:55  *** checksauce has joined #bitcoin-core-dev
155 2017-11-16T04:13:49  *** go1111111 has joined #bitcoin-core-dev
156 2017-11-16T04:26:11  *** justan0theruser has joined #bitcoin-core-dev
157 2017-11-16T04:26:46  *** justan0theruser has joined #bitcoin-core-dev
158 2017-11-16T04:27:00  *** justanotheruser has quit IRC
159 2017-11-16T04:28:51  *** lex11 has joined #bitcoin-core-dev
160 2017-11-16T04:31:48  *** lex11 has quit IRC
161 2017-11-16T04:32:01  *** grio_ has quit IRC
162 2017-11-16T04:32:50  *** BCBot has quit IRC
163 2017-11-16T04:49:18  *** checksauce has quit IRC
164 2017-11-16T04:49:45  *** checksauce has joined #bitcoin-core-dev
165 2017-11-16T04:50:27  *** nickler has quit IRC
166 2017-11-16T04:51:45  *** nickler has joined #bitcoin-core-dev
167 2017-11-16T05:00:10  <bitcoin-git> [bitcoin] jamesob opened pull request #11702: [build] Add a script for installing db4 (master...install-db4-script) https://github.com/bitcoin/bitcoin/pull/11702
168 2017-11-16T05:00:24  *** grio has joined #bitcoin-core-dev
169 2017-11-16T05:08:28  *** bule has quit IRC
170 2017-11-16T05:11:54  *** jsfour has joined #bitcoin-core-dev
171 2017-11-16T05:23:00  *** BCBot has joined #bitcoin-core-dev
172 2017-11-16T05:27:51  *** checksauce has quit IRC
173 2017-11-16T05:32:47  *** go1111111 has quit IRC
174 2017-11-16T05:39:00  *** intcat has quit IRC
175 2017-11-16T05:39:50  *** intcat has joined #bitcoin-core-dev
176 2017-11-16T06:05:21  *** tripleslash has quit IRC
177 2017-11-16T06:08:46  *** tripleslash has joined #bitcoin-core-dev
178 2017-11-16T06:09:29  *** go1111111 has joined #bitcoin-core-dev
179 2017-11-16T06:14:47  *** Taco has joined #bitcoin-core-dev
180 2017-11-16T06:19:10  *** Taco has quit IRC
181 2017-11-16T06:34:15  *** Ylbam has joined #bitcoin-core-dev
182 2017-11-16T06:35:39  *** checksauce has joined #bitcoin-core-dev
183 2017-11-16T06:41:49  *** indistylo has joined #bitcoin-core-dev
184 2017-11-16T07:21:51  *** jl2012 has quit IRC
185 2017-11-16T07:23:02  *** sugarpuff has quit IRC
186 2017-11-16T07:25:08  *** sugarpuff has joined #bitcoin-core-dev
187 2017-11-16T07:33:20  *** Victor_sueca is now known as Victorsueca
188 2017-11-16T07:44:57  *** jsfour has quit IRC
189 2017-11-16T07:51:08  *** intcat has quit IRC
190 2017-11-16T07:51:09  *** arubi has quit IRC
191 2017-11-16T07:53:38  *** intcat has joined #bitcoin-core-dev
192 2017-11-16T07:55:39  *** BashCo has quit IRC
193 2017-11-16T07:56:15  *** BashCo has joined #bitcoin-core-dev
194 2017-11-16T07:56:25  *** arubi has joined #bitcoin-core-dev
195 2017-11-16T07:58:57  *** checksauce has quit IRC
196 2017-11-16T08:00:27  *** BashCo has quit IRC
197 2017-11-16T08:06:31  *** dat_ has joined #bitcoin-core-dev
198 2017-11-16T08:23:55  *** checksauce has joined #bitcoin-core-dev
199 2017-11-16T08:25:35  *** LumberCartel has quit IRC
200 2017-11-16T08:25:59  *** BashCo has joined #bitcoin-core-dev
201 2017-11-16T08:26:14  <eck> random question: if leveldb already has its own WAL, then why does bitcoin need an in-memory cache/buffer for utxo updates
202 2017-11-16T08:27:04  <wumpus> eck: because we can do it more efficient because we know the properties of the data and access patterns
203 2017-11-16T08:27:21  <wumpus> eck: leveldb's caching is virtually useless for our use case, so we minimize their caches
204 2017-11-16T08:28:19  <wumpus> why does a database implement its own caching if the OS already has a page cache'
205 2017-11-16T08:28:24  *** qoqsj has joined #bitcoin-core-dev
206 2017-11-16T08:28:28  <wumpus> would be similar :)
207 2017-11-16T08:28:50  <eck> i don't care/know that much about how leveldb implements read caches, but for writes, it seems really problematic you have multiple caching layrs
208 2017-11-16T08:28:53  *** LumberCartel has joined #bitcoin-core-dev
209 2017-11-16T08:29:19  <eck> in this example you have bitcoin cache + leveldb cache + kernel page cache
210 2017-11-16T08:29:20  *** checksauce has quit IRC
211 2017-11-16T08:29:27  <eck> that's a lot of caching layers
212 2017-11-16T08:29:58  <wumpus> why would that be problematic? it's pretty much how modern platforms work, your CPU also has caches as different levels
213 2017-11-16T08:30:12  <sipa> eck: there are a number of reasons why our caching provides functionality on top of LevelDB's
214 2017-11-16T08:30:37  <sipa> one is that LevelDB inherently deals with serialized records, while our caching layer stores information using our native in-memory formats
215 2017-11-16T08:30:46  <sipa> serialization and deserialization have a nontrivial cost
216 2017-11-16T08:30:53  <wumpus> in any case, if you find a way to improve performance, go for it
217 2017-11-16T08:31:45  <sipa> furthermore, we're able to exploit a very significant property of our data set: entries are written once, and deleted once - and almost never overwritten in between
218 2017-11-16T08:32:02  <eck> maybe I can or cannot, just trying to make sure I understand the current state of affaris:
219 2017-11-16T08:32:26  <eck> as i understand it, by default a full node today will flush to disk as infrequently as once every 24h
220 2017-11-16T08:32:35  <sipa> we keep track in our cache whether an entry exists in the caching layer below, and if not (= it's a freshly created entry) which gets deleted (utxo gets spent), we simply delete it from memory, without it ever needing to touch disk
221 2017-11-16T08:33:11  <sipa> this optimization saves us 90% of all I/O or more, depending on cache sizes
222 2017-11-16T08:33:27  <sipa> because many UTXO entries get spent quickly after being created
223 2017-11-16T08:33:41  <bitcoin-git> [bitcoin] laanwj pushed 9 new commits to master: https://github.com/bitcoin/bitcoin/compare/54aedc013744...3c098a8aa078
224 2017-11-16T08:33:42  <bitcoin-git> bitcoin/master 1a44534 MeshCollider: scripted-diff: Replace #include "" with #include <> (ryanofsky)...
225 2017-11-16T08:33:43  <bitcoin-git> bitcoin/master 5b56ec9 Wladimir J. van der Laan: qt: refactor: Use absolute include paths in .ui files
226 2017-11-16T08:33:43  <eck> could you point me (rougly) to the part of the code i woudl oook at?
227 2017-11-16T08:33:44  <bitcoin-git> bitcoin/master 0c71521 Wladimir J. van der Laan: build: Remove -I for everything but project root...
228 2017-11-16T08:33:46  <wumpus> yes, leveldb does not do this for batches
229 2017-11-16T08:34:07  <sipa> eck: coins.cpp
230 2017-11-16T08:34:12  <wumpus> so if you queue both adds, updates, and deletes, they will actually get executed
231 2017-11-16T08:34:16  <bitcoin-git> [bitcoin] laanwj closed pull request #11651: refactor: Make all #includes relative to project root (laanwj, MeshCollider, ryanofsky) (master...201711_absolute_includes) https://github.com/bitcoin/bitcoin/pull/11651
232 2017-11-16T08:34:42  <sipa> yes, LevelDB's "batch" structure is literally an std::string
233 2017-11-16T08:34:47  <wumpus> also leveldb's batch storage format is really inefficient with regard to memroy allocation, large consecutive memory areas
234 2017-11-16T08:34:48  <wumpus> right
235 2017-11-16T08:34:51  <sipa> with serialized write/erase operations in it
236 2017-11-16T08:35:22  <gmaxwell> eck: the infrequency is basically unrelated to the structure, it could continiously flush, now that we have non-atomic flushing... that just hasn't been implemented yet.
237 2017-11-16T08:35:48  <eck> thanks! in the past I wrote my own (simpler obviously) C++ implementation of an ss-table-like structure, and I'm trying to confirm that the whole caching i/o layer in bitcoind works as I expect it would
238 2017-11-16T08:35:48  <gmaxwell> and in general we want to delay flushing as much as possible in order to suppress writes from ever happening in the first place.
239 2017-11-16T08:35:50  <sipa> right, the idea is to move the flushing to a background process that continuously flushes the oldest dirty cache entries
240 2017-11-16T08:36:24  <sipa> but never gets too close to 'now', as to not interfere with our freshness optimization
241 2017-11-16T08:36:39  <sipa> eck: cool
242 2017-11-16T08:36:57  *** jouke has quit IRC
243 2017-11-16T08:39:48  *** jouke has joined #bitcoin-core-dev
244 2017-11-16T08:39:55  <eck> thanks everyone for your help, I will report back if I have further conclusions or questions
245 2017-11-16T08:41:28  <sipa> yw!
246 2017-11-16T08:50:09  *** ekrok_ has quit IRC
247 2017-11-16T08:50:28  *** ekrok_ has joined #bitcoin-core-dev
248 2017-11-16T08:51:30  *** laurentmt has joined #bitcoin-core-dev
249 2017-11-16T08:52:11  *** whphhg has quit IRC
250 2017-11-16T08:52:19  *** whphhg_ has joined #bitcoin-core-dev
251 2017-11-16T08:52:45  *** whphhg_ is now known as whphhg
252 2017-11-16T08:53:46  <eck> related, if I wanted to follow up with someone speifically about the storage layer, who is an expert?
253 2017-11-16T08:54:11  <gmaxwell> just ask here.
254 2017-11-16T08:54:16  <gmaxwell> other people would learn from your questions.
255 2017-11-16T08:55:58  <eck> alright, at a high level i want to know why bitcoin has a caching layer at all: there is already some caching in the kernel page cache, leveldb has some caching og it its own, and then bitcoin itself will cache data for up to 24h
256 2017-11-16T08:56:56  <wumpus> that's what sipa just explained
257 2017-11-16T08:57:00  <eck> from my superfcial understanding, i would just write directly to leveldb and let it take care of the rest
258 2017-11-16T08:57:20  <wumpus> that was the first thing that was tried, and perf was abysmal
259 2017-11-16T08:57:36  <gmaxwell> eck: you can do that, it takes about a week to sync the chain that way on typical hardware.
260 2017-11-16T08:57:58  <wumpus> that might work better with other databases, but not leveldb
261 2017-11-16T08:58:15  <gmaxwell> everything else we've tried was basically an order of magnitude slower than leveldb.
262 2017-11-16T08:58:28  <wumpus> the best (only) way to get feeling for it is to experiment, it's really hard to beat the performance of the current solution
263 2017-11-16T08:58:33  <eck> interesting, I would like to / am willin g to repeat the experiment to verify it locally
264 2017-11-16T08:59:08  <gmaxwell> eck: you can just set the dbcache to a minimal value and see the result for 95% of the effect.
265 2017-11-16T08:59:42  <eck> ok, thanks
266 2017-11-16T08:59:58  <wumpus> I have some old utxo database experiments here: https://github.com/laanwj/bitcoin
267 2017-11-16T09:00:02  <gmaxwell> eck: syncing the chain with every operation going to the database requires about 1 billion database updates.
268 2017-11-16T09:00:10  <wumpus> maybe that's useful at least for seeing what files to touch...
269 2017-11-16T09:00:45  *** qoqsj has quit IRC
270 2017-11-16T09:01:10  <eck> generally though,i would account for the wallet db taking basically 0 time, right?
271 2017-11-16T09:01:21  <wumpus> LMDB seemed promising but I never got spectacular results, maybe I was just using it wrong, and maybe the approach of not caching at the bitcoin level would work better with it
272 2017-11-16T09:02:02  <gmaxwell> eck: I can't figure out why you think the wallet database would be involved at all.
273 2017-11-16T09:02:12  <wumpus> wallet has nothing to do with this, when you benchmark this, I suggest you disable it compile-time
274 2017-11-16T09:02:21  <gmaxwell> The wallet database uses reasources when you have wallets with loads of transactions and keys, otherwise it does nothing.
275 2017-11-16T09:02:43  <eck> i don't care about the wallet database, it's slow but it's smalll
276 2017-11-16T09:03:13  <wumpus> it's also only read at startup, and written when the wallet actually changes, which is infrequent for most people
277 2017-11-16T09:03:15  <eck> my interest recently is syncing a g1-small GCE instance
278 2017-11-16T09:03:35  <gmaxwell> in any case, to sync in three hours (which we do, on a fast desktop at least with dbcache cranked) without a dbcache would require sustaining 100k operations per second, which is not realistic except on specialized hardware (e.g. nvme raid or whatever).
279 2017-11-16T09:03:42  <eck> it has 1.5GB of memory, and essentially 0 disk I/O
280 2017-11-16T09:03:44  *** inimaxedwards has joined #bitcoin-core-dev
281 2017-11-16T09:04:32  <wumpus> it has terrible I/O, disabling caching is certainly not what you should look at
282 2017-11-16T09:04:41  <eck> my instance on core right now is synicin at < 5% a *day*, which seems like there is some fundamental horkage in some layer
283 2017-11-16T09:04:42  <wumpus> sync on a faster machine then copy over the state
284 2017-11-16T09:04:50  <eck> yeah maybe
285 2017-11-16T09:05:37  <gmaxwell> I think you might just be underestimating how much work is involved in syncing and how slow those instances are. :)
286 2017-11-16T09:05:38  <wumpus> w/ the cloud thing, you could just hire one of the large instances for a day or so
287 2017-11-16T09:06:00  <gmaxwell> with 1.5GB memory you may be able to increase the cache further without crashing, it will speed it up.
288 2017-11-16T09:06:44  <eck> from what I can tell, on GCE they give you I/O capacity base on the disk size
289 2017-11-16T09:07:02  <wumpus> this is comparable to running a node on e.g. rpi, they can keep up, but doing initial sync on them takes extreme amounts of patience
290 2017-11-16T09:07:09  <eck> so if you want 200 GB, you're basically on teh bottom wrung, even if you have a ton of cpu/memory
291 2017-11-16T09:07:34  <gmaxwell> But there is just an utterly stupendous amount of work required, ... the software is highly optimized (and sure, there are also still many things that could be done to make it faster). .. but e.g. compared to ethereum the amount of blockchain bitcoin core processes per second is something like two orders of magnitude greater. ( https://anduck.net/bitcoincore_vs_geth_full_node_stats.png )
292 2017-11-16T09:07:48  <wumpus> if it's a vm maybe you can temporary increase the amount if memory, then sync with dbcache of 4000, then i/o will not be touched until it's done
293 2017-11-16T09:09:06  <eck> thanks all, I will definitely be in here asking about this gain
294 2017-11-16T09:09:12  <gmaxwell> indeed, the only IO with a huge db cache is just writing blocks out to disk and the final flush.
295 2017-11-16T09:09:43  <eck> I will ask more once I've done more research
296 2017-11-16T09:10:16  <wumpus> oh yes it will need to write out the blocks, but that's linear and granular, not seek-heavy database i/o I meant
297 2017-11-16T09:11:32  *** Antoinette31Kuhl has quit IRC
298 2017-11-16T09:17:57  *** jl2012 has joined #bitcoin-core-dev
299 2017-11-16T09:19:04  *** Edward48Beier has joined #bitcoin-core-dev
300 2017-11-16T09:20:44  *** intcat has quit IRC
301 2017-11-16T09:21:57  *** intcat has joined #bitcoin-core-dev
302 2017-11-16T09:29:43  *** jadox has joined #bitcoin-core-dev
303 2017-11-16T09:30:52  *** Edward48Beier has quit IRC
304 2017-11-16T09:40:56  *** jsfour has joined #bitcoin-core-dev
305 2017-11-16T09:43:39  *** Yasmeen76Emard has joined #bitcoin-core-dev
306 2017-11-16T09:45:09  *** jsfour has quit IRC
307 2017-11-16T09:45:54  *** AaronvanW has joined #bitcoin-core-dev
308 2017-11-16T09:50:28  *** qposdf has joined #bitcoin-core-dev
309 2017-11-16T10:01:02  *** Ylbam has quit IRC
310 2017-11-16T10:02:56  *** promag has joined #bitcoin-core-dev
311 2017-11-16T10:04:15  *** roconnor_ has quit IRC
312 2017-11-16T10:05:29  *** timothy has joined #bitcoin-core-dev
313 2017-11-16T10:14:19  *** dabura667 has quit IRC
314 2017-11-16T10:14:50  *** jadox has quit IRC
315 2017-11-16T10:28:16  <meshcollider> To add witnessScript to listunspent output, how do we retrieve a CScript from the wallet using WitnessV0ScriptHash if the CScripts are indexed by CScriptID which is Hash160 not SHA?
316 2017-11-16T10:28:46  <sipa> aha!
317 2017-11-16T10:29:04  <sipa> Hash160(x) = RIPEMD160(SHA256(x))
318 2017-11-16T10:29:14  *** Yasmeen76Emard has quit IRC
319 2017-11-16T10:29:27  <sipa> so given y=SHA256(x) you can compute Hash160(x) as RIPEMD160(y)
320 2017-11-16T10:29:31  <meshcollider> Oh! Perfect :)
321 2017-11-16T10:36:10  *** Edgardo10Toy has joined #bitcoin-core-dev
322 2017-11-16T10:49:13  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3c098a8aa078...66d46c7901b7
323 2017-11-16T10:49:14  <bitcoin-git> bitcoin/master ec85248 John Newbery: [travis-ci] Only run linters on Pull Requests...
324 2017-11-16T10:49:14  <bitcoin-git> bitcoin/master 66d46c7 Wladimir J. van der Laan: Merge #11699: [travis-ci] Only run linters on Pull Requests...
325 2017-11-16T10:49:46  <bitcoin-git> [bitcoin] laanwj closed pull request #11699: [travis-ci] Only run linters on Pull Requests (master...lint_only_prs) https://github.com/bitcoin/bitcoin/pull/11699
326 2017-11-16T10:50:06  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/66d46c7901b7...084f52f38dc2
327 2017-11-16T10:50:06  <bitcoin-git> bitcoin/master 069215e practicalswift: Initialize recently introduced non-static class member lastCycles to zero in constructor...
328 2017-11-16T10:50:06  <bitcoin-git> bitcoin/master 084f52f Wladimir J. van der Laan: Merge #11654: tests: Initialize recently introduced non-static class member lastCycles to zero in constructor...
329 2017-11-16T10:50:37  <bitcoin-git> [bitcoin] laanwj closed pull request #11654: tests: Initialize recently introduced non-static class member lastCycles to zero in constructor (master...uninitialized-members) https://github.com/bitcoin/bitcoin/pull/11654
330 2017-11-16T10:57:39  *** asu has joined #bitcoin-core-dev
331 2017-11-16T10:58:25  <asu> hi, all
332 2017-11-16T10:58:44  <asu> i has a question.
333 2017-11-16T10:59:50  <asu> i want to build a bitcoin p2p trade web
334 2017-11-16T11:00:48  <asu> buy i found if my customer send 0.01bitcoin to another, the fee is 0.00027 bitcoin
335 2017-11-16T11:01:59  <asu> but in localbitcoins.com, the fee is zero
336 2017-11-16T11:02:27  <asu> how can i do it? thanks
337 2017-11-16T11:02:46  <asu> who can help me
338 2017-11-16T11:03:07  <kinlo> asu: that question is for #bitcoin, not for here
339 2017-11-16T11:03:43  <asu> ok, thanks
340 2017-11-16T11:04:18  <asu> where is the bitcoin irc?
341 2017-11-16T11:04:29  <kinlo> just join #bitcoin
342 2017-11-16T11:04:39  <asu> join #bitcoin
343 2017-11-16T11:05:05  <kinlo> it's /join #bitcoin
344 2017-11-16T11:05:35  <asu> thanks
345 2017-11-16T11:11:02  <kgc> Hi, I updated issue https://github.com/bitcoin/bitcoin/issues/11693 with some more information, while I can proceed with my implementation it's slightly tricky/confusing to use.
346 2017-11-16T11:11:49  <meshcollider> sipa: It appears there is a way to use signrawtransaction with P2SH-P2WSH as-is, check out https://bitcoin.stackexchange.com/a/62746/51948
347 2017-11-16T11:12:48  <meshcollider> basically put the same input twice, once with the P2SH redeemScript and once with the witness redeemScript
348 2017-11-16T11:13:20  <meshcollider> which is good to know but certainly not the cleanest way to do it
349 2017-11-16T11:14:09  <kgc> yeah
350 2017-11-16T11:14:39  <kgc> I made a suggestion how to potentially clean it up a bit, but that's already for you to decide change or not :)
351 2017-11-16T11:16:02  <meshcollider> kgc: I'm working on something right now here: github.com/MeshCollider/bitcoin/tree/201711_signrawtransaction_wsh
352 2017-11-16T11:16:20  <meshcollider> haven't tested yet
353 2017-11-16T11:18:21  <kgc> good to know
354 2017-11-16T11:19:37  <kgc> if it makes it to official release most probably will switch it using to avoid confusion in the future when looking at my own code
355 2017-11-16T11:20:00  *** ghost43 has quit IRC
356 2017-11-16T11:20:22  *** ghost43 has joined #bitcoin-core-dev
357 2017-11-16T11:21:57  *** shesek has quit IRC
358 2017-11-16T11:24:28  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/084f52f38dc2...99bc0b428b03
359 2017-11-16T11:24:29  <bitcoin-git> bitcoin/master 28f8b66 Eelis: Diagnose unsuitable outputs in lockunspent()....
360 2017-11-16T11:24:29  <bitcoin-git> bitcoin/master 99bc0b4 Wladimir J. van der Laan: Merge #11087: Diagnose unsuitable outputs in lockunspent()....
361 2017-11-16T11:24:53  <bitcoin-git> [bitcoin] laanwj closed pull request #11087: Diagnose unsuitable outputs in lockunspent(). (master...lockunspent) https://github.com/bitcoin/bitcoin/pull/11087
362 2017-11-16T11:41:46  *** jsfour has joined #bitcoin-core-dev
363 2017-11-16T11:43:06  <wumpus> re: https://github.com/bitcoin/bitcoin/pull/11281#discussion_r151390218
364 2017-11-16T11:43:25  <wumpus> is there anything that makes scanning blocks out of order go wrong?
365 2017-11-16T11:43:49  <wumpus> (maybe when there are chains of transactions depending on each other?)
366 2017-11-16T11:44:56  <wumpus> in that case we'd want to disable scanning of the wallet of incoming blocks when it is rescanning
367 2017-11-16T11:45:55  <wumpus> and we would need logic to backtrack in case of reorgs
368 2017-11-16T11:45:57  *** jsfour has quit IRC
369 2017-11-16T11:48:10  *** asu has quit IRC
370 2017-11-16T11:56:20  *** qposdf has quit IRC
371 2017-11-16T12:02:47  *** jsfour has joined #bitcoin-core-dev
372 2017-11-16T12:05:03  *** shesek has joined #bitcoin-core-dev
373 2017-11-16T12:05:03  *** shesek has joined #bitcoin-core-dev
374 2017-11-16T12:07:15  *** jsfour has quit IRC
375 2017-11-16T12:18:21  *** jsfour has joined #bitcoin-core-dev
376 2017-11-16T12:20:36  *** shesek has quit IRC
377 2017-11-16T12:21:55  *** mxg has joined #bitcoin-core-dev
378 2017-11-16T12:22:57  *** jsfour has quit IRC
379 2017-11-16T12:40:52  *** SopaXorzTaker has joined #bitcoin-core-dev
380 2017-11-16T12:45:00  *** lex11 has joined #bitcoin-core-dev
381 2017-11-16T12:47:45  *** lex11 has quit IRC
382 2017-11-16T12:51:04  *** mxg has quit IRC
383 2017-11-16T12:51:11  *** d9b4bef9 has quit IRC
384 2017-11-16T12:52:24  *** d9b4bef9 has joined #bitcoin-core-dev
385 2017-11-16T12:57:35  <bitcoin-git> [bitcoin] sipsorcery opened pull request #11704: Windows build doc update (master...windoc) https://github.com/bitcoin/bitcoin/pull/11704
386 2017-11-16T13:02:47  *** promag has quit IRC
387 2017-11-16T13:18:35  *** Gnof has joined #bitcoin-core-dev
388 2017-11-16T13:39:40  *** goatpig has joined #bitcoin-core-dev
389 2017-11-16T13:44:59  *** Gnof has quit IRC
390 2017-11-16T13:46:22  *** jtimon has joined #bitcoin-core-dev
391 2017-11-16T13:49:03  *** roconnor_ has joined #bitcoin-core-dev
392 2017-11-16T14:00:18  *** promag has joined #bitcoin-core-dev
393 2017-11-16T14:14:32  *** bitstr3am has joined #bitcoin-core-dev
394 2017-11-16T14:15:45  *** dat_ has quit IRC
395 2017-11-16T14:16:46  *** blockchain has joined #bitcoin-core-dev
396 2017-11-16T14:17:15  *** bitstr3am has quit IRC
397 2017-11-16T14:19:12  *** jsfour has joined #bitcoin-core-dev
398 2017-11-16T14:21:53  *** indistylo has quit IRC
399 2017-11-16T14:22:56  *** meshcollider has quit IRC
400 2017-11-16T14:23:08  *** lukedashjr has joined #bitcoin-core-dev
401 2017-11-16T14:23:24  *** jsfour has quit IRC
402 2017-11-16T14:24:01  *** luke-jr has quit IRC
403 2017-11-16T14:27:32  *** lukedashjr is now known as luke-jr
404 2017-11-16T14:45:43  *** jsfour has joined #bitcoin-core-dev
405 2017-11-16T14:46:22  <bitcoin-git> [bitcoin] laanwj closed pull request #10772: Implementation of BIP8 (master...bip8-height) https://github.com/bitcoin/bitcoin/pull/10772
406 2017-11-16T14:47:25  *** andytoshi has quit IRC
407 2017-11-16T14:47:25  *** andytoshi has joined #bitcoin-core-dev
408 2017-11-16T14:49:16  *** Chris_Stewart_5 has joined #bitcoin-core-dev
409 2017-11-16T14:49:41  *** coin_trader has quit IRC
410 2017-11-16T14:50:07  *** jsfour has quit IRC
411 2017-11-16T14:50:22  *** coin_trader has joined #bitcoin-core-dev
412 2017-11-16T14:51:45  *** promag has quit IRC
413 2017-11-16T14:59:41  *** lex11 has joined #bitcoin-core-dev
414 2017-11-16T15:02:33  *** lex11 has quit IRC
415 2017-11-16T15:16:07  <bitcoin-git> [bitcoin] zhaokexun opened pull request #11705: 0.15 (master...0.15) https://github.com/bitcoin/bitcoin/pull/11705
416 2017-11-16T15:17:27  <bitcoin-git> [bitcoin] laanwj closed pull request #11705: 0.15 (master...0.15) https://github.com/bitcoin/bitcoin/pull/11705
417 2017-11-16T15:19:29  *** Chris_Stewart_5 has quit IRC
418 2017-11-16T15:27:24  *** lex11 has joined #bitcoin-core-dev
419 2017-11-16T15:33:10  *** lex11 has quit IRC
420 2017-11-16T15:33:55  *** lex11 has joined #bitcoin-core-dev
421 2017-11-16T15:34:57  *** jcorgan has quit IRC
422 2017-11-16T15:34:57  *** jcorgan has joined #bitcoin-core-dev
423 2017-11-16T15:36:12  *** lex11 has quit IRC
424 2017-11-16T15:40:08  *** Chris_Stewart_5 has joined #bitcoin-core-dev
425 2017-11-16T15:57:57  *** jb55 has quit IRC
426 2017-11-16T16:01:21  *** Chris_Stewart_5 has quit IRC
427 2017-11-16T16:21:04  *** indistylo has joined #bitcoin-core-dev
428 2017-11-16T16:26:36  <BlueMatt> wumpus: do you want a followup pr to #11686 ?
429 2017-11-16T16:26:38  <gribble> https://github.com/bitcoin/bitcoin/issues/11686 | Make ISSUE_TEMPLATE a bit shorter, mention hardware tests by TheBlueMatt · Pull Request #11686 · bitcoin/bitcoin · GitHub
430 2017-11-16T16:28:11  *** d9b4bef9 has quit IRC
431 2017-11-16T16:28:30  <wumpus> BlueMatt: I think it'd make sense, but I don't really want to turn it into a controversial topic
432 2017-11-16T16:29:01  <BlueMatt> oh I dont think anyone cares *that* much, I was just curious if you want more pr volume
433 2017-11-16T16:29:12  <BlueMatt> its also a github md file, not like we need to have big review cycles on it.......
434 2017-11-16T16:29:16  *** d9b4bef9 has joined #bitcoin-core-dev
435 2017-11-16T16:34:54  *** Khunbish has joined #bitcoin-core-dev
436 2017-11-16T16:35:41  *** m8tion01 has quit IRC
437 2017-11-16T16:37:07  *** LumberCartel has quit IRC
438 2017-11-16T16:37:25  <wumpus> right, a change to a github md isn't so bad with regards to PR volume
439 2017-11-16T16:37:47  <BlueMatt> k, I'll tweak it again
440 2017-11-16T16:37:56  <wumpus> thanks :)
441 2017-11-16T16:44:13  *** cbag has joined #bitcoin-core-dev
442 2017-11-16T16:46:09  *** jsfour has joined #bitcoin-core-dev
443 2017-11-16T16:50:32  *** jsfour has quit IRC
444 2017-11-16T16:50:59  *** sunday-afternoon has joined #bitcoin-core-dev
445 2017-11-16T16:51:44  <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #11706: Make default issue text all comments to make issues more readable (master...2017-11-shorter-default-issue-redux) https://github.com/bitcoin/bitcoin/pull/11706
446 2017-11-16T16:51:46  <BlueMatt> wumpus: ^
447 2017-11-16T16:55:46  *** chartractegg has joined #bitcoin-core-dev
448 2017-11-16T17:04:12  *** BashCo has quit IRC
449 2017-11-16T17:04:34  *** indistylo has quit IRC
450 2017-11-16T17:06:18  *** LumberCartel has joined #bitcoin-core-dev
451 2017-11-16T17:24:06  *** jsfour has joined #bitcoin-core-dev
452 2017-11-16T17:25:39  *** Chris_Stewart_5 has joined #bitcoin-core-dev
453 2017-11-16T17:29:02  *** jsfour has quit IRC
454 2017-11-16T17:29:05  *** kk_ has joined #bitcoin-core-dev
455 2017-11-16T17:33:15  *** BashCo has joined #bitcoin-core-dev
456 2017-11-16T17:43:01  *** LumberCartel has quit IRC
457 2017-11-16T17:44:07  *** RubenSomsen has joined #bitcoin-core-dev
458 2017-11-16T17:47:21  *** jsfour has joined #bitcoin-core-dev
459 2017-11-16T17:55:05  *** cbag has quit IRC
460 2017-11-16T18:01:55  *** LumberCartel has joined #bitcoin-core-dev
461 2017-11-16T18:04:38  *** Murch has joined #bitcoin-core-dev
462 2017-11-16T18:34:12  *** kk_ has quit IRC
463 2017-11-16T18:36:26  *** timothy has quit IRC
464 2017-11-16T18:36:37  *** sh4ft has joined #bitcoin-core-dev
465 2017-11-16T18:39:34  *** Ylbam has joined #bitcoin-core-dev
466 2017-11-16T18:42:03  *** RubenSomsen has quit IRC
467 2017-11-16T18:42:08  *** sipa has quit IRC
468 2017-11-16T18:42:09  *** sipa has joined #bitcoin-core-dev
469 2017-11-16T18:43:01  *** tux3do has joined #bitcoin-core-dev
470 2017-11-16T18:43:32  *** laurentmt has quit IRC
471 2017-11-16T18:54:00  *** JackH has joined #bitcoin-core-dev
472 2017-11-16T18:55:26  <bitcoin-git> [bitcoin] jnewbery opened pull request #11707: [tests] Fix sendheaders (master...fix_sendheaders) https://github.com/bitcoin/bitcoin/pull/11707
473 2017-11-16T18:55:34  *** meshcollider has joined #bitcoin-core-dev
474 2017-11-16T19:00:18  <achow101> meeting?
475 2017-11-16T19:00:45  <luke-jr> hajimemashite?
476 2017-11-16T19:01:04  <wumpus> #startmeeting
477 2017-11-16T19:01:04  <lightningbot> Meeting started Thu Nov 16 19:01:04 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
478 2017-11-16T19:01:04  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
479 2017-11-16T19:01:22  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag
480 2017-11-16T19:01:30  <sipa> present
481 2017-11-16T19:01:33  <achow101> hi
482 2017-11-16T19:01:34  <jtimon> hi
483 2017-11-16T19:01:35  <meshcollider> hello
484 2017-11-16T19:01:41  <gmaxwell> hi
485 2017-11-16T19:01:45  <sdaftuar> ack
486 2017-11-16T19:01:48  <jonasschnelli> hi
487 2017-11-16T19:01:56  <gmaxwell> will be back in 10 minutes, maybe the meeting won't be over by then. :P
488 2017-11-16T19:02:11  <wumpus> #topic high priority for review
489 2017-11-16T19:02:16  <BlueMatt> new high-priority for me: #11639
490 2017-11-16T19:02:18  <gribble> https://github.com/bitcoin/bitcoin/issues/11639 | Rewrite the interface between validation and net_processing wrt DoS by TheBlueMatt · Pull Request #11639 · bitcoin/bitcoin · GitHub
491 2017-11-16T19:02:32  <kanzure> hi.
492 2017-11-16T19:02:55  <wumpus> only four things left https://github.com/bitcoin/bitcoin/projects/8
493 2017-11-16T19:03:23  <BlueMatt> also probably worth a post-merge review: #10286 (note that this will likely make lots of open wallet-rpc change conflict silently - you need to add the new BlockUntilSyncedToCurrentChain call in some wallet rpc functions as boiler plate, see dev docs for more)
494 2017-11-16T19:03:27  <gribble> https://github.com/bitcoin/bitcoin/issues/10286 | Call wallet notify callbacks in scheduler thread (without cs_main) by TheBlueMatt · Pull Request #10286 · bitcoin/bitcoin · GitHub
495 2017-11-16T19:03:31  *** promag has joined #bitcoin-core-dev
496 2017-11-16T19:03:37  <wumpus> added 11639
497 2017-11-16T19:03:42  <promag> Hi
498 2017-11-16T19:04:15  <luke-jr> should #11383 be on there? I can rebase after the meeting
499 2017-11-16T19:04:18  <gribble> https://github.com/bitcoin/bitcoin/issues/11383 | Basic Multiwallet GUI support by luke-jr · Pull Request #11383 · bitcoin/bitcoin · GitHub
500 2017-11-16T19:04:34  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
501 2017-11-16T19:04:45  *** Cogito_Ergo_Sum has joined #bitcoin-core-dev
502 2017-11-16T19:05:12  <wumpus> luke-jr: added
503 2017-11-16T19:05:16  <bitcoin-git> [bitcoin] luke-jr closed pull request #10391: OP_CHECKBLOCKATHEIGHT anti-replay (BIP 115; logic only) (master...cbah) https://github.com/bitcoin/bitcoin/pull/10391
504 2017-11-16T19:06:07  <promag> Rpc console still only for 1st wallet right?
505 2017-11-16T19:06:33  <luke-jr> promag: that PR has an independent combobox for the debug window
506 2017-11-16T19:06:46  <luke-jr> (including a "no wallet" option)
507 2017-11-16T19:07:01  <promag> Should rebase on dynamic wallet loading? Or vice-versa?
508 2017-11-16T19:07:01  <wumpus> #topic rpc console for multi wallet
509 2017-11-16T19:07:16  <jonasschnelli> The dropdown seems okay isch.
510 2017-11-16T19:07:23  <luke-jr> promag: IMO GUI should go before dynamic loading
511 2017-11-16T19:07:28  <jonasschnelli> Ideally we would have a higher-level visual selector
512 2017-11-16T19:07:31  <promag> Kk
513 2017-11-16T19:07:38  <luke-jr> jonasschnelli: ?
514 2017-11-16T19:07:41  <jonasschnelli> luke-jr: agree
515 2017-11-16T19:08:09  <jonasschnelli> luke-jr: it confusing to have a wallet level switch in the console
516 2017-11-16T19:08:24  <jtimon> what's wrong with the combobox ?
517 2017-11-16T19:08:30  <jonasschnelli> But I don't see another simple way
518 2017-11-16T19:08:41  <promag> One thing that bothers me with the combo is that the gui state is lost
519 2017-11-16T19:08:45  <luke-jr> maybe improvements there can be made after merge, if someone thinks of a better way
520 2017-11-16T19:08:52  <luke-jr> promag: ?
521 2017-11-16T19:08:56  <achow101> I think it might be confusing to users to have the debug window possibly be for a different wallet than the main wallet gui
522 2017-11-16T19:09:12  <wumpus> the combobox is ok
523 2017-11-16T19:09:16  <jonasschnelli> I think its an acceptable first step
524 2017-11-16T19:09:22  <promag> Like list scroll position, selection, focus, etc
525 2017-11-16T19:09:24  <sipa> pieter was here
526 2017-11-16T19:09:38  <wumpus> the debug window is supposed to be separate from the main GUI, having it influence what wallet is selected is even more confusing
527 2017-11-16T19:09:40  <jtimon> I think it's perfectly fine for the debug console to be flexible like this. seems just handy to put it there
528 2017-11-16T19:09:50  <wumpus> yes
529 2017-11-16T19:09:56  <promag> Another option is one tab per wallet
530 2017-11-16T19:10:02  <wumpus> no, please not
531 2017-11-16T19:10:14  <luke-jr> maybe (post-merge) an idea might be to have a red alert icon next to the combobox if it doesn't match the main window
532 2017-11-16T19:10:43  <achow101> I was thinking that when you first opened the debug window it could default to the wallet that was in use in the main window
533 2017-11-16T19:10:44  <wumpus> meh
534 2017-11-16T19:10:54  <achow101> then users can change the wallet if they want to
535 2017-11-16T19:11:13  <jonasschnelli> I think the dropbox is still the best solution on the table,... (even if not ideal)
536 2017-11-16T19:11:14  <jtimon> that sounds reasonable to me
537 2017-11-16T19:11:18  <wumpus> I really think having the debug window and main window interact in that way is a mess both in code and in interaction, but anyhow
538 2017-11-16T19:11:40  <wumpus> okay, any other topic?
539 2017-11-16T19:11:41  <luke-jr> sounds like we at least agree it's a post-merge topic XD
540 2017-11-16T19:11:45  <jonasschnelli> ack
541 2017-11-16T19:11:48  <jtimon> oh, if it's a mess in the code, I'm not sure it's worth it. I'll shut up
542 2017-11-16T19:11:52  <promag> wumpus: btw why not tabs?
543 2017-11-16T19:12:07  <wumpus> promag: multiple tabs with the same console just pointing at a different wallet sounds terrible to me
544 2017-11-16T19:12:08  <jtimon> spaces are better
545 2017-11-16T19:12:11  <jonasschnelli> promag: most calls are pure node calls...
546 2017-11-16T19:12:11  * jtimon hidfdes
547 2017-11-16T19:12:22  <wumpus> promag: the tabs are supposed to be for essentially different things
548 2017-11-16T19:12:33  <promag> At least you keep track of the correct log
549 2017-11-16T19:12:36  <wumpus> e.g. more charts, more pages of debug info, etc
550 2017-11-16T19:12:42  <jnewbery> promag: multiwallet comes first, dynamic loading later
551 2017-11-16T19:13:04  <jnewbery> *multiwallet GUI comes first
552 2017-11-16T19:13:17  <promag> Anyway, ack on the order
553 2017-11-16T19:13:24  <luke-jr> promag: perhaps a log entry after you execute a command on a different wallet than previously (post-merge stuff)
554 2017-11-16T19:13:42  <promag> Ok ok
555 2017-11-16T19:13:47  <wumpus> why not a command to switch between wallets, btw?
556 2017-11-16T19:14:10  <wumpus> the combobox is great to show what the current wallet is, but shouldn't the wallet be switchable with typing?
557 2017-11-16T19:14:12  <luke-jr> /wallet <name> ?
558 2017-11-16T19:14:16  <wumpus> for ex.
559 2017-11-16T19:14:23  <jonasschnelli> yes... ideally it would be stateless.
560 2017-11-16T19:14:30  <jonasschnelli> to ensure one is not executing on the wrong wallet
561 2017-11-16T19:14:31  <achow101> so it would be a gui only command?
562 2017-11-16T19:14:37  <jonasschnelli> wallet:xyz getnewaddress
563 2017-11-16T19:14:38  <wumpus> achow101: yes
564 2017-11-16T19:14:54  <jonasschnelli> if wallet:<filename> is missing, we get the standard rpcish reject
565 2017-11-16T19:14:55  <wumpus> jonasschnelli: type the wallet name for every command? yes, maybe
566 2017-11-16T19:14:57  <promag> It could be part of the "prompt"
567 2017-11-16T19:15:08  <MarcoFalke> Needs autocomplete!
568 2017-11-16T19:15:17  <jonasschnelli> I think the wallet-selected-state can be dangerous
569 2017-11-16T19:15:18  <wumpus> jonasschnelli: that's absolutely safest
570 2017-11-16T19:15:23  <wumpus> jonasschnelli: agree
571 2017-11-16T19:15:26  <jonasschnelli> and it's RPC like
572 2017-11-16T19:15:35  <wumpus> yes
573 2017-11-16T19:15:39  <jonasschnelli> one can still use arrow-up edit
574 2017-11-16T19:15:43  <jtimon> a gui only command doesn't feel right
575 2017-11-16T19:16:07  <luke-jr> nesting is already GUI-only
576 2017-11-16T19:16:21  <wumpus> jtimon: no, agree, jonasschnelli's proposal to make it stateless and have to provide it for every command is better, that's the same as needs tobe done for bitcoin-cli
577 2017-11-16T19:16:22  <jonasschnelli> yes. It's fine
578 2017-11-16T19:16:59  <luke-jr> is this still post-merge, or have we un-concept-ack'd the MW GUI PR?
579 2017-11-16T19:17:01  <promag> Even when there is 1 wallet only?
580 2017-11-16T19:17:13  <MarcoFalke> promag: No
581 2017-11-16T19:17:17  <wumpus> promag: that's the exception
582 2017-11-16T19:17:22  <jonasschnelli> luke-jr: both would be okay for me (post merge or now)
583 2017-11-16T19:17:24  <wumpus> if it is unambigious then why not
584 2017-11-16T19:17:36  <wumpus> wallet needs to be provided if multiple wallets are loaded
585 2017-11-16T19:17:47  <promag> Ack
586 2017-11-16T19:17:53  <luke-jr> wumpus: because it'd be really annoying to use?
587 2017-11-16T19:17:54  <wumpus> if no wallet is loaded, there's no problem, if one wallet is loaded, then it's clear which one is meant
588 2017-11-16T19:18:14  <wumpus> if mutliple are loaded then wallet commands are ambigious
589 2017-11-16T19:18:19  <promag> It's the same with cli
590 2017-11-16T19:18:26  <wumpus> yes, it's the same with bitcoin-cli
591 2017-11-16T19:18:28  <jonasschnelli> It's maybe annoying... but it's the wallet. Safety first
592 2017-11-16T19:18:36  <luke-jr> cli is just a testing tool though; it doesn't need to be convenient
593 2017-11-16T19:18:36  <gmaxwell> why wouldn't the debug window just have a combo box
594 2017-11-16T19:18:49  <luke-jr> gmaxwell: that's the current code
595 2017-11-16T19:18:57  <jonasschnelli> gmaxwell: I think you will quickly choose the wrong wallet
596 2017-11-16T19:19:11  <wumpus> gmaxwell: it's somewhat dangerous; easy to type a command with the wrong one selected
597 2017-11-16T19:19:17  <jtimon> gmaxwell: some people are worried about a state, not sure what the problem is either
598 2017-11-16T19:19:23  <gmaxwell> luke-jr: that is not true. cli is probably about as frequently used for using the software as the gui (this probably says some unfortunate things about the gui, but.. :P )
599 2017-11-16T19:19:33  <luke-jr> could do both, I guess
600 2017-11-16T19:19:37  <luke-jr> gmaxwell: I highly doubt that!
601 2017-11-16T19:19:46  <achow101> luke-jr: I think there could be some weird interactions with doing both
602 2017-11-16T19:19:55  <wumpus> gmaxwell: there is no clear visual link between what you type and the combobox, though it could be somehow improved by logging in big colorful letters when a different wallet is selected
603 2017-11-16T19:19:58  <luke-jr> both = combobox with in-command override only when no-wallet selected
604 2017-11-16T19:20:03  <wumpus> e.g. ============ current wallet: blabla.dat ===============
605 2017-11-16T19:20:10  <gmaxwell> wumpus: thats a point, the prompt to could also show the wallet.
606 2017-11-16T19:20:21  <wumpus> gmaxwell: yes, indeed
607 2017-11-16T19:20:21  <gmaxwell> and there could be a line written in when it chages, like that.
608 2017-11-16T19:20:31  <jtimon> how is it going to be with the cli again?
609 2017-11-16T19:20:41  <luke-jr> jtimon: no changes needed there
610 2017-11-16T19:20:57  <wumpus> jtimon: for the cli you have to provide the wallet name on every call to select the endpoint ,if it's ambigious, nothing will change there
611 2017-11-16T19:21:11  <gmaxwell> jtimon: cli makes you specify it as a dashed argument to bitcoin-cli, which is a bit obnoxious but works.
612 2017-11-16T19:21:12  <wumpus> decision is to be made about the console
613 2017-11-16T19:21:16  <wumpus> but seems a combobox will do for now
614 2017-11-16T19:21:21  <wumpus> so leave it like that for now luke-jr
615 2017-11-16T19:21:24  <luke-jr> k
616 2017-11-16T19:21:33  *** laurentmt has joined #bitcoin-core-dev
617 2017-11-16T19:21:41  <promag> Next?
618 2017-11-16T19:21:41  <jtimon> I see, thanks. just like you have to provide testnet or regtest every time but you don't need that in the GUI
619 2017-11-16T19:21:49  <wumpus> jtimon: yep
620 2017-11-16T19:21:58  <wumpus> GUI can keep state for you that the cli cannot
621 2017-11-16T19:22:11  <wumpus> because it 'captures' the user, unlike a command that's launched every time
622 2017-11-16T19:22:25  <wumpus> yes, other topics?
623 2017-11-16T19:22:50  <achow101> topic suggestion: encrypted wallets by default
624 2017-11-16T19:22:51  <promag> Flat options in rpc?
625 2017-11-16T19:23:11  <wumpus> #topic encrypted wallets by default
626 2017-11-16T19:23:18  <jtimon> I wanted to ask jl2012 about #11398
627 2017-11-16T19:23:20  <gribble> https://github.com/bitcoin/bitcoin/issues/11398 | Hardcode CSV and SEGWIT deployment by jl2012 · Pull Request #11398 · bitcoin/bitcoin · GitHub
628 2017-11-16T19:23:26  <wumpus> ... why??
629 2017-11-16T19:23:38  <morcos> can someone open an issue about deciding wallet access from the console, i think shipping with it as i understand it to be now seems terrible, but i agree no reason to hold up progress on merging
630 2017-11-16T19:23:43  <jonasschnelli> achow101: with an option to unencrypt later?
631 2017-11-16T19:23:59  <achow101> jonasschnelli: I guess?
632 2017-11-16T19:24:03  <sipa> wumpus: why what?
633 2017-11-16T19:24:09  <wumpus> why encrypt the wallet by default?
634 2017-11-16T19:24:14  <jonasschnelli> achow101: I think that would be great.
635 2017-11-16T19:24:24  <gmaxwell> If you have users encrypt wallets when they open one without any value in it they will reliably lose the key.  The positive confirmation that the user is backed up like electrum has reduces that sort of risk.
636 2017-11-16T19:24:31  <wumpus> it forces people to choose a passphrase which they'll probably forget
637 2017-11-16T19:24:33  <achow101> a lot of wallet software do this now and I don't think people necessarily realize that their wallets are unencrypted until they go to the encrypt wallet option or rpc
638 2017-11-16T19:24:40  <wumpus> I think most people lose money because of losing wallets or losing passphrases not theft
639 2017-11-16T19:24:51  <wumpus> what thread model does encrypting wallets protect against anyhow?
640 2017-11-16T19:24:52  <jonasschnelli> that true on the other hand
641 2017-11-16T19:25:10  <jonasschnelli> Those who have access to support ticket systems of consumer wallets do know that
642 2017-11-16T19:25:21  <luke-jr> wumpus: bad PR
643 2017-11-16T19:25:29  <gmaxwell> Wallet encryption is mostly a tool for people to lose their money but feel better about it because its their own fault.    The great advantage of wallet encryption by default, as I'd see it, is resolving this mess of having to preserve unencrypted keys.
644 2017-11-16T19:25:34  <morcos> couldn't we encrypt the wallet by default but not create the wallet by default
645 2017-11-16T19:25:45  <morcos> so you solve the problem of them just clicking through the encryption aspect
646 2017-11-16T19:25:50  <achow101> morcos: that was the idea I was thinking about
647 2017-11-16T19:25:57  <gmaxwell> But for that advantage I would recommend a late initilization that doesn't create a wallet until you ask for an address... or go to encrypt it.
648 2017-11-16T19:26:06  <achow101> you don't make the wallet until it is actually used, and only then do you prompt the user to make a wallet
649 2017-11-16T19:26:11  <wumpus> I mean, the only use for encrypting wallets I see is: other people use your computer, and you're afraid of them copying the wallet but not installing a keylogger
650 2017-11-16T19:26:20  <gmaxwell> +1 on the late initilization.
651 2017-11-16T19:26:24  <wumpus> I don't think it protects against any other attacks
652 2017-11-16T19:26:46  <morcos> wumpus: you dont think its useful for backups?
653 2017-11-16T19:26:48  <gmaxwell> wumpus: well I really like encryption so that I know that I'm not accidentally going to send funds, but for that it's sufficient to make the key "yes" :P
654 2017-11-16T19:26:59  <luke-jr> morcos: for backups you really want to encrypt the whole thing anyway
655 2017-11-16T19:27:06  <gmaxwell> morcos: ^
656 2017-11-16T19:27:07  <achow101> I have a branch for late initialiation: https://github.com/achow101/bitcoin/tree/start-no-wallet
657 2017-11-16T19:27:09  <morcos> i suppose, maybe backups wasn't the right word
658 2017-11-16T19:27:10  <achow101> it doesn't work right now
659 2017-11-16T19:27:30  <morcos> maybe i meant having the wallet to check on things but not worrying too much about it
660 2017-11-16T19:27:34  <wumpus> or maybe the case where e.g. malware in the browser sandbox can grab a fixed file from your computer, but there's no persistent access
661 2017-11-16T19:27:39  <achow101> also encryption reduces the file size by like half because unencrypted keys are massive for some reason
662 2017-11-16T19:28:08  <wumpus> another thing that will cause confusion is that for other wallets, the passphrase is the seed
663 2017-11-16T19:28:15  <luke-jr> wumpus: even when it was introduced, it was acknowledged as mostly just a PR stunt
664 2017-11-16T19:28:20  <wumpus> so people will think that only keeping the passphrase is enough to keep access to their funds
665 2017-11-16T19:28:20  <jtimon> gmaxwell: I was actually scared to suggest a default key for "resolving this mess of having to preserve unencrypted keys"
666 2017-11-16T19:28:31  <wumpus> there are already peple making that mistake now but it's rarer
667 2017-11-16T19:28:37  <wumpus> (because you only have to choose it explicitly)
668 2017-11-16T19:28:40  <luke-jr> achow101: huh? how?
669 2017-11-16T19:28:44  *** pugsh has joined #bitcoin-core-dev
670 2017-11-16T19:28:45  <gmaxwell> +1 for late init,  +1 for positive confirmation recovery backup (like electrum);  -1 for more pressure to encrypt unless the last step is done, +1 for it if the last step is done.
671 2017-11-16T19:28:50  <morcos> also, this might sound stupid, but if you have a Core-encrypted wallet, you at least know the balance, so you know whether it's worth trying to figure out how to unencrypt it
672 2017-11-16T19:29:01  <wumpus> so no, I think focing people to choose a passphrase when first creating their wallet is a bad idea
673 2017-11-16T19:29:07  <achow101> luke-jr: encrypted keys are way smaller than unencrypted ones
674 2017-11-16T19:29:14  <morcos> +1 gmaxwells +/-1's
675 2017-11-16T19:29:20  <luke-jr> how is that even possible?
676 2017-11-16T19:29:33  <promag> Sorry have to be afk
677 2017-11-16T19:29:38  <gmaxwell> luke-jr: because the unencryted keys use some brain damaged openssl encoding
678 2017-11-16T19:29:43  <gmaxwell> that encludes all the curve parameters.
679 2017-11-16T19:29:44  <wumpus> that's just an implementation detail htough; unencrypted keys could be stored smaller, too
680 2017-11-16T19:29:51  <achow101> luke-jr: the format. unencrypted keys are DER format or something. they have the curve params in them
681 2017-11-16T19:29:55  <wumpus> we could encrypt the wallet by default, with an empty passphrase
682 2017-11-16T19:30:02  <luke-jr> ew
683 2017-11-16T19:30:04  <gmaxwell> right, thats a reason to change the format, not a reason to encrypt.
684 2017-11-16T19:30:09  <sipa> achow101: they have field params, curve params, generator, public key and private key in them :)
685 2017-11-16T19:30:17  <sipa> and all of that in inefficient DER
686 2017-11-16T19:30:24  <sipa> 279 bytes total, iirc
687 2017-11-16T19:30:45  <wumpus> yes it's terrible
688 2017-11-16T19:31:26  <wumpus> and doesn't help with anything, if you're going to store the keys in redundant format at least pad it with something that provides error correction
689 2017-11-16T19:31:54  <luke-jr> XD
690 2017-11-16T19:31:54  <BlueMatt> I mean its error correction in case we forget our curve parameters...or something
691 2017-11-16T19:32:03  <sipa> BlueMatt: we actually hardly look at it
692 2017-11-16T19:32:05  <gmaxwell> wumpus: What are your thoughts on, long term:  delayed creation, at create time in the GUI force the user to write down a recovery code (like electrum does; force via reentry and copy/paste jamming).. and have a checkbox to encrypt there too?   recovery code would greatly offset all risks of loss, including lost the passphrase.
693 2017-11-16T19:32:17  <luke-jr> at that size, just store 8 copies of it
694 2017-11-16T19:32:48  <wumpus> gmaxwell: the recovery code would be the HD seed?
695 2017-11-16T19:32:53  <gmaxwell> luke-jr: storing N copies of a key right next to each other hardly helps since disks tend to die a physical sector at a time.
696 2017-11-16T19:32:56  <achow101> gmaxwell: recovery code as in something like bip39?
697 2017-11-16T19:32:58  <gmaxwell> wumpus: yea, an encoding of the HD seed.
698 2017-11-16T19:33:03  <wumpus> gmaxwell: that sounds great to me
699 2017-11-16T19:33:06  <morcos> gmaxwell: encryption using recovery code?
700 2017-11-16T19:33:17  <luke-jr> gmaxwell: sure, but in that case you're screwed with checksums too
701 2017-11-16T19:33:20  <gmaxwell> achow101: not bip39 as it's a brainwallet scheme that can't encode arbritary data, but yes.
702 2017-11-16T19:33:30  <morcos> i also like that idea, but i worry about the importing of private keys...  we'd have to put in a whole lot of warnings about that
703 2017-11-16T19:33:36  <wumpus> achow101: more like other wallets lke electrum's seed phrase
704 2017-11-16T19:33:49  <achow101> wumpus: yes, I would prefer using Electrum's scheme
705 2017-11-16T19:33:53  <wumpus> achow101: (there's a BIP for it but I don't know the number)
706 2017-11-16T19:33:54  <gmaxwell> morcos: I think we need to get to having an import tainted flag on wallets, and warnings about that.
707 2017-11-16T19:33:54  <achow101> that's what we plan to do for Armory
708 2017-11-16T19:34:08  <luke-jr> morcos: importing private keys is already considered dangerous and "never do this"
709 2017-11-16T19:34:21  <wumpus> gmaxwell: I also greatly like the idea of not creating a wallet by default, so starting in no-wallet mode
710 2017-11-16T19:34:22  <jtimon> so what's wrong with the "yes"/default/empty passphrase/key?
711 2017-11-16T19:34:32  <jonasschnelli> the recovery phrase would be unencrypted?
712 2017-11-16T19:34:40  <gmaxwell> achow101: ugg electrum itself. can't encode arbritary data, so it can't work with existing wallets. at least it's better than bip39.
713 2017-11-16T19:34:55  <achow101> jonasschnelli: it would have to be to be able to recover from forgotten passwords
714 2017-11-16T19:35:25  <achow101> gmaxwell: it can't? (I haven't really looked at it)
715 2017-11-16T19:35:27  <jtimon> jonasschnelli: yes, would be public knowledge (and for the user it would be like if none was set) unless you actively set one
716 2017-11-16T19:35:32  <jonasschnelli> achow101: I just worry about people storing those recovery phrases on phones and "plaintext "papers
717 2017-11-16T19:35:34  <gmaxwell> jonasschnelli: I have mixed feelings about that.  I think a best practice is to have your recovery keys encrypted with a WEAK key,  like that insecure password your whole family knows; and there is no risk of it being forgotten... but which a burgler would likely be thwarted, but thats too complex to communicate.
718 2017-11-16T19:36:12  <gmaxwell> jonasschnelli: but we should realize that risk of users losing a strong password is likely orders of magnitude more likely than a local in person attack.
719 2017-11-16T19:36:27  <wumpus> it gets quite complex to manage if the recovery key is encrypted too
720 2017-11-16T19:36:33  * BlueMatt notes that if we do some kind of default-encryption-with-weak-password we should have a clear tag on keys to help out the "shitsihitshit, please scan entire raw disk for anything that looks like a key" use-case
721 2017-11-16T19:36:37  <jonasschnelli> gmaxwell: Indeed. Though people who can take care of a passphase should not be punished
722 2017-11-16T19:36:40  <wumpus> there's the recovery key passphrase, the wallet passphrase,...
723 2017-11-16T19:36:46  <gmaxwell> achow101: unless I'm confused (always likely) it's just a minor fixup of BIP39.
724 2017-11-16T19:37:15  <luke-jr> BlueMatt: +1
725 2017-11-16T19:37:16  <wumpus> BlueMatt: that's where the redundant key format is useful :)
726 2017-11-16T19:37:32  <wumpus> BlueMatt: it greatly helps efficiently scanning for private keys on a disk :p
727 2017-11-16T19:37:38  <BlueMatt> heh, I know
728 2017-11-16T19:37:49  <gmaxwell> BlueMatt: yea, sure, anything key format should have e.g. somethin like the network magic then the private key then a 64 bit crc... and then its cheap to scan the media looking for it.
729 2017-11-16T19:37:54  <wumpus> I don't think you can do a similar thing for the encrypted keys right now
730 2017-11-16T19:38:00  <wumpus> not that they're any use without the master key
731 2017-11-16T19:38:27  <BlueMatt> i mean ideally we'd have a clear tag on both so that such software can prompt the user with "found a wallet, please enter passphrase"
732 2017-11-16T19:38:35  <BlueMatt> but now we're going down a rewrite-wallet-format rabbit hole
733 2017-11-16T19:38:38  <gmaxwell> jonasschnelli: I don't know how to manage the multiple keys case. One possiblity would be to make the recovery key unencrypted by default, and have an advanced dialog that lets you set encryption for it. And support reading in encrypted ones.
734 2017-11-16T19:38:54  <jonasschnelli> Yes. That would be great
735 2017-11-16T19:39:04  <gmaxwell> jonasschnelli: I have a lovely suggestion for hardware wallet friendly KDFs for these things too.
736 2017-11-16T19:39:06  <achow101> BlueMatt: I propose that we just deprecate the wallet :p
737 2017-11-16T19:39:09  <morcos> May I make a meta suggestion.. I think we often lose progress on ideas like this by not having someone document what we discussed.  could we ask for volunteer every time we have a good discussion like this to draft up a plan.
738 2017-11-16T19:39:16  <luke-jr> achow101: I get the feeling often
739 2017-11-16T19:39:16  <jtimon> ack on starting in no-wallet mode
740 2017-11-16T19:39:22  <jonasschnelli> gmaxwell: +1 (happy to hear)
741 2017-11-16T19:39:33  <achow101> morcos: meeting notes writer
742 2017-11-16T19:39:49  <achow101> morcos: he'll write the meeting notes sometime after exams this week
743 2017-11-16T19:39:52  <wumpus> achow101: and then what, change it into an art project where you can look at blocks drifting by, without being able to do anything? :p
744 2017-11-16T19:40:22  <luke-jr> wumpus: write a new one :p
745 2017-11-16T19:40:36  <morcos> yeah but i mena more a focused thing... like after SF devcore -> plan for Segwit wallet  ;   this meeting -> plan for wallet encryption recovery code
746 2017-11-16T19:40:36  <wumpus> luke-jr: you can do that without deprecating anything
747 2017-11-16T19:40:51  <gmaxwell> the block drifting UI should play https://www.youtube.com/watch?v=8Z-fyNdnOKE in a loop.
748 2017-11-16T19:41:00  <achow101> I'm scared to click that link
749 2017-11-16T19:41:05  <gmaxwell> it's just music.
750 2017-11-16T19:41:17  <gmaxwell> but we've trained you well.
751 2017-11-16T19:41:50  *** promag has quit IRC
752 2017-11-16T19:41:52  <sipa> morcos: i've just posted a bit of a writeup/rant on wallet design and segwit support: https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2
753 2017-11-16T19:41:54  <wumpus> morcos: yes, we shouldn't forget segwit wallet
754 2017-11-16T19:41:59  <morcos> woohoo!
755 2017-11-16T19:42:02  <wumpus> morcos: that's the thing people are actually waiting for now :)
756 2017-11-16T19:42:05  <sdaftuar> sipa: thanks!
757 2017-11-16T19:42:08  <gmaxwell> FWIW, sipa has been working on a stronger base=32 BCH code for things like private keys and stealth addresses; which could be an option for recovery codes.
758 2017-11-16T19:42:12  <wumpus> sipa: nice!
759 2017-11-16T19:42:13  <luke-jr> clicking that link won't have permissions for my audio :p
760 2017-11-16T19:42:22  <BlueMatt> when segwit wallet
761 2017-11-16T19:42:27  *** d_t has joined #bitcoin-core-dev
762 2017-11-16T19:42:30  <achow101> sipa: cool!
763 2017-11-16T19:42:51  <achow101> BlueMatt: soon(tm)
764 2017-11-16T19:43:20  <luke-jr> (mini rant: using #include <…> for our own files is stupid)
765 2017-11-16T19:44:23  <wumpus> luke-jr: sigh, the other alternative would have been to fix all relative includes, but that was discussed in detail in the PR and the one before it
766 2017-11-16T19:44:29  *** go1111111 has quit IRC
767 2017-11-16T19:45:23  <wumpus> luke-jr: so using #include "../primitive/block.h" in e.g. the wallet. This roots everything at the project root, which is just as unambigious and shorter...
768 2017-11-16T19:45:32  *** Dizzle has joined #bitcoin-core-dev
769 2017-11-16T19:45:54  <luke-jr> I just hope /usr/include/primitive/block.h gets ignored
770 2017-11-16T19:46:08  <wumpus> that doesn't get ignored either with ""
771 2017-11-16T19:46:20  <luke-jr> :|
772 2017-11-16T19:47:06  <gmaxwell> obviously we need to rename every header file to filename_bitcoin_core_is_awesome.h
773 2017-11-16T19:47:08  <wumpus> at least not the way we were using it, which is essentially as <>, I think if you use "" relatively you can avoid it
774 2017-11-16T19:48:03  <wumpus> yep!
775 2017-11-16T19:48:03  <luke-jr> gmaxwell: I'm thinking more of malware infecting builds this way
776 2017-11-16T19:48:19  <wumpus> well if your build root is infected you're fucked anyway
777 2017-11-16T19:48:29  <jnewbery> luke-jr: in any case, the PRs were open for a few months (much longer than I would have liked in fact). There was opportunity to comment on those PRs. I think the ship has sailed now.
778 2017-11-16T19:48:40  <luke-jr> true
779 2017-11-16T19:48:57  <wumpus> protecting against that is even more questionable than encrypting your wallet, against any possible realistic threat model
780 2017-11-16T19:49:45  <gmaxwell> luke-jr: well if your host is compromised it's pretty unlikely that it would be limited to only tripping you up with shadowed include files.
781 2017-11-16T19:49:47  <wumpus> jnewbery: indeed, it's almost as if he waited for it to be merged
782 2017-11-16T19:49:57  * BlueMatt nominates #11363 for cfields' high-priority-for-review, cause he apparently isnt here...
783 2017-11-16T19:49:59  <gribble> https://github.com/bitcoin/bitcoin/issues/11363 | net: Split socket create/connect by theuni · Pull Request #11363 · bitcoin/bitcoin · GitHub
784 2017-11-16T19:50:10  <luke-jr> wumpus: just didn't notice it until rebasing on top of the merged code
785 2017-11-16T19:51:05  <luke-jr> anyhow, #11383 rebase is done
786 2017-11-16T19:51:07  <gribble> https://github.com/bitcoin/bitcoin/issues/11383 | Basic Multiwallet GUI support by luke-jr · Pull Request #11383 · bitcoin/bitcoin · GitHub
787 2017-11-16T19:51:16  <jonasschnelli> ^^
788 2017-11-16T19:52:33  <jonasschnelli> thanks.. will test
789 2017-11-16T19:52:34  <Dizzle> I like multiwallet. Thanks for working on it, luke-jr. I miss the classic multibit bulk walletting.
790 2017-11-16T19:52:41  <wumpus> luke-jr: anyhow C/C++ including is fragile that way; possible modules https://clang.llvm.org/docs/Modules.html will improve that in the future
791 2017-11-16T19:53:13  <sipa> yay c++20... which we'll switch to in 20125?
792 2017-11-16T19:53:19  <sipa> *2025
793 2017-11-16T19:53:24  <wumpus> yes, in 20125
794 2017-11-16T19:53:31  <luke-jr> :x
795 2017-11-16T19:53:33  <Chris_Stewart_5> ack
796 2017-11-16T19:53:36  <meshcollider>  lol
797 2017-11-16T19:53:39  <jonasschnelli> heh
798 2017-11-16T19:53:53  <gmaxwell> change in topic, anyone have recent stats for the number of remaining btc1 nodes-- which are likely about to become a distributed DOS attack on the bitcoin network?
799 2017-11-16T19:53:56  <wumpus> BlueMatt: will add that one
800 2017-11-16T19:54:40  <wumpus> #topic DDoS network stats
801 2017-11-16T19:54:59  <meshcollider> gmaxwell:  https://coin.dance/nodes says 139 but maybe not what you're after?
802 2017-11-16T19:55:12  <luke-jr> (I'm going to drop as soon as the meeting is officially over. I'll be back a few minutes later in case there's stuff to talk about)
803 2017-11-16T19:55:19  <jonasschnelli> I can filter my seed crawler for uagent string?
804 2017-11-16T19:55:23  <gmaxwell> meshcollider: ha. I didn't expect them to shut off that fast, I guess they were really almost all just a couple people sybling.
805 2017-11-16T19:55:35  <gmaxwell> meshcollider: okay, probably not much to worry about.
806 2017-11-16T19:56:24  <meshcollider> gmaxwell yeah lol there was a Reddit post which went into some detail showing 90% were hosted by AWS
807 2017-11-16T19:56:35  <wumpus> PSA before the meeting is over: I want to collect corrupted leveldb files, if you have a leveldb corruption please patch https://github.com/bitcoin/bitcoin/pull/11674 and send me the indicated corrupted file.
808 2017-11-16T19:57:10  <jonasschnelli> 861 peers with "Bitcoin ABC" and 100% uptime during last two hours.
809 2017-11-16T19:57:24  <luke-jr> jonasschnelli: that's just BCH
810 2017-11-16T19:57:25  <achow101> gmaxwell: my btc1 node is connected to 34 other btc1 nodes, so at least 35
811 2017-11-16T19:57:26  <meshcollider> ABC is not btc1
812 2017-11-16T19:57:35  <BlueMatt> i mean its what we did 0.15.1 for, no?
813 2017-11-16T19:58:10  <gmaxwell> BlueMatt: yes, sure I wanted to know how many there weer because if there were thousands I'd make a post on reddit to urge people to upgrade to 0.15+ seems it might not be needed.
814 2017-11-16T19:58:11  <jonasschnelli> meshcollider: what uagent does btc1 uses?
815 2017-11-16T19:58:25  <luke-jr> jonasschnelli: /Satoshi:1.*/
816 2017-11-16T19:58:30  <jonasschnelli> ah
817 2017-11-16T19:58:41  <achow101> jonasschnelli: most have a uacomment with "2x"
818 2017-11-16T19:58:46  <jonasschnelli> 107
819 2017-11-16T19:58:55  <gmaxwell> with only 140ish it's pretty unlikely many nodes will get isolated behind them.
820 2017-11-16T19:59:25  <achow101> what block were they forking at?
821 2017-11-16T19:59:30  <achow101> (I need to add it to my site)
822 2017-11-16T19:59:34  <gmaxwell> 494784
823 2017-11-16T19:59:55  <jtimon> weren't they using a naming just the same as bitcoin core but increasing a version? (ie 0.14.3)
824 2017-11-16T20:00:03  <gmaxwell> hopefully someone will mine a couple blocks on that fork to help get those nodes disconnected.
825 2017-11-16T20:00:21  <gmaxwell> jtimon: they made the major version 1.
826 2017-11-16T20:00:26  <achow101> gmaxwell: a mining pool announced that they would go with the 2x fork regardless
827 2017-11-16T20:00:27  <jtimon> oh, right
828 2017-11-16T20:00:27  <jnewbery> won't they disconnect themselves once a valid block is found?
829 2017-11-16T20:00:34  <wumpus> ding dong
830 2017-11-16T20:00:40  <gmaxwell> achow101: that was 'bitpico' who is crazy.
831 2017-11-16T20:00:53  <gmaxwell> it's meaningless.
832 2017-11-16T20:00:59  <achow101> oh
833 2017-11-16T20:01:11  <wumpus> #endmeeting
834 2017-11-16T20:01:11  <lightningbot> Meeting ended Thu Nov 16 20:01:11 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
835 2017-11-16T20:01:11  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-11-16-19.01.html
836 2017-11-16T20:01:11  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-11-16-19.01.txt
837 2017-11-16T20:01:11  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-11-16-19.01.log.html
838 2017-11-16T20:01:52  *** Dizzle has quit IRC
839 2017-11-16T20:25:01  *** luke-jr has quit IRC
840 2017-11-16T20:26:16  *** luke-jr has joined #bitcoin-core-dev
841 2017-11-16T20:27:12  *** jtimon has quit IRC
842 2017-11-16T20:30:50  *** roadcrap has quit IRC
843 2017-11-16T20:54:00  *** LumberCartel has quit IRC
844 2017-11-16T21:01:03  *** laurentmt has quit IRC
845 2017-11-16T21:01:08  *** BashCo_ has joined #bitcoin-core-dev
846 2017-11-16T21:04:10  *** BashCo has quit IRC
847 2017-11-16T21:05:33  *** Cheeseo has joined #bitcoin-core-dev
848 2017-11-16T21:06:18  *** wunpunch has joined #bitcoin-core-dev
849 2017-11-16T21:06:28  *** promag has joined #bitcoin-core-dev
850 2017-11-16T21:11:06  *** promag has joined #bitcoin-core-dev
851 2017-11-16T21:11:55  *** promag has quit IRC
852 2017-11-16T21:12:15  *** sh4ft has quit IRC
853 2017-11-16T21:21:10  *** promag has joined #bitcoin-core-dev
854 2017-11-16T21:22:27  *** promag has quit IRC
855 2017-11-16T21:31:24  *** torkel_ has joined #bitcoin-core-dev
856 2017-11-16T21:38:58  *** torkel_ has quit IRC
857 2017-11-16T21:39:43  *** PaulCapestany has quit IRC
858 2017-11-16T21:39:48  *** torkel_ has joined #bitcoin-core-dev
859 2017-11-16T21:43:22  *** AaronvanW has quit IRC
860 2017-11-16T21:43:58  *** AaronvanW has joined #bitcoin-core-dev
861 2017-11-16T21:44:46  <jonasschnelli> gmaxwell: I'm happy to hear your bip39 successor HWW KDF idea...
862 2017-11-16T21:44:57  *** cryptapus has quit IRC
863 2017-11-16T21:45:06  *** torkel_ is now known as torkelrogstad
864 2017-11-16T21:45:16  <jonasschnelli> PBKDF2 with 2048 rounds seems not ideal (BIP39)
865 2017-11-16T21:46:16  <sipa> jonasschnelli: the idea is a mechanism that allows you to enter the passphrase on a HW device, have the HW device outsource the hardening to a desktop computer (with more power) without revealing the passphrase
866 2017-11-16T21:46:26  <sipa> and then being able to verify the computer did the hardening correctly
867 2017-11-16T21:46:51  <jonasschnelli> +1
868 2017-11-16T21:47:02  <sipa> adam back proposed a scheme for this a while ago, but it's purely CPU dependent
869 2017-11-16T21:47:17  <sipa> whether it can be combined with memory hard hardening is an open question i think
870 2017-11-16T21:47:33  <jonasschnelli> purely CPU is still much better then 2048-PBKDF
871 2017-11-16T21:47:40  <sipa> haha, yes
872 2017-11-16T21:49:17  <jonasschnelli> Somethine we (HWW company) do discuss regularly is how we can make the backup situation better.. a lot of things are involved. Bip39, sdcard, shamir's secret, notary services, etc.
873 2017-11-16T21:49:37  <jonasschnelli> I'm not sure if a plain text seed dump (or BIP39) is something you want in a bank tresor
874 2017-11-16T21:54:10  *** PaulCapestany has joined #bitcoin-core-dev
875 2017-11-16T21:55:02  *** jsfour has quit IRC
876 2017-11-16T22:10:04  *** LumberCartel has joined #bitcoin-core-dev
877 2017-11-16T22:11:47  <goatpig> mdisc?
878 2017-11-16T22:12:19  <jonasschnelli> mdisc?
879 2017-11-16T22:12:31  <goatpig> it's basically a cdrom made out of rock
880 2017-11-16T22:12:35  <goatpig> really really durable
881 2017-11-16T22:12:42  *** Dizzle has joined #bitcoin-core-dev
882 2017-11-16T22:14:38  *** ok7685 has joined #bitcoin-core-dev
883 2017-11-16T22:16:33  *** LumberCartel has quit IRC
884 2017-11-16T22:18:45  *** sunday-afternoon has quit IRC
885 2017-11-16T22:21:07  <jcorgan> i'm not sure if the durability is really demonstrated, but i do have quite a few encrypted live boot images burned to them
886 2017-11-16T22:21:29  *** harrymm has quit IRC
887 2017-11-16T22:22:16  <goatpig> it's hard to demonstrate in practice
888 2017-11-16T22:24:27  *** jsfour has joined #bitcoin-core-dev
889 2017-11-16T22:25:58  *** harrymm has joined #bitcoin-core-dev
890 2017-11-16T22:29:14  <jcorgan> it's a bit like closed-source software, the media is trade secret, but independent testing was pretty good
891 2017-11-16T22:29:24  <jcorgan> http://www.esystor.com/images/China_Lake_Full_Report.pdf
892 2017-11-16T22:30:11  <goatpig> at any rate it's far superior to plain cd/dvds or nvram
893 2017-11-16T22:31:39  <jcorgan> certainly. they're great for "encrypted live boot cold-storage resurrection system" discs scattered in a few places
894 2017-11-16T22:34:07  *** Chris_Stewart_5 has quit IRC
895 2017-11-16T22:38:06  *** JackH has quit IRC
896 2017-11-16T22:38:34  *** JackH has joined #bitcoin-core-dev
897 2017-11-16T22:45:21  *** wxss has quit IRC
898 2017-11-16T22:47:54  <cfields> whoops, totally forgot about today's meeting :\
899 2017-11-16T22:56:54  *** wxss has joined #bitcoin-core-dev
900 2017-11-16T22:58:33  *** PaulCapestany has quit IRC
901 2017-11-16T22:58:38  *** PaulCape_ has joined #bitcoin-core-dev
902 2017-11-16T22:58:46  *** cryptapus has joined #bitcoin-core-dev
903 2017-11-16T22:58:47  *** cryptapus has joined #bitcoin-core-dev
904 2017-11-16T23:02:56  *** meshcollider has quit IRC
905 2017-11-16T23:10:30  *** torkelrogstad has quit IRC
906 2017-11-16T23:10:47  *** torkelrogstad has joined #bitcoin-core-dev
907 2017-11-16T23:11:39  *** tux3do has quit IRC
908 2017-11-16T23:20:25  *** torkelrogstad has quit IRC
909 2017-11-16T23:20:46  *** torkelrogstad has joined #bitcoin-core-dev
910 2017-11-16T23:24:16  *** LumberCartel has joined #bitcoin-core-dev
911 2017-11-16T23:32:33  *** jtimon has joined #bitcoin-core-dev
912 2017-11-16T23:33:57  *** torkelrogstad has quit IRC
913 2017-11-16T23:41:00  *** torkelrogstad has joined #bitcoin-core-dev
914 2017-11-16T23:41:27  *** LumberCartel has quit IRC
915 2017-11-16T23:42:09  *** Cogito_Ergo_Sum has quit IRC
916 2017-11-16T23:50:49  *** goatpig has quit IRC
917 2017-11-16T23:51:20  *** torkelrogstad has quit IRC
918 2017-11-16T23:51:34  *** torkelrogstad has joined #bitcoin-core-dev
919 2017-11-16T23:51:50  *** meshcollider has joined #bitcoin-core-dev
920 2017-11-16T23:56:33  *** jsfour has quit IRC