1 2017-06-29T00:13:08  *** IRC-Source_13446 has joined #bitcoin-core-dev
  2 2017-06-29T00:21:52  *** Victorsueca has quit IRC
  3 2017-06-29T00:32:52  *** Chris_Stewart_5 has quit IRC
  4 2017-06-29T00:35:26  *** Victorsueca has joined #bitcoin-core-dev
  5 2017-06-29T00:36:07  *** mol has joined #bitcoin-core-dev
  6 2017-06-29T01:00:48  *** dabura667 has joined #bitcoin-core-dev
  7 2017-06-29T01:16:21  *** dabura667 has quit IRC
  8 2017-06-29T01:16:34  *** dabura667 has joined #bitcoin-core-dev
  9 2017-06-29T01:18:35  <phantomcircuit> gmaxwell, btw master compiles for me
 10 2017-06-29T01:31:05  <phantomcircuit> why did the pull testing stuff get changed?
 11 2017-06-29T01:31:11  <phantomcircuit> oh travis changed
 12 2017-06-29T02:04:40  *** elkalamar has joined #bitcoin-core-dev
 13 2017-06-29T02:19:02  *** mol has quit IRC
 14 2017-06-29T02:30:35  *** goatpig has joined #bitcoin-core-dev
 15 2017-06-29T03:11:17  *** belcher_ has quit IRC
 16 2017-06-29T03:27:52  *** SopaXorzTaker has quit IRC
 17 2017-06-29T03:32:06  *** SopaXorzTaker has joined #bitcoin-core-dev
 18 2017-06-29T03:37:33  *** jamesob has quit IRC
 19 2017-06-29T03:38:07  *** jamesob has joined #bitcoin-core-dev
 20 2017-06-29T03:42:22  *** jamesob has quit IRC
 21 2017-06-29T04:11:22  *** movrcx has quit IRC
 22 2017-06-29T04:12:27  *** fizzwont has quit IRC
 23 2017-06-29T04:12:30  *** j_ybt has joined #bitcoin-core-dev
 24 2017-06-29T04:13:39  *** fizzwont has joined #bitcoin-core-dev
 25 2017-06-29T04:14:03  *** fizzwont is now known as Guest7920
 26 2017-06-29T04:16:46  *** owowo has quit IRC
 27 2017-06-29T04:18:01  *** d9b4bef9 has quit IRC
 28 2017-06-29T04:19:07  *** d9b4bef9 has joined #bitcoin-core-dev
 29 2017-06-29T04:20:01  *** d9b4bef9 has quit IRC
 30 2017-06-29T04:21:09  *** d9b4bef9 has joined #bitcoin-core-dev
 31 2017-06-29T04:23:14  *** owowo has joined #bitcoin-core-dev
 32 2017-06-29T04:26:40  *** Guest7920 has quit IRC
 33 2017-06-29T04:27:09  *** fizzwont_ has joined #bitcoin-core-dev
 34 2017-06-29T04:28:29  *** j_ybt has quit IRC
 35 2017-06-29T04:30:16  *** j_ybt has joined #bitcoin-core-dev
 36 2017-06-29T04:54:38  *** Giszmo has quit IRC
 37 2017-06-29T05:00:50  *** fizzwont_ is now known as fizzwont
 38 2017-06-29T05:00:56  *** fizzwont has joined #bitcoin-core-dev
 39 2017-06-29T05:03:46  *** ProfMac has joined #bitcoin-core-dev
 40 2017-06-29T05:10:31  *** unholymachine has quit IRC
 41 2017-06-29T06:03:52  *** harrymm has quit IRC
 42 2017-06-29T06:25:44  *** harrymm has joined #bitcoin-core-dev
 43 2017-06-29T06:40:36  <bitcoin-git> [bitcoin] jonasschnelli closed pull request #9722: GUI: Display warning when attempting address reuse (master...reuse) https://github.com/bitcoin/bitcoin/pull/9722
 44 2017-06-29T07:00:05  *** dermoth has quit IRC
 45 2017-06-29T07:00:35  *** dermoth has joined #bitcoin-core-dev
 46 2017-06-29T07:04:24  *** j_ybt has quit IRC
 47 2017-06-29T07:09:03  *** j_ybt has joined #bitcoin-core-dev
 48 2017-06-29T07:19:40  *** harrymm1 has joined #bitcoin-core-dev
 49 2017-06-29T07:19:48  *** j_ybt has quit IRC
 50 2017-06-29T07:21:09  *** harrymm has quit IRC
 51 2017-06-29T07:24:05  *** arowser has joined #bitcoin-core-dev
 52 2017-06-29T07:24:22  *** harrymm1 has quit IRC
 53 2017-06-29T07:39:29  *** harrymm has joined #bitcoin-core-dev
 54 2017-06-29T07:41:31  *** Ylbam has joined #bitcoin-core-dev
 55 2017-06-29T07:43:26  *** baldur has quit IRC
 56 2017-06-29T08:02:50  *** chjj has quit IRC
 57 2017-06-29T08:04:54  *** chjj has joined #bitcoin-core-dev
 58 2017-06-29T08:14:37  *** timothy has joined #bitcoin-core-dev
 59 2017-06-29T08:16:05  *** AaronvanW has joined #bitcoin-core-dev
 60 2017-06-29T08:18:06  *** Aaronvan_ has joined #bitcoin-core-dev
 61 2017-06-29T08:21:31  *** AaronvanW has quit IRC
 62 2017-06-29T08:30:07  *** JackH has joined #bitcoin-core-dev
 63 2017-06-29T08:42:32  *** riemann has joined #bitcoin-core-dev
 64 2017-06-29T08:53:21  *** sjums has quit IRC
 65 2017-06-29T08:56:30  *** sjums has joined #bitcoin-core-dev
 66 2017-06-29T08:57:58  *** timothy has quit IRC
 67 2017-06-29T08:59:31  *** timothy has joined #bitcoin-core-dev
 68 2017-06-29T09:05:35  <bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/90a002ea647d...080ec5209172
 69 2017-06-29T09:05:35  <bitcoin-git> bitcoin/master 3c85332 Wladimir J. van der Laan: contrib: Update laanwj key...
 70 2017-06-29T09:05:36  <bitcoin-git> bitcoin/master 080ec52 MarcoFalke: Merge #10688: contrib: Update laanwj key...
 71 2017-06-29T09:06:06  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #10688: contrib: Update laanwj key (master...2017_06_laanwj_key) https://github.com/bitcoin/bitcoin/pull/10688
 72 2017-06-29T09:13:36  *** jannes has joined #bitcoin-core-dev
 73 2017-06-29T09:50:31  *** baldur has joined #bitcoin-core-dev
 74 2017-06-29T09:56:09  <wumpus> cfields: btw any idea what to do here? https://github.com/bitcoin/bitcoin/issues/10670   arguably the openbsd case is worst-case, they use an ancient GNU binutils
 75 2017-06-29T09:57:39  <wumpus> would be possible to detect this in configure, I guess, and then not disable the sse-reliant stuff
 76 2017-06-29T09:57:47  <wumpus> s/disable/enable
 77 2017-06-29T10:10:08  *** bincap has joined #bitcoin-core-dev
 78 2017-06-29T10:10:28  *** bincap has quit IRC
 79 2017-06-29T10:25:13  *** LeMiner2 has joined #bitcoin-core-dev
 80 2017-06-29T10:26:28  *** LeMiner has quit IRC
 81 2017-06-29T10:26:28  *** LeMiner2 is now known as LeMiner
 82 2017-06-29T10:36:20  *** Guest20283 is now known as haakonn
 83 2017-06-29T10:36:31  *** haakonn has joined #bitcoin-core-dev
 84 2017-06-29T11:09:03  <bitcoin-git> [bitcoin] practicalswift opened pull request #10701: Remove the virtual specifier for functions with the override specifier (master...virtual-override) https://github.com/bitcoin/bitcoin/pull/10701
 85 2017-06-29T11:22:55  *** talmai has joined #bitcoin-core-dev
 86 2017-06-29T11:35:33  *** jannes has quit IRC
 87 2017-06-29T11:35:59  *** jannes has joined #bitcoin-core-dev
 88 2017-06-29T11:38:06  *** Guyver2 has joined #bitcoin-core-dev
 89 2017-06-29T11:44:16  *** laurentmt has joined #bitcoin-core-dev
 90 2017-06-29T11:44:55  *** laurentmt has quit IRC
 91 2017-06-29T11:45:20  *** jannes has quit IRC
 92 2017-06-29T11:46:17  *** dabura667 has quit IRC
 93 2017-06-29T11:54:59  *** jannes has joined #bitcoin-core-dev
 94 2017-06-29T11:59:58  *** jannes has quit IRC
 95 2017-06-29T12:00:15  *** jannes has joined #bitcoin-core-dev
 96 2017-06-29T12:05:30  *** unholymachine has joined #bitcoin-core-dev
 97 2017-06-29T12:09:40  *** jannes has quit IRC
 98 2017-06-29T12:09:57  *** jannes has joined #bitcoin-core-dev
 99 2017-06-29T12:26:07  <bitcoin-git> [bitcoin] MeshCollider opened pull request #10702: [Trivial] Improve end-of-namespace comment consistency (master...improve-end-of-namespace-comment-consistence) https://github.com/bitcoin/bitcoin/pull/10702
100 2017-06-29T12:28:38  *** arubi has quit IRC
101 2017-06-29T12:29:02  *** arubi has joined #bitcoin-core-dev
102 2017-06-29T12:33:14  <wumpus> what is people's obsession with en-of-namespace comments
103 2017-06-29T12:33:52  <wumpus> we've just merged one three days ago (#9544), and now we need another PR?
104 2017-06-29T12:33:53  <gribble> https://github.com/bitcoin/bitcoin/issues/9544 | [trivial] Add end of namespace comments. Improve consistency. by practicalswift · Pull Request #9544 · bitcoin/bitcoin · GitHub
105 2017-06-29T12:55:37  *** belcher_ has joined #bitcoin-core-dev
106 2017-06-29T13:04:02  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/080ec5209172...4c72cc33ebcc
107 2017-06-29T13:04:02  <bitcoin-git> bitcoin/master fd9599b practicalswift: [qt] Avoid potential null pointer dereference in TransactionView::exportClicked()
108 2017-06-29T13:04:03  <bitcoin-git> bitcoin/master 4c72cc3 Wladimir J. van der Laan: Merge #10673: [qt] Avoid potential null pointer dereference in TransactionView::exportClicked()...
109 2017-06-29T13:04:32  <bitcoin-git> [bitcoin] laanwj closed pull request #10673: [qt] Avoid potential null pointer dereference in TransactionView::exportClicked() (master...null-pointer-dereference) https://github.com/bitcoin/bitcoin/pull/10673
110 2017-06-29T13:05:18  *** Chris_Stewart_5 has joined #bitcoin-core-dev
111 2017-06-29T13:36:06  <bitcoin-git> [bitcoin] jnewbery opened pull request #10703: [tests] Allow tests to pass when stderr is non-empty (master...test_stderr) https://github.com/bitcoin/bitcoin/pull/10703
112 2017-06-29T13:57:49  <bitcoin-git> [bitcoin] jnewbery opened pull request #10704: [tests] nits in dbcrash.py (master...dbcrashtestnits) https://github.com/bitcoin/bitcoin/pull/10704
113 2017-06-29T14:11:25  *** nemgun has joined #bitcoin-core-dev
114 2017-06-29T14:16:56  *** Giszmo has joined #bitcoin-core-dev
115 2017-06-29T14:23:46  *** Guest8908 has quit IRC
116 2017-06-29T14:24:49  *** arubi has quit IRC
117 2017-06-29T14:24:49  *** dermoth has quit IRC
118 2017-06-29T14:25:04  *** tripleslash has joined #bitcoin-core-dev
119 2017-06-29T14:25:27  *** tripleslash is now known as Guest86952
120 2017-06-29T14:28:40  *** arubi has joined #bitcoin-core-dev
121 2017-06-29T14:29:32  *** dermoth has joined #bitcoin-core-dev
122 2017-06-29T14:30:17  *** mol has joined #bitcoin-core-dev
123 2017-06-29T14:36:52  *** Dyaheon has quit IRC
124 2017-06-29T14:37:39  *** Dyaheon has joined #bitcoin-core-dev
125 2017-06-29T14:40:22  *** mol- has joined #bitcoin-core-dev
126 2017-06-29T14:43:07  *** mol has quit IRC
127 2017-06-29T14:43:08  *** jannes has quit IRC
128 2017-06-29T14:43:53  *** jannes has joined #bitcoin-core-dev
129 2017-06-29T14:58:28  *** jannes has quit IRC
130 2017-06-29T14:59:53  *** Murch has joined #bitcoin-core-dev
131 2017-06-29T15:03:50  *** talmai has quit IRC
132 2017-06-29T15:11:17  *** jannes has joined #bitcoin-core-dev
133 2017-06-29T15:14:14  *** mol- has quit IRC
134 2017-06-29T15:15:40  *** Dizzle has joined #bitcoin-core-dev
135 2017-06-29T15:18:28  *** riemann has quit IRC
136 2017-06-29T15:18:52  *** jannes has quit IRC
137 2017-06-29T15:20:02  <jonasschnelli> Should we crate a "trivial" label?
138 2017-06-29T15:20:17  *** molz has joined #bitcoin-core-dev
139 2017-06-29T15:20:39  <jonasschnelli> or something that point concludes "typo only fixes / comments / etc."?
140 2017-06-29T15:25:13  <wumpus> we had a trivial label in the past, I removed that at some point because it didn't add anything that 'docs/output' or 'refactoring' doesn't do
141 2017-06-29T15:25:46  <wumpus> it made people have the idea that labeling something 'trivial' would make it accepted sooner, prompting the creating of tons of trivial changes
142 2017-06-29T15:26:17  <wumpus> in any case, better to label specifically. 'trivial' doesn't really tell anything about the change
143 2017-06-29T15:27:19  <wumpus> e.g. a comment change would be a documentation change
144 2017-06-29T15:31:12  *** jannes has joined #bitcoin-core-dev
145 2017-06-29T15:32:56  *** eenoch_ has quit IRC
146 2017-06-29T15:33:49  *** eenoch has joined #bitcoin-core-dev
147 2017-06-29T15:37:32  *** sanada has quit IRC
148 2017-06-29T15:37:41  *** sanada has joined #bitcoin-core-dev
149 2017-06-29T15:39:29  <wumpus> dbcrash.py travis troubles https://github.com/bitcoin/bitcoin/pull/10704#issuecomment-312005841
150 2017-06-29T15:40:58  <bitcoin-git> [bitcoin] MarcoFalke pushed 7 new commits to master: https://github.com/bitcoin/bitcoin/compare/4c72cc33ebcc...65cc7aacfbfc
151 2017-06-29T15:41:00  <bitcoin-git> bitcoin/master 37065d2 John Newbery: [tests] remove unused imports from utils.py
152 2017-06-29T15:41:00  <bitcoin-git> bitcoin/master f1fe536 John Newbery: [tests] fix flake8 warnings in test_framework.py and util.py
153 2017-06-29T15:41:01  <bitcoin-git> bitcoin/master cad967a John Newbery: [tests] Move stop_node and start_node methods to BitcoinTestFramework...
154 2017-06-29T15:41:23  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #10556: Move stop/start functions from utils.py into BitcoinTestFramework (master...testframeworkstopstart) https://github.com/bitcoin/bitcoin/pull/10556
155 2017-06-29T15:44:39  *** abpa has joined #bitcoin-core-dev
156 2017-06-29T15:49:18  <jonasschnelli> wumpus: I see, good point about the "trivial" label.
157 2017-06-29T15:59:01  <sdaftuar> wumpus: thanks for the heads up, i'll investigate the dbcrash issue
158 2017-06-29T16:00:20  *** talmai has joined #bitcoin-core-dev
159 2017-06-29T16:00:22  <wumpus> sdaftuar: they look like different problems; 248398016 seems a race issue (sending command to terminated process), 248398018 is a timeout while running `generate`. It's a tad strange that both problems happen to be in dbcrash.
160 2017-06-29T16:00:57  <sdaftuar> wumpus: it looks like the issue in 248398016 may just be that i got the exception name wrong somehow
161 2017-06-29T16:01:08  <sdaftuar> https://travis-ci.org/bitcoin/bitcoin/jobs/248398016#L3321
162 2017-06-29T16:03:50  <sdaftuar> we can bump the rpctimeout for the test to fix that second problem
163 2017-06-29T16:04:59  <wumpus> sdaftuar: oops, good catch, didn't notice that between all the errors
164 2017-06-29T16:06:47  <wumpus> yes that seems just a case of 'travis VM too slow for timeout'
165 2017-06-29T16:10:18  *** mol has joined #bitcoin-core-dev
166 2017-06-29T16:12:49  *** molz has quit IRC
167 2017-06-29T16:24:41  *** talmai has quit IRC
168 2017-06-29T16:32:34  *** Guyver2_ has joined #bitcoin-core-dev
169 2017-06-29T16:34:48  *** Guyver2 has quit IRC
170 2017-06-29T16:36:24  <cfields> wumpus: hmm
171 2017-06-29T16:36:59  <cfields> wumpus: re the openbsd assembler, i think we could patch it to use the hex, same as rdrand? :(
172 2017-06-29T16:38:21  <cfields> (i don't know enough about assemblers, but i figured that was done that way for exactly this reason)
173 2017-06-29T16:38:54  <cfields> but yes, we could also check during configure
174 2017-06-29T16:39:29  *** abpa has quit IRC
175 2017-06-29T16:42:08  *** bincap has joined #bitcoin-core-dev
176 2017-06-29T16:48:58  <sipa> cfields: yeah, i tried using rdrand as a mnemonic, but the osx assembler didn't accept it on travis
177 2017-06-29T16:49:26  <sipa> hex always works, but the downside is that you must hardcode the registers
178 2017-06-29T16:50:33  <wumpus> cfields: I don't particularly care if the sse-accelerated code is not used on openbsd
179 2017-06-29T16:50:43  <wumpus> cfields: they have to use openssl -no-asm as well
180 2017-06-29T16:51:15  <cfields> heh, really? That's a big hit
181 2017-06-29T16:51:30  <wumpus> we definitely shouldn't dumb down the code just for this case, there's a large chance that openbsd will switch to clang at some pointa different as
182 2017-06-29T16:51:33  <wumpus> yes, really
183 2017-06-29T16:52:20  <cfields> yikes, ok
184 2017-06-29T16:52:56  <wumpus> I don't want to get involved in the politics here, they choose to use an old as that doens't support modern instructions, they get slower code. It does need to compile, though.
185 2017-06-29T16:53:42  *** timothy has quit IRC
186 2017-06-29T16:53:42  <cfields> ok. let's just add an inline-compile check rather than trying to hunt down the assembler though. some (clang, at least) let you choose between an internal assembler or external
187 2017-06-29T16:53:53  <sipa> using hex asm also has the advantage of not needing separately compiled objects just to have access to one asm instruction
188 2017-06-29T16:53:58  <wumpus> their gcc is also -  by choice - an ancient version
189 2017-06-29T16:54:17  <wumpus> ok, just don't do this for openbsd, it's likely a temporary issue
190 2017-06-29T16:54:24  <sipa> agree
191 2017-06-29T17:12:37  *** JackH has quit IRC
192 2017-06-29T17:19:10  *** Aaronvan_ is now known as AaronvanW
193 2017-06-29T17:21:43  *** Guyver2_ has quit IRC
194 2017-06-29T17:24:57  *** owowo has quit IRC
195 2017-06-29T17:30:06  *** owowo has joined #bitcoin-core-dev
196 2017-06-29T17:35:00  *** Yogaqueef has joined #bitcoin-core-dev
197 2017-06-29T17:49:13  *** sdaftuar has quit IRC
198 2017-06-29T17:49:29  <bitcoin-git> [bitcoin] ka7 opened pull request #10705: Trivial: spelling fixes (master...feature/spelling) https://github.com/bitcoin/bitcoin/pull/10705
199 2017-06-29T17:56:27  <bitcoin-git> [bitcoin] laanwj pushed 7 new commits to master: https://github.com/bitcoin/bitcoin/compare/65cc7aacfbfc...0c3542e5dec3
200 2017-06-29T17:56:28  <bitcoin-git> bitcoin/master 00cb69b Jonas Schnelli: [Qt] allow to execute a callback during splashscreen progress
201 2017-06-29T17:56:28  <bitcoin-git> bitcoin/master ae09d45 Jonas Schnelli: Allow to shut down during txdb upgrade
202 2017-06-29T17:56:29  <bitcoin-git> bitcoin/master 316fcb5 Jonas Schnelli: Allow to cancel the txdb upgrade via splashscreen callback
203 2017-06-29T17:57:01  <bitcoin-git> [bitcoin] laanwj closed pull request #10660: Allow to cancel the txdb upgrade via splashscreen keypress 'q' (master...2017/06/chainstate_update_prog) https://github.com/bitcoin/bitcoin/pull/10660
204 2017-06-29T18:17:50  *** shesek has quit IRC
205 2017-06-29T18:18:37  <bitcoin-git> [bitcoin] morcos opened pull request #10706: Improve wallet fee logic and fix GUI bugs (master...improveWalletFeeLogic) https://github.com/bitcoin/bitcoin/pull/10706
206 2017-06-29T18:20:06  <bitcoin-git> [bitcoin] laanwj pushed 8 new commits to master: https://github.com/bitcoin/bitcoin/compare/0c3542e5dec3...2935b469ae96
207 2017-06-29T18:20:07  <bitcoin-git> bitcoin/master 6d22b2b Matt Corallo: Pull script verify flags calculation out of ConnectBlock
208 2017-06-29T18:20:07  <bitcoin-git> bitcoin/master b5fea8d Matt Corallo: Cache full script execution results in addition to signatures...
209 2017-06-29T18:20:08  <bitcoin-git> bitcoin/master eada04e Matt Corallo: Do not print soft-fork-script warning with -promiscuousmempool
210 2017-06-29T18:20:19  <bitcoin-git> [bitcoin] laanwj closed pull request #10192: Cache full script execution results in addition to signatures (master...2017-04-cache-scriptchecks) https://github.com/bitcoin/bitcoin/pull/10192
211 2017-06-29T18:24:34  <sipa> w00t
212 2017-06-29T18:24:47  <BlueMatt> hey, neat
213 2017-06-29T18:24:52  <BlueMatt> 0.15 is coming together :)
214 2017-06-29T18:25:27  <Dizzle> Definitely. I'm sad unix sockets aren't in. Looking forward to that for my electrum server.
215 2017-06-29T18:29:00  <wumpus> Dizzle: good to hear at least someone was waiting for the UNIX sockets stuff, did you review/test the PR?
216 2017-06-29T18:29:33  <Dizzle> wumpus: I will be this weekend, am getting an electrumx patch ready for it.
217 2017-06-29T18:29:47  <wumpus> awesome
218 2017-06-29T18:33:21  <wumpus> Dizzle: are you going to use P2P or RPC over unix sockets?
219 2017-06-29T18:33:29  <wumpus> (or both)
220 2017-06-29T18:33:29  <Dizzle> RPC
221 2017-06-29T18:35:21  <Dizzle> Electrum servers tend to maintain their own utxo DB. Syncing with your node's db over RPC is a bit of a bottleneck. Using asynchronous i/o speeds things along but there is plenty of overhead using the TCP loopback when both softwares are in the same operating environment.
222 2017-06-29T18:37:04  <bitcoin-git> [bitcoin] morcos opened pull request #10707: Better API for estimatesmartfee RPC  (master...bettersmartfeeapi) https://github.com/bitcoin/bitcoin/pull/10707
223 2017-06-29T18:37:12  <wumpus> oh interesting, I had mostly seen the UNIX sockets as security improvement, not so much for performance, but yes avoiding local TCP might save some overhead
224 2017-06-29T18:48:42  <gmaxwell> wumpus: well for p2p it would be a performance improvement if we changed the protocol and dropped the 'crc', but I don't think we were planning on doing that.
225 2017-06-29T18:49:26  <jcorgan> i have a non-bitcoin related application that communicates between two processes using a unix socket at a continuous 5Gbps data rate
226 2017-06-29T18:49:43  <jcorgan> uses zmq over unix socket
227 2017-06-29T18:50:10  <gmaxwell> jcorgan: yea, right now though for us the fact that every p2p message gets sha2ed is way more overhead than TCP.
228 2017-06-29T18:50:31  <wumpus> http://bhavin.directi.com/unix-domain-sockets-vs-tcp-sockets/
229 2017-06-29T18:50:38  <jcorgan> oh, sure, i was just commenting that unix sockets are pretty fast
230 2017-06-29T18:51:00  <wumpus> so there's really a performance gain in both latency and throughput because no local network needs to be emulates, I never realized that
231 2017-06-29T18:51:16  <wumpus> makes sense though
232 2017-06-29T18:51:33  <gmaxwell> perhaps an input to the BIP151 stuff... that it should be possible to run it in a mode that turns off the auth for use over a domain socket at least.
233 2017-06-29T18:52:16  <wumpus> well one of the uses for UNIX would be for tor; in that case we certainly don't want to disable the crcing, or auth. But agree it'd be nice to have it as possibility.
234 2017-06-29T18:52:57  <gmaxwell> yea. and one always needs to worry about downgrade attacks.
235 2017-06-29T18:53:03  <jonasschnelli> gmaxwell: great idea
236 2017-06-29T18:53:30  <wumpus> it'd be another kind of whitebind 'nocrcbind' 'noauthbind'
237 2017-06-29T18:53:37  <jonasschnelli> gmaxwell: But then there would be no checksum... if you disable the poly1305 mac?
238 2017-06-29T18:53:39  <wumpus> bindflags extension
239 2017-06-29T18:53:58  <gmaxwell> jonasschnelli: right but there isn't any need for one with a purely local unix domain socket.
240 2017-06-29T18:54:07  <wumpus> jonasschnelli: over a local socket, when communicating with local software, there's no point
241 2017-06-29T18:54:20  <jonasschnelli> Indeed.
242 2017-06-29T18:54:25  *** Murch has quit IRC
243 2017-06-29T18:54:29  <jonasschnelli> Corruption through socket coms is not possible I guess?
244 2017-06-29T18:54:38  <wumpus> and the permissions of the socket itself are used for authentication
245 2017-06-29T18:54:40  <jonasschnelli> *domain sockets
246 2017-06-29T18:54:44  *** SopaXorzTaker has quit IRC
247 2017-06-29T18:54:49  <gmaxwell> jonasschnelli: no, not any more than any random memory anywhere.
248 2017-06-29T18:54:53  <wumpus> no, unless memory/cpu corruption, in which case there's other issues
249 2017-06-29T18:55:23  <wumpus> this is a SOCK_STREAM AF_UNIX socket, so neither reordering nor corruption nor packet loss should happen
250 2017-06-29T18:55:28  <jonasschnelli> I see... yes. That mode would be awesome... especially for wallets (stuff I'm writing into libbtc) that downloads everything from the local peer
251 2017-06-29T18:55:31  *** talmai has joined #bitcoin-core-dev
252 2017-06-29T18:56:15  <wumpus> it's theoretically possible for SOCK_DGRAM AF_UNIX sockets to deliver packets out of order, though I've never heard of an OS that does that (but anyhow not an issue for us)
253 2017-06-29T18:57:20  <jcorgan> one nice thing about ZMQ is that with a parameter change I can do the same thing between network endpoints on different hosts or two processes on the same host over a unix socket
254 2017-06-29T18:57:24  <jcorgan> no code changes
255 2017-06-29T18:57:29  *** lifeofguenter has quit IRC
256 2017-06-29T18:57:46  <jcorgan> (or even two threads using zmq over shared memory)
257 2017-06-29T18:57:58  <wumpus> yes, that's nice
258 2017-06-29T18:58:28  <jcorgan> anyway, carry on
259 2017-06-29T18:59:03  *** lifeofguenter has joined #bitcoin-core-dev
260 2017-06-29T18:59:14  *** Murch has joined #bitcoin-core-dev
261 2017-06-29T19:00:02  <gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
262 2017-06-29T19:00:12  <wumpus> #startmeeting
263 2017-06-29T19:00:12  <lightningbot> Meeting started Thu Jun 29 19:00:12 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
264 2017-06-29T19:00:12  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
265 2017-06-29T19:00:41  <sipa> oi
266 2017-06-29T19:00:46  <instagibbs> #meetingbegin
267 2017-06-29T19:00:58  <wumpus> topics?
268 2017-06-29T19:00:59  <morcos> i'd like to discuss the fee changes needed for 0.15
269 2017-06-29T19:01:06  <sipa> i have a few topics
270 2017-06-29T19:01:16  <instagibbs> morcos, ack
271 2017-06-29T19:01:19  <sipa> short update on signature aggregation
272 2017-06-29T19:01:25  *** sdaftuar has joined #bitcoin-core-dev
273 2017-06-29T19:01:29  * BlueMatt wants to change priority review to #10652
274 2017-06-29T19:01:31  <gribble> https://github.com/bitcoin/bitcoin/issues/10652 | Small step towards demangling cs_main from CNodeState by TheBlueMatt · Pull Request #10652 · bitcoin/bitcoin · GitHub
275 2017-06-29T19:01:34  <gmaxwell> hurray for merges.
276 2017-06-29T19:01:44  <sipa> the need for the watchonly rpc flag after multiwallet
277 2017-06-29T19:01:47  <cfields> hi, here
278 2017-06-29T19:02:03  *** praxeology has joined #bitcoin-core-dev
279 2017-06-29T19:02:09  <sipa> rolling utxo hashes
280 2017-06-29T19:02:22  <wumpus> thanks for the topic suggestions, yes let's as usual start with high priority for review
281 2017-06-29T19:02:33  <wumpus> #topic high priority for review
282 2017-06-29T19:02:35  <wumpus> https://github.com/bitcoin/bitcoin/projects/8
283 2017-06-29T19:02:38  <jonasschnelli> suggesting: again multiwallet endpoint vs json parameter
284 2017-06-29T19:02:58  *** shesek has joined #bitcoin-core-dev
285 2017-06-29T19:03:08  <wumpus> BlueMatt: instead of #10179?
286 2017-06-29T19:03:11  <gribble> https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
287 2017-06-29T19:03:15  <BlueMatt> correct
288 2017-06-29T19:03:17  <kanzure> hi.
289 2017-06-29T19:03:22  <BlueMatt> well, actually, its built on
290 2017-06-29T19:03:22  <jonasschnelli> Replaced BlueMatt's 10179 with 10652
291 2017-06-29T19:03:23  <BlueMatt> so...ehh
292 2017-06-29T19:03:29  <BlueMatt> but, yea
293 2017-06-29T19:03:46  <jonasschnelli> aha.. double pull binding strategy. :)
294 2017-06-29T19:04:04  <BlueMatt> i mean 10179 is like one ack away, just want cfields to confirm i addressed his feedback sufficiently
295 2017-06-29T19:04:25  <morcos> So I don't think I've had any there for a couple weeks, if I could add two?  It would be the first two of the fee changes, both have been open a little while, #10543 and #10589
296 2017-06-29T19:04:26  <gribble> https://github.com/bitcoin/bitcoin/issues/10543 | Change API to estimaterawfee by morcos · Pull Request #10543 · bitcoin/bitcoin · GitHub
297 2017-06-29T19:04:27  <gribble> https://github.com/bitcoin/bitcoin/issues/10589 | More economical fee estimates for RBF and RPC options to control by morcos · Pull Request #10589 · bitcoin/bitcoin · GitHub
298 2017-06-29T19:04:35  <morcos> I apologize I have not been around to do more reviewing recently
299 2017-06-29T19:05:09  <wumpus> BlueMatt: yes, as we discussed: it should still be merged, but it's no longer high-priority because you don't expect the dependent PR to get in in time to be safe for 0.15
300 2017-06-29T19:05:22  <jonasschnelli> morcos: which one di you want to add to the high-prio list?
301 2017-06-29T19:05:29  <jonasschnelli> *do
302 2017-06-29T19:05:31  <wumpus> both
303 2017-06-29T19:05:55  <morcos> both! :)  but i suppose 10589, if i can only have one
304 2017-06-29T19:05:56  <jonasschnelli> Good
305 2017-06-29T19:06:00  <BlueMatt> wumpus: well I want some glances at 10652 pre-15 to see if its too much or if it can go ahead...if its small enough for 15 I do want it for 15
306 2017-06-29T19:06:19  <cfields> BlueMatt: yes, good enough. Will ACK it.
307 2017-06-29T19:06:19  <jonasschnelli> We need both for 0.15
308 2017-06-29T19:06:20  <BlueMatt> (since it fixes the kinda-not-a-big-deal provide-invalid-block attack thing)
309 2017-06-29T19:07:15  <wumpus> ok - any other suggestions?
310 2017-06-29T19:07:23  <wumpus> enough other topics otherwise
311 2017-06-29T19:07:40  <wumpus> #topic short update on signature aggregation
312 2017-06-29T19:07:43  <sipa> hi
313 2017-06-29T19:07:43  <wumpus> (sipa)
314 2017-06-29T19:07:54  <praxeology> Whats the status on the mempool data structure change?
315 2017-06-29T19:08:05  <praxeology> woops not mempool
316 2017-06-29T19:08:07  <sipa> this is just a status update of what gmaxwell, apoelstra and me have been working on lately
317 2017-06-29T19:08:08  <praxeology> utxo
318 2017-06-29T19:08:11  <wumpus> praxeology: you're interrupting a meeting
319 2017-06-29T19:08:31  <sipa> i presented on this in milan, and later we wrote a paper for bitcoin17
320 2017-06-29T19:08:37  <gmaxwell> praxeology: long since done.
321 2017-06-29T19:08:50  <sipa> the paper was rejected with the very valuable feedback that a solution already existed
322 2017-06-29T19:09:05  <sipa> namely a paper by Bellare & Neven from 2006
323 2017-06-29T19:09:42  <sipa> it only solves one of the problems we were trying to solve (signature aggregation, not key aggregation)... but that's the only consensus-critical part if we'd want aggregation in bitcoin trnasactions
324 2017-06-29T19:09:53  <gmaxwell> (which irritatingly never turned up in eons of searching for us)
325 2017-06-29T19:10:15  <wumpus> so that solution is usable for bitcoin?
326 2017-06-29T19:10:18  <sipa> yes
327 2017-06-29T19:10:24  <sipa> the advantage is that this is peer-reviewed scheme with a strong security proof under very wide assumptions
328 2017-06-29T19:10:26  <wumpus> nice!
329 2017-06-29T19:10:29  <gmaxwell> Their solution is almost equivilent to ours (or is equivient with the right kind of squinting about hash function definitions).
330 2017-06-29T19:10:31  <jonasschnelli> https://eprint.iacr.org/2006/285.pdf
331 2017-06-29T19:10:37  *** ivan has joined #bitcoin-core-dev
332 2017-06-29T19:10:38  <gmaxwell> so thats good too.
333 2017-06-29T19:11:08  <wumpus> yes
334 2017-06-29T19:11:12  <wumpus> good news
335 2017-06-29T19:11:16  <gmaxwell> jonasschnelli: doesn't look like the right paper (though maybe its one they published to another venue)
336 2017-06-29T19:11:17  <BlueMatt> cool!
337 2017-06-29T19:11:48  <sipa> so what this scheme gives us is a way for transactions to have a single signature (as long as all signers cooperate, so even in the case of coinjoin) overall... regardless of the number of inputs or multisig
338 2017-06-29T19:11:51  <wumpus> do we have a good link?
339 2017-06-29T19:12:08  <sipa> https://cseweb.ucsd.edu/~mihir/papers/multisignatures-ccs.pdf
340 2017-06-29T19:12:14  <sipa> ^
341 2017-06-29T19:12:24  <gmaxwell> ^ thats it.
342 2017-06-29T19:12:26  <wumpus> #link https://cseweb.ucsd.edu/~mihir/papers/multisignatures-ccs.pdf
343 2017-06-29T19:12:35  <sipa> what it does not do is an ability to turn multisig into signle sig (but that could be added on top later, as it's purely a wallet interaction thing)
344 2017-06-29T19:12:43  <sipa> it also supports batch validation
345 2017-06-29T19:12:53  <cfields> ooh
346 2017-06-29T19:12:56  <sipa> meaning that a whole block (or even multiple blocks) could be validated at once
347 2017-06-29T19:13:16  <sipa> the speedup depends on the size of the batch, but may go as high as 5x (for 4000 signatures)
348 2017-06-29T19:13:21  <gmaxwell> Unfortunately our paper isn't available because we need to update it to reflect that work,  but it is much more targeted for the Bitcoin application (and would probably be much more clear for people here).
349 2017-06-29T19:13:50  <sipa> in the batch validation case (without aggregated signatures) the speedup would likely be restricted to 3.5x or so
350 2017-06-29T19:13:55  <morcos> gmaxwell: is that something that'll happen?  can we just wait to read yours?
351 2017-06-29T19:14:08  <sipa> yes, we'll definitely finish up the paper
352 2017-06-29T19:14:18  <sipa> and discuss the change more widely
353 2017-06-29T19:14:24  <sipa> just wanted to give a heads up here
354 2017-06-29T19:14:31  <wumpus> yes, thanks for the update!
355 2017-06-29T19:14:37  <morcos> if i could have next topic, i have to leave early
356 2017-06-29T19:14:40  <cfields> sipa: what about that per-block aggregation that was briefly discussed? does this get us any closer to that?
357 2017-06-29T19:14:46  *** jannes has quit IRC
358 2017-06-29T19:14:50  <cfields> nm, will follow-up after meeting
359 2017-06-29T19:15:07  <wumpus> morcos: what was your topic?
360 2017-06-29T19:15:13  <gmaxwell> ~2.3x speedup for 32 signatures in the aggregate, fwiw.
361 2017-06-29T19:15:17  <morcos> Fee changes for 0.15
362 2017-06-29T19:15:27  <wumpus> #topic fee changes needed for 0.15
363 2017-06-29T19:15:31  <wumpus> morcos: sorry, missed that one
364 2017-06-29T19:15:50  <wumpus> morcos: you were actually first to propose a topic :)
365 2017-06-29T19:16:05  <morcos> I'll be relatively quick for my part, I think I've got all the PR's out now that I think need to go in for 0.15, but I want to encourage people to think about a bunch of the RPC API changes so they are good in their first release
366 2017-06-29T19:16:24  <morcos> But the other thign is there is one piece of missing functionality wheich I think is needed
367 2017-06-29T19:16:26  *** goatpig has quit IRC
368 2017-06-29T19:16:36  <morcos> #10590
369 2017-06-29T19:16:36  <gribble> https://github.com/bitcoin/bitcoin/issues/10590 | Access to longer fee estimates using GUI · Issue #10590 · bitcoin/bitcoin · GitHub
370 2017-06-29T19:17:15  <morcos> Given how volatile fee estimates are and how much they change between short targets and long, I think it's important to give the GUI access to longer fee estimates
371 2017-06-29T19:17:31  <morcos> But someone more familar with QT can probably whip that up a lot quicker than me
372 2017-06-29T19:17:53  <morcos> Might be best to build it on top of all my other changes, #10707 shoudl have everything in one
373 2017-06-29T19:17:54  <gribble> https://github.com/bitcoin/bitcoin/issues/10707 | Better API for estimatesmartfee RPC by morcos · Pull Request #10707 · bitcoin/bitcoin · GitHub
374 2017-06-29T19:17:55  <jonasschnelli> I'll have a look.
375 2017-06-29T19:18:13  <morcos> Thanks!  That's what I was hoping for. :)  instagibbs might have more on this topic?
376 2017-06-29T19:18:21  <instagibbs> #10333 for fee bug fix :)
377 2017-06-29T19:18:22  <gribble> https://github.com/bitcoin/bitcoin/issues/10333 | [wallet] fee fixes: always create change, adjust value, and p… by instagibbs · Pull Request #10333 · bitcoin/bitcoin · GitHub
378 2017-06-29T19:18:36  <instagibbs> not much else, maybe I'll summon energy to review your PRs
379 2017-06-29T19:18:39  <morcos> is that the only thing you feel is critical for 0.15?
380 2017-06-29T19:18:46  <instagibbs> only realistic merge yeah
381 2017-06-29T19:18:51  <morcos> most of mine are now easy to review i think
382 2017-06-29T19:18:59  <gmaxwell> Do we have a feel for when bump will be generally usable enough to start making replacability a default? (or at least more visible?)
383 2017-06-29T19:19:03  <instagibbs> some of my other work has been sucked up by achow101 so waiting on 0.16 for that stuff
384 2017-06-29T19:19:54  <morcos> gmaxwell: at least it'll be easily accessible to choose RBF given the RPC and GUI options in 0.15
385 2017-06-29T19:20:09  <instagibbs> I'd really like it to be able to add additional inputs as needed
386 2017-06-29T19:20:13  <gmaxwell> Oh!  I often miss gui changes, I'll check that out.
387 2017-06-29T19:20:20  <jonasschnelli> Yes. Not sure if we persist the RBF state across restarts (in the GUI)
388 2017-06-29T19:20:21  <morcos> Might help us learn if there are other bump features needed...
389 2017-06-29T19:20:23  <instagibbs> but it seems to me the logic is much simpler with effective value....
390 2017-06-29T19:20:26  <jonasschnelli> Ideally we should
391 2017-06-29T19:20:28  <instagibbs> some disagree
392 2017-06-29T19:20:43  <gmaxwell> instagibbs: IIRC that was the big usability blocker for further use.
393 2017-06-29T19:21:05  <wumpus> gmaxwell: https://github.com/bitcoin/bitcoin/pull/9527#issuecomment-311659024
394 2017-06-29T19:21:09  <instagibbs> it randomly doesn't work which is disappointing UX
395 2017-06-29T19:21:20  <morcos> gmaxwell: the ability to addother inputs?  isn't it pretty rare to not have change?
396 2017-06-29T19:21:45  <jonasschnelli> But can happen...
397 2017-06-29T19:21:46  <wumpus> no, we don't persist RBF state, it has to be selected per transaction
398 2017-06-29T19:22:02  <jonasschnelli> wumpus: maybe the GUI should remember it
399 2017-06-29T19:22:03  <instagibbs> morcos, we are going to target more exact matches in future, fwiw
400 2017-06-29T19:22:04  <wumpus> the only way to make it persist is the command line option
401 2017-06-29T19:22:15  <morcos> wumpus: the gui initializes with the the command line argument, and then persists during the session
402 2017-06-29T19:22:15  <wumpus> jonasschnelli: meh, better to have it as "option" then
403 2017-06-29T19:22:24  <gmaxwell> FWIW, I believe electrum defaults to replacable now and pushes pretty hard in that direction, though users can flip it off on a per tx basis.
404 2017-06-29T19:22:29  <morcos> via checkbox
405 2017-06-29T19:22:47  <wumpus> jonasschnelli: persisting non-option settings between restarts would be unexpected
406 2017-06-29T19:23:03  <jonasschnelli> Yes. I guess your right..
407 2017-06-29T19:23:03  <gmaxwell> In any case, I think the default is kind of moot until bumping is sufficiently mature.
408 2017-06-29T19:23:21  <wumpus> between transactions in the same session makes sense I guess
409 2017-06-29T19:23:25  <morcos> I suppose I have one more question on that
410 2017-06-29T19:23:31  <jonasschnelli> Yes. If the bump won't work because it can't add another input the default should remain at the current state
411 2017-06-29T19:23:37  <wumpus> yes
412 2017-06-29T19:23:51  <jonasschnelli> It can happen quickly when fees are rising
413 2017-06-29T19:23:52  *** DrOlmer has joined #bitcoin-core-dev
414 2017-06-29T19:23:59  <achow101> hi. I'm late
415 2017-06-29T19:24:01  <morcos> Right now there are no options to the "Increase transaction fee" option in the GUI and it uses the default tx confirm target.  Should it instead use whatever the slider is set to?
416 2017-06-29T19:24:19  <jonasschnelli> Yes
417 2017-06-29T19:24:19  <morcos> If the slider is not in use and custom fee is set, shoudl it use that?
418 2017-06-29T19:24:23  <wumpus> morcos: the slider is on another tab
419 2017-06-29T19:24:24  <jonasschnelli> I'd like to work on the replacability in the GUI for 0.16
420 2017-06-29T19:24:31  <morcos> Those would be easy changes to make after my PR
421 2017-06-29T19:24:39  <BlueMatt> the slider is in another tab, thats strange
422 2017-06-29T19:24:47  <wumpus> morcos: not sure that would be intuitive, people assume the slider is for new transactions, the bump option should probably have its own choice dialog
423 2017-06-29T19:24:50  <jonasschnelli> First I though of bringing back to tx to the original send-tx screen (you could even add recipients... ) but meh
424 2017-06-29T19:25:01  <morcos> wumpus: that seems maybe too much optionality
425 2017-06-29T19:25:08  <jonasschnelli> The bump window should just be lager and has the slider
426 2017-06-29T19:25:16  <wumpus> jonasschnelli: yes
427 2017-06-29T19:25:27  <morcos> ok, thats fine.. so leave it as the wallet default confirm target for now?
428 2017-06-29T19:25:35  <wumpus> yes
429 2017-06-29T19:25:41  <BlueMatt> yea, sucks, but its easy and reasonable
430 2017-06-29T19:25:47  <jonasschnelli> And also we have never really discussed the pre-signed bumps.. but that we should probably do in another meeting
431 2017-06-29T19:25:57  <BlueMatt> yea, that sounds like a 16
432 2017-06-29T19:25:59  <instagibbs> jonasschnelli, that will involve new strategy
433 2017-06-29T19:26:00  <instagibbs> :)
434 2017-06-29T19:26:17  <instagibbs> reasonably diff from after the fact fix imo
435 2017-06-29T19:26:43  <jonasschnelli> I'd say focus on fee opt. in 0.15, rbf in 0.16
436 2017-06-29T19:27:15  <wumpus> agreed
437 2017-06-29T19:27:22  <wumpus> #topic the need for the watchonly rpc flag after multiwallet (sipa)
438 2017-06-29T19:27:27  <sipa> hi!
439 2017-06-29T19:27:38  <wumpus> (we need to move forward a bit, lots of topics)
440 2017-06-29T19:27:43  <sipa> currently many RPCs have an optional flag "include watchonly"
441 2017-06-29T19:27:44  <jonasschnelli> is that similar to the -disablehot?
442 2017-06-29T19:27:53  * jonasschnelli is listening
443 2017-06-29T19:28:14  <sipa> at the time the need for that flag existed because of a desire to keep your "hot" wallet separated from your "watch only" wallet
444 2017-06-29T19:28:20  <sipa> i think that was a mistake
445 2017-06-29T19:28:24  <wumpus> disablehot: #9662
446 2017-06-29T19:28:25  <gribble> https://github.com/bitcoin/bitcoin/issues/9662 | Add `-disablehot` mode: a sane mode for watchonly-wallets by jonasschnelli · Pull Request #9662 · bitcoin/bitcoin · GitHub
447 2017-06-29T19:28:41  <wumpus> sipa: yes, on the long term I agree with you
448 2017-06-29T19:28:50  <jonasschnelli> sipa: you think with multiwallet the wallet should either be watch or hot?
449 2017-06-29T19:28:55  <sipa> jonasschnelli: no
450 2017-06-29T19:29:03  <wumpus> sipa: makes more sense to have a wallet either full-watchonly or has-keys
451 2017-06-29T19:29:16  <sipa> wumpus: perhaps, but that's orthogonal
452 2017-06-29T19:29:22  <wumpus> sipa: I don't understand you then
453 2017-06-29T19:29:24  <instagibbs> ok get to the point :)
454 2017-06-29T19:29:25  <BlueMatt> why is that a mistake?
455 2017-06-29T19:29:32  <jonasschnelli> Let sipa explain...
456 2017-06-29T19:29:33  <sipa> what i'm trying to get at is that the within-a-wallet separation is no longer needed
457 2017-06-29T19:29:51  <wumpus> how is that different from what I said?
458 2017-06-29T19:30:11  <wumpus> instead of watchonly within a wallet you'd have a watchonly wallet and a normal wallet
459 2017-06-29T19:30:18  <sipa> i'm not arguing to remove the ability to have both keys and watchonly in one wallet
460 2017-06-29T19:30:18  <gmaxwell> because if you want to have a mixed thing thats fine too, then you just have a mixed thing. No need to flag, if you want seperation, use two wallets.
461 2017-06-29T19:30:19  <jonasschnelli> but I fail to see the difference then between only allowing watch-only or hot
462 2017-06-29T19:30:32  <sipa> just that there is no need to just select coins that affect one part
463 2017-06-29T19:30:38  <gmaxwell> you're suggesting an extra restriction.
464 2017-06-29T19:30:38  <sipa> or see a 'balance' of just one part
465 2017-06-29T19:30:51  <sipa> a wallet is a wallet, and has a single balance
466 2017-06-29T19:31:01  <sipa> some of the keys may require decrypting your wallet
467 2017-06-29T19:31:05  <wumpus> oh, right
468 2017-06-29T19:31:06  <sipa> some of the keys may require a hardware wallet
469 2017-06-29T19:31:18  <jonasschnelli> I see... yes.
470 2017-06-29T19:31:21  <sipa> some of the key may be just watchonly and you need to use raw transactions to interact with thing
471 2017-06-29T19:31:26  <BlueMatt> fair, this sounds like an 0.17 or 0.18 thing, though
472 2017-06-29T19:31:30  <gmaxwell> Now, logically you probably will seperate or something, for convience, but I don't see a particular reason to require that right now.
473 2017-06-29T19:31:32  <BlueMatt> are you asking if we should deprecate?
474 2017-06-29T19:31:33  <sipa> i was hoping 0.15
475 2017-06-29T19:31:33  <wumpus> BlueMatt: agree, long term
476 2017-06-29T19:31:47  <sipa> just make the watchonly flag ignored and always set it to true
477 2017-06-29T19:31:49  <wumpus> this is not something we're going to change in the RPC interface pre-0.15
478 2017-06-29T19:31:54  <sipa> ok
479 2017-06-29T19:31:55  <wumpus> peopel rely on this
480 2017-06-29T19:32:01  <wumpus> we could document it as deprecated
481 2017-06-29T19:32:03  <BlueMatt> we'd need to mark it deprecated
482 2017-06-29T19:32:04  <morcos> sipa: that seems reasonable except what about identifying which things you have keys for and which you dont..
483 2017-06-29T19:32:11  <BlueMatt> probably deprecate after we have working multiwallet that is stable
484 2017-06-29T19:32:19  <wumpus> then remove the flag for 0.16 or 0.17, but this seems over-hurried
485 2017-06-29T19:32:20  <BlueMatt> so maybe deprecate in 0.16...
486 2017-06-29T19:32:24  <morcos> that seems a useful distinction to keep to me
487 2017-06-29T19:32:25  <gmaxwell> with 0.15 and multiwallet we can start deprication at least-- e.g. advise that this will happen in the future, suggest people use seperate wallets. . The one problem with that however is that your seperate watchonly wallet still needs the stupid flag everywhere. :(
488 2017-06-29T19:32:26  <BlueMatt> remove in 17 or 18
489 2017-06-29T19:32:46  <wumpus> let's focus on actually getting multiwallet into 0.15
490 2017-06-29T19:32:48  <jonasschnelli> I somehow think mixed wallets can be a footgun source... but right, it orthogonal
491 2017-06-29T19:32:48  <instagibbs> related topic: some way to signal that the funds are "safe" when you expect a hardware wallet to have the privkey
492 2017-06-29T19:32:52  <instagibbs> post-0.15 ofc
493 2017-06-29T19:32:57  <sipa> maybe i haven't made this clear, but how do you deal with hardware wallets, for example?
494 2017-06-29T19:33:07  <jonasschnelli> we need a standard!
495 2017-06-29T19:33:08  <sipa> add a 2nd option everyone 'include hw wallet keys'
496 2017-06-29T19:33:09  <morcos> +1 better support for hardware wallets!
497 2017-06-29T19:33:14  <sipa> jonasschnelli: orthogonal
498 2017-06-29T19:33:18  <wumpus> hardware wallets in bitcoin core is a different topic
499 2017-06-29T19:33:23  <BlueMatt> we dont need to add a flag for hw wallets
500 2017-06-29T19:33:30  <sipa> BlueMatt: then why do we need a flag for watchonly?
501 2017-06-29T19:33:35  <wumpus> important, but certainly not one that's going to make it into 0.15
502 2017-06-29T19:33:38  <BlueMatt> we can say "hw wallets are always included in balance, flag for watchonly is deprecated" starting in the version that supports hw wallets
503 2017-06-29T19:33:40  <gmaxwell> sipa is pointing out that the model of 'watch only' when applied to also having hardware wallets starts adding combinitoric blowup.
504 2017-06-29T19:33:54  <sipa> BlueMatt: fair enough
505 2017-06-29T19:34:01  <jonasschnelli> If a wallet has no clear cur between hot and cold (watch-only), a code-level guarantee, I would not use it for hot funds...
506 2017-06-29T19:34:02  <BlueMatt> yes, agreed, we should not make it worse, but we dont need to worry about this until at least 16, I think
507 2017-06-29T19:34:06  <jonasschnelli> *cut
508 2017-06-29T19:34:14  <wumpus> agree on not making it worse
509 2017-06-29T19:34:15  <BlueMatt> need useable working good multiwallet first, which likely wont be 15
510 2017-06-29T19:34:18  <gmaxwell> BlueMatt: thats a point. now just give me a flag for importmulti that gives me a watching key imported that way and it's good to go. :P
511 2017-06-29T19:34:19  <sipa> jonasschnelli: again, orthogonal
512 2017-06-29T19:34:34  <instagibbs> I have a working Core+Ledger system, and have a couple thoughts, but this is a different topic yep
513 2017-06-29T19:34:39  <gmaxwell> BlueMatt: uhh, it's like done.
514 2017-06-29T19:34:50  <jonasschnelli> sipa: but why not just separating pure watch-only wallets from hot wallets? Why would that be orthogonal?
515 2017-06-29T19:35:03  <BlueMatt> gmaxwell: I know, but we need a cycle of finding more use-cases and making sure we've got it all covered, was my piont
516 2017-06-29T19:35:11  <wumpus> yes multiwallet is almost done, but in 0.15 it will at least be experimental
517 2017-06-29T19:35:15  <BlueMatt> eg createwallet flows within rpc, disconectwallet, etc
518 2017-06-29T19:35:16  <sipa> jonasschnelli: "orthogonal" means you can still do that
519 2017-06-29T19:35:23  <sipa> jonasschnelli: it has nothing to do with this issue
520 2017-06-29T19:35:24  <wumpus> it's the first release it is in, after all
521 2017-06-29T19:35:27  <gmaxwell> jonasschnelli: because that is an additional restriction that AFAIK isn't needed. maybe later its needed to not support mixed but it seems like a seperate issue to me.
522 2017-06-29T19:35:46  <jonasschnelli> Okay
523 2017-06-29T19:36:01  <BlueMatt> ok, so we all agree, eventually push people towards multiwallet away from watchonly :)
524 2017-06-29T19:36:05  <BlueMatt> next topic? :p
525 2017-06-29T19:36:07  <sipa> what i want to get add is that a wallet is just a collection of keys it considers "mine" - independent of its ability to actually fully sign
526 2017-06-29T19:36:12  <sipa> BlueMatt: yes, agree
527 2017-06-29T19:36:19  <wumpus> #topic rolling utxo hashes
528 2017-06-29T19:36:22  <wumpus> (sipa again)
529 2017-06-29T19:36:22  <sipa> hi!
530 2017-06-29T19:36:30  <instagibbs> sipa, ISMINE_* tho :)
531 2017-06-29T19:36:34  <instagibbs> ok next topic
532 2017-06-29T19:36:54  <sipa> with pertxout we changed the serialized_hash because the new format no longer maintains the tx version of the utxo
533 2017-06-29T19:37:11  <sipa> i posted about rolling utxo hashes a while ago on the ML
534 2017-06-29T19:37:35  <sipa> i'm not proposing actually implementing that, but would it be worthwhile to immediately switch to a scheme that is compatible with it?
535 2017-06-29T19:37:43  <sipa> so that there is no need to break the API again
536 2017-06-29T19:38:01  <gmaxwell> sipa: as in don't do the rolling thing, but have the oneshot thing compute the same hash?
537 2017-06-29T19:38:06  <sipa> yes
538 2017-06-29T19:38:16  <sipa> downside: makes gettxoutsetinfo slower
539 2017-06-29T19:38:30  <wumpus> how much slower?
540 2017-06-29T19:38:32  <sipa> upside: allows us to make gettxoutsetinfo super fast in the future
541 2017-06-29T19:38:42  <gmaxwell> lots slower.
542 2017-06-29T19:38:44  <sipa> several times
543 2017-06-29T19:38:56  <wumpus> could add a new RPC for it
544 2017-06-29T19:39:02  <gmaxwell> sipa: Well a challenge there is that I'm not sure that we've settled on the field. So that isn't a guarentee of compatiblity.
545 2017-06-29T19:39:03  <wumpus> instead of gettxoutsetinfo
546 2017-06-29T19:39:27  <sipa> interesting, i hadn't considered that
547 2017-06-29T19:39:30  <sipa> gmaxwell: yeah, i know
548 2017-06-29T19:39:47  <gmaxwell> actually if we drop the hash from gettxoutsetinfo i think thats the only thing now that requires scanning the whole thing.
549 2017-06-29T19:39:55  <sipa> no, everything does
550 2017-06-29T19:40:01  <sipa> (txout count etc)
551 2017-06-29T19:40:03  <wumpus> yes it's all aggregate statistics
552 2017-06-29T19:40:06  <gmaxwell> yes but it wouldn't have to with rather trivial changes.
553 2017-06-29T19:40:08  <sipa> though those things can be maintained on the fly
554 2017-06-29T19:40:20  <gmaxwell> which would be robust and wouldn't change.
555 2017-06-29T19:40:27  <sdaftuar> i think we will want an RPC that can scan the disk to calculate the answer, even if we are able to calculate everything on the fly
556 2017-06-29T19:40:34  <sdaftuar> so that we know our on-disk data is correct
557 2017-06-29T19:40:35  <sipa> sdaftuar: good point
558 2017-06-29T19:40:39  <gmaxwell> sdaftuar: restart your node. :P
559 2017-06-29T19:41:02  <sipa> an advantage of the fast hash is that you can compare it with a recompute-the-whole-thing
560 2017-06-29T19:41:17  <wumpus> that'd be very nice
561 2017-06-29T19:41:17  <gmaxwell> okay interesting points.
562 2017-06-29T19:41:45  <wumpus> a utxo hash that would be quick to compute for every block would be very nice to have
563 2017-06-29T19:42:20  <gmaxwell> (I was momentarily overestimating how easy it would be to switch to summary statistics, I forgot that they have to be saved and loaded across restart... or otherwise every startup needs the equal of a stats call)
564 2017-06-29T19:43:03  <gmaxwell> wumpus: right thats the goal of pieter's work. It's just a bit immature now, and if we implmenet it at the moment we may want to switch to an incompatible version later.
565 2017-06-29T19:43:39  <wumpus> I like to check that all my nodes have the same utxo hash, but calling getutxosetinfo for every block takes too much time, I've tried and given up :)
566 2017-06-29T19:43:52  <gmaxwell> Assuming we stay with the multiplicative group hash,  we need to pick a prime where multiplication mod that prime is as fast as possible.  Sipa has done some work there, but it's a research project that can sink as much time as we want to put into it.
567 2017-06-29T19:44:32  <sipa> or we could just use the elliptic curve version, which can probably be made ~2 slower than the GMP-based MuHash
568 2017-06-29T19:44:38  <sipa> which is just a few lines of code
569 2017-06-29T19:44:57  <wumpus> now doing it intermittently, but that means that when it fails we don't know exacly where it started to diverge
570 2017-06-29T19:45:17  <gmaxwell> right, I want to have UpdateTip log the value.
571 2017-06-29T19:45:23  <sipa> ^ that
572 2017-06-29T19:45:31  <sdaftuar> wumpus: it's actually not clear to me how much the fast utxo hash calculation helps in comparing running nodes
573 2017-06-29T19:46:11  <sipa> well the fast utxo hash lets you do a consistency check on just a single node
574 2017-06-29T19:46:31  <sdaftuar> but what is exactly being compared as consistent?
575 2017-06-29T19:46:32  <sipa> by having a fast incrementally-updated version, and a slow recompute-from-scratch one
576 2017-06-29T19:46:33  <gmaxwell> sdaftuar: because you can log the utxo hash at each point, and so if they diverge in a way that the hash sees (e.g. not underlying disk corruption) you'll learn.   Also you could run a command that checks the disk against the running value to catch that disk corrouption.
577 2017-06-29T19:46:56  <gmaxwell> so  your disk <> your running <> my running <> my disk
578 2017-06-29T19:47:13  <sdaftuar> yes, i agree if you do the comparison with disk, then you get something valuable
579 2017-06-29T19:47:25  <sdaftuar> but just comparing the fast calculation between nodes doesn't seem like it does much, does it?
580 2017-06-29T19:47:28  <wumpus> hm yes good point
581 2017-06-29T19:47:31  <gmaxwell> right now it is a PITA to compare you and I at disk because we have to do it at the same time (and hope there isn't a block at that instant. :P )
582 2017-06-29T19:47:40  <sdaftuar> gmaxwell: agreed
583 2017-06-29T19:47:47  <gmaxwell> sdaftuar: it depends on where the errors you're concerned about are happening.
584 2017-06-29T19:48:23  <wumpus> gmaxwell: yes, even when you time the RPC command on blocknotify, it sometimes misses the block :)
585 2017-06-29T19:48:31  <gmaxwell> if they're below the layer where the running hash runs you only gain if you also do periodic checks between it and the disk.  Above it, however, you have constant checking.
586 2017-06-29T19:49:08  <gmaxwell> but the nice thing is that disk and running can be async checked... You and I don't need to do our disk comparisons at the same time.
587 2017-06-29T19:49:36  <wumpus> indeed
588 2017-06-29T19:49:49  <gmaxwell> sdaftuar: this is all also machinery we almost certantly need for a reasonable UTXO-assume-valid kind of sync in the future.
589 2017-06-29T19:49:55  <wumpus> all in all a rolling utxo hash is an improvement, it creates more options, but you can still do the same as now if you want
590 2017-06-29T19:50:03  <sdaftuar> gmaxwell: yeah i agree and that's the use case i'm most excited about :)
591 2017-06-29T19:50:19  <sdaftuar> i was just trying to figure out exactly how i'd use to compare my own nodes, and wasn't sure of the utility
592 2017-06-29T19:50:44  <gmaxwell> wumpus: the challange though is that it isn't free. muhash on the whole utxo set takes CPU minutes.
593 2017-06-29T19:51:02  <wumpus> gmaxwell: yes I'm not sure it should replace the faster hash
594 2017-06-29T19:51:15  <wumpus> maybe it should justb e an additional thing
595 2017-06-29T19:51:23  <gmaxwell> well once it's a running hash its very fast. :P
596 2017-06-29T19:51:41  <sdaftuar> hash_serialized_3? :P
597 2017-06-29T19:51:41  <wumpus> OTOH we're already breaking the hash for 0.15
598 2017-06-29T19:51:56  <wumpus> (which is kind of sad, as it makes it impossible to compare against older versions)
599 2017-06-29T19:52:32  <gmaxwell> sipa backported the new hash to the old system for development testing, FWIW.
600 2017-06-29T19:52:54  <gmaxwell> (it's a pretty trivial change, IIRC, just drop the version from it)
601 2017-06-29T19:53:03  <wumpus> cool, that'd be useful, especially with the 0.14 to 0.15 database change it's important to be able to check synchronization
602 2017-06-29T19:54:05  <gmaxwell> This patch existed at one point already, dunno if sipa still has it.
603 2017-06-29T19:54:07  <sipa> the problem is that #10434 is quite a bit of intricate code
604 2017-06-29T19:54:09  <gribble> https://github.com/bitcoin/bitcoin/issues/10434 | [WIP] 3072-bit MuHash based hash_serialized by sipa · Pull Request #10434 · bitcoin/bitcoin · GitHub
605 2017-06-29T19:54:43  <sipa> the EC version would be many times less code (given that we already have secp256k1), but be a few times slower
606 2017-06-29T19:55:36  <wumpus> I don't have a strong opinion on it
607 2017-06-29T19:55:37  <sipa> on the other hand, MuHash is very simple to implement in anything that already has big integers (it's a few lines in python)
608 2017-06-29T19:55:58  <sipa> ok
609 2017-06-29T19:56:00  *** tmddzk has joined #bitcoin-core-dev
610 2017-06-29T19:56:03  *** justanotheruser has quit IRC
611 2017-06-29T19:56:06  <wumpus> though in general I'd say higher performance seems preferable to the ability to re-use code
612 2017-06-29T19:56:23  <sipa> in that case, some review would be welcome :)
613 2017-06-29T19:56:32  <wumpus> but I haven't seen the code
614 2017-06-29T19:56:42  <wumpus> yeah, hope to get around to it
615 2017-06-29T19:56:59  <sipa> i can drop the asm optimized version from the first PR if wanted
616 2017-06-29T19:57:06  <praxeology> Couldn't you put a delay on insert/remove from the rolling hash... say only for utxos that are 1 day of blocks old? isn't a hash for N blocks ago just about as good as the current hash?
617 2017-06-29T19:57:18  <sipa> praxeology: totally irrelevant
618 2017-06-29T19:57:43  <sipa> that would mean you need to keep those utxos around for processing later
619 2017-06-29T19:57:59  <sipa> we have an approach that can combine them into a running hash in _microseconds_
620 2017-06-29T19:58:06  <gmaxwell> All doing that does is perhaps saves you 1% of computation for blocks that are reorged out but at the expense of complexifying everything because the data is inconsistently available.
621 2017-06-29T19:58:16  *** justanotheruser has joined #bitcoin-core-dev
622 2017-06-29T19:58:29  <praxeology> What percent of utxos are spent within a day?
623 2017-06-29T19:58:37  <instagibbs> 2 minutes left
624 2017-06-29T19:58:41  <instagibbs> if anyone has microtopic
625 2017-06-29T19:58:43  <wumpus> that seems irrelevant to this discussion
626 2017-06-29T19:58:54  <wumpus> (although it's interesting to know in its own right)
627 2017-06-29T19:59:07  <praxeology> Sounds like you guys are concerned about performance on the rolling hash
628 2017-06-29T19:59:24  <wumpus> #endmeeting
629 2017-06-29T19:59:24  <lightningbot> Meeting ended Thu Jun 29 19:59:24 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
630 2017-06-29T19:59:24  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-29-19.00.html
631 2017-06-29T19:59:24  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-29-19.00.txt
632 2017-06-29T19:59:24  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-29-19.00.log.html
633 2017-06-29T19:59:31  <gmaxwell> praxeology: delaying it doesn't change that.
634 2017-06-29T19:59:32  <instagibbs> sipa, you still around?
635 2017-06-29T19:59:36  <sipa> instagibbs: sure
636 2017-06-29T19:59:45  <instagibbs> ok let me toss a wallet q to you
637 2017-06-29T19:59:54  <gmaxwell> praxeology: its the same amount of computation if you do it now or 100 blocks ago.
638 2017-06-29T20:00:00  <midnightmagic> Man I wish people would end meetings elsewhere with that kind of precision..
639 2017-06-29T20:00:04  <wumpus> yes it'd just move the computation in time
640 2017-06-29T20:00:04  * midnightmagic shudders
641 2017-06-29T20:00:22  <wumpus> midnightmagic: I guess they should hire me as meeting chair :p
642 2017-06-29T20:00:27  <praxeology> gmaxwell:  delaying will reduce the number of things that are xored (err whatever math op you doing), since utxos that were spent before the delay window are never added
643 2017-06-29T20:00:29  <gmaxwell> praxeology: if you imagine something which is never consistent with any actual state of the system, then that really isn't all that useful.
644 2017-06-29T20:00:36  <BlueMatt> I was gonna say something about the swiss leading the meeting, but its wumpus, not jonasschnelli
645 2017-06-29T20:00:42  <instagibbs> sipa, so for hw wallets, would it be reasonable to have a new ISMINE type that means the wallet expects the output to be spendable by the wallet even though privkeys aren't physically present in the wallet.dat
646 2017-06-29T20:00:58  <sipa> instagibbs: in my opinion, ISMINE should become a bool
647 2017-06-29T20:01:03  <sipa> it's yours or not
648 2017-06-29T20:01:07  <gmaxwell> praxeology: that isn't delayed.
649 2017-06-29T20:01:08  <instagibbs> ah ok, so no
650 2017-06-29T20:01:14  <midnightmagic> wumpus: :-)  Your nickname shall be The Guillotine
651 2017-06-29T20:01:19  <instagibbs> well in that case at least some of the logic could be moved elsewhere
652 2017-06-29T20:01:22  <sipa> instagibbs: the ability to spend should be independent from what is considered yours
653 2017-06-29T20:01:30  <sipa> instagibbs: (note, that's my personal opinion)
654 2017-06-29T20:01:43  <gmaxwell> praxeology: I believe what you are suggesting doing is computing it sparsely so that there isn't a value computed at every block.
655 2017-06-29T20:01:43  <wumpus> midnightmagic: :-)
656 2017-06-29T20:01:45  <instagibbs> no it's fair, I'm just trying to get my wallet to understand what I consider mine
657 2017-06-29T20:01:49  <instagibbs> and right now it's janky mess
658 2017-06-29T20:02:05  <sipa> instagibbs: perhaps the solvable distinction is still useful
659 2017-06-29T20:02:30  <instagibbs> so a new "consider yours" function would fix my current issue in a cleaner way
660 2017-06-29T20:02:49  <sipa> cfields: still around?
661 2017-06-29T20:02:52  <achow101> instagibbs: isn't there some combination of ISMINE_ types that indicate "no privkey in wallet but I can still spend it"
662 2017-06-29T20:02:53  <gmaxwell> praxeology: this would reduce computation somewhat, but at the expense creating coordination points... and then you only perhaps get a 2x speedup, but you also add bursts of letency rather than being able to compute it continually.
663 2017-06-29T20:02:57  <cfields> sipa: yes
664 2017-06-29T20:03:00  <praxeology> gmaxwell: I'm suggesting only adding a utxo to the hash on the 144th confirmation
665 2017-06-29T20:03:27  <instagibbs> achow101, there is solvable+watchonly
666 2017-06-29T20:03:31  <instagibbs> but that doesn't mean you can spend it
667 2017-06-29T20:03:37  <gmaxwell> praxeology: that would make something which is _never_ consistent with the utxo set at any point, I think this would be completely useless to us.
668 2017-06-29T20:03:49  <instagibbs> so if you have a hw wallet, you expect to be able to sign for it, but there's no enum for it
669 2017-06-29T20:03:54  <gmaxwell> praxeology: it couldn't be used to check database consistency, it couldn't be used to perform a sync from utxo.
670 2017-06-29T20:04:07  <sipa> praxeology: that sort of approach may be useful for UTXO/TXO commitment like approaches, where updating the commitment is very expensive and cheaper when batched
671 2017-06-29T20:04:08  <instagibbs> so in corner cases you consider that output untrusted, and get "insufficient amount"
672 2017-06-29T20:04:22  <sipa> praxeology: but the rolling UTXO idea was specifically intended to not need that, because it's so fast
673 2017-06-29T20:04:30  <praxeology> gmaxwell: if you have the last 144 blocks then you can do the remainder of the utxos from those blocks to finish the hash at a particular point
674 2017-06-29T20:04:33  <achow101> instagibbs: oh
675 2017-06-29T20:04:47  <bitcoin-git> [bitcoin] narula opened pull request #10708: Connecttrace fewer blocks (master...connecttrace-fewer-blocks) https://github.com/bitcoin/bitcoin/pull/10708
676 2017-06-29T20:05:01  <sipa> praxeology: just looking up a utxo spent from disk is more work than updating the hash
677 2017-06-29T20:05:08  *** Dyaheon has quit IRC
678 2017-06-29T20:05:17  <cfields> neha__: eh? :)
679 2017-06-29T20:05:20  <instagibbs> achow101, I coded a fix for this corner issue, but only for p2sh multisig... current methodology kind of forces me to do janky fix or make another ismine enum :/
680 2017-06-29T20:06:12  <praxeology> sipa: yes, well not sure the use case of when such would be needed
681 2017-06-29T20:06:15  <achow101> just make ismine a bool :p
682 2017-06-29T20:07:07  *** Dyaheon has joined #bitcoin-core-dev
683 2017-06-29T20:08:17  <praxeology> earlier you guys were talking about a tx just having one signature, even for multiple things that need to be signed.  You talked about computation performance.  What about impact on tx size?
684 2017-06-29T20:08:45  <praxeology> particularly since... i hear that network bandwidth is the main bottlneck
685 2017-06-29T20:09:23  <instagibbs> N signatures in 1 signature space possible, across inputs. Still need the pubkeys
686 2017-06-29T20:10:52  *** neha__ is now known as neha
687 2017-06-29T20:11:19  <praxeology> sure, still need public keys.  but what about the signature size?
688 2017-06-29T20:11:42  <neha> cfields: BlueMatt gave me an issue to fix!
689 2017-06-29T20:11:46  <sipa> 64 bytes instead of N*72
690 2017-06-29T20:11:48  <sipa> praxeology: ^
691 2017-06-29T20:11:53  <gmaxwell> praxeology: instagibbs said: one signature.
692 2017-06-29T20:12:28  <sipa> praxeology: and bandwidth isn't all that much of a factor anymore single compact blocks etc
693 2017-06-29T20:12:31  <cfields> neha: good to see! that's a rabbit hole. I'd be nervous if you didn't find more things to fix while you're down there :)
694 2017-06-29T20:12:40  <instagibbs> achow101, sounds simple, but let's see all the interaction with current wallet stuff
695 2017-06-29T20:12:47  <gmaxwell> (the signatures in bitcoin today are 72 instead of 64.125 just because they use a dumb encoding.
696 2017-06-29T20:13:05  <gmaxwell> )
697 2017-06-29T20:13:44  <praxeology> sipa: do you have a layman link for "single compact block"? or anyone?
698 2017-06-29T20:13:52  <sipa> praxeology: bip 152
699 2017-06-29T20:14:12  <sipa> i'm sure there are better explanation online than the bip, though
700 2017-06-29T20:14:21  <gmaxwell> I dunno the bit is pretty good.
701 2017-06-29T20:14:27  <gmaxwell> bip
702 2017-06-29T20:14:38  <instagibbs> gmaxwell, speaking of which how is 0.5RTT going these days, any change?
703 2017-06-29T20:14:44  *** justanotheruser has quit IRC
704 2017-06-29T20:14:53  <sipa> cfields: so the block-wide aggregation that adiabat proposed a while ago on the ML still applies to Bellare-Neven... allowing to have only 32 bytes of the signature in every tx, and another 32 byte block wide
705 2017-06-29T20:14:54  <instagibbs> (if you're monitoring)
706 2017-06-29T20:14:57  *** AaronvanW has quit IRC
707 2017-06-29T20:15:06  <sipa> cfields: the downside is that it doesn't play nicely with cached signature validation
708 2017-06-29T20:15:12  *** justanotheruser has joined #bitcoin-core-dev
709 2017-06-29T20:15:18  <gmaxwell> instagibbs: meh, we need the skip-recent-txn things in mining.  It's gone up and down (in particular utility of the extrapool has gone up and down)
710 2017-06-29T20:15:23  <sipa> cfields: as wtxids would change after inclusion in a block
711 2017-06-29T20:15:34  *** AaronvanW has joined #bitcoin-core-dev
712 2017-06-29T20:15:44  <gmaxwell> instagibbs: during periods of long backlogs the extrapool is too small-- I see misses that my node had seen before.
713 2017-06-29T20:15:49  *** RoyceX has joined #bitcoin-core-dev
714 2017-06-29T20:16:11  *** cheese_ has quit IRC
715 2017-06-29T20:16:33  <cfields> sipa: hmm. Does parallel validation still apply as well?
716 2017-06-29T20:16:39  <praxeology> sipa: oh, that is just where txs are not re-relayed with blocks.  Something  like a 1/2 bandwidth used improvement.
717 2017-06-29T20:16:51  <cfields> *the parallel validation improvements
718 2017-06-29T20:17:04  <praxeology> or is there something else I missed when skimming? something on the order of mimblewimble improvement?
719 2017-06-29T20:17:18  <sipa> praxeology: yes, but the bandwidth needed to propagate a block quickly is massively reduced
720 2017-06-29T20:17:27  <sipa> praxeology: overall data volume is reduced by 2
721 2017-06-29T20:17:42  <gmaxwell> praxeology: typically at the tip blocks are transmitted with about 16kbytes.
722 2017-06-29T20:18:08  <sipa> cfields: yes
723 2017-06-29T20:18:29  <sipa> cfields: you can just shard and do the computation for a number of groups
724 2017-06-29T20:18:44  <sipa> and then do a simple cheap combine operation
725 2017-06-29T20:19:08  <gmaxwell> you lose some of the asymtoptic gains though we've been expirementing with parallel versions of the aggregate validation operation.
726 2017-06-29T20:20:30  <gmaxwell> instagibbs: in the last 144 blocks I see 23% requiring a round trip. 13% were saved from needing one by the extra pool.  A week ago the extrapool saves were about half that I think.
727 2017-06-29T20:21:21  *** owowo has quit IRC
728 2017-06-29T20:22:43  <cfields> sipa: if we cached as much as possible otherwise (hashing mainly, i suppose) and completely dropped the pre-validated cache, do you have a sense of how it'd compare? I realize it'd be worth keeping the cache as it'd still have a good hit rate, i'm just curious.
729 2017-06-29T20:23:17  <sipa> cfields: i don't understand
730 2017-06-29T20:23:19  <gmaxwell> cfields: I'm unclear about what you're asking.
731 2017-06-29T20:24:18  <cfields> trying to weight the benefits of parallel validation against losing some pre-cached hits
732 2017-06-29T20:24:26  <sipa> well, do both
733 2017-06-29T20:24:45  <gmaxwell> yea, there isn't any conflict. You parallel validate the things that miss the cache.
734 2017-06-29T20:24:57  <sipa> oh, you mean with the block-wide aggregation
735 2017-06-29T20:24:59  <gmaxwell> do you mean batch validation instead of parallel btw?
736 2017-06-29T20:25:06  <sipa> my concern there is mostly the layering violation
737 2017-06-29T20:25:13  <gmaxwell> the block wide aggregation stuff is ugh
738 2017-06-29T20:25:27  <cfields> yes, sorry. i meant batch.
739 2017-06-29T20:26:22  <BlueMatt> instagibbs: I think we'd actually see a bigger improvement by responding to getblocktxn requests in the background while connecting a block than making 0.5rtt more common
740 2017-06-29T20:26:29  <BlueMatt> network-wide that is
741 2017-06-29T20:26:39  <BlueMatt> though 0.15 may speed up block connection enough.....anyway
742 2017-06-29T20:26:52  *** owowo has joined #bitcoin-core-dev
743 2017-06-29T20:28:00  <instagibbs> BlueMatt, i was hoping for lazy improvements, like "it got better" :P
744 2017-06-29T20:28:09  <instagibbs> yeah I reviewed a PR for that a long while ago
745 2017-06-29T20:29:05  <sipa> cfields: so for batch validation... batch validate the txn in a block you haven't seen yet, and ignore the rest
746 2017-06-29T20:29:15  <sipa> (assuming there is no block-wide anything going on)
747 2017-06-29T20:29:38  <gmaxwell> BlueMatt: if you want to make that even faster: create a ReadBlockFromDisk that returns a blockblob (don't deseralize the transactions in it).
748 2017-06-29T20:29:56  <gmaxwell> and use that for all getblocky kinds of requests.
749 2017-06-29T20:30:39  <gmaxwell> (the cases not covered by the cached getblocktxn are whole block requests...)
750 2017-06-29T20:30:51  <gmaxwell> (and we waste time deseralizing then reserializing the blocks...)
751 2017-06-29T20:30:59  <cfields> sipa: ah, makes sense
752 2017-06-29T20:31:37  <gmaxwell> cfields: there is a bit of a conflict right now because batch and parallel are in competition with each other... bigger batches get more speedup, but you want to use all your cores....
753 2017-06-29T20:34:47  <cfields> is the batch operation itself not parallelizable?
754 2017-06-29T20:34:55  <BlueMatt> gmaxwell: I'm less concerned about that w/ parallell ProcessMessages - if you ask for an old block its gonna be slow (though I agree that we should fix that, just less interesting for the latency improvements)
755 2017-06-29T20:35:03  <BlueMatt> instagibbs: well that one got dropped in favor of "doing it cleanly"
756 2017-06-29T20:35:15  <cfields> i suspect I'm misunderstanding the conflict
757 2017-06-29T20:35:28  <BlueMatt> instagibbs: now its 10652 + the two other PRs that make up that branch, then an actual parallellization PR
758 2017-06-29T20:35:33  <BlueMatt> so....16? maybe 17
759 2017-06-29T20:35:39  <instagibbs> BlueMatt, ah k
760 2017-06-29T20:35:53  <instagibbs> clearly behind the times
761 2017-06-29T20:38:20  <sipa> cfields: a batch of 4000 signatures takes less than 8 times as much CPU as a batch of 500 signatures
762 2017-06-29T20:38:46  <sipa> cfields: if you split the batch up in 8, and run those 8 on separate CPUs, you're going to do 8*batch(500) work, not 1*batch(4000) work
763 2017-06-29T20:39:26  <gmaxwell> cfields: the algorithim is not naturally parallizable, though with lots of synchronization traffic it can be made parallel.
764 2017-06-29T20:40:00  <gmaxwell> how much of the batch(4000) speed we can get out of something pushed to be made N-way parallel is an open question.
765 2017-06-29T20:40:12  <gmaxwell> If synchronization between threads is free the answer is "almost all of it"
766 2017-06-29T20:40:23  <cfields> ok that's what i was missing, thanks
767 2017-06-29T20:40:27  <gmaxwell> If it is very expensive, the answer appears to be "almost none of it".
768 2017-06-29T20:40:31  *** Yogaqueef has quit IRC
769 2017-06-29T20:40:56  <cfields> i'll read the paper before discussing further
770 2017-06-29T20:41:01  *** Murch has quit IRC
771 2017-06-29T20:41:07  <gmaxwell> None of this is in the paper.
772 2017-06-29T20:41:41  <cfields> oh. in that case, I already read the paper :p
773 2017-06-29T20:41:47  <sipa> cfields: the 'hard' part of the computation is doing a huge n1*P1 + n2*P2 + n3*P3 + ... (where the n's are integers and the Ps are EC points)
774 2017-06-29T20:42:21  <sipa> turns out, there is a very neat algorithm (multiple, in fact) that do this whole computation many times faster than just multiplying individually and adding
775 2017-06-29T20:42:57  <gmaxwell> Same as for a normal signature validation except there it's  just n1 * P + n2 * G  ... so only two ecpoints.  in batch and aggregate validation there is  pubkeys + 1.
776 2017-06-29T20:46:36  <gmaxwell> done simply, a n1*P requires 256 EC additions (technically 256 doublings and 128 additions, but doublings are about twice as fast as an addition)-- using grade school long multiplication (in base 2).  The batch computation of n1*P1 + n2*P2 + n3*P3 + ... can do the job in about 26 additions per point for 4096 inputs.
777 2017-06-29T20:46:46  *** Murch has joined #bitcoin-core-dev
778 2017-06-29T20:47:10  <cfields> whoa
779 2017-06-29T20:47:28  <cfields> oh, misread :)
780 2017-06-29T20:47:56  <gmaxwell> yes per point, but still thats almost a 10x speedup over a dumb algorithim.
781 2017-06-29T20:50:24  <cfields> are there further speedups if the result is known ahead of time and you're just attempting to verify correctness?
782 2017-06-29T20:50:56  <gmaxwell> What we do in secp256k1 for validation (which is two points) is far from simplisic and takes much less than 256 adds worth of work per each.  I believe it's about equal to about 84 per point.
783 2017-06-29T20:51:19  <gmaxwell> cfields: the aggregate and batch both count on a property like that.
784 2017-06-29T20:51:40  <gmaxwell> the R value in the signature is the result of this calculation.
785 2017-06-29T20:52:08  <cfields> ah, so that's the 32bytes-per-block
786 2017-06-29T20:54:01  <sipa> cfields: eh, i think you're confused
787 2017-06-29T20:54:07  <sipa> cfields: perhaps we should move to #secp256k1
788 2017-06-29T20:54:22  <cfields> sure
789 2017-06-29T20:55:14  <gmaxwell> To be clear: Batch validation exploits ' the result is known ahead of time and you're just attempting to verify correctness'   --- because the signature (or each signature) has an R value that comes with it, and the signature validation is trying to verify that an R value it computes is the same as the provided one.
790 2017-06-29T20:56:22  <gmaxwell> You can also encode signatures another way, like the wikipedia article on schnorr signatures does-- using "e,s" which is a hash and a scalar; and this form cannot be batch verified because you don't know the result of that multiexp equation in advance.
791 2017-06-29T20:58:36  *** DrOlmer has quit IRC
792 2017-06-29T21:07:52  *** awsom82 has joined #bitcoin-core-dev
793 2017-06-29T21:36:58  *** Chris_Stewart_5 has quit IRC
794 2017-06-29T21:39:41  *** laurentmt has joined #bitcoin-core-dev
795 2017-06-29T21:41:54  *** laurentmt has quit IRC
796 2017-06-29T21:53:36  <cfields> BlueMatt: ok, i have the shared_ptr change hacked in, and it's pretty huge. Lots of stuff has to change at the same time... it's kinda hard to avoid a giant commit.
797 2017-06-29T21:54:15  <cfields> BlueMatt: i have no problem doing your refcount change first, then undoing it with this big change next if you'd prefer.
798 2017-06-29T22:07:39  *** justan0theruser has joined #bitcoin-core-dev
799 2017-06-29T22:10:40  *** justanotheruser has quit IRC
800 2017-06-29T22:15:27  *** vicenteH has quit IRC
801 2017-06-29T22:22:40  *** AaronvanW has quit IRC
802 2017-06-29T22:27:16  *** AaronvanW has joined #bitcoin-core-dev
803 2017-06-29T22:41:33  *** justan0theruser has quit IRC
804 2017-06-29T22:41:51  *** justanotheruser has joined #bitcoin-core-dev
805 2017-06-29T22:42:38  *** Murch has quit IRC
806 2017-06-29T22:44:11  *** Murch has joined #bitcoin-core-dev
807 2017-06-29T22:50:21  *** nemgun has quit IRC
808 2017-06-29T23:01:03  *** awsom82 has left #bitcoin-core-dev
809 2017-06-29T23:22:17  *** jimhash has joined #bitcoin-core-dev
810 2017-06-29T23:23:27  *** rabidus has quit IRC
811 2017-06-29T23:25:23  *** rabidus has joined #bitcoin-core-dev
812 2017-06-29T23:28:29  *** Dizzle has quit IRC
813 2017-06-29T23:41:37  *** jimhash has left #bitcoin-core-dev
814 2017-06-29T23:42:35  *** Giszmo has quit IRC
815 2017-06-29T23:57:33  *** Giszmo has joined #bitcoin-core-dev