1 2018-08-16T00:14:49  *** AaronvanW has quit IRC
  2 2018-08-16T00:19:14  *** masonicboom has quit IRC
  3 2018-08-16T00:20:07  *** masonicboom has joined #bitcoin-core-dev
  4 2018-08-16T00:22:34  *** grubles has quit IRC
  5 2018-08-16T00:31:38  *** grubles has joined #bitcoin-core-dev
  6 2018-08-16T00:42:34  *** jhfrontz has joined #bitcoin-core-dev
  7 2018-08-16T00:53:45  *** grubles has quit IRC
  8 2018-08-16T00:54:01  *** grubles has joined #bitcoin-core-dev
  9 2018-08-16T00:55:38  *** roasbeef has quit IRC
 10 2018-08-16T00:57:18  *** masonicboom has quit IRC
 11 2018-08-16T01:06:01  *** d9b4bef9 has quit IRC
 12 2018-08-16T01:07:07  *** d9b4bef9 has joined #bitcoin-core-dev
 13 2018-08-16T01:09:46  *** Chris_Stewart_5 has quit IRC
 14 2018-08-16T01:10:28  <gmaxwell> jonasschnelli: maybe when you post a patch for the encryption, I'll go and quickly staple newhope to it and we see how we feel about the combination.
 15 2018-08-16T01:22:26  *** roasbeef has joined #bitcoin-core-dev
 16 2018-08-16T01:26:21  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 17 2018-08-16T01:59:55  *** thaumavorio has quit IRC
 18 2018-08-16T02:02:09  *** goatpig has quit IRC
 19 2018-08-16T02:07:54  *** Giszmo has quit IRC
 20 2018-08-16T02:32:58  *** Chris_Stewart_5 has quit IRC
 21 2018-08-16T02:46:54  *** reallll has joined #bitcoin-core-dev
 22 2018-08-16T02:51:02  *** belcher_ has quit IRC
 23 2018-08-16T02:58:05  *** reallll has quit IRC
 24 2018-08-16T02:58:32  *** reallll has joined #bitcoin-core-dev
 25 2018-08-16T03:02:26  *** rls has joined #bitcoin-core-dev
 26 2018-08-16T03:27:41  *** justan0theruser is now known as justanotheruser
 27 2018-08-16T03:32:45  *** Randolf has joined #bitcoin-core-dev
 28 2018-08-16T04:04:19  *** Krellan has quit IRC
 29 2018-08-16T04:05:27  *** Krellan has joined #bitcoin-core-dev
 30 2018-08-16T04:19:40  *** vexbuy has quit IRC
 31 2018-08-16T04:20:02  *** vexbuy has joined #bitcoin-core-dev
 32 2018-08-16T04:20:44  *** vexbuy has joined #bitcoin-core-dev
 33 2018-08-16T04:21:15  *** vexbuy has quit IRC
 34 2018-08-16T04:21:34  *** vexbuy has joined #bitcoin-core-dev
 35 2018-08-16T04:22:22  *** vexbuy has joined #bitcoin-core-dev
 36 2018-08-16T04:23:04  *** vexbuy has joined #bitcoin-core-dev
 37 2018-08-16T04:23:35  *** vexbuy has quit IRC
 38 2018-08-16T04:23:54  *** vexbuy has joined #bitcoin-core-dev
 39 2018-08-16T04:36:01  *** minskey has joined #bitcoin-core-dev
 40 2018-08-16T04:55:02  *** d9b4bef9 has quit IRC
 41 2018-08-16T04:56:17  *** d9b4bef9 has joined #bitcoin-core-dev
 42 2018-08-16T05:28:36  *** AaronvanW has joined #bitcoin-core-dev
 43 2018-08-16T05:33:49  *** AaronvanW has quit IRC
 44 2018-08-16T05:47:31  *** Victorsueca has quit IRC
 45 2018-08-16T05:48:45  *** Victorsueca has joined #bitcoin-core-dev
 46 2018-08-16T06:00:28  *** Krellan has quit IRC
 47 2018-08-16T06:05:51  *** Krellan has joined #bitcoin-core-dev
 48 2018-08-16T06:19:05  *** ken2812221_ has joined #bitcoin-core-dev
 49 2018-08-16T06:21:57  *** ken2812221 has quit IRC
 50 2018-08-16T06:25:08  *** murrayn has quit IRC
 51 2018-08-16T06:29:55  <jonasschnelli> gmaxwell: Sure. My current priority is encryption, then auth, then NH/PQ. I can also try the NH stuff earlier..
 52 2018-08-16T06:47:50  *** face has quit IRC
 53 2018-08-16T06:59:26  *** vexbuy has joined #bitcoin-core-dev
 54 2018-08-16T07:06:25  *** plankers has joined #bitcoin-core-dev
 55 2018-08-16T07:18:47  *** Guyver2 has joined #bitcoin-core-dev
 56 2018-08-16T07:26:29  *** da2ce7 has quit IRC
 57 2018-08-16T07:28:49  *** da2ce7 has joined #bitcoin-core-dev
 58 2018-08-16T07:49:36  <gmaxwell> sipa: 13989 is doing avx512 sha256d64, and gets a 19% speedup for the MerkleRoot benchmark.
 59 2018-08-16T07:57:08  *** SopaXorzTaker has joined #bitcoin-core-dev
 60 2018-08-16T08:11:14  *** kewde[m] has quit IRC
 61 2018-08-16T08:11:19  *** joshb[m] has quit IRC
 62 2018-08-16T08:11:22  *** squarfed[m] has quit IRC
 63 2018-08-16T08:17:30  *** savil has joined #bitcoin-core-dev
 64 2018-08-16T08:19:00  *** vexbuy has quit IRC
 65 2018-08-16T08:20:19  *** Guyver2 has quit IRC
 66 2018-08-16T08:25:33  <wumpus> time to tag rc1?
 67 2018-08-16T08:27:39  <wumpus> I'm sure enough issues will come up during gitian building (although I was able to do so succesfully) for the first time with bionic, but I think it makes sense to start testing?
 68 2018-08-16T08:33:33  *** promag has joined #bitcoin-core-dev
 69 2018-08-16T08:34:01  *** csknk has joined #bitcoin-core-dev
 70 2018-08-16T08:37:57  *** reallll has quit IRC
 71 2018-08-16T08:38:33  *** belcher_ has joined #bitcoin-core-dev
 72 2018-08-16T08:47:18  *** ExtraCrispy has joined #bitcoin-core-dev
 73 2018-08-16T08:56:09  *** murrayn has joined #bitcoin-core-dev
 74 2018-08-16T08:57:01  *** d9b4bef9 has quit IRC
 75 2018-08-16T08:58:08  *** d9b4bef9 has joined #bitcoin-core-dev
 76 2018-08-16T09:05:53  *** promag has quit IRC
 77 2018-08-16T09:40:03  *** e4xit has quit IRC
 78 2018-08-16T09:42:01  *** Jmabsd has quit IRC
 79 2018-08-16T09:44:45  *** fanquake has joined #bitcoin-core-dev
 80 2018-08-16T09:44:52  <fanquake> wumpus sounds good
 81 2018-08-16T09:48:39  <wumpus> ok! let's do it then
 82 2018-08-16T09:56:31  *** AaronvanW has joined #bitcoin-core-dev
 83 2018-08-16T09:56:34  <wumpus> yesterday no one protested either so...
 84 2018-08-16T09:56:58  *** dcousens has quit IRC
 85 2018-08-16T09:57:44  *** dcousens has joined #bitcoin-core-dev
 86 2018-08-16T09:58:24  <fanquake> Just take that as silent optimism/agreement or something
 87 2018-08-16T10:01:14  *** plankers has quit IRC
 88 2018-08-16T10:02:52  <wumpus> I'm just trying to avoid forgetting a dumb step (yes, version has been bumped)
 89 2018-08-16T10:08:15  <wumpus>  * [new tag]                                                                           v0.17.0rc1 -> v0.17.0rc1
 90 2018-08-16T10:10:04  <wumpus> there we go
 91 2018-08-16T10:11:22  <fanquake> That's what rc's are for anyways. Wouldn't be the first time we've fixed something up quickly with an rc2 etc.
 92 2018-08-16T10:11:24  <fanquake> wew!
 93 2018-08-16T10:13:30  *** e4xit has joined #bitcoin-core-dev
 94 2018-08-16T11:08:32  *** vexbuy has joined #bitcoin-core-dev
 95 2018-08-16T11:16:53  *** rex4539 has joined #bitcoin-core-dev
 96 2018-08-16T11:17:15  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 97 2018-08-16T11:36:59  *** vexbuy has quit IRC
 98 2018-08-16T11:38:42  *** vexbuy has joined #bitcoin-core-dev
 99 2018-08-16T11:53:48  <wumpus> 0.17.0rc1 gitian sigs (unsigned) up, wonder if anyone reproduces
100 2018-08-16T11:57:21  <jonasschnelli> wumpus: still building: https://bitcoin.jonasschnelli.ch/build/743
101 2018-08-16T11:58:46  <jonasschnelli> windows and osx match though... linux: will see
102 2018-08-16T12:06:32  *** no_input_found has quit IRC
103 2018-08-16T12:06:52  *** no_input_found has joined #bitcoin-core-dev
104 2018-08-16T12:14:04  <wumpus> awesome !
105 2018-08-16T12:18:27  *** Chris_Stewart_5 has quit IRC
106 2018-08-16T12:22:38  *** goatpig has joined #bitcoin-core-dev
107 2018-08-16T12:25:02  *** d9b4bef9 has quit IRC
108 2018-08-16T12:25:07  <jonasschnelli> linux is a match as well
109 2018-08-16T12:26:05  *** Krellan has quit IRC
110 2018-08-16T12:26:14  *** d9b4bef9 has joined #bitcoin-core-dev
111 2018-08-16T12:27:17  *** Krellan has joined #bitcoin-core-dev
112 2018-08-16T12:58:36  <jonasschnelli> Benchmarks for v2 message format composing (encrypted):
113 2018-08-16T12:59:07  <jonasschnelli> 100 blocks: V1 legacy (dblSHA): 1.43978, V2 (ChaCha20/Poly1305): 1.42594
114 2018-08-16T12:59:33  <jonasschnelli> (and this is with SHA SSE4.1/AVX versus non NI acceletarted chacha)
115 2018-08-16T13:20:38  *** Chris_Stewart_5 has joined #bitcoin-core-dev
116 2018-08-16T13:22:33  <fanquake> thanks for the quick build ken2812221, more matching sigs
117 2018-08-16T13:28:01  *** ken2812221_ is now known as ken2812221
118 2018-08-16T13:28:10  <ken2812221> fanquake: nice
119 2018-08-16T13:32:33  *** arubi has quit IRC
120 2018-08-16T13:32:55  *** arubi has joined #bitcoin-core-dev
121 2018-08-16T13:34:15  *** vexbuy has quit IRC
122 2018-08-16T13:34:53  *** vexbuy has joined #bitcoin-core-dev
123 2018-08-16T13:36:15  <wumpus> jonasschnelli: nice
124 2018-08-16T13:36:31  *** asoltys has quit IRC
125 2018-08-16T13:39:43  *** vexbuy has quit IRC
126 2018-08-16T13:48:27  *** fanquake has quit IRC
127 2018-08-16T13:58:39  *** vexbuy has joined #bitcoin-core-dev
128 2018-08-16T14:13:00  *** promag has joined #bitcoin-core-dev
129 2018-08-16T14:20:23  *** michaelsdunn1 has joined #bitcoin-core-dev
130 2018-08-16T14:22:58  <promag> with 0.17 branched, #13529 can be reviewed
131 2018-08-16T14:23:01  <gribble> https://github.com/bitcoin/bitcoin/issues/13529 | Use new Qt5 connect syntax by promag · Pull Request #13529 · bitcoin/bitcoin · GitHub
132 2018-08-16T14:23:08  <promag> wumpus: ^
133 2018-08-16T14:23:20  <promag> maybe I should push a rebased commit?
134 2018-08-16T14:25:03  *** promag has quit IRC
135 2018-08-16T14:25:04  *** michaelsdunn1 has quit IRC
136 2018-08-16T14:35:34  *** michaelsdunn1 has joined #bitcoin-core-dev
137 2018-08-16T14:37:41  *** promag has joined #bitcoin-core-dev
138 2018-08-16T14:38:43  <wumpus> promag: will review
139 2018-08-16T14:39:30  <wumpus> looks like you first need to address the issues, apparently you did break some things!
140 2018-08-16T15:05:03  *** vexbuy has quit IRC
141 2018-08-16T15:05:41  *** vexbuy has joined #bitcoin-core-dev
142 2018-08-16T15:09:09  *** vexbuy_ has joined #bitcoin-core-dev
143 2018-08-16T15:10:05  *** vexbuy has quit IRC
144 2018-08-16T15:11:21  <promag> wumpus: yeah I saw that after writing the above :(
145 2018-08-16T15:15:00  *** csknk has quit IRC
146 2018-08-16T15:15:28  *** Dizzle has joined #bitcoin-core-dev
147 2018-08-16T15:33:49  <gmaxwell> jonasschnelli: well the reason I was offering to try sticking on newhope is that I _think_ I can add it in with only a few lines code change.  If we're of the view that we'll want it long term, then maybe it'll make sense to do up front if it really does turn out to be that simple.
148 2018-08-16T15:35:15  <jonasschnelli> gmaxwell: Yes. Agree. I just too deep into implementing the ecdh/chachapoly stuff right now... but as soon as some tests are finished, I'm happy to try the NH implementation.
149 2018-08-16T15:35:36  <jonasschnelli> I need to research (or ask you) a bit more, though
150 2018-08-16T15:35:42  <gmaxwell> OKAY but thats why I was offering to help. :P
151 2018-08-16T15:35:57  <gmaxwell> Sure. Fortunately it's really simple.
152 2018-08-16T15:36:10  <jonasschnelli> What implementation are you looking at?
153 2018-08-16T15:37:27  <jonasschnelli> Doesn't it also require SHA3?
154 2018-08-16T15:37:31  <gmaxwell> https://github.com/newhopecrypto/newhope-usenix/tree/master/ref
155 2018-08-16T15:37:33  <gmaxwell> No.
156 2018-08-16T15:37:50  <jonasschnelli> ok
157 2018-08-16T15:37:53  <gmaxwell> it uses chacha20 internally just as a random number generator.
158 2018-08-16T15:38:25  <gmaxwell> oh actually there is a sha3 impl there too. I guess it's using that to hash the final state.
159 2018-08-16T15:38:32  <jonasschnelli> gmaxwell: replace that with sha2? -> https://github.com/newhopecrypto/newhope-usenix/blob/master/ref/newhope.c#L58?
160 2018-08-16T15:39:41  <gmaxwell> yes, we can just replace that with sha2.
161 2018-08-16T15:49:39  *** SopaXorzTaker has quit IRC
162 2018-08-16T15:50:41  *** SopaXorzTaker has joined #bitcoin-core-dev
163 2018-08-16T15:51:12  <gmaxwell> You see how it works though? https://github.com/newhopecrypto/newhope-usenix/blob/master/ref/test/test_newhope.c#L36  Initator runs newhope_keygen and sticks senda on their message, responder runs newhope_sharedb with that as an argument and gets the shared keyb and sendb to send, intiator gets sendb and feeds it to newhope_shareda and gets the same shared secret out.
164 2018-08-16T15:53:41  <gmaxwell> The messages are constant length (1824 and 2048 bytes IIRC).
165 2018-08-16T15:55:47  <gmaxwell> And we'd probably go and change the API slightly so that it takes randomness as an argument rather than reading /dev/urandom itself.
166 2018-08-16T15:57:51  <gmaxwell> though for an intial test that could be ignored.
167 2018-08-16T16:01:08  *** masonicboom has joined #bitcoin-core-dev
168 2018-08-16T16:06:39  *** ExtraCrispy has quit IRC
169 2018-08-16T16:08:55  <jonasschnelli> gmaxwell: so it basically works the same as DH (in terms of network interactions)?
170 2018-08-16T16:09:28  <jonasschnelli> Could we append send_a (the message) to the ecdh-32byte-pubkey?
171 2018-08-16T16:10:03  <jonasschnelli> I guess the NH handshake doesn't have to follow under the ECDH encrypted channel?
172 2018-08-16T16:12:33  *** jhfrontz has quit IRC
173 2018-08-16T16:13:04  <gmaxwell> Correct on all counts.
174 2018-08-16T16:13:54  <gmaxwell> It's not _exactly_ the same as DH in terms of network interactions because in DH  both alice and bob can send their pubkeys at the same time, but in newhope alice sends then bob sends. But we're not using DH that way anways.
175 2018-08-16T16:14:26  <gmaxwell> So we can implement it just by appending send_a to the initator pubkey, and send_b to the responder pubkey.
176 2018-08-16T16:15:02  <gmaxwell> then we just hash the secrets that come out of both DH and newhope.
177 2018-08-16T16:16:06  *** LeMiner has quit IRC
178 2018-08-16T16:17:34  <jonasschnelli> gmaxwell: I see. Whats the length of the newhope secret output? 32b?
179 2018-08-16T16:21:26  <gmaxwell> 32 bytes, yes.
180 2018-08-16T16:23:49  <jonasschnelli> Okay. From a protocol level, this seems easy.
181 2018-08-16T16:24:12  <jonasschnelli> Are the NH messagedsidentifiable (DPIish)?
182 2018-08-16T16:25:55  <gmaxwell> No, not more than the secp256k1 public keys.
183 2018-08-16T16:26:15  <jonasschnelli> ok. then ECDH doesn't need to go first
184 2018-08-16T16:27:02  *** d9b4bef9 has quit IRC
185 2018-08-16T16:28:16  *** d9b4bef9 has joined #bitcoin-core-dev
186 2018-08-16T16:30:58  *** plankers has joined #bitcoin-core-dev
187 2018-08-16T16:32:12  <jonasschnelli> yeah. I don't see a reason why we should not add newhope to the handshake (unless I completely must understand it)
188 2018-08-16T16:35:38  <gmaxwell> The only arguments I can make against it are that (1) it might turn out to not add much security (e.g. if newhope gets broken), and then we're stuck carrying it around buring bytes, loc, and cpu cycles on it  (2) implementers that insist on writing everything from scratch instead of using public domain C code will have a harder time, (3) more stuff to integrate.
189 2018-08-16T16:36:49  <gmaxwell> For other applications, like using it in SSL the size and speed impacts matter more. For us, I think they're basically irrelevant. Newhope is as fast as (or maybe even faster than, with AVX in use) ECDH.
190 2018-08-16T16:39:20  <jonasschnelli> I hope my implementation for detecting a version message or a key-handshake will still work since it's now longer then 32b
191 2018-08-16T16:40:32  <gmaxwell> You should just be able to read the first N bytes and decide, then keep reading.
192 2018-08-16T16:44:08  <jonasschnelli> yeah... avoid pubkeys with netmagic, read 4 bytes and continue with handshake when not equal the net magic
193 2018-08-16T16:46:38  <wumpus> RISC-V node (on actual hw) started syncing
194 2018-08-16T16:49:29  <sipa> wooho
195 2018-08-16T16:49:31  <gmaxwell> wumpus: \O/
196 2018-08-16T16:50:15  <gmaxwell> now just submit some patches upstream to add hardware SHA2 and ECC so that the next one will finish in reasonable time. :P
197 2018-08-16T16:53:31  <sipa> just add a HW instruction vrfybtcnblk
198 2018-08-16T16:53:39  <sipa> wait, what did the R in RISC stand for again?
199 2018-08-16T16:53:45  <wumpus> hehe
200 2018-08-16T16:54:10  <gmaxwell> well, brainfuck has fewer operations, so clearly RISCV isn't RISC? :P
201 2018-08-16T16:54:21  <wumpus> it stands for Rich Instruction Set Computing
202 2018-08-16T16:54:25  <sipa> ah.
203 2018-08-16T16:54:31  <sipa> gmaxwell: try whitespace
204 2018-08-16T16:55:07  <sipa> i guess whitespace has more instructions, but just encoded in 3 characters
205 2018-08-16T16:55:11  <gmaxwell> wumpus: as opposed to constrained instruction set computing?
206 2018-08-16T16:55:17  *** jhfrontz has joined #bitcoin-core-dev
207 2018-08-16T16:56:00  <wumpus> though tbh at this point I'm less worried about performance than about obscure chip and compiler issues
208 2018-08-16T16:56:03  <wumpus> gmaxwell: exactly!
209 2018-08-16T16:57:08  *** thebiglq12345 has joined #bitcoin-core-dev
210 2018-08-16T16:59:29  <gmaxwell> jonasschnelli: in any case, newhope is a member of the class of fast PQ schemes that are newer and more likely to turn out to be insecure against even classical computers.  But the only alternatives that aren't have properties that would make us not use them.. e.g. the McEliece public keys are like 1MB.
211 2018-08-16T17:14:04  *** Emcy has quit IRC
212 2018-08-16T17:33:30  <gmaxwell> jonasschnelli: oh, actually we'd want to be using this implementation https://github.com/newhopecrypto/newhope/tree/master/ref  as it's their NIST submission, and has a substantial simplifcation to the protocol innards.
213 2018-08-16T18:00:53  <jonasschnelli> ok. thanks... I'll look into it.
214 2018-08-16T18:01:08  *** vexbuy has joined #bitcoin-core-dev
215 2018-08-16T18:02:08  <wumpus> apparently the MMC interface on the HiFive unleashed is really slow, either due to the current kernel version, or due to some hardware limitation
216 2018-08-16T18:04:38  *** vexbuy_ has quit IRC
217 2018-08-16T18:04:48  <wumpus> the debian/fedora developers use nbd, though currently I have a 100mbit switch connected for the embedded LAN so that's not going to be great either :-)
218 2018-08-16T18:08:02  <wumpus> but all in all I'm quite happy with this, for the first larger-scale ASIC implementation of a new architecture it's cool how far this gets
219 2018-08-16T18:08:51  <sipa> the risc-v instrunction encoding is pretty cool
220 2018-08-16T18:09:21  <sipa> everything is natively a 32 bit instruction, but there are optional extensions that compress certain instructions down
221 2018-08-16T18:09:22  <wumpus> yes how they do variable-length is interesting
222 2018-08-16T18:09:43  <sipa> but that compression can be implemented as a pure postprocessing step in the assembler
223 2018-08-16T18:10:55  <wumpus> isa : rv64imafdc  I think -c is the 16-bit compressed instruction extension, so it has that
224 2018-08-16T18:11:17  *** ken2812221 has quit IRC
225 2018-08-16T18:11:48  *** Randolf has quit IRC
226 2018-08-16T18:12:02  *** Krellan has quit IRC
227 2018-08-16T18:12:05  <wumpus> M=Integer
228 2018-08-16T18:13:26  <wumpus> Multiplication and Division, A=Atomic instructions, F=32-bit fp, D=64-bit fp, C=Compressed instructions
229 2018-08-16T18:13:28  *** Krellan has joined #bitcoin-core-dev
230 2018-08-16T18:13:36  <sipa> no B=bit manipulation?
231 2018-08-16T18:13:50  <wumpus> apparently not!
232 2018-08-16T18:14:37  <wumpus> "This chapter is a placeholder for a future standard extension to provide bit manipulation instruc-
233 2018-08-16T18:15:05  <sipa> ah
234 2018-08-16T18:15:07  <wumpus> the version of the spec I have calls it future, at least
235 2018-08-16T18:15:15  <sipa> i guess basic bit manipulation is available in the base set of instructions
236 2018-08-16T18:16:53  <wumpus> yes: ANDI/ORI/XORI
237 2018-08-16T18:17:47  <wumpus> they're in the mandatory part
238 2018-08-16T18:18:41  <wumpus> I guess the idea of -B will be to provide a more extensive set, say for direct injection/extraction of bit sequences, popcount, count leading/trailing bits, etc
239 2018-08-16T18:21:20  *** plankers has quit IRC
240 2018-08-16T18:22:12  *** Emcy has joined #bitcoin-core-dev
241 2018-08-16T18:22:36  *** Victorsueca has quit IRC
242 2018-08-16T18:23:45  *** Victorsueca has joined #bitcoin-core-dev
243 2018-08-16T18:35:15  *** grafcaps has joined #bitcoin-core-dev
244 2018-08-16T18:35:21  *** promag has quit IRC
245 2018-08-16T18:35:54  *** promag has joined #bitcoin-core-dev
246 2018-08-16T18:39:47  *** promag has quit IRC
247 2018-08-16T18:42:21  <jonasschnelli> gmaxwell: I guess a configuration option that would allow ecdh only handshake would make little sense?
248 2018-08-16T18:42:48  <jonasschnelli> I down side with newhope could be the implementation burden if we also want SPV clients to adopt it.
249 2018-08-16T18:44:55  *** davex__ has left #bitcoin-core-dev
250 2018-08-16T18:45:05  *** davex__ has quit IRC
251 2018-08-16T18:45:08  *** Krellan has quit IRC
252 2018-08-16T18:45:12  <gmaxwell> jonasschnelli: I don't think it would make sense to make it optional.
253 2018-08-16T18:45:34  *** davex__ has joined #bitcoin-core-dev
254 2018-08-16T18:46:25  <gmaxwell> jonasschnelli: there are, for example, java implementations and whatnot.
255 2018-08-16T18:46:41  *** jtimon has joined #bitcoin-core-dev
256 2018-08-16T18:47:57  *** thaumavorio has joined #bitcoin-core-dev
257 2018-08-16T18:48:34  *** promag has joined #bitcoin-core-dev
258 2018-08-16T18:50:22  *** dgenr8 has joined #bitcoin-core-dev
259 2018-08-16T19:01:06  <promag> meeting?
260 2018-08-16T19:01:18  * luke-jr pokes wumpus
261 2018-08-16T19:03:23  *** clarkmoody has joined #bitcoin-core-dev
262 2018-08-16T19:03:48  <wumpus> hello
263 2018-08-16T19:03:51  <sdaftuar> hi
264 2018-08-16T19:03:54  <wumpus> #startmeeting
265 2018-08-16T19:03:54  <lightningbot> Meeting started Thu Aug 16 19:03:54 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
266 2018-08-16T19:03:54  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
267 2018-08-16T19:03:56  <jonasschnelli> Yeah... hi
268 2018-08-16T19:04:03  <meshcollider> hi
269 2018-08-16T19:04:07  <promag> hi
270 2018-08-16T19:04:12  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark mi
271 2018-08-16T19:04:16  <wumpus> chagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
272 2018-08-16T19:04:24  <gmaxwell> darn, just beat me to it.
273 2018-08-16T19:04:29  <achow101> hi
274 2018-08-16T19:04:33  <sipa> hi
275 2018-08-16T19:04:43  <wumpus> apparently I divided michagogo in two
276 2018-08-16T19:04:55  <sdaftuar> ouch
277 2018-08-16T19:05:24  <meshcollider> Hope he's not too cut up about it
278 2018-08-16T19:05:25  <wumpus> PSA: we've tagged 0.17.0rc1, let us know if you have any trouble gitian-building due to the upgrade of the guest to Ubuntu 18.04/bionic
279 2018-08-16T19:05:53  *** ken2812221 has joined #bitcoin-core-dev
280 2018-08-16T19:06:31  <jonasschnelli> As mentioned its a bit sad that we dropped debian 9 (newest version) as gitian host,... but apparently compiling lxc was simpler then expected
281 2018-08-16T19:06:36  <wumpus> now that 0.17 release cycle is started, it might be time to bring back "high priority for review" as a topic
282 2018-08-16T19:06:50  *** clarkmoody has quit IRC
283 2018-08-16T19:07:10  <wumpus> I have some instructions for debian 9 here: https://gist.github.com/laanwj/c62e101bfd68718f0686926dfd10666b
284 2018-08-16T19:07:27  <jtimon> hi
285 2018-08-16T19:07:29  <wumpus> (yes, you need to build lxc and debootstrap from source)
286 2018-08-16T19:07:32  <jonasschnelli> Nice wumpus
287 2018-08-16T19:07:43  <wumpus> we should probably integrate that into the documentation
288 2018-08-16T19:08:25  <jonasschnelli> Yes. Until lxc 2.1.1 is supported by debians apt
289 2018-08-16T19:08:40  <wumpus> yes, 3.x is probably overkill
290 2018-08-16T19:08:59  <luke-jr> wumpus: did we ever get a solution for making a bionic base VM? :/
291 2018-08-16T19:08:59  <jonasschnelli> Works fine here with 2.1.1
292 2018-08-16T19:09:09  <luke-jr> (for qemu)
293 2018-08-16T19:09:17  <achow101> also, with the suite bump, don't forget to do bin/make-base-vm again
294 2018-08-16T19:09:29  <wumpus> luke-jr: I don't think so, I think LXC and Docker are the only options at the moment
295 2018-08-16T19:09:34  <luke-jr> achow101: last I checked that doesn't work
296 2018-08-16T19:09:42  <achow101> luke-jr: there's a fork of vmbuilder that works for bionic
297 2018-08-16T19:09:57  *** viaj3ro has joined #bitcoin-core-dev
298 2018-08-16T19:09:59  <achow101> luke-jr: https://github.com/newroco/vmbuilder
299 2018-08-16T19:10:06  <luke-jr> #link https://github.com/newroco/vmbuilder
300 2018-08-16T19:10:09  <wumpus> but the fork might be worth a try !
301 2018-08-16T19:10:12  <achow101> luke-jr: see also https://github.com/devrandom/gitian-builder/issues/191
302 2018-08-16T19:11:54  <wumpus> anyhow -- please ask in this channel if you have any problems getting gitian running
303 2018-08-16T19:11:58  *** clarkmoody has joined #bitcoin-core-dev
304 2018-08-16T19:12:04  <wumpus> #topic High priority for review
305 2018-08-16T19:12:30  <wumpus> https://github.com/bitcoin/bitcoin/projects/8  there's only one PR in there at the moment, #13100
306 2018-08-16T19:12:35  <gribble> https://github.com/bitcoin/bitcoin/issues/13100 | gui: Add dynamic wallets support by promag · Pull Request #13100 · bitcoin/bitcoin · GitHub
307 2018-08-16T19:12:51  <sipa> i'd like #13723
308 2018-08-16T19:12:53  <gribble> https://github.com/bitcoin/bitcoin/issues/13723 | PSBT key path cleanups by sipa · Pull Request #13723 · bitcoin/bitcoin · GitHub
309 2018-08-16T19:13:05  <sipa> (it's the basis for further psbt/descript integration)
310 2018-08-16T19:13:24  <instagibbs> https://github.com/bitcoin/bitcoin/pull/13968 bugfixes for psbt stuff too(0.17 backport)
311 2018-08-16T19:13:54  <instagibbs> #13968
312 2018-08-16T19:13:56  <gribble> https://github.com/bitcoin/bitcoin/issues/13968 | [wallet] couple of walletcreatefundedpsbt fixes by instagibbs · Pull Request #13968 · bitcoin/bitcoin · GitHub
313 2018-08-16T19:14:42  <ken2812221> #13866
314 2018-08-16T19:14:44  <gribble> https://github.com/bitcoin/bitcoin/issues/13866 | utils: Use _wfopen and _wfreopen on Windows by ken2812221 · Pull Request #13866 · bitcoin/bitcoin · GitHub
315 2018-08-16T19:14:48  <wumpus> 13723 added
316 2018-08-16T19:15:33  <jtimon> if high priority is still for blockers, https://github.com/bitcoin/bitcoin/pull/13311 is kind of a blocker for https://github.com/bitcoin/bitcoin/pull/8994 which is itself a blocker for toher things I wanted to do
317 2018-08-16T19:15:38  <wumpus> the other two as well
318 2018-08-16T19:15:55  <wumpus> #13311
319 2018-08-16T19:15:57  <gribble> https://github.com/bitcoin/bitcoin/issues/13311 | Dont edit Chainparams after initialization by jtimon · Pull Request #13311 · bitcoin/bitcoin · GitHub
320 2018-08-16T19:16:12  <promag> wumpus: can you replace 13100 with #13529?
321 2018-08-16T19:16:14  <gribble> https://github.com/bitcoin/bitcoin/issues/13529 | Use new Qt5 connect syntax by promag · Pull Request #13529 · bitcoin/bitcoin · GitHub
322 2018-08-16T19:17:05  <wumpus> promag: ok
323 2018-08-16T19:17:09  <wumpus> jtimon: added
324 2018-08-16T19:17:16  <jtimon> yeah, thanks
325 2018-08-16T19:17:20  <achow101> I would like #12493
326 2018-08-16T19:17:22  <gribble> https://github.com/bitcoin/bitcoin/issues/12493 | [wallet] Reopen CDBEnv after encryption instead of shutting down by achow101 · Pull Request #12493 · bitcoin/bitcoin · GitHub
327 2018-08-16T19:17:41  <promag> ty
328 2018-08-16T19:17:56  <wumpus> achow101: ok, yes, probably makes sense to merge that early in the 0.18 cycle
329 2018-08-16T19:18:11  <wumpus> achow101: it sounds sort of--risky
330 2018-08-16T19:18:14  <gmaxwell> achow101: did we ever work through the problems that were preventing "don't create a wallet until a key is requested or until encryption is added" ?
331 2018-08-16T19:18:32  <gmaxwell> As that would also get rid of the shutdown on encryption for many users.
332 2018-08-16T19:18:37  <achow101> gmaxwell: the problems with that were backups IIRC
333 2018-08-16T19:19:10  <achow101> gmaxwell: actually that problem was with generate keys on use, not create wallet on use
334 2018-08-16T19:19:59  <achow101> in that case, I don't think so. but I also don't remember the problems with create wallet on use. I think I just never got around to implementing it
335 2018-08-16T19:20:17  <gmaxwell> right. I thought there were some dumb bugs with create on use that were going to get fixed as a side effect of pending multiwallet work.
336 2018-08-16T19:20:31  *** ken2812221 has quit IRC
337 2018-08-16T19:21:51  <wumpus> any other proposed topics?
338 2018-08-16T19:21:52  *** ken2812221 has joined #bitcoin-core-dev
339 2018-08-16T19:22:28  *** Guyver2 has joined #bitcoin-core-dev
340 2018-08-16T19:23:59  <sipa> short announcement: we're working on an extension to descriptors to support nested and/or/threshold constructions
341 2018-08-16T19:25:03  <wumpus> cool!
342 2018-08-16T19:25:38  <sdaftuar> topic suggestion: open floor for people to share what they
343 2018-08-16T19:25:40  <jonasschnelli> nice
344 2018-08-16T19:25:41  <sdaftuar> re wokring on
345 2018-08-16T19:26:01  <sdaftuar> (since we don't have any other topics apparently :P)
346 2018-08-16T19:26:29  <sipa> i like wok rings
347 2018-08-16T19:26:33  <wumpus> #topic open floor for people to share what they are working on
348 2018-08-16T19:26:40  <jonasschnelli> Working on p2p level encryption since a couple of weeks (thats why I'm pretty quite on github). Will open PR in 1-2 weeks.
349 2018-08-16T19:27:16  <wumpus> sipa: so this is for further improvements to scantxoutset I suppose?
350 2018-08-16T19:27:24  <sipa> wumpus: and all the things :)
351 2018-08-16T19:27:34  <wumpus> right
352 2018-08-16T19:27:48  <sipa> wumpus: so you can import and(xpub/...,or(xpub/...,xpub/...)) into your wallet as watch-only chain for example
353 2018-08-16T19:27:53  <achow101> wumpus: hopefully for eventually a replacement-ish thing to the wallet
354 2018-08-16T19:27:54  <sipa> and get psbt to sign for it
355 2018-08-16T19:28:20  <instagibbs> so excite
356 2018-08-16T19:28:22  <wumpus> that's neat
357 2018-08-16T19:28:25  <jonasschnelli> sipa: how would/could that lead to xpub watch only wallets?
358 2018-08-16T19:28:57  <sipa> jonasschnelli: yes
359 2018-08-16T19:29:02  <sipa> that's the goal of the descriptors
360 2018-08-16T19:29:45  <instagibbs> achow101, not sure if he was joking, but he said he wouldnt have to implement it if I did the HWI version. I'll keep leaning on him
361 2018-08-16T19:29:52  <jonasschnelli> That would be a great feature... could also be extended to the GUI including coin selection (send screen), but don't sign/broadcast (obviously) but will create a PSBT (file)
362 2018-08-16T19:30:16  <instagibbs> my guess is for any hope of Core support of HWW, we want to not have to support a bunch of PSBT drivers...
363 2018-08-16T19:30:52  <instagibbs> oh sorry wrong window
364 2018-08-16T19:30:56  <jonasschnelli> instagibbs: you mean more support then the PSBT?
365 2018-08-16T19:31:15  <sipa> jonasschnelli: seems reasonable yes
366 2018-08-16T19:31:37  <sipa> i need to think through how to integrate things in the wallet itself, so you can import descriptors
367 2018-08-16T19:31:46  <sipa> and how to make it compatible with all existing RPCs etc
368 2018-08-16T19:32:16  <gmaxwell> sipa: not just and/or/threshold but also CSV and hashlocks?
369 2018-08-16T19:32:22  <instagibbs> gmaxwell, yes
370 2018-08-16T19:32:22  <sipa> gmaxwell: oh yes
371 2018-08-16T19:32:27  <instagibbs> (oh rhetorical)
372 2018-08-16T19:32:32  <sipa> but my goal is that the wallet eventually consists of a bunch of descriptors with metadata (and labels and transactions, and precomputed pubkeys to replace keypools)
373 2018-08-16T19:33:41  <gmaxwell> If lots of people can help sipa finish that work it would be good. :P
374 2018-08-16T19:34:15  <sipa> but independently andytoshi and i started looking into how we can efficiently compile arbitrary and/or/k-of-n/locktime/hash expressions to script
375 2018-08-16T19:34:44  <sipa> which would just plug into descriptors, and then be available to everything that uses them
376 2018-08-16T19:35:40  *** ken2812221 has quit IRC
377 2018-08-16T19:35:54  *** ken2812221 has joined #bitcoin-core-dev
378 2018-08-16T19:36:12  <wumpus> so I"ve been working on the RISC-V support, today I was able to do basic bring-up of the hardware (HiFive Unleashed) and test the gitian-built executables, which work!
379 2018-08-16T19:36:28  <wumpus> been able to run the test_bitcoin succesfully and sync part of the chain, I'll keep the node running
380 2018-08-16T19:36:34  <wumpus> probably the first RISC-V bitcoin node in the world
381 2018-08-16T19:37:03  <jonasschnelli> \o/ nice!
382 2018-08-16T19:37:39  <gmaxwell> nmkgl and I have been working on reconciliation based transaction relay.  (With sipa's help too, that why I want people to help finish the descriptor work, ... :P )
383 2018-08-16T19:38:26  <sdaftuar> gmaxwell: as in set reconciliation, eg to sync mempools?
384 2018-08-16T19:38:52  <sipa> sdaftuar: yup
385 2018-08-16T19:39:04  <gmaxwell> sdaftuar: Yes, but I believed we found a better design that doesn't sync the mempools directly.
386 2018-08-16T19:39:46  <sdaftuar> neat, i'd be interested in learning more if you guys have a summary at some point.
387 2018-08-16T19:39:49  <sipa> sdaftuar: it uses a more computationally expensive set reconcilation protocol than IBLT, but it's much more space efficient
388 2018-08-16T19:40:06  <gmaxwell> muuuch more.
389 2018-08-16T19:40:11  <sipa> (basically the amount of data transferred is equal to the expected difference between the two sets)
390 2018-08-16T19:41:15  <sipa> but it's in the order of 40ms here to find 300 differences
391 2018-08-16T19:41:24  <gmaxwell> I also came up with a new reconcilation protocol with a different performance tradeoff (much faster to decode, but slightly less space efficient), though it doesn't look like we'll have cause to use it.
392 2018-08-16T19:42:04  <gmaxwell> sdaftuar: in any case the short summary of what we're thinking now vs before is that instead of reconciling mempools, reconcile the transactions that the peers would have otherwise INVed to each other.
393 2018-08-16T19:42:42  <gmaxwell> this avoids cases like a mempool policy difference causing transations to get 'stuck' in the difference set forever until they get mined.
394 2018-08-16T19:42:51  <sipa> oooh nice
395 2018-08-16T19:42:53  <sipa> i missed that
396 2018-08-16T19:43:23  <gmaxwell> Then it can just be coupled with some simple mechenism to fast-start an empty or stale mempool.
397 2018-08-16T19:43:53  <gmaxwell> (I have a proposal for that too, just a simple one/two message protocol)
398 2018-08-16T19:44:19  *** ken2812221 has quit IRC
399 2018-08-16T19:44:22  <instagibbs> would this be complementary to a ip-hiding protocol like dandelion?
400 2018-08-16T19:44:25  <sdaftuar> ah, that sounds neat
401 2018-08-16T19:44:25  <BlueMatt> hmm, and then you could bucket and do reconciliation slowly for low-feerate buckets, or no reason to do that anymore?
402 2018-08-16T19:44:46  <BlueMatt> instagibbs: I'd presume so, this would be during fluff-faze :p
403 2018-08-16T19:44:55  <sipa> instagibbs: yes, orthogonal
404 2018-08-16T19:45:04  <sipa> this is changing the diffusion phase
405 2018-08-16T19:45:06  <sipa> not the stem
406 2018-08-16T19:45:09  <instagibbs> BlueMatt, not exactly fluffing anymore :)
407 2018-08-16T19:45:16  <gmaxwell> instagibbs: it's orthorgonal, the main motivation is getting rid of the really high bandwidth overhead for rumoring.
408 2018-08-16T19:45:17  <instagibbs> sipa understood
409 2018-08-16T19:45:28  <gmaxwell> Probably some differences in context here. :)
410 2018-08-16T19:45:52  <gmaxwell> Right now, ignoring peers that IBD, the vast majority of node bandwidth is wasted on INV rumoring.
411 2018-08-16T19:45:54  <BlueMatt> gmaxwell: wait, was that a response to my question?
412 2018-08-16T19:46:30  <gmaxwell> BlueMatt: sure, it could be split by feerate too, though I'm not sure we'll have a reason to because the reconcilation is so efficient.
413 2018-08-16T19:46:41  *** SopaXorzTaker has quit IRC
414 2018-08-16T19:46:52  <BlueMatt> hmm, guess I need to think about this more
415 2018-08-16T19:46:54  <sdaftuar> do you have an estimate on the bandwidth reduction?
416 2018-08-16T19:47:38  <gmaxwell> No, don't have a mature enough simulator yet. Also I haven't measured overheads since we improved inv batching.
417 2018-08-16T19:49:04  <gmaxwell> But previously INV overhead was something like 80% of node bandwidth (excluding IBDing peers).
418 2018-08-16T19:49:33  <gmaxwell> And this should mostly eliminate that overhead.
419 2018-08-16T19:50:16  *** BashCo has joined #bitcoin-core-dev
420 2018-08-16T19:52:22  <wumpus> looks like we've run out of topics, but not out of time yet
421 2018-08-16T19:52:34  <gmaxwell> In any case, we've been largely off in number theory land optimizing the recon itself... but I think we've got what we need there now. :)
422 2018-08-16T19:53:38  <sdaftuar> btw i've been thinking about dandelion recently, trying to work through anti-DoS measures for stem routing
423 2018-08-16T19:57:17  <gmaxwell> sdaftuar: Good!  This seems to be one of those things that is easy in theory (if either you ignore getting it right or ignore how hard it is to implement) but hard in practice. :)
424 2018-08-16T19:57:37  <wumpus> would be good to reduce the DoS surface there, yes
425 2018-08-16T20:00:02  <sipa> PLOINKIDOOF
426 2018-08-16T20:00:03  <wumpus> #endmeeting
427 2018-08-16T20:00:03  <lightningbot> Meeting ended Thu Aug 16 20:00:03 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
428 2018-08-16T20:00:03  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-08-16-19.03.html
429 2018-08-16T20:00:03  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-08-16-19.03.txt
430 2018-08-16T20:00:03  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-08-16-19.03.log.html
431 2018-08-16T20:01:08  <sdaftuar> ploinkidoof?
432 2018-08-16T20:07:31  *** BashCo has quit IRC
433 2018-08-16T20:09:28  *** Krellan has joined #bitcoin-core-dev
434 2018-08-16T20:12:18  *** promag has quit IRC
435 2018-08-16T20:12:24  *** BashCo has joined #bitcoin-core-dev
436 2018-08-16T20:12:51  *** promag has joined #bitcoin-core-dev
437 2018-08-16T20:13:08  *** clarkmoody has quit IRC
438 2018-08-16T20:13:23  <jonasschnelli> rekeying and message queues is a nightmare
439 2018-08-16T20:16:58  <sipa> it would be much easier if it wrre deterministic rather than negotiated
440 2018-08-16T20:17:16  *** promag has quit IRC
441 2018-08-16T20:18:17  <jonasschnelli> sipa: not sure. Since you need to decrypt the length in chacha20poly1305@openssh
442 2018-08-16T20:19:01  <jonasschnelli> Or maybe whenever a message exceeds the limit, the following one will be encrypted with the new key
443 2018-08-16T20:19:20  <jonasschnelli> but the time limit could be tricky,... the byte limit probably not
444 2018-08-16T20:22:40  <gmaxwell> wait why is rekeying and message queues a nightmare.
445 2018-08-16T20:23:10  <sipa> gmaxwell: because parsing protocol messages happens before processing them
446 2018-08-16T20:23:26  <jonasschnelli> gmaxwell: because decomposing and queuing (where you decrypt) is done before processing
447 2018-08-16T20:23:32  <sipa> a rekey means you need to undo the parsing for unprocessed things
448 2018-08-16T20:23:33  <gmaxwell> the rekeying should be handled at the decryption layer, when you decrypt and find a rekey message then the very next bytes out you decrypt with the new stuff.
449 2018-08-16T20:24:17  <jonasschnelli> I currently try to approach where I pause the read channel when I'm detecting a rekey during parsing
450 2018-08-16T20:25:04  <sipa> i guess it would be easier if rekeying was done at a meta layer
451 2018-08-16T20:25:12  <sipa> say a flag bit in the encrypted packet
452 2018-08-16T20:25:21  <sipa> rather than an actual protocol message
453 2018-08-16T20:25:40  <gmaxwell> Is the e.g. an out of range length.
454 2018-08-16T20:26:08  <jonasschnelli> Rekeying when the decrypted length is oor seems fragile though,...
455 2018-08-16T20:26:23  <jonasschnelli> I kinda like gmaxwell approach of putting the rekeyin logic in the decryption handler
456 2018-08-16T20:26:23  <gmaxwell> Why?
457 2018-08-16T20:26:54  <jonasschnelli> gmaxwell: isn't it technically possible that you get a valid length (<MAX_LENGTH) even with an invalid key?
458 2018-08-16T20:27:02  *** d9b4bef9 has quit IRC
459 2018-08-16T20:27:57  <gmaxwell> so? if your stream is corrupted you'll dsync, fail auth, and disconnect.
460 2018-08-16T20:28:15  *** d9b4bef9 has joined #bitcoin-core-dev
461 2018-08-16T20:28:50  *** masonicboom has quit IRC
462 2018-08-16T20:28:50  *** BashCo has quit IRC
463 2018-08-16T20:29:01  *** d9b4bef9 has quit IRC
464 2018-08-16T20:30:08  *** d9b4bef9 has joined #bitcoin-core-dev
465 2018-08-16T20:30:28  *** masonicboom has joined #bitcoin-core-dev
466 2018-08-16T20:30:39  *** prod_ has joined #bitcoin-core-dev
467 2018-08-16T20:30:40  <jonasschnelli> gmaxwell: is the probability very of an valid length with an unexpected key that low that a reconnect would be an acceptable workaround?
468 2018-08-16T20:30:41  <jonasschnelli> *very low
469 2018-08-16T20:31:02  *** d9b4bef9 has quit IRC
470 2018-08-16T20:31:07  <gmaxwell> I think we're probably talking past each other.
471 2018-08-16T20:31:15  <jonasschnelli> heh...
472 2018-08-16T20:31:31  *** BashCo has joined #bitcoin-core-dev
473 2018-08-16T20:31:58  <gmaxwell> what I'm trying to suggest is that to signal rekeying, under the old key, you send a specific length value which we would otherwise never use (due to it being out of range)
474 2018-08-16T20:32:08  *** d9b4bef9 has joined #bitcoin-core-dev
475 2018-08-16T20:32:14  <jonasschnelli> Assume when I rekey in case of decrypting to an invalid length... i guess there is a probability that the decrypted length with the now invalid key is within the MAX_SIZE boundary.... right?
476 2018-08-16T20:32:33  <gmaxwell> No.
477 2018-08-16T20:32:41  <jonasschnelli> aha! I see
478 2018-08-16T20:33:01  <jonasschnelli> Wait..
479 2018-08-16T20:33:01  *** d9b4bef9 has quit IRC
480 2018-08-16T20:33:05  <gmaxwell> You don't decrypt the same data again. You see an invalid length, you rekey and throw out that length and continue.
481 2018-08-16T20:34:00  <jonasschnelli> So... peer A asking for a rekey by setting an invalid length encrypted under the old key, then next message will use the new key?`
482 2018-08-16T20:34:08  *** d9b4bef9 has joined #bitcoin-core-dev
483 2018-08-16T20:34:16  <gmaxwell> Yes.
484 2018-08-16T20:34:28  <jonasschnelli> So the message with the invalid length is a dummy, right?
485 2018-08-16T20:34:31  <gmaxwell> yes.
486 2018-08-16T20:34:48  <gmaxwell> I dunno if thats easier than just handling the rekey message at the decryption layer.
487 2018-08-16T20:34:52  <jonasschnelli> Could the dummy message be a rekey message. :)
488 2018-08-16T20:35:01  <gmaxwell> I was just presenting it as an alternative. :)
489 2018-08-16T20:35:28  <gmaxwell> sure. But it could also, for example be an empty message (length 0) which maybe is easier structurally to handle.
490 2018-08-16T20:35:29  <jonasschnelli> Yes. I like the invalid length approach since it doesn't require the parsing message logic
491 2018-08-16T20:36:14  *** BashCo has quit IRC
492 2018-08-16T20:36:25  <jonasschnelli> Length 0 would be an option and would not be confused with a real invalid decryption
493 2018-08-16T20:36:34  <jonasschnelli> But length 0 would also be prone to DPI I guess
494 2018-08-16T20:36:43  <jonasschnelli> Since it would be the smalles package always
495 2018-08-16T20:36:53  <jonasschnelli> (could artificial blow up though)
496 2018-08-16T20:37:14  <sipa> sdaftuar: my one line summary: we have a way to compute a 'sketch' from a set of N bit elements, with a size of M*N (so equal in size to M elements), in such a way that you can recover the contents from a sketch as long as there aren't more than M elements in it. Now, XORing two sketches gives you a sketch of the set of elements that are in one of the two input sets (but not both)
497 2018-08-16T20:37:17  <gmaxwell> this protocol doesn't resist traffic analysis.
498 2018-08-16T20:37:35  <jonasschnelli> Yes... right,..
499 2018-08-16T20:38:04  <jonasschnelli> But with the 32byte pure pubkey handshake and the encrypted length, we are pretty stealth and make lives a bit harder for DPI configurators
500 2018-08-16T20:38:21  <gmaxwell> jonasschnelli: I don't see how the length0 thing changes anything.
501 2018-08-16T20:38:41  <gmaxwell> the length is still encrypted.
502 2018-08-16T20:38:48  <jonasschnelli> I think the length 0 package could still contain a payload of the size of an inv
503 2018-08-16T20:39:47  *** vexbuy_ has joined #bitcoin-core-dev
504 2018-08-16T20:39:54  <jonasschnelli> Yes. I just though the size of that package (if someone analyses package burst) a rekey if the payload would be length 0 would be easy to identify ... but probably so other commands.
505 2018-08-16T20:40:09  <jonasschnelli> *with
506 2018-08-16T20:40:54  <jonasschnelli> But I guess my point is weak... length 0 seems to be the most advance idea how to tigger a rekey on the encryption layer
507 2018-08-16T20:40:56  <gmaxwell> Sending a minimum length (just len and auth tag) message is no more or less identifable than any other size. If socket handling merges multiple messages, they're not identifyable at all, and if socket handling splits every message, the lengths are implicitly visible.
508 2018-08-16T20:41:23  <gmaxwell> the only downside I see to length 0 is that we might otherwise want to use them for keepalives. :)
509 2018-08-16T20:41:45  <gmaxwell> (which you might want to send more often than once per ten minutes)
510 2018-08-16T20:41:56  <gmaxwell> but I guess we ping at the protocol level, so nevermind. :P
511 2018-08-16T20:42:18  <jonasschnelli> We could also use MAX_INT32 for the rekey
512 2018-08-16T20:42:57  <gmaxwell> I forget how the length is encoded. Is it encoded with a variable length encoding?
513 2018-08-16T20:43:19  *** vexbuy has quit IRC
514 2018-08-16T20:44:17  *** masonicboom has quit IRC
515 2018-08-16T20:44:39  <gmaxwell> sipa, sdaftuar: An astute reader might notice that a set of M  N bit elements can be communicated in log2(2^N choose M) bits. so this scheme is not quite perfectly efficient, but it's close.
516 2018-08-16T20:45:15  <jonasschnelli> gmaxwell: length is a fix 4 byte uint32
517 2018-08-16T20:45:32  <jonasschnelli> encrypted with a key only used for chacha20-crypt the length
518 2018-08-16T20:46:29  <gmaxwell> jonasschnelli: oh, okay, for some reason I thought we had something with lower overhead there. I guess it doesn't matter much since most of our messages are fairly long.
519 2018-08-16T20:47:03  <gmaxwell> jonasschnelli: indeed MAX_UINT32 could just be a length 0 message that triggers rekey.
520 2018-08-16T20:47:30  *** masonicboom has joined #bitcoin-core-dev
521 2018-08-16T20:47:30  <sdaftuar> sipa gmaxwell: what's the failure mode/fallback scenario if a sketch can't be reconciled?
522 2018-08-16T20:47:32  *** Victorsueca has quit IRC
523 2018-08-16T20:48:18  <gmaxwell> sdaftuar: Sketch data can be incrementally sent. so if M wasn't enough I can just send you one more value and now you have M+1.
524 2018-08-16T20:48:45  *** Victorsueca has joined #bitcoin-core-dev
525 2018-08-16T20:48:47  <sdaftuar> oh neat
526 2018-08-16T20:49:06  <sipa> it can even be incrementally computed
527 2018-08-16T20:49:07  <sdaftuar> does that require saving the old sketch?
528 2018-08-16T20:49:11  <sdaftuar> oh
529 2018-08-16T20:49:26  <sipa> (but computing a sketch is very cheap, recovering data form one is expensive)
530 2018-08-16T20:49:38  <jonasschnelli> gmaxwell: ideally, there would be a way to flag the rekey in a non-extra message,.. so flag in the first message using the new key
531 2018-08-16T20:49:48  <gmaxwell> In practice we'll have a limit on the maximum M, both for memory reasons storing the sketches and computation (decode is quadratic). ... so if we exceeded that, you'd just fail to relay those transactions since the last reconcile with that peer, better luck next time around.
532 2018-08-16T20:49:59  <jonasschnelli> (to avoid accessing the push message logic from with the encryption logic)
533 2018-08-16T20:50:53  <gmaxwell> sdaftuar: The sketch decode has two steps, the first step is perfectly incremental. So we waste no cpu getting the sketch data a little at a time.  The second step is not incremental, though we can arrange things so that we can be pretty sure if it'll be successful or not before starting it.
534 2018-08-16T20:51:43  <gmaxwell> jonasschnelli: well just steal the most significant bit of the length.
535 2018-08-16T20:51:56  <gmaxwell> 2^31 byte messages should be enough for anyone.
536 2018-08-16T20:52:09  <jonasschnelli> But megablocks!
537 2018-08-16T20:52:16  <jonasschnelli> Yes. Great idea
538 2018-08-16T20:52:23  <gmaxwell> 2^31 is still 2GB. :P
539 2018-08-16T20:52:59  * jonasschnelli got called to bed
540 2018-08-16T20:53:04  <gmaxwell> night
541 2018-08-16T20:56:20  <gmaxwell> sdaftuar: the actual construction of this system is a binary BCH code, with a message 2^n bits long which has up to M 'errors' (the set difference) in it that we need to correct.  So obviously, we should call it something like BCH faulting transaction relay. :P
542 2018-08-16T20:58:06  <sdaftuar> just bch relay sounds nice and concise
543 2018-08-16T20:59:58  * warren blinked hard upon reading BCH
544 2018-08-16T21:00:57  <sdaftuar> anyway that sounds awesome, i assume it's close enough that we can hope to have it for 0.18?
545 2018-08-16T21:01:54  <gmaxwell> I don't see why it couldn't be done in a couple months, assuming that as we implement we don't get stuck on stupid protocol issues.
546 2018-08-16T21:02:39  <gmaxwell> or fall too far down the number theory rathole of microoptimizing the set reconcillation.
547 2018-08-16T21:02:45  <sdaftuar> :)
548 2018-08-16T21:02:51  *** Chris_Stewart_5 has quit IRC
549 2018-08-16T21:02:59  <sipa> sdaftuar: the algorithm is based on a project called 'pinsketch', which relies on a library called NTL, which is LGPL and a lot of code
550 2018-08-16T21:03:03  <gmaxwell> it's really quite a lovely problem to work on. :)
551 2018-08-16T21:03:21  <sipa> sdaftuar: so we reimplemented inlt from scratch in a few hundred lines, and it's faster too :)
552 2018-08-16T21:03:27  <sdaftuar> yeah i look forward to being nerdsniped when you share your results
553 2018-08-16T21:03:32  <sdaftuar> sipa: lol
554 2018-08-16T21:03:38  <gmaxwell> and came up with a bunch of moderately smart optimizations...
555 2018-08-16T21:04:17  <sdaftuar> i was about to groan about dealing with a 3rd party library and lgpl, i should have known better than to think you'd let that be a problem :)
556 2018-08-16T21:05:26  <gmaxwell> sdaftuar: that was what held me off from doing more of this a year ago.
557 2018-08-16T21:13:25  *** thebiglq12345 has quit IRC
558 2018-08-16T21:21:20  *** vexbuy_ has quit IRC
559 2018-08-16T21:23:39  <sipa> sdaftuar: curiously, the author of pinsketch is someone i've met at real world crypto; he did a talk on UTXO commitment data structures
560 2018-08-16T21:46:52  *** jhfrontz has quit IRC
561 2018-08-16T21:59:43  *** Aaronvan_ has joined #bitcoin-core-dev
562 2018-08-16T22:02:10  *** AaronvanW has quit IRC
563 2018-08-16T22:03:14  *** Chris_Stewart_5 has joined #bitcoin-core-dev
564 2018-08-16T22:08:57  *** jhfrontz has joined #bitcoin-core-dev
565 2018-08-16T22:20:27  *** jhfrontz has quit IRC
566 2018-08-16T22:27:01  *** jhfrontz has joined #bitcoin-core-dev
567 2018-08-16T22:32:13  *** jhfrontz has quit IRC
568 2018-08-16T22:32:36  *** jhfrontz has joined #bitcoin-core-dev
569 2018-08-16T22:38:08  *** jhfrontz has quit IRC
570 2018-08-16T22:38:46  *** jhfrontz has joined #bitcoin-core-dev
571 2018-08-16T22:46:16  *** michaelsdunn1 has quit IRC
572 2018-08-16T22:48:27  *** grafcaps has quit IRC
573 2018-08-16T22:58:33  *** jhfrontz has quit IRC
574 2018-08-16T23:04:28  *** Victorsueca has quit IRC
575 2018-08-16T23:05:51  *** Victorsueca has joined #bitcoin-core-dev
576 2018-08-16T23:07:21  *** esotericnonsense has quit IRC
577 2018-08-16T23:07:24  <luke-jr> achow101: MarcoFalke: that VMBuilder fork doesn't actually work :<
578 2018-08-16T23:07:32  <luke-jr> at least not for bionic
579 2018-08-16T23:08:23  *** Emcy has quit IRC
580 2018-08-16T23:08:59  *** Emcy has joined #bitcoin-core-dev
581 2018-08-16T23:09:00  *** esotericnonsense has joined #bitcoin-core-dev
582 2018-08-16T23:09:23  *** Guyver2 has quit IRC
583 2018-08-16T23:13:42  *** plankers has joined #bitcoin-core-dev
584 2018-08-16T23:20:52  *** kevink has joined #bitcoin-core-dev
585 2018-08-16T23:24:35  <kevink> I'm using Bitcoin Core's API and am wondering whats the difference between `duplicate-inconclusive` and `duplicate` when calling `submitblock`. I'm just trying to verify that a block exists in the blockchain and `submitblock` seems like the simplest way to do that.
586 2018-08-16T23:26:05  *** promag has joined #bitcoin-core-dev
587 2018-08-16T23:27:50  <sipa> kevink: try getblockheader instead
588 2018-08-16T23:29:29  <phantomcircuit> kevink, submit block is definitely not the way to do that, getblockheader is
589 2018-08-16T23:29:32  *** Emcy has quit IRC
590 2018-08-16T23:31:14  *** promag has quit IRC
591 2018-08-16T23:36:15  <kevink> The reason I wanted to use submitblock was because I'm encoding and decoding the blockdata into an image using https://gist.github.com/laanwj/51f276c44ba9882bb4b27cc6f3a499a4 and wanted to check that it also remained a valid block after being decoded.
592 2018-08-16T23:44:19  <sipa> compute its hash, and query for it
593 2018-08-16T23:48:30  <gmaxwell> I guess we don't have a decoderawblock rpc that would construct the hash for you!
594 2018-08-16T23:49:40  *** viaj3ro has quit IRC
595 2018-08-16T23:54:57  *** justanotheruser has quit IRC