12018-08-16T00:14:49 *** AaronvanW has quit IRC
22018-08-16T00:19:14 *** masonicboom has quit IRC
32018-08-16T00:20:07 *** masonicboom has joined #bitcoin-core-dev
42018-08-16T00:22:34 *** grubles has quit IRC
52018-08-16T00:31:38 *** grubles has joined #bitcoin-core-dev
62018-08-16T00:42:34 *** jhfrontz has joined #bitcoin-core-dev
72018-08-16T00:53:45 *** grubles has quit IRC
82018-08-16T00:54:01 *** grubles has joined #bitcoin-core-dev
92018-08-16T00:55:38 *** roasbeef has quit IRC
102018-08-16T00:57:18 *** masonicboom has quit IRC
112018-08-16T01:06:01 *** d9b4bef9 has quit IRC
122018-08-16T01:07:07 *** d9b4bef9 has joined #bitcoin-core-dev
132018-08-16T01:09:46 *** Chris_Stewart_5 has quit IRC
142018-08-16T01:10:28 <gmaxwell> jonasschnelli: maybe when you post a patch for the encryption, I'll go and quickly staple newhope to it and we see how we feel about the combination.
152018-08-16T01:22:26 *** roasbeef has joined #bitcoin-core-dev
162018-08-16T01:26:21 *** Chris_Stewart_5 has joined #bitcoin-core-dev
172018-08-16T01:59:55 *** thaumavorio has quit IRC
182018-08-16T02:02:09 *** goatpig has quit IRC
192018-08-16T02:07:54 *** Giszmo has quit IRC
202018-08-16T02:32:58 *** Chris_Stewart_5 has quit IRC
212018-08-16T02:46:54 *** reallll has joined #bitcoin-core-dev
222018-08-16T02:51:02 *** belcher_ has quit IRC
232018-08-16T02:58:05 *** reallll has quit IRC
242018-08-16T02:58:32 *** reallll has joined #bitcoin-core-dev
252018-08-16T03:02:26 *** rls has joined #bitcoin-core-dev
262018-08-16T03:27:41 *** justan0theruser is now known as justanotheruser
272018-08-16T03:32:45 *** Randolf has joined #bitcoin-core-dev
282018-08-16T04:04:19 *** Krellan has quit IRC
292018-08-16T04:05:27 *** Krellan has joined #bitcoin-core-dev
302018-08-16T04:19:40 *** vexbuy has quit IRC
312018-08-16T04:20:02 *** vexbuy has joined #bitcoin-core-dev
322018-08-16T04:20:44 *** vexbuy has joined #bitcoin-core-dev
332018-08-16T04:21:15 *** vexbuy has quit IRC
342018-08-16T04:21:34 *** vexbuy has joined #bitcoin-core-dev
352018-08-16T04:22:22 *** vexbuy has joined #bitcoin-core-dev
362018-08-16T04:23:04 *** vexbuy has joined #bitcoin-core-dev
372018-08-16T04:23:35 *** vexbuy has quit IRC
382018-08-16T04:23:54 *** vexbuy has joined #bitcoin-core-dev
392018-08-16T04:36:01 *** minskey has joined #bitcoin-core-dev
402018-08-16T04:55:02 *** d9b4bef9 has quit IRC
412018-08-16T04:56:17 *** d9b4bef9 has joined #bitcoin-core-dev
422018-08-16T05:28:36 *** AaronvanW has joined #bitcoin-core-dev
432018-08-16T05:33:49 *** AaronvanW has quit IRC
442018-08-16T05:47:31 *** Victorsueca has quit IRC
452018-08-16T05:48:45 *** Victorsueca has joined #bitcoin-core-dev
462018-08-16T06:00:28 *** Krellan has quit IRC
472018-08-16T06:05:51 *** Krellan has joined #bitcoin-core-dev
482018-08-16T06:19:05 *** ken2812221_ has joined #bitcoin-core-dev
492018-08-16T06:21:57 *** ken2812221 has quit IRC
502018-08-16T06:25:08 *** murrayn has quit IRC
512018-08-16T06:29:55 <jonasschnelli> gmaxwell: Sure. My current priority is encryption, then auth, then NH/PQ. I can also try the NH stuff earlier..
522018-08-16T06:47:50 *** face has quit IRC
532018-08-16T06:59:26 *** vexbuy has joined #bitcoin-core-dev
542018-08-16T07:06:25 *** plankers has joined #bitcoin-core-dev
552018-08-16T07:18:47 *** Guyver2 has joined #bitcoin-core-dev
562018-08-16T07:26:29 *** da2ce7 has quit IRC
572018-08-16T07:28:49 *** da2ce7 has joined #bitcoin-core-dev
582018-08-16T07:49:36 <gmaxwell> sipa: 13989 is doing avx512 sha256d64, and gets a 19% speedup for the MerkleRoot benchmark.
592018-08-16T07:57:08 *** SopaXorzTaker has joined #bitcoin-core-dev
602018-08-16T08:11:14 *** kewde[m] has quit IRC
612018-08-16T08:11:19 *** joshb[m] has quit IRC
622018-08-16T08:11:22 *** squarfed[m] has quit IRC
632018-08-16T08:17:30 *** savil has joined #bitcoin-core-dev
642018-08-16T08:19:00 *** vexbuy has quit IRC
652018-08-16T08:20:19 *** Guyver2 has quit IRC
662018-08-16T08:25:33 <wumpus> time to tag rc1?
672018-08-16T08:27:39 <wumpus> I'm sure enough issues will come up during gitian building (although I was able to do so succesfully) for the first time with bionic, but I think it makes sense to start testing?
682018-08-16T08:33:33 *** promag has joined #bitcoin-core-dev
692018-08-16T08:34:01 *** csknk has joined #bitcoin-core-dev
702018-08-16T08:37:57 *** reallll has quit IRC
712018-08-16T08:38:33 *** belcher_ has joined #bitcoin-core-dev
722018-08-16T08:47:18 *** ExtraCrispy has joined #bitcoin-core-dev
732018-08-16T08:56:09 *** murrayn has joined #bitcoin-core-dev
742018-08-16T08:57:01 *** d9b4bef9 has quit IRC
752018-08-16T08:58:08 *** d9b4bef9 has joined #bitcoin-core-dev
762018-08-16T09:05:53 *** promag has quit IRC
772018-08-16T09:40:03 *** e4xit has quit IRC
782018-08-16T09:42:01 *** Jmabsd has quit IRC
792018-08-16T09:44:45 *** fanquake has joined #bitcoin-core-dev
802018-08-16T09:44:52 <fanquake> wumpus sounds good
812018-08-16T09:48:39 <wumpus> ok! let's do it then
822018-08-16T09:56:31 *** AaronvanW has joined #bitcoin-core-dev
832018-08-16T09:56:34 <wumpus> yesterday no one protested either so...
842018-08-16T09:56:58 *** dcousens has quit IRC
852018-08-16T09:57:44 *** dcousens has joined #bitcoin-core-dev
862018-08-16T09:58:24 <fanquake> Just take that as silent optimism/agreement or something
872018-08-16T10:01:14 *** plankers has quit IRC
882018-08-16T10:02:52 <wumpus> I'm just trying to avoid forgetting a dumb step (yes, version has been bumped)
892018-08-16T10:08:15 <wumpus> * [new tag] v0.17.0rc1 -> v0.17.0rc1
902018-08-16T10:10:04 <wumpus> there we go
912018-08-16T10:11:22 <fanquake> That's what rc's are for anyways. Wouldn't be the first time we've fixed something up quickly with an rc2 etc.
922018-08-16T10:11:24 <fanquake> wew!
932018-08-16T10:13:30 *** e4xit has joined #bitcoin-core-dev
942018-08-16T11:08:32 *** vexbuy has joined #bitcoin-core-dev
952018-08-16T11:16:53 *** rex4539 has joined #bitcoin-core-dev
962018-08-16T11:17:15 *** Chris_Stewart_5 has joined #bitcoin-core-dev
972018-08-16T11:36:59 *** vexbuy has quit IRC
982018-08-16T11:38:42 *** vexbuy has joined #bitcoin-core-dev
992018-08-16T11:53:48 <wumpus> 0.17.0rc1 gitian sigs (unsigned) up, wonder if anyone reproduces
1002018-08-16T11:57:21 <jonasschnelli> wumpus: still building: https://bitcoin.jonasschnelli.ch/build/743
1012018-08-16T11:58:46 <jonasschnelli> windows and osx match though... linux: will see
1022018-08-16T12:06:32 *** no_input_found has quit IRC
1032018-08-16T12:06:52 *** no_input_found has joined #bitcoin-core-dev
1042018-08-16T12:14:04 <wumpus> awesome !
1052018-08-16T12:18:27 *** Chris_Stewart_5 has quit IRC
1062018-08-16T12:22:38 *** goatpig has joined #bitcoin-core-dev
1072018-08-16T12:25:02 *** d9b4bef9 has quit IRC
1082018-08-16T12:25:07 <jonasschnelli> linux is a match as well
1092018-08-16T12:26:05 *** Krellan has quit IRC
1102018-08-16T12:26:14 *** d9b4bef9 has joined #bitcoin-core-dev
1112018-08-16T12:27:17 *** Krellan has joined #bitcoin-core-dev
1122018-08-16T12:58:36 <jonasschnelli> Benchmarks for v2 message format composing (encrypted):
1132018-08-16T12:59:07 <jonasschnelli> 100 blocks: V1 legacy (dblSHA): 1.43978, V2 (ChaCha20/Poly1305): 1.42594
1142018-08-16T12:59:33 <jonasschnelli> (and this is with SHA SSE4.1/AVX versus non NI acceletarted chacha)
1152018-08-16T13:20:38 *** Chris_Stewart_5 has joined #bitcoin-core-dev
1162018-08-16T13:22:33 <fanquake> thanks for the quick build ken2812221, more matching sigs
1172018-08-16T13:28:01 *** ken2812221_ is now known as ken2812221
1182018-08-16T13:28:10 <ken2812221> fanquake: nice
1192018-08-16T13:32:33 *** arubi has quit IRC
1202018-08-16T13:32:55 *** arubi has joined #bitcoin-core-dev
1212018-08-16T13:34:15 *** vexbuy has quit IRC
1222018-08-16T13:34:53 *** vexbuy has joined #bitcoin-core-dev
1232018-08-16T13:36:15 <wumpus> jonasschnelli: nice
1242018-08-16T13:36:31 *** asoltys has quit IRC
1252018-08-16T13:39:43 *** vexbuy has quit IRC
1262018-08-16T13:48:27 *** fanquake has quit IRC
1272018-08-16T13:58:39 *** vexbuy has joined #bitcoin-core-dev
1282018-08-16T14:13:00 *** promag has joined #bitcoin-core-dev
1292018-08-16T14:20:23 *** michaelsdunn1 has joined #bitcoin-core-dev
1302018-08-16T14:22:58 <promag> with 0.17 branched, #13529 can be reviewed
1312018-08-16T14:23:01 <gribble> https://github.com/bitcoin/bitcoin/issues/13529 | Use new Qt5 connect syntax by promag · Pull Request #13529 · bitcoin/bitcoin · GitHub
1322018-08-16T14:23:08 <promag> wumpus: ^
1332018-08-16T14:23:20 <promag> maybe I should push a rebased commit?
1342018-08-16T14:25:03 *** promag has quit IRC
1352018-08-16T14:25:04 *** michaelsdunn1 has quit IRC
1362018-08-16T14:35:34 *** michaelsdunn1 has joined #bitcoin-core-dev
1372018-08-16T14:37:41 *** promag has joined #bitcoin-core-dev
1382018-08-16T14:38:43 <wumpus> promag: will review
1392018-08-16T14:39:30 <wumpus> looks like you first need to address the issues, apparently you did break some things!
1402018-08-16T15:05:03 *** vexbuy has quit IRC
1412018-08-16T15:05:41 *** vexbuy has joined #bitcoin-core-dev
1422018-08-16T15:09:09 *** vexbuy_ has joined #bitcoin-core-dev
1432018-08-16T15:10:05 *** vexbuy has quit IRC
1442018-08-16T15:11:21 <promag> wumpus: yeah I saw that after writing the above :(
1452018-08-16T15:15:00 *** csknk has quit IRC
1462018-08-16T15:15:28 *** Dizzle has joined #bitcoin-core-dev
1472018-08-16T15:33:49 <gmaxwell> jonasschnelli: well the reason I was offering to try sticking on newhope is that I _think_ I can add it in with only a few lines code change. If we're of the view that we'll want it long term, then maybe it'll make sense to do up front if it really does turn out to be that simple.
1482018-08-16T15:35:15 <jonasschnelli> gmaxwell: Yes. Agree. I just too deep into implementing the ecdh/chachapoly stuff right now... but as soon as some tests are finished, I'm happy to try the NH implementation.
1492018-08-16T15:35:36 <jonasschnelli> I need to research (or ask you) a bit more, though
1502018-08-16T15:35:42 <gmaxwell> OKAY but thats why I was offering to help. :P
1512018-08-16T15:35:57 <gmaxwell> Sure. Fortunately it's really simple.
1522018-08-16T15:36:10 <jonasschnelli> What implementation are you looking at?
1532018-08-16T15:37:27 <jonasschnelli> Doesn't it also require SHA3?
1542018-08-16T15:37:31 <gmaxwell> https://github.com/newhopecrypto/newhope-usenix/tree/master/ref
1552018-08-16T15:37:33 <gmaxwell> No.
1562018-08-16T15:37:50 <jonasschnelli> ok
1572018-08-16T15:37:53 <gmaxwell> it uses chacha20 internally just as a random number generator.
1582018-08-16T15:38:25 <gmaxwell> oh actually there is a sha3 impl there too. I guess it's using that to hash the final state.
1592018-08-16T15:38:32 <jonasschnelli> gmaxwell: replace that with sha2? -> https://github.com/newhopecrypto/newhope-usenix/blob/master/ref/newhope.c#L58?
1602018-08-16T15:39:41 <gmaxwell> yes, we can just replace that with sha2.
1612018-08-16T15:49:39 *** SopaXorzTaker has quit IRC
1622018-08-16T15:50:41 *** SopaXorzTaker has joined #bitcoin-core-dev
1632018-08-16T15:51:12 <gmaxwell> You see how it works though? https://github.com/newhopecrypto/newhope-usenix/blob/master/ref/test/test_newhope.c#L36 Initator runs newhope_keygen and sticks senda on their message, responder runs newhope_sharedb with that as an argument and gets the shared keyb and sendb to send, intiator gets sendb and feeds it to newhope_shareda and gets the same shared secret out.
1642018-08-16T15:53:41 <gmaxwell> The messages are constant length (1824 and 2048 bytes IIRC).
1652018-08-16T15:55:47 <gmaxwell> And we'd probably go and change the API slightly so that it takes randomness as an argument rather than reading /dev/urandom itself.
1662018-08-16T15:57:51 <gmaxwell> though for an intial test that could be ignored.
1672018-08-16T16:01:08 *** masonicboom has joined #bitcoin-core-dev
1682018-08-16T16:06:39 *** ExtraCrispy has quit IRC
1692018-08-16T16:08:55 <jonasschnelli> gmaxwell: so it basically works the same as DH (in terms of network interactions)?
1702018-08-16T16:09:28 <jonasschnelli> Could we append send_a (the message) to the ecdh-32byte-pubkey?
1712018-08-16T16:10:03 <jonasschnelli> I guess the NH handshake doesn't have to follow under the ECDH encrypted channel?
1722018-08-16T16:12:33 *** jhfrontz has quit IRC
1732018-08-16T16:13:04 <gmaxwell> Correct on all counts.
1742018-08-16T16:13:54 <gmaxwell> It's not _exactly_ the same as DH in terms of network interactions because in DH both alice and bob can send their pubkeys at the same time, but in newhope alice sends then bob sends. But we're not using DH that way anways.
1752018-08-16T16:14:26 <gmaxwell> So we can implement it just by appending send_a to the initator pubkey, and send_b to the responder pubkey.
1762018-08-16T16:15:02 <gmaxwell> then we just hash the secrets that come out of both DH and newhope.
1772018-08-16T16:16:06 *** LeMiner has quit IRC
1782018-08-16T16:17:34 <jonasschnelli> gmaxwell: I see. Whats the length of the newhope secret output? 32b?
1792018-08-16T16:21:26 <gmaxwell> 32 bytes, yes.
1802018-08-16T16:23:49 <jonasschnelli> Okay. From a protocol level, this seems easy.
1812018-08-16T16:24:12 <jonasschnelli> Are the NH messagedsidentifiable (DPIish)?
1822018-08-16T16:25:55 <gmaxwell> No, not more than the secp256k1 public keys.
1832018-08-16T16:26:15 <jonasschnelli> ok. then ECDH doesn't need to go first
1842018-08-16T16:27:02 *** d9b4bef9 has quit IRC
1852018-08-16T16:28:16 *** d9b4bef9 has joined #bitcoin-core-dev
1862018-08-16T16:30:58 *** plankers has joined #bitcoin-core-dev
1872018-08-16T16:32:12 <jonasschnelli> yeah. I don't see a reason why we should not add newhope to the handshake (unless I completely must understand it)
1882018-08-16T16:35:38 <gmaxwell> The only arguments I can make against it are that (1) it might turn out to not add much security (e.g. if newhope gets broken), and then we're stuck carrying it around buring bytes, loc, and cpu cycles on it (2) implementers that insist on writing everything from scratch instead of using public domain C code will have a harder time, (3) more stuff to integrate.
1892018-08-16T16:36:49 <gmaxwell> For other applications, like using it in SSL the size and speed impacts matter more. For us, I think they're basically irrelevant. Newhope is as fast as (or maybe even faster than, with AVX in use) ECDH.
1902018-08-16T16:39:20 <jonasschnelli> I hope my implementation for detecting a version message or a key-handshake will still work since it's now longer then 32b
1912018-08-16T16:40:32 <gmaxwell> You should just be able to read the first N bytes and decide, then keep reading.
1922018-08-16T16:44:08 <jonasschnelli> yeah... avoid pubkeys with netmagic, read 4 bytes and continue with handshake when not equal the net magic
1932018-08-16T16:46:38 <wumpus> RISC-V node (on actual hw) started syncing
1942018-08-16T16:49:29 <sipa> wooho
1952018-08-16T16:49:31 <gmaxwell> wumpus: \O/
1962018-08-16T16:50:15 <gmaxwell> now just submit some patches upstream to add hardware SHA2 and ECC so that the next one will finish in reasonable time. :P
1972018-08-16T16:53:31 <sipa> just add a HW instruction vrfybtcnblk
1982018-08-16T16:53:39 <sipa> wait, what did the R in RISC stand for again?
1992018-08-16T16:53:45 <wumpus> hehe
2002018-08-16T16:54:10 <gmaxwell> well, brainfuck has fewer operations, so clearly RISCV isn't RISC? :P
2012018-08-16T16:54:21 <wumpus> it stands for Rich Instruction Set Computing
2022018-08-16T16:54:25 <sipa> ah.
2032018-08-16T16:54:31 <sipa> gmaxwell: try whitespace
2042018-08-16T16:55:07 <sipa> i guess whitespace has more instructions, but just encoded in 3 characters
2052018-08-16T16:55:11 <gmaxwell> wumpus: as opposed to constrained instruction set computing?
2062018-08-16T16:55:17 *** jhfrontz has joined #bitcoin-core-dev
2072018-08-16T16:56:00 <wumpus> though tbh at this point I'm less worried about performance than about obscure chip and compiler issues
2082018-08-16T16:56:03 <wumpus> gmaxwell: exactly!
2092018-08-16T16:57:08 *** thebiglq12345 has joined #bitcoin-core-dev
2102018-08-16T16:59:29 <gmaxwell> jonasschnelli: in any case, newhope is a member of the class of fast PQ schemes that are newer and more likely to turn out to be insecure against even classical computers. But the only alternatives that aren't have properties that would make us not use them.. e.g. the McEliece public keys are like 1MB.
2112018-08-16T17:14:04 *** Emcy has quit IRC
2122018-08-16T17:33:30 <gmaxwell> jonasschnelli: oh, actually we'd want to be using this implementation https://github.com/newhopecrypto/newhope/tree/master/ref as it's their NIST submission, and has a substantial simplifcation to the protocol innards.
2132018-08-16T18:00:53 <jonasschnelli> ok. thanks... I'll look into it.
2142018-08-16T18:01:08 *** vexbuy has joined #bitcoin-core-dev
2152018-08-16T18:02:08 <wumpus> apparently the MMC interface on the HiFive unleashed is really slow, either due to the current kernel version, or due to some hardware limitation
2162018-08-16T18:04:38 *** vexbuy_ has quit IRC
2172018-08-16T18:04:48 <wumpus> the debian/fedora developers use nbd, though currently I have a 100mbit switch connected for the embedded LAN so that's not going to be great either :-)
2182018-08-16T18:08:02 <wumpus> but all in all I'm quite happy with this, for the first larger-scale ASIC implementation of a new architecture it's cool how far this gets
2192018-08-16T18:08:51 <sipa> the risc-v instrunction encoding is pretty cool
2202018-08-16T18:09:21 <sipa> everything is natively a 32 bit instruction, but there are optional extensions that compress certain instructions down
2212018-08-16T18:09:22 <wumpus> yes how they do variable-length is interesting
2222018-08-16T18:09:43 <sipa> but that compression can be implemented as a pure postprocessing step in the assembler
2232018-08-16T18:10:55 <wumpus> isa : rv64imafdc I think -c is the 16-bit compressed instruction extension, so it has that
2242018-08-16T18:11:17 *** ken2812221 has quit IRC
2252018-08-16T18:11:48 *** Randolf has quit IRC
2262018-08-16T18:12:02 *** Krellan has quit IRC
2272018-08-16T18:12:05 <wumpus> M=Integer
2282018-08-16T18:13:26 <wumpus> Multiplication and Division, A=Atomic instructions, F=32-bit fp, D=64-bit fp, C=Compressed instructions
2292018-08-16T18:13:28 *** Krellan has joined #bitcoin-core-dev
2302018-08-16T18:13:36 <sipa> no B=bit manipulation?
2312018-08-16T18:13:50 <wumpus> apparently not!
2322018-08-16T18:14:37 <wumpus> "This chapter is a placeholder for a future standard extension to provide bit manipulation instruc-
2332018-08-16T18:15:05 <sipa> ah
2342018-08-16T18:15:07 <wumpus> the version of the spec I have calls it future, at least
2352018-08-16T18:15:15 <sipa> i guess basic bit manipulation is available in the base set of instructions
2362018-08-16T18:16:53 <wumpus> yes: ANDI/ORI/XORI
2372018-08-16T18:17:47 <wumpus> they're in the mandatory part
2382018-08-16T18:18:41 <wumpus> I guess the idea of -B will be to provide a more extensive set, say for direct injection/extraction of bit sequences, popcount, count leading/trailing bits, etc
2392018-08-16T18:21:20 *** plankers has quit IRC
2402018-08-16T18:22:12 *** Emcy has joined #bitcoin-core-dev
2412018-08-16T18:22:36 *** Victorsueca has quit IRC
2422018-08-16T18:23:45 *** Victorsueca has joined #bitcoin-core-dev
2432018-08-16T18:35:15 *** grafcaps has joined #bitcoin-core-dev
2442018-08-16T18:35:21 *** promag has quit IRC
2452018-08-16T18:35:54 *** promag has joined #bitcoin-core-dev
2462018-08-16T18:39:47 *** promag has quit IRC
2472018-08-16T18:42:21 <jonasschnelli> gmaxwell: I guess a configuration option that would allow ecdh only handshake would make little sense?
2482018-08-16T18:42:48 <jonasschnelli> I down side with newhope could be the implementation burden if we also want SPV clients to adopt it.
2492018-08-16T18:44:55 *** davex__ has left #bitcoin-core-dev
2502018-08-16T18:45:05 *** davex__ has quit IRC
2512018-08-16T18:45:08 *** Krellan has quit IRC
2522018-08-16T18:45:12 <gmaxwell> jonasschnelli: I don't think it would make sense to make it optional.
2532018-08-16T18:45:34 *** davex__ has joined #bitcoin-core-dev
2542018-08-16T18:46:25 <gmaxwell> jonasschnelli: there are, for example, java implementations and whatnot.
2552018-08-16T18:46:41 *** jtimon has joined #bitcoin-core-dev
2562018-08-16T18:47:57 *** thaumavorio has joined #bitcoin-core-dev
2572018-08-16T18:48:34 *** promag has joined #bitcoin-core-dev
2582018-08-16T18:50:22 *** dgenr8 has joined #bitcoin-core-dev
2592018-08-16T19:01:06 <promag> meeting?
2602018-08-16T19:01:18 * luke-jr pokes wumpus
2612018-08-16T19:03:23 *** clarkmoody has joined #bitcoin-core-dev
2622018-08-16T19:03:48 <wumpus> hello
2632018-08-16T19:03:51 <sdaftuar> hi
2642018-08-16T19:03:54 <wumpus> #startmeeting
2652018-08-16T19:03:54 <lightningbot> Meeting started Thu Aug 16 19:03:54 2018 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
2662018-08-16T19:03:54 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
2672018-08-16T19:03:56 <jonasschnelli> Yeah... hi
2682018-08-16T19:04:03 <meshcollider> hi
2692018-08-16T19:04:07 <promag> hi
2702018-08-16T19:04:12 <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark mi
2712018-08-16T19:04:16 <wumpus> chagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
2722018-08-16T19:04:24 <gmaxwell> darn, just beat me to it.
2732018-08-16T19:04:29 <achow101> hi
2742018-08-16T19:04:33 <sipa> hi
2752018-08-16T19:04:43 <wumpus> apparently I divided michagogo in two
2762018-08-16T19:04:55 <sdaftuar> ouch
2772018-08-16T19:05:24 <meshcollider> Hope he's not too cut up about it
2782018-08-16T19:05:25 <wumpus> PSA: we've tagged 0.17.0rc1, let us know if you have any trouble gitian-building due to the upgrade of the guest to Ubuntu 18.04/bionic
2792018-08-16T19:05:53 *** ken2812221 has joined #bitcoin-core-dev
2802018-08-16T19:06:31 <jonasschnelli> As mentioned its a bit sad that we dropped debian 9 (newest version) as gitian host,... but apparently compiling lxc was simpler then expected
2812018-08-16T19:06:36 <wumpus> now that 0.17 release cycle is started, it might be time to bring back "high priority for review" as a topic
2822018-08-16T19:06:50 *** clarkmoody has quit IRC
2832018-08-16T19:07:10 <wumpus> I have some instructions for debian 9 here: https://gist.github.com/laanwj/c62e101bfd68718f0686926dfd10666b
2842018-08-16T19:07:27 <jtimon> hi
2852018-08-16T19:07:29 <wumpus> (yes, you need to build lxc and debootstrap from source)
2862018-08-16T19:07:32 <jonasschnelli> Nice wumpus
2872018-08-16T19:07:43 <wumpus> we should probably integrate that into the documentation
2882018-08-16T19:08:25 <jonasschnelli> Yes. Until lxc 2.1.1 is supported by debians apt
2892018-08-16T19:08:40 <wumpus> yes, 3.x is probably overkill
2902018-08-16T19:08:59 <luke-jr> wumpus: did we ever get a solution for making a bionic base VM? :/
2912018-08-16T19:08:59 <jonasschnelli> Works fine here with 2.1.1
2922018-08-16T19:09:09 <luke-jr> (for qemu)
2932018-08-16T19:09:17 <achow101> also, with the suite bump, don't forget to do bin/make-base-vm again
2942018-08-16T19:09:29 <wumpus> luke-jr: I don't think so, I think LXC and Docker are the only options at the moment
2952018-08-16T19:09:34 <luke-jr> achow101: last I checked that doesn't work
2962018-08-16T19:09:42 <achow101> luke-jr: there's a fork of vmbuilder that works for bionic
2972018-08-16T19:09:57 *** viaj3ro has joined #bitcoin-core-dev
2982018-08-16T19:09:59 <achow101> luke-jr: https://github.com/newroco/vmbuilder
2992018-08-16T19:10:06 <luke-jr> #link https://github.com/newroco/vmbuilder
3002018-08-16T19:10:09 <wumpus> but the fork might be worth a try !
3012018-08-16T19:10:12 <achow101> luke-jr: see also https://github.com/devrandom/gitian-builder/issues/191
3022018-08-16T19:11:54 <wumpus> anyhow -- please ask in this channel if you have any problems getting gitian running
3032018-08-16T19:11:58 *** clarkmoody has joined #bitcoin-core-dev
3042018-08-16T19:12:04 <wumpus> #topic High priority for review
3052018-08-16T19:12:30 <wumpus> https://github.com/bitcoin/bitcoin/projects/8 there's only one PR in there at the moment, #13100
3062018-08-16T19:12:35 <gribble> https://github.com/bitcoin/bitcoin/issues/13100 | gui: Add dynamic wallets support by promag · Pull Request #13100 · bitcoin/bitcoin · GitHub
3072018-08-16T19:12:51 <sipa> i'd like #13723
3082018-08-16T19:12:53 <gribble> https://github.com/bitcoin/bitcoin/issues/13723 | PSBT key path cleanups by sipa · Pull Request #13723 · bitcoin/bitcoin · GitHub
3092018-08-16T19:13:05 <sipa> (it's the basis for further psbt/descript integration)
3102018-08-16T19:13:24 <instagibbs> https://github.com/bitcoin/bitcoin/pull/13968 bugfixes for psbt stuff too(0.17 backport)
3112018-08-16T19:13:54 <instagibbs> #13968
3122018-08-16T19:13:56 <gribble> https://github.com/bitcoin/bitcoin/issues/13968 | [wallet] couple of walletcreatefundedpsbt fixes by instagibbs · Pull Request #13968 · bitcoin/bitcoin · GitHub
3132018-08-16T19:14:42 <ken2812221> #13866
3142018-08-16T19:14:44 <gribble> https://github.com/bitcoin/bitcoin/issues/13866 | utils: Use _wfopen and _wfreopen on Windows by ken2812221 · Pull Request #13866 · bitcoin/bitcoin · GitHub
3152018-08-16T19:14:48 <wumpus> 13723 added
3162018-08-16T19:15:33 <jtimon> if high priority is still for blockers, https://github.com/bitcoin/bitcoin/pull/13311 is kind of a blocker for https://github.com/bitcoin/bitcoin/pull/8994 which is itself a blocker for toher things I wanted to do
3172018-08-16T19:15:38 <wumpus> the other two as well
3182018-08-16T19:15:55 <wumpus> #13311
3192018-08-16T19:15:57 <gribble> https://github.com/bitcoin/bitcoin/issues/13311 | Dont edit Chainparams after initialization by jtimon · Pull Request #13311 · bitcoin/bitcoin · GitHub
3202018-08-16T19:16:12 <promag> wumpus: can you replace 13100 with #13529?
3212018-08-16T19:16:14 <gribble> https://github.com/bitcoin/bitcoin/issues/13529 | Use new Qt5 connect syntax by promag · Pull Request #13529 · bitcoin/bitcoin · GitHub
3222018-08-16T19:17:05 <wumpus> promag: ok
3232018-08-16T19:17:09 <wumpus> jtimon: added
3242018-08-16T19:17:16 <jtimon> yeah, thanks
3252018-08-16T19:17:20 <achow101> I would like #12493
3262018-08-16T19:17:22 <gribble> https://github.com/bitcoin/bitcoin/issues/12493 | [wallet] Reopen CDBEnv after encryption instead of shutting down by achow101 · Pull Request #12493 · bitcoin/bitcoin · GitHub
3272018-08-16T19:17:41 <promag> ty
3282018-08-16T19:17:56 <wumpus> achow101: ok, yes, probably makes sense to merge that early in the 0.18 cycle
3292018-08-16T19:18:11 <wumpus> achow101: it sounds sort of--risky
3302018-08-16T19:18:14 <gmaxwell> achow101: did we ever work through the problems that were preventing "don't create a wallet until a key is requested or until encryption is added" ?
3312018-08-16T19:18:32 <gmaxwell> As that would also get rid of the shutdown on encryption for many users.
3322018-08-16T19:18:37 <achow101> gmaxwell: the problems with that were backups IIRC
3332018-08-16T19:19:10 <achow101> gmaxwell: actually that problem was with generate keys on use, not create wallet on use
3342018-08-16T19:19:59 <achow101> in that case, I don't think so. but I also don't remember the problems with create wallet on use. I think I just never got around to implementing it
3352018-08-16T19:20:17 <gmaxwell> right. I thought there were some dumb bugs with create on use that were going to get fixed as a side effect of pending multiwallet work.
3362018-08-16T19:20:31 *** ken2812221 has quit IRC
3372018-08-16T19:21:51 <wumpus> any other proposed topics?
3382018-08-16T19:21:52 *** ken2812221 has joined #bitcoin-core-dev
3392018-08-16T19:22:28 *** Guyver2 has joined #bitcoin-core-dev
3402018-08-16T19:23:59 <sipa> short announcement: we're working on an extension to descriptors to support nested and/or/threshold constructions
3412018-08-16T19:25:03 <wumpus> cool!
3422018-08-16T19:25:38 <sdaftuar> topic suggestion: open floor for people to share what they
3432018-08-16T19:25:40 <jonasschnelli> nice
3442018-08-16T19:25:41 <sdaftuar> re wokring on
3452018-08-16T19:26:01 <sdaftuar> (since we don't have any other topics apparently :P)
3462018-08-16T19:26:29 <sipa> i like wok rings
3472018-08-16T19:26:33 <wumpus> #topic open floor for people to share what they are working on
3482018-08-16T19:26:40 <jonasschnelli> Working on p2p level encryption since a couple of weeks (thats why I'm pretty quite on github). Will open PR in 1-2 weeks.
3492018-08-16T19:27:16 <wumpus> sipa: so this is for further improvements to scantxoutset I suppose?
3502018-08-16T19:27:24 <sipa> wumpus: and all the things :)
3512018-08-16T19:27:34 <wumpus> right
3522018-08-16T19:27:48 <sipa> wumpus: so you can import and(xpub/...,or(xpub/...,xpub/...)) into your wallet as watch-only chain for example
3532018-08-16T19:27:53 <achow101> wumpus: hopefully for eventually a replacement-ish thing to the wallet
3542018-08-16T19:27:54 <sipa> and get psbt to sign for it
3552018-08-16T19:28:20 <instagibbs> so excite
3562018-08-16T19:28:22 <wumpus> that's neat
3572018-08-16T19:28:25 <jonasschnelli> sipa: how would/could that lead to xpub watch only wallets?
3582018-08-16T19:28:57 <sipa> jonasschnelli: yes
3592018-08-16T19:29:02 <sipa> that's the goal of the descriptors
3602018-08-16T19:29:45 <instagibbs> achow101, not sure if he was joking, but he said he wouldnt have to implement it if I did the HWI version. I'll keep leaning on him
3612018-08-16T19:29:52 <jonasschnelli> That would be a great feature... could also be extended to the GUI including coin selection (send screen), but don't sign/broadcast (obviously) but will create a PSBT (file)
3622018-08-16T19:30:16 <instagibbs> my guess is for any hope of Core support of HWW, we want to not have to support a bunch of PSBT drivers...
3632018-08-16T19:30:52 <instagibbs> oh sorry wrong window
3642018-08-16T19:30:56 <jonasschnelli> instagibbs: you mean more support then the PSBT?
3652018-08-16T19:31:15 <sipa> jonasschnelli: seems reasonable yes
3662018-08-16T19:31:37 <sipa> i need to think through how to integrate things in the wallet itself, so you can import descriptors
3672018-08-16T19:31:46 <sipa> and how to make it compatible with all existing RPCs etc
3682018-08-16T19:32:16 <gmaxwell> sipa: not just and/or/threshold but also CSV and hashlocks?
3692018-08-16T19:32:22 <instagibbs> gmaxwell, yes
3702018-08-16T19:32:22 <sipa> gmaxwell: oh yes
3712018-08-16T19:32:27 <instagibbs> (oh rhetorical)
3722018-08-16T19:32:32 <sipa> but my goal is that the wallet eventually consists of a bunch of descriptors with metadata (and labels and transactions, and precomputed pubkeys to replace keypools)
3732018-08-16T19:33:41 <gmaxwell> If lots of people can help sipa finish that work it would be good. :P
3742018-08-16T19:34:15 <sipa> but independently andytoshi and i started looking into how we can efficiently compile arbitrary and/or/k-of-n/locktime/hash expressions to script
3752018-08-16T19:34:44 <sipa> which would just plug into descriptors, and then be available to everything that uses them
3762018-08-16T19:35:40 *** ken2812221 has quit IRC
3772018-08-16T19:35:54 *** ken2812221 has joined #bitcoin-core-dev
3782018-08-16T19:36:12 <wumpus> so I"ve been working on the RISC-V support, today I was able to do basic bring-up of the hardware (HiFive Unleashed) and test the gitian-built executables, which work!
3792018-08-16T19:36:28 <wumpus> been able to run the test_bitcoin succesfully and sync part of the chain, I'll keep the node running
3802018-08-16T19:36:34 <wumpus> probably the first RISC-V bitcoin node in the world
3812018-08-16T19:37:03 <jonasschnelli> \o/ nice!
3822018-08-16T19:37:39 <gmaxwell> nmkgl and I have been working on reconciliation based transaction relay. (With sipa's help too, that why I want people to help finish the descriptor work, ... :P )
3832018-08-16T19:38:26 <sdaftuar> gmaxwell: as in set reconciliation, eg to sync mempools?
3842018-08-16T19:38:52 <sipa> sdaftuar: yup
3852018-08-16T19:39:04 <gmaxwell> sdaftuar: Yes, but I believed we found a better design that doesn't sync the mempools directly.
3862018-08-16T19:39:46 <sdaftuar> neat, i'd be interested in learning more if you guys have a summary at some point.
3872018-08-16T19:39:49 <sipa> sdaftuar: it uses a more computationally expensive set reconcilation protocol than IBLT, but it's much more space efficient
3882018-08-16T19:40:06 <gmaxwell> muuuch more.
3892018-08-16T19:40:11 <sipa> (basically the amount of data transferred is equal to the expected difference between the two sets)
3902018-08-16T19:41:15 <sipa> but it's in the order of 40ms here to find 300 differences
3912018-08-16T19:41:24 <gmaxwell> I also came up with a new reconcilation protocol with a different performance tradeoff (much faster to decode, but slightly less space efficient), though it doesn't look like we'll have cause to use it.
3922018-08-16T19:42:04 <gmaxwell> sdaftuar: in any case the short summary of what we're thinking now vs before is that instead of reconciling mempools, reconcile the transactions that the peers would have otherwise INVed to each other.
3932018-08-16T19:42:42 <gmaxwell> this avoids cases like a mempool policy difference causing transations to get 'stuck' in the difference set forever until they get mined.
3942018-08-16T19:42:51 <sipa> oooh nice
3952018-08-16T19:42:53 <sipa> i missed that
3962018-08-16T19:43:23 <gmaxwell> Then it can just be coupled with some simple mechenism to fast-start an empty or stale mempool.
3972018-08-16T19:43:53 <gmaxwell> (I have a proposal for that too, just a simple one/two message protocol)
3982018-08-16T19:44:19 *** ken2812221 has quit IRC
3992018-08-16T19:44:22 <instagibbs> would this be complementary to a ip-hiding protocol like dandelion?
4002018-08-16T19:44:25 <sdaftuar> ah, that sounds neat
4012018-08-16T19:44:25 <BlueMatt> hmm, and then you could bucket and do reconciliation slowly for low-feerate buckets, or no reason to do that anymore?
4022018-08-16T19:44:46 <BlueMatt> instagibbs: I'd presume so, this would be during fluff-faze :p
4032018-08-16T19:44:55 <sipa> instagibbs: yes, orthogonal
4042018-08-16T19:45:04 <sipa> this is changing the diffusion phase
4052018-08-16T19:45:06 <sipa> not the stem
4062018-08-16T19:45:09 <instagibbs> BlueMatt, not exactly fluffing anymore :)
4072018-08-16T19:45:16 <gmaxwell> instagibbs: it's orthorgonal, the main motivation is getting rid of the really high bandwidth overhead for rumoring.
4082018-08-16T19:45:17 <instagibbs> sipa understood
4092018-08-16T19:45:28 <gmaxwell> Probably some differences in context here. :)
4102018-08-16T19:45:52 <gmaxwell> Right now, ignoring peers that IBD, the vast majority of node bandwidth is wasted on INV rumoring.
4112018-08-16T19:45:54 <BlueMatt> gmaxwell: wait, was that a response to my question?
4122018-08-16T19:46:30 <gmaxwell> BlueMatt: sure, it could be split by feerate too, though I'm not sure we'll have a reason to because the reconcilation is so efficient.
4132018-08-16T19:46:41 *** SopaXorzTaker has quit IRC
4142018-08-16T19:46:52 <BlueMatt> hmm, guess I need to think about this more
4152018-08-16T19:46:54 <sdaftuar> do you have an estimate on the bandwidth reduction?
4162018-08-16T19:47:38 <gmaxwell> No, don't have a mature enough simulator yet. Also I haven't measured overheads since we improved inv batching.
4172018-08-16T19:49:04 <gmaxwell> But previously INV overhead was something like 80% of node bandwidth (excluding IBDing peers).
4182018-08-16T19:49:33 <gmaxwell> And this should mostly eliminate that overhead.
4192018-08-16T19:50:16 *** BashCo has joined #bitcoin-core-dev
4202018-08-16T19:52:22 <wumpus> looks like we've run out of topics, but not out of time yet
4212018-08-16T19:52:34 <gmaxwell> In any case, we've been largely off in number theory land optimizing the recon itself... but I think we've got what we need there now. :)
4222018-08-16T19:53:38 <sdaftuar> btw i've been thinking about dandelion recently, trying to work through anti-DoS measures for stem routing
4232018-08-16T19:57:17 <gmaxwell> sdaftuar: Good! This seems to be one of those things that is easy in theory (if either you ignore getting it right or ignore how hard it is to implement) but hard in practice. :)
4242018-08-16T19:57:37 <wumpus> would be good to reduce the DoS surface there, yes
4252018-08-16T20:00:02 <sipa> PLOINKIDOOF
4262018-08-16T20:00:03 <wumpus> #endmeeting
4272018-08-16T20:00:03 <lightningbot> Meeting ended Thu Aug 16 20:00:03 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
4282018-08-16T20:00:03 <lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-08-16-19.03.html
4292018-08-16T20:00:03 <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-08-16-19.03.txt
4302018-08-16T20:00:03 <lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-08-16-19.03.log.html
4312018-08-16T20:01:08 <sdaftuar> ploinkidoof?
4322018-08-16T20:07:31 *** BashCo has quit IRC
4332018-08-16T20:09:28 *** Krellan has joined #bitcoin-core-dev
4342018-08-16T20:12:18 *** promag has quit IRC
4352018-08-16T20:12:24 *** BashCo has joined #bitcoin-core-dev
4362018-08-16T20:12:51 *** promag has joined #bitcoin-core-dev
4372018-08-16T20:13:08 *** clarkmoody has quit IRC
4382018-08-16T20:13:23 <jonasschnelli> rekeying and message queues is a nightmare
4392018-08-16T20:16:58 <sipa> it would be much easier if it wrre deterministic rather than negotiated
4402018-08-16T20:17:16 *** promag has quit IRC
4412018-08-16T20:18:17 <jonasschnelli> sipa: not sure. Since you need to decrypt the length in chacha20poly1305@openssh
4422018-08-16T20:19:01 <jonasschnelli> Or maybe whenever a message exceeds the limit, the following one will be encrypted with the new key
4432018-08-16T20:19:20 <jonasschnelli> but the time limit could be tricky,... the byte limit probably not
4442018-08-16T20:22:40 <gmaxwell> wait why is rekeying and message queues a nightmare.
4452018-08-16T20:23:10 <sipa> gmaxwell: because parsing protocol messages happens before processing them
4462018-08-16T20:23:26 <jonasschnelli> gmaxwell: because decomposing and queuing (where you decrypt) is done before processing
4472018-08-16T20:23:32 <sipa> a rekey means you need to undo the parsing for unprocessed things
4482018-08-16T20:23:33 <gmaxwell> the rekeying should be handled at the decryption layer, when you decrypt and find a rekey message then the very next bytes out you decrypt with the new stuff.
4492018-08-16T20:24:17 <jonasschnelli> I currently try to approach where I pause the read channel when I'm detecting a rekey during parsing
4502018-08-16T20:25:04 <sipa> i guess it would be easier if rekeying was done at a meta layer
4512018-08-16T20:25:12 <sipa> say a flag bit in the encrypted packet
4522018-08-16T20:25:21 <sipa> rather than an actual protocol message
4532018-08-16T20:25:40 <gmaxwell> Is the e.g. an out of range length.
4542018-08-16T20:26:08 <jonasschnelli> Rekeying when the decrypted length is oor seems fragile though,...
4552018-08-16T20:26:23 <jonasschnelli> I kinda like gmaxwell approach of putting the rekeyin logic in the decryption handler
4562018-08-16T20:26:23 <gmaxwell> Why?
4572018-08-16T20:26:54 <jonasschnelli> gmaxwell: isn't it technically possible that you get a valid length (<MAX_LENGTH) even with an invalid key?
4582018-08-16T20:27:02 *** d9b4bef9 has quit IRC
4592018-08-16T20:27:57 <gmaxwell> so? if your stream is corrupted you'll dsync, fail auth, and disconnect.
4602018-08-16T20:28:15 *** d9b4bef9 has joined #bitcoin-core-dev
4612018-08-16T20:28:50 *** masonicboom has quit IRC
4622018-08-16T20:28:50 *** BashCo has quit IRC
4632018-08-16T20:29:01 *** d9b4bef9 has quit IRC
4642018-08-16T20:30:08 *** d9b4bef9 has joined #bitcoin-core-dev
4652018-08-16T20:30:28 *** masonicboom has joined #bitcoin-core-dev
4662018-08-16T20:30:39 *** prod_ has joined #bitcoin-core-dev
4672018-08-16T20:30:40 <jonasschnelli> gmaxwell: is the probability very of an valid length with an unexpected key that low that a reconnect would be an acceptable workaround?
4682018-08-16T20:30:41 <jonasschnelli> *very low
4692018-08-16T20:31:02 *** d9b4bef9 has quit IRC
4702018-08-16T20:31:07 <gmaxwell> I think we're probably talking past each other.
4712018-08-16T20:31:15 <jonasschnelli> heh...
4722018-08-16T20:31:31 *** BashCo has joined #bitcoin-core-dev
4732018-08-16T20:31:58 <gmaxwell> what I'm trying to suggest is that to signal rekeying, under the old key, you send a specific length value which we would otherwise never use (due to it being out of range)
4742018-08-16T20:32:08 *** d9b4bef9 has joined #bitcoin-core-dev
4752018-08-16T20:32:14 <jonasschnelli> Assume when I rekey in case of decrypting to an invalid length... i guess there is a probability that the decrypted length with the now invalid key is within the MAX_SIZE boundary.... right?
4762018-08-16T20:32:33 <gmaxwell> No.
4772018-08-16T20:32:41 <jonasschnelli> aha! I see
4782018-08-16T20:33:01 <jonasschnelli> Wait..
4792018-08-16T20:33:01 *** d9b4bef9 has quit IRC
4802018-08-16T20:33:05 <gmaxwell> You don't decrypt the same data again. You see an invalid length, you rekey and throw out that length and continue.
4812018-08-16T20:34:00 <jonasschnelli> So... peer A asking for a rekey by setting an invalid length encrypted under the old key, then next message will use the new key?`
4822018-08-16T20:34:08 *** d9b4bef9 has joined #bitcoin-core-dev
4832018-08-16T20:34:16 <gmaxwell> Yes.
4842018-08-16T20:34:28 <jonasschnelli> So the message with the invalid length is a dummy, right?
4852018-08-16T20:34:31 <gmaxwell> yes.
4862018-08-16T20:34:48 <gmaxwell> I dunno if thats easier than just handling the rekey message at the decryption layer.
4872018-08-16T20:34:52 <jonasschnelli> Could the dummy message be a rekey message. :)
4882018-08-16T20:35:01 <gmaxwell> I was just presenting it as an alternative. :)
4892018-08-16T20:35:28 <gmaxwell> sure. But it could also, for example be an empty message (length 0) which maybe is easier structurally to handle.
4902018-08-16T20:35:29 <jonasschnelli> Yes. I like the invalid length approach since it doesn't require the parsing message logic
4912018-08-16T20:36:14 *** BashCo has quit IRC
4922018-08-16T20:36:25 <jonasschnelli> Length 0 would be an option and would not be confused with a real invalid decryption
4932018-08-16T20:36:34 <jonasschnelli> But length 0 would also be prone to DPI I guess
4942018-08-16T20:36:43 <jonasschnelli> Since it would be the smalles package always
4952018-08-16T20:36:53 <jonasschnelli> (could artificial blow up though)
4962018-08-16T20:37:14 <sipa> sdaftuar: my one line summary: we have a way to compute a 'sketch' from a set of N bit elements, with a size of M*N (so equal in size to M elements), in such a way that you can recover the contents from a sketch as long as there aren't more than M elements in it. Now, XORing two sketches gives you a sketch of the set of elements that are in one of the two input sets (but not both)
4972018-08-16T20:37:17 <gmaxwell> this protocol doesn't resist traffic analysis.
4982018-08-16T20:37:35 <jonasschnelli> Yes... right,..
4992018-08-16T20:38:04 <jonasschnelli> But with the 32byte pure pubkey handshake and the encrypted length, we are pretty stealth and make lives a bit harder for DPI configurators
5002018-08-16T20:38:21 <gmaxwell> jonasschnelli: I don't see how the length0 thing changes anything.
5012018-08-16T20:38:41 <gmaxwell> the length is still encrypted.
5022018-08-16T20:38:48 <jonasschnelli> I think the length 0 package could still contain a payload of the size of an inv
5032018-08-16T20:39:47 *** vexbuy_ has joined #bitcoin-core-dev
5042018-08-16T20:39:54 <jonasschnelli> Yes. I just though the size of that package (if someone analyses package burst) a rekey if the payload would be length 0 would be easy to identify ... but probably so other commands.
5052018-08-16T20:40:09 <jonasschnelli> *with
5062018-08-16T20:40:54 <jonasschnelli> But I guess my point is weak... length 0 seems to be the most advance idea how to tigger a rekey on the encryption layer
5072018-08-16T20:40:56 <gmaxwell> Sending a minimum length (just len and auth tag) message is no more or less identifable than any other size. If socket handling merges multiple messages, they're not identifyable at all, and if socket handling splits every message, the lengths are implicitly visible.
5082018-08-16T20:41:23 <gmaxwell> the only downside I see to length 0 is that we might otherwise want to use them for keepalives. :)
5092018-08-16T20:41:45 <gmaxwell> (which you might want to send more often than once per ten minutes)
5102018-08-16T20:41:56 <gmaxwell> but I guess we ping at the protocol level, so nevermind. :P
5112018-08-16T20:42:18 <jonasschnelli> We could also use MAX_INT32 for the rekey
5122018-08-16T20:42:57 <gmaxwell> I forget how the length is encoded. Is it encoded with a variable length encoding?
5132018-08-16T20:43:19 *** vexbuy has quit IRC
5142018-08-16T20:44:17 *** masonicboom has quit IRC
5152018-08-16T20:44:39 <gmaxwell> sipa, sdaftuar: An astute reader might notice that a set of M N bit elements can be communicated in log2(2^N choose M) bits. so this scheme is not quite perfectly efficient, but it's close.
5162018-08-16T20:45:15 <jonasschnelli> gmaxwell: length is a fix 4 byte uint32
5172018-08-16T20:45:32 <jonasschnelli> encrypted with a key only used for chacha20-crypt the length
5182018-08-16T20:46:29 <gmaxwell> jonasschnelli: oh, okay, for some reason I thought we had something with lower overhead there. I guess it doesn't matter much since most of our messages are fairly long.
5192018-08-16T20:47:03 <gmaxwell> jonasschnelli: indeed MAX_UINT32 could just be a length 0 message that triggers rekey.
5202018-08-16T20:47:30 *** masonicboom has joined #bitcoin-core-dev
5212018-08-16T20:47:30 <sdaftuar> sipa gmaxwell: what's the failure mode/fallback scenario if a sketch can't be reconciled?
5222018-08-16T20:47:32 *** Victorsueca has quit IRC
5232018-08-16T20:48:18 <gmaxwell> sdaftuar: Sketch data can be incrementally sent. so if M wasn't enough I can just send you one more value and now you have M+1.
5242018-08-16T20:48:45 *** Victorsueca has joined #bitcoin-core-dev
5252018-08-16T20:48:47 <sdaftuar> oh neat
5262018-08-16T20:49:06 <sipa> it can even be incrementally computed
5272018-08-16T20:49:07 <sdaftuar> does that require saving the old sketch?
5282018-08-16T20:49:11 <sdaftuar> oh
5292018-08-16T20:49:26 <sipa> (but computing a sketch is very cheap, recovering data form one is expensive)
5302018-08-16T20:49:38 <jonasschnelli> gmaxwell: ideally, there would be a way to flag the rekey in a non-extra message,.. so flag in the first message using the new key
5312018-08-16T20:49:48 <gmaxwell> In practice we'll have a limit on the maximum M, both for memory reasons storing the sketches and computation (decode is quadratic). ... so if we exceeded that, you'd just fail to relay those transactions since the last reconcile with that peer, better luck next time around.
5322018-08-16T20:49:59 <jonasschnelli> (to avoid accessing the push message logic from with the encryption logic)
5332018-08-16T20:50:53 <gmaxwell> sdaftuar: The sketch decode has two steps, the first step is perfectly incremental. So we waste no cpu getting the sketch data a little at a time. The second step is not incremental, though we can arrange things so that we can be pretty sure if it'll be successful or not before starting it.
5342018-08-16T20:51:43 <gmaxwell> jonasschnelli: well just steal the most significant bit of the length.
5352018-08-16T20:51:56 <gmaxwell> 2^31 byte messages should be enough for anyone.
5362018-08-16T20:52:09 <jonasschnelli> But megablocks!
5372018-08-16T20:52:16 <jonasschnelli> Yes. Great idea
5382018-08-16T20:52:23 <gmaxwell> 2^31 is still 2GB. :P
5392018-08-16T20:52:59 * jonasschnelli got called to bed
5402018-08-16T20:53:04 <gmaxwell> night
5412018-08-16T20:56:20 <gmaxwell> sdaftuar: the actual construction of this system is a binary BCH code, with a message 2^n bits long which has up to M 'errors' (the set difference) in it that we need to correct. So obviously, we should call it something like BCH faulting transaction relay. :P
5422018-08-16T20:58:06 <sdaftuar> just bch relay sounds nice and concise
5432018-08-16T20:59:58 * warren blinked hard upon reading BCH
5442018-08-16T21:00:57 <sdaftuar> anyway that sounds awesome, i assume it's close enough that we can hope to have it for 0.18?
5452018-08-16T21:01:54 <gmaxwell> I don't see why it couldn't be done in a couple months, assuming that as we implement we don't get stuck on stupid protocol issues.
5462018-08-16T21:02:39 <gmaxwell> or fall too far down the number theory rathole of microoptimizing the set reconcillation.
5472018-08-16T21:02:45 <sdaftuar> :)
5482018-08-16T21:02:51 *** Chris_Stewart_5 has quit IRC
5492018-08-16T21:02:59 <sipa> sdaftuar: the algorithm is based on a project called 'pinsketch', which relies on a library called NTL, which is LGPL and a lot of code
5502018-08-16T21:03:03 <gmaxwell> it's really quite a lovely problem to work on. :)
5512018-08-16T21:03:21 <sipa> sdaftuar: so we reimplemented inlt from scratch in a few hundred lines, and it's faster too :)
5522018-08-16T21:03:27 <sdaftuar> yeah i look forward to being nerdsniped when you share your results
5532018-08-16T21:03:32 <sdaftuar> sipa: lol
5542018-08-16T21:03:38 <gmaxwell> and came up with a bunch of moderately smart optimizations...
5552018-08-16T21:04:17 <sdaftuar> i was about to groan about dealing with a 3rd party library and lgpl, i should have known better than to think you'd let that be a problem :)
5562018-08-16T21:05:26 <gmaxwell> sdaftuar: that was what held me off from doing more of this a year ago.
5572018-08-16T21:13:25 *** thebiglq12345 has quit IRC
5582018-08-16T21:21:20 *** vexbuy_ has quit IRC
5592018-08-16T21:23:39 <sipa> sdaftuar: curiously, the author of pinsketch is someone i've met at real world crypto; he did a talk on UTXO commitment data structures
5602018-08-16T21:46:52 *** jhfrontz has quit IRC
5612018-08-16T21:59:43 *** Aaronvan_ has joined #bitcoin-core-dev
5622018-08-16T22:02:10 *** AaronvanW has quit IRC
5632018-08-16T22:03:14 *** Chris_Stewart_5 has joined #bitcoin-core-dev
5642018-08-16T22:08:57 *** jhfrontz has joined #bitcoin-core-dev
5652018-08-16T22:20:27 *** jhfrontz has quit IRC
5662018-08-16T22:27:01 *** jhfrontz has joined #bitcoin-core-dev
5672018-08-16T22:32:13 *** jhfrontz has quit IRC
5682018-08-16T22:32:36 *** jhfrontz has joined #bitcoin-core-dev
5692018-08-16T22:38:08 *** jhfrontz has quit IRC
5702018-08-16T22:38:46 *** jhfrontz has joined #bitcoin-core-dev
5712018-08-16T22:46:16 *** michaelsdunn1 has quit IRC
5722018-08-16T22:48:27 *** grafcaps has quit IRC
5732018-08-16T22:58:33 *** jhfrontz has quit IRC
5742018-08-16T23:04:28 *** Victorsueca has quit IRC
5752018-08-16T23:05:51 *** Victorsueca has joined #bitcoin-core-dev
5762018-08-16T23:07:21 *** esotericnonsense has quit IRC
5772018-08-16T23:07:24 <luke-jr> achow101: MarcoFalke: that VMBuilder fork doesn't actually work :<
5782018-08-16T23:07:32 <luke-jr> at least not for bionic
5792018-08-16T23:08:23 *** Emcy has quit IRC
5802018-08-16T23:08:59 *** Emcy has joined #bitcoin-core-dev
5812018-08-16T23:09:00 *** esotericnonsense has joined #bitcoin-core-dev
5822018-08-16T23:09:23 *** Guyver2 has quit IRC
5832018-08-16T23:13:42 *** plankers has joined #bitcoin-core-dev
5842018-08-16T23:20:52 *** kevink has joined #bitcoin-core-dev
5852018-08-16T23:24:35 <kevink> I'm using Bitcoin Core's API and am wondering whats the difference between `duplicate-inconclusive` and `duplicate` when calling `submitblock`. I'm just trying to verify that a block exists in the blockchain and `submitblock` seems like the simplest way to do that.
5862018-08-16T23:26:05 *** promag has joined #bitcoin-core-dev
5872018-08-16T23:27:50 <sipa> kevink: try getblockheader instead
5882018-08-16T23:29:29 <phantomcircuit> kevink, submit block is definitely not the way to do that, getblockheader is
5892018-08-16T23:29:32 *** Emcy has quit IRC
5902018-08-16T23:31:14 *** promag has quit IRC
5912018-08-16T23:36:15 <kevink> The reason I wanted to use submitblock was because I'm encoding and decoding the blockdata into an image using https://gist.github.com/laanwj/51f276c44ba9882bb4b27cc6f3a499a4 and wanted to check that it also remained a valid block after being decoded.
5922018-08-16T23:44:19 <sipa> compute its hash, and query for it
5932018-08-16T23:48:30 <gmaxwell> I guess we don't have a decoderawblock rpc that would construct the hash for you!
5942018-08-16T23:49:40 *** viaj3ro has quit IRC
5952018-08-16T23:54:57 *** justanotheruser has quit IRC