12016-03-24T00:00:45  <gmaxwell> oh hmph. made some pgoress but stuck again. :(
   22016-03-24T00:01:17  <gmaxwell> now this looks like local corruption from running out of space. 2016-03-24 00:00:19 ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=-1, nPos=0)
   32016-03-24T00:01:50  <sipax> ugh!
   42016-03-24T00:02:04  <sipax> that means the block data wasn't written but the index entry for it was
   52016-03-24T00:02:20  <sipax> remind me to investigate the write order during flush in such a case
   62016-03-24T00:02:29  * sipax goes into standby mode
   72016-03-24T00:03:17  <gmaxwell> well it may have been out of space and the write could have failed. Our out of space handling is not so great always.
   82016-03-24T00:03:37  <sipax> it should not go write a database entry for a file it failed to create
   92016-03-24T00:03:46  <sipax> is it a pruned node?
  102016-03-24T00:03:50  <gmaxwell> no.
  112016-03-24T00:03:56  <sipax> in that case, certainly not
  122016-03-24T00:04:44  <gmaxwell> what the heck, even after logging that a bunch of times it's making progress again.
  132016-03-24T00:06:01  <gmaxwell> okay, actually the readblockfromdisk is being triggered by getblock rpcs being called by p2pool
  142016-03-24T00:06:40  <gmaxwell> so this might just be an artifact of running getblock on a block we have headers for but no data.
  152016-03-24T00:07:32  <sipax> gmaxwell: no, it shouldn't
  162016-03-24T00:07:45  <sipax> getblock RPC checks whether the BLOCK_HAVE_DATA flag is present for that block index entry
  172016-03-24T00:14:33  <gmaxwell> well my log is full of
  182016-03-24T00:14:33  <gmaxwell> 2016-03-23 23:58:18 Received a POST request for / from 127.0.0.1:51001
  192016-03-24T00:14:34  <gmaxwell> 2016-03-23 23:58:18 ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=-1, nPos=0)
  202016-03-24T00:14:36  <gmaxwell> 2016-03-23 23:58:18 ThreadRPCServer method=getblock
  212016-03-24T00:32:55  <gmaxwell> and now that it's caught up, no more..
  222016-03-24T00:34:32  *** afk11 has quit IRC
  232016-03-24T00:37:17  *** afk11 has joined #bitcoin-core-dev
  242016-03-24T01:09:49  *** murch has quit IRC
  252016-03-24T01:11:01  *** Ylbam has quit IRC
  262016-03-24T01:12:32  *** fengling has joined #bitcoin-core-dev
  272016-03-24T01:56:42  *** supasonic has quit IRC
  282016-03-24T01:57:10  *** supasonic has joined #bitcoin-core-dev
  292016-03-24T02:09:02  *** Alopex has quit IRC
  302016-03-24T02:10:07  *** Alopex has joined #bitcoin-core-dev
  312016-03-24T02:19:06  <morcos> gmaxwell: sipax: looks like thats just the error that happens, the RPC only checks the BLOCK_HAVE_DATA flag if you're pruning.  Ha!
  322016-03-24T02:20:23  *** baldur has quit IRC
  332016-03-24T02:20:41  *** baldur has joined #bitcoin-core-dev
  342016-03-24T02:21:10  *** afk11 has quit IRC
  352016-03-24T02:24:04  *** zooko has joined #bitcoin-core-dev
  362016-03-24T02:28:57  *** afk11 has joined #bitcoin-core-dev
  372016-03-24T02:52:51  *** Chris_Stewart_5 has quit IRC
  382016-03-24T02:53:59  *** AdrianG_ is now known as AdrianG
  392016-03-24T02:54:29  *** AdrianG is now known as Guest82537
  402016-03-24T02:54:43  *** Guest82537 is now known as AdrianG_
  412016-03-24T03:01:02  *** AdrianG_ has quit IRC
  422016-03-24T03:01:02  *** AdrianG_ has joined #bitcoin-core-dev
  432016-03-24T03:01:16  *** AdrianG_ is now known as Aleph0
  442016-03-24T03:08:34  *** afk11 has quit IRC
  452016-03-24T03:16:08  *** PRab has quit IRC
  462016-03-24T03:17:08  *** PRab has joined #bitcoin-core-dev
  472016-03-24T03:22:01  *** afk11 has joined #bitcoin-core-dev
  482016-03-24T03:26:56  *** zooko has quit IRC
  492016-03-24T03:29:10  *** xiangfu has joined #bitcoin-core-dev
  502016-03-24T03:31:54  *** molz has joined #bitcoin-core-dev
  512016-03-24T03:34:05  *** mrkent has quit IRC
  522016-03-24T03:34:51  *** moli has quit IRC
  532016-03-24T03:37:10  *** frankenmint has quit IRC
  542016-03-24T03:39:36  *** achow101 has quit IRC
  552016-03-24T03:41:35  *** supasonic has quit IRC
  562016-03-24T03:54:25  *** supasonic has joined #bitcoin-core-dev
  572016-03-24T03:56:47  *** mrkent has joined #bitcoin-core-dev
  582016-03-24T04:49:01  *** Alopex has quit IRC
  592016-03-24T04:50:06  *** Alopex has joined #bitcoin-core-dev
  602016-03-24T04:50:58  *** frankenmint has joined #bitcoin-core-dev
  612016-03-24T04:55:17  *** BCB has quit IRC
  622016-03-24T05:09:24  *** mrkent has quit IRC
  632016-03-24T05:11:46  *** mrkent has joined #bitcoin-core-dev
  642016-03-24T05:21:57  *** Don_John has quit IRC
  652016-03-24T05:23:49  *** mrkent has quit IRC
  662016-03-24T05:24:02  *** mrkent has joined #bitcoin-core-dev
  672016-03-24T05:44:15  *** schmidty has quit IRC
  682016-03-24T05:48:01  *** Alopex has quit IRC
  692016-03-24T05:49:06  *** Alopex has joined #bitcoin-core-dev
  702016-03-24T05:51:15  *** supasonic has quit IRC
  712016-03-24T05:52:25  *** d_t has joined #bitcoin-core-dev
  722016-03-24T05:55:02  *** d_t has quit IRC
  732016-03-24T05:56:25  *** mrkent has quit IRC
  742016-03-24T05:59:36  *** fengling has quit IRC
  752016-03-24T06:00:03  *** xiangfu has quit IRC
  762016-03-24T06:00:04  *** dermoth has quit IRC
  772016-03-24T06:00:48  *** dermoth has joined #bitcoin-core-dev
  782016-03-24T06:02:04  *** xiangfu has joined #bitcoin-core-dev
  792016-03-24T06:04:01  *** Alopex has quit IRC
  802016-03-24T06:05:06  *** Alopex has joined #bitcoin-core-dev
  812016-03-24T06:07:05  *** mrkent has joined #bitcoin-core-dev
  822016-03-24T06:08:23  *** fengling has joined #bitcoin-core-dev
  832016-03-24T06:17:34  *** molly has joined #bitcoin-core-dev
  842016-03-24T06:20:58  *** molz has quit IRC
  852016-03-24T06:45:23  <jonasschnelli> encryption: would it be stupid to use the identity keypair (used for Auth / MITM protection) in a ECDH scheme to encrypt a dh key-exchange for further encryption?
  862016-03-24T06:45:47  <jonasschnelli> I think we should not use the "static" identity key for encryption.
  872016-03-24T06:46:16  <jonasschnelli> It would expose the shared ecdh secred (AES256key) to known-plaintext-attacks.
  882016-03-24T06:46:59  <jonasschnelli> But not sure if using the identity key for a quick ECDH keyexchange (known-plaintext-attacks are pretty impossible) would make sense.
  892016-03-24T06:51:35  <gmaxwell> It would be stupid beyond all comprehension.
  902016-03-24T06:52:14  <gmaxwell> The encryption should be ephemerally keyed. There is no need for the key to outlive the session.
  912016-03-24T06:57:35  *** Ylbam has joined #bitcoin-core-dev
  922016-03-24T06:57:40  <gmaxwell> What I suggested earlier-- which is a pretty standard and well studied architecture is to use randomly generated keys to establish a secure channel (with authenticated encryption like AES-GCM or chacha20-poly1305); then inside if-- if some identity is in use-- you use that identity to authenticate the secure session you just established. E.g. by signing the hash of the shared session key.
  932016-03-24T07:00:44  <sipax> gmaxwell: that is MitMable?
  942016-03-24T07:01:02  <gmaxwell> ... No.
  952016-03-24T07:01:36  <sipax> oh, no!
  962016-03-24T07:01:37  <jonasschnelli> gmaxwell: how would you pass the random generated symmetric cipher key to "the other side"? ECDH channel with the identity key?
  972016-03-24T07:03:06  <gmaxwell> Please please stop trying to use ecdh with long lived keys. This is almost always a severe design mistake.
  982016-03-24T07:04:21  <gmaxwell> Each side sends a new, randomly generated public key to the other side. The session keys (there will be more than one, likely four-- auth and encrypt in each direction) are the output of ECDH with those keys, fed through a KDF.
  992016-03-24T07:06:16  <jonasschnelli> gmaxwell: Okay. How would you protect from MITM. By auth with the identity key AFTER the encryption channel is setup?
 1002016-03-24T07:08:03  <gmaxwell> if identification is in use, inside the encrypted channel you send a signature of the session id (a hash of the ephemerial key from ecdh which also commits to all the channel parameters) using the relevant identity.
 1012016-03-24T07:09:20  <sipax> what do you mean by channel parameters?
 1022016-03-24T07:09:47  <jonasschnelli> gmaxwell: how would you protect from attacking(DOSing) the encryption request (key generation!) if the auth is after the enc?
 1032016-03-24T07:10:02  <jonasschnelli> Also fingerprinting
 1042016-03-24T07:10:28  <jonasschnelli> (the later is probably fine if the 4 key are generated randomly)
 1052016-03-24T07:10:31  <gmaxwell> key generation is very fast, faster then verifying your digital signatures you propose for authentication.
 1062016-03-24T07:11:01  <gmaxwell> Fingerprinting what?
 1072016-03-24T07:11:02  * jonasschnelli always though key generation is much more CPU intense then ecdsa verify.
 1082016-03-24T07:11:12  <sipax> jonasschnelli: if you always use freshly generated ecdh keys, there is no fongerprinting
 1092016-03-24T07:11:14  <jonasschnelli> gmaxwell: fingerprinting the node.
 1102016-03-24T07:11:44  <jonasschnelli> Well,.. right. We could ban a node after a failed encryption request.
 1112016-03-24T07:12:11  <sipax> i don't think it can fail
 1122016-03-24T07:12:24  <jonasschnelli> sipax: I mean the identity check afterwards
 1132016-03-24T07:13:04  <sipax> jonasschnelli: it's much cheaper for them to go ask us a bunch of random blocks, with far higher costs
 1142016-03-24T07:13:09  <sipax> if they want to dos
 1152016-03-24T07:13:11  <gmaxwell> jonasschnelli: no, it's fundimentally easier to generate a new keypair than to verify a ecdh signature. They're pretty close in timings, because we use a constant time implementation for the former.
 1162016-03-24T07:13:14  <jonasschnelli> But this would basically mean, every peer can request encrypted sessions... (which i'm not sure if its good or bad).
 1172016-03-24T07:13:44  <gmaxwell> they should, and they should all be encrypted so as not to distingush the ones using authentication.
 1182016-03-24T07:14:10  <gmaxwell> and this can be made faster than the existing transport (or at least not considerably worse).
 1192016-03-24T07:14:38  <sipax> jonasschnelli: for ECDSA, verify > sign >>> keygen
 1202016-03-24T07:14:59  <jonasschnelli> sipax: I see. Thanks!
 1212016-03-24T07:15:17  <sipax> jonasschnelli: computing the pubkey for a generated private key is similar to signing
 1222016-03-24T07:15:24  <gmaxwell> well if by keygen you mean generate a pubkey too, then sign == keygen, pretty much.
 1232016-03-24T07:15:48  <gmaxwell> and this is faster than verify.
 1242016-03-24T07:16:01  <sipax> same order of magnitude in any case
 1252016-03-24T07:16:05  <jonasschnelli> gmaxwell: So. The peer would pass one pubkey for the dh secret calculation and the (static) identity pubkey (static) for auth to the other peer. Right?
 1262016-03-24T07:16:52  <jonasschnelli> (doesn't the current[and most] ec keygen algos do a dummy sign/verify to ensure validity?)
 1272016-03-24T07:17:05  <sipax> jonasschnelli: if by auth you mean the signing of the session key, indeed
 1282016-03-24T07:17:12  <sipax> jonasschnelli: bitcoin does
 1292016-03-24T07:17:14  <gmaxwell> the only software ever created that does that is bitcoin core.
 1302016-03-24T07:17:19  <gmaxwell> as far as I can tell.
 1312016-03-24T07:17:34  <gmaxwell> and it's only sensible to do that for payment addresses, it wouldn't be done here.
 1322016-03-24T07:18:15  <gmaxwell> first an encrypted, authenticated channel would be brought up using a ephemeral pubkey sent in each direction.  Then _inside_ that channel, the participants could identify themselves by signing the session-id.
 1332016-03-24T07:18:24  <jonasschnelli> only signing the session key would not allow the remote peer to identify which identity is signing. Wouldn't it require at least the hash of the identity pubkey?
 1342016-03-24T07:19:45  <gmaxwell> Techincally the signature and message its signing is enough, you can just get the pubkey from that. Though it could send the pubkey its using.
 1352016-03-24T07:20:14  <jonasschnelli> Ah. Right pubkey recover.
 1362016-03-24T07:20:20  <gmaxwell> but there are better protocols for identifying than just sending a signature-- ones that do not leak the identity of at least one participant.
 1372016-03-24T07:20:24  *** mrkent has quit IRC
 1382016-03-24T07:20:35  <jonasschnelli> 0-knowlage?
 1392016-03-24T07:21:32  *** mrkent has joined #bitcoin-core-dev
 1402016-03-24T07:21:49  *** PaulCape_ has joined #bitcoin-core-dev
 1412016-03-24T07:22:53  <jonasschnelli> What about allowing auth without enc? We probably presume encryption when authenticating.
 1422016-03-24T07:23:12  * gmaxwell looks through logs.
 1432016-03-24T07:23:14  <gmaxwell> 13:14 < gmaxwell> There are protocols that avoid the fingerprinting; e.g. (the server protectng mode of) https://www.ietf.org/proceedings/54/I-D/draft-ietf-ipsec-jfk-04.txt
 1442016-03-24T07:23:39  <gmaxwell> jonasschnelli: I think there is no utility to identifying without encryption.
 1452016-03-24T07:24:21  *** PaulCapestany has quit IRC
 1462016-03-24T07:24:29  <jonasschnelli> Agree. The current auth proposal would also require signing (auth) each message (which is resource hungry).
 1472016-03-24T07:24:42  <gmaxwell> thats unacceptably slow.
 1482016-03-24T07:24:45  <jonasschnelli> encryption before authentication allows to keep a auth-session.
 1492016-03-24T07:25:34  <sipax> jonasschnelli: note that authentication is used in two meanings here; there is identification (done by signing the session key) and MAC (which you still need, but ks covered by using an authenticated mode of encryption such as AES-GCM)
 1502016-03-24T07:25:40  <gmaxwell> (and signining indivigual messages provides non-repudiation, which is usually not what you want; except in specific instances where you do want it.)
 1512016-03-24T07:25:42  *** sipax is now known as sipa
 1522016-03-24T07:26:14  <jonasschnelli> I see.
 1532016-03-24T07:26:38  <sipa> jonasschnelli: so there is "auth" on each message, but it's done with a symmetric shared session key, rather than the identity key
 1542016-03-24T07:26:48  <jonasschnelli> And I guess it would be acceptable if the remote peer can fingerprint the requesting peer because it reveals its identity after the encryption channel is established.
 1552016-03-24T07:27:29  <gmaxwell> jonasschnelli: it's better than the other way around at least. preferably no one could fingerprint anyone; but it's not clear how possible that is.
 1562016-03-24T07:27:32  <jonasschnelli> sipa: Right. But we just need to make sure the symmetric shared session key was shared between the right parties and not Mitmdled.
 1572016-03-24T07:27:42  <sipa> jonasschnelli: i think you probably want a per-IP identity, or perhaps just randomly generated identities by default, and allow a peer to ask "hey, you're identity X, right? prove it"
 1582016-03-24T07:27:52  <gmaxwell> Other than the "connect to my own server" case it's not clear to me exactly how the identification would be used.
 1592016-03-24T07:28:03  <jonasschnelli> sipa: But that would not work for DHCP nodes (SPV)?
 1602016-03-24T07:29:03  <sipa> jonasschnelli: so perhaps a separate private non-random configurable host key, which is only used if requested by the peer
 1612016-03-24T07:29:29  <sipa> that means no TOFU, though
 1622016-03-24T07:29:32  <gmaxwell> we don't want to create cases where nodes can be tracked easily, we've worked to remove identifying information from the protocol...  having a protocol where there is mutual auth, say between your own 'server' and 'client'  or between your own hosts,  having the requester leak something the allows them to be correlated, but only inside an encrypted channel which they think is to their trusted pe
 1632016-03-24T07:29:38  <gmaxwell> er, would be minimally harmful.
 1642016-03-24T07:30:40  <gmaxwell> sipa: I don't know what use tofu would have... and what happens when a node goes away (say, gets deleted) and a new one is at that address? now you need an expiration mechenism.
 1652016-03-24T07:31:03  <jonasschnelli> gmaxwell: at least you could setup a honeypot-like peer that waits until other peers connect, request encryption and identify themself. You could than combine this data with other fingerprint possibilities (mempool and stuff like that).
 1662016-03-24T07:32:13  <gmaxwell> jonasschnelli: the party initating authentication would be the only one to lose privacy, so a honey pot would not work; unless it also successfully mitmed hosts that clients were attempting to connect to.
 1672016-03-24T07:32:33  <jonasschnelli> But somehow I think either you have a MITM attack possibility or you reveal ones identity. Maybe some zero knowledge prove method could be used here (not familiar with these)
 1682016-03-24T07:32:52  <gmaxwell> There are probably multiple usecases which would identify the channel in different ways; but the same secure channel mechenism would work.
 1692016-03-24T07:33:58  <gmaxwell> jonasschnelli: I linked a likely sutable identification protocol above.
 1702016-03-24T07:34:20  * jonasschnelli is reading the JFK proposal. Thanks to gmaxwell 
 1712016-03-24T07:35:28  <gmaxwell> I am not suggesting we specifically use that, as much as pointing out that it's possible to do better... but even a very simple protocol does what we want, I think.
 1722016-03-24T07:35:42  <sipa> jonasschnelli: you could also avoid the problem, and by default not have identification, and only have a "prove to me you're X before continuing" message inside the encryptee channel
 1732016-03-24T07:36:37  <sipa> perhaps even using symmetric crypto, so it only works with a preshared symmetric key
 1742016-03-24T07:36:44  <jonasschnelli> sipa: Yes. Agreed. Auth would only be necessary or useful if the both peers know each other (preshared key, same owner, SPV<->node stuff).
 1752016-03-24T07:37:31  <gmaxwell> identification, when you say auth it easily gets confused with the authentication part of encryption which is necessary and cannot be omitted when encryption is used. :)
 1762016-03-24T07:37:32  *** PaulCapestany has joined #bitcoin-core-dev
 1772016-03-24T07:37:48  <jonasschnelli> A preshared preshared symmetric key would also work. Right. IMO its just easier to share two pubkeys instead of one "symmetric key".
 1782016-03-24T07:38:59  <jonasschnelli> gmaxwell: Can you explain the differences between the identity authentication and the encryption authentication? Why would the later be required if both parties stay anonyme to each other?
 1792016-03-24T07:39:37  *** PaulCape_ has quit IRC
 1802016-03-24T07:39:55  <jonasschnelli> Once you exchanged encryption ephemeral pubkeys, MITM should no longer be possible?
 1812016-03-24T07:40:23  <gmaxwell> jonasschnelli: any use of encryption _must_ come with symmetric-key authentication, or otherwise the encryption will be mallible and an attacker can corrupt the channel in ways that leak information about the content.
 1822016-03-24T07:41:35  <gmaxwell> e.g. they flip a bit, and instead of instantly hanging up, the application is fed some gibberish, and then based on exactly how it responseds/doesn't respond the attacker learns something about the state of the connection and the prior encrypted data.
 1832016-03-24T07:42:26  <gmaxwell> so the encryption must comes with a message-authentication-code keyed at the same time as the encryption, so any corruption is detected and the connection terminated without revealing anything, ideally before even decrypting the data.
 1842016-03-24T07:42:55  <jonasschnelli> gmaxwell: sharing the encryption ephemeral pubkeys, calculate symmetric secret and hash the complete communication (starting with both pubkey in deterministic order), compare the hashes after decrypt..
 1852016-03-24T07:43:20  <gmaxwell> jonasschnelli: that isn't sufficient.
 1862016-03-24T07:43:50  <sipa> jonasschnelli: encryption provides confidentiality, not integrity
 1872016-03-24T07:44:31  <jonasschnelli> hmm....
 1882016-03-24T07:44:55  * jonasschnelli is reading...
 1892016-03-24T07:45:05  <Luke-Jr> jonasschnelli: it might or might not make sense to coordinate with the new payment protocol encryption stuff
 1902016-03-24T07:45:20  <Luke-Jr> in BIP 75
 1912016-03-24T07:47:22  <jonasschnelli> Luke-Jr: thanks. Will check. Agree on the p2p 16x num range.
 1922016-03-24T07:49:18  <sipa> jonasschnelli, gmaxwell: the paper my const time aes is based on actually discusses aes-gcm (and easily has 4x the performance of the cbc case, as it can do 4 parallel encryption operations)
 1932016-03-24T07:50:27  <jonasschnelli> gmaxwell sipa: hmm... (having a hard time to understand this correctly): so you would require to generate two keypairs at the same time, one for the encryption sym. cipher key, one for the MAC key. Then you sould sha256_hmac the complete message traffic on both sides and compare to validate the integrity?
 1942016-03-24T07:50:52  <sipa> jonasschnelli: no, you use aes-gcm
 1952016-03-24T07:51:03  <sipa> which does encryption and mac simultanelusly
 1962016-03-24T07:51:47  <jonasschnelli> sipa: okay. I see. But the overall concept would be correct (just to understand the basics).
 1972016-03-24T07:51:56  <jonasschnelli> s/./?
 1982016-03-24T07:52:10  <sipa> though, conceptually, yes, you can use aes for encryption, and then do hmac-sha256 on the encrypted data for integrity
 1992016-03-24T07:52:56  <gmaxwell> There should be two "keys" if using authenticated encryption (like AES-GCM, chacha20-poly, or things coming out of CAESAR) one key in each direction;  or four keys if constructing autenticated encryption from a cipher and hmac.
 2002016-03-24T07:53:15  <jonasschnelli> What I have problems to understand is how attacks would be possible if you would take the encryption ephemeral publey calculated sym. key for the MAC...
 2012016-03-24T07:53:49  <sipa> jonasschnelli: let's say you use aes-ctr
 2022016-03-24T07:53:57  <sipa> you know ctr?
 2032016-03-24T07:54:13  * jonasschnelli reading... got the essence. Yes.
 2042016-03-24T07:54:18  <jonasschnelli> streaming.
 2052016-03-24T07:55:26  <sipa> someone may not be able to decrypt without having the key, but if they can guess what the contents is of a particular message, they can modify it by xoring it with both what they think the decrypted data is, and xoring it with what they want the decrypted data to be
 2062016-03-24T07:56:33  <sipa> there are more complex bad things you can do with other encryption schemes
 2072016-03-24T07:56:52  <jonasschnelli> sipa: Got it! Thanks!
 2082016-03-24T07:57:18  <sipa> but ctr is a nice example, because it perfectly covers confidentially, but provides almost nothing else
 2092016-03-24T07:57:36  <sipa> so it clearly shows the distinction
 2102016-03-24T07:58:35  * jonasschnelli is reading the details of AES-GCM
 2112016-03-24T07:59:39  <jonasschnelli> sipa: But IIRC, your AES PR is CBC. So GCM would require another implementation while our codebase already supports SHA2 HMAC?
 2122016-03-24T07:59:57  <sipa> jonasschnelli: GCM is easy :)
 2132016-03-24T08:00:19  <jonasschnelli> sipa: for you or for all other lazy programmers.. :)
 2142016-03-24T08:00:35  <gmaxwell> sha2 is slow.
 2152016-03-24T08:01:05  <jonasschnelli> And I guess we could drop the message headers 4byte sha256 checksum once encrypted and MACed
 2162016-03-24T08:01:49  <gmaxwell> (AES-GCM is also not astonishingly fast without hardware AES, but should be faster than sha2)
 2172016-03-24T08:02:09  <gmaxwell> jonasschnelli: yes, and the result, with sutiable primitive selection would be _faster_ than the existing protocol, even though it was encrypted.
 2182016-03-24T08:02:25  <jonasschnelli> hah. Thats an argument.
 2192016-03-24T08:02:32  <gmaxwell> (faster in terms of cpu usage, it would likely have some more bandwidth overhead, but not a ton)
 2202016-03-24T08:03:15  <jonasschnelli> But I'm mostly thinking "in implementations". So,... I could write a first implementation that drops the 4byte header sha and uses SHA2 HMAC for the MAC.. just to have an implementation basepoint.
 2212016-03-24T08:03:37  <jonasschnelli> (in addition to the whole encryption keysharing stuff)
 2222016-03-24T08:03:47  *** mrkent has quit IRC
 2232016-03-24T08:04:58  <jonasschnelli> I could be the guy that works on the code frontend while I require analysis and conceptual support from gmaxwell/sipa and others. This could lead to a pratical p2p encryption IMO.
 2242016-03-24T08:06:26  <sipa> gmaxwell: my current const aes needs 270 cycles/byte; making it do 4 in parallel is trivial and would make it need 72; researchers claim they can do ~10 using simd
 2252016-03-24T08:06:37  <sipa> and ~3 with aes-ni
 2262016-03-24T08:06:39  *** mrkent has joined #bitcoin-core-dev
 2272016-03-24T08:06:41  <gmaxwell> sipa: I think with a.. right.
 2282016-03-24T08:07:13  <gmaxwell> See also the motivationes behind the CAESAR competition.
 2292016-03-24T08:11:16  *** asdfdsas has quit IRC
 2302016-03-24T08:13:20  <gmaxwell> sipa: lol one of the CAESAR entries is based on keccak and is named "keyak". That totally isn't going to cause any confusion...
 2312016-03-24T08:16:15  <GitHub117> [bitcoin] laanwj closed pull request #7694: Rename AcceptBlock/AcceptBlockHeader to StoreBlock/StoreBlockHeader (master...2016-03-15-naming) https://github.com/bitcoin/bitcoin/pull/7694
 2322016-03-24T08:16:45  <sipa> gmaxwell: i was just reading the keyak v2 paper
 2332016-03-24T08:18:03  <btcdrak> I've asked a cryptographer to look at #7689
 2342016-03-24T08:22:59  <jonasschnelli> Could we ask apoelstra?
 2352016-03-24T08:23:41  <jonasschnelli> I'm not sure if a standalone AES library (move sipas PR to a custom library) would make sense.
 2362016-03-24T08:24:59  <jonasschnelli> Adding more non-consensus stuff there will very likely lead to a openssl clone?!
 2372016-03-24T08:25:05  <jonasschnelli> And adding SHA512 but not SHA256?
 2382016-03-24T08:25:27  *** cjcj has joined #bitcoin-core-dev
 2392016-03-24T08:25:40  <wumpus> jonasschnelli: correct
 2402016-03-24T08:26:06  <wumpus> I don't think petertodd's concern is so much about the library, but more about 'how much review doees this get'
 2412016-03-24T08:26:22  *** mrkent has quit IRC
 2422016-03-24T08:28:19  <jonasschnelli> wumpus: after thinking about it. maybe a simple C bases AES library could have a reason to exists.
 2432016-03-24T08:28:22  <sipa> gmaxwell: what is the fastest authenticated encryption scheme implementation that you know? (assume no hardware support, but constant time)?
 2442016-03-24T08:28:52  <wumpus> jonasschnelli: sure, it could, but this is specifically a constant-time AES for wallet encryption
 2452016-03-24T08:29:20  *** Arnavion has quit IRC
 2462016-03-24T08:29:38  <jonasschnelli> wumpus: Right. If the p2p encryption will be implemented once, it could be extended there (AES CGM).
 2472016-03-24T08:29:46  <gmaxwell> sipa: people who want that are using chacha20-poly1305.  AES-GCM with hardware support is apparently somewhat faster (and much less power hungry); but without it the chacha20-poly1305 stuff is faster.
 2482016-03-24T08:29:47  <wumpus> it's e.g. not the fastest way to implement AES, because we specifically don't need that. There may not be many projects that want to tuse it at all, and there's a long tradition of copy/pasting crypto code anyhow....
 2492016-03-24T08:30:31  <gmaxwell> people who care about fast AES use hardware or use these parallel implementations which cannot be used for CBC mode, which compatiblity with the wallet encryption needs.
 2502016-03-24T08:31:11  <wumpus> gmaxwell: yes, exactly
 2512016-03-24T08:31:48  <wumpus> for the AES code I really have only one concern: is it correct
 2522016-03-24T08:32:10  <wumpus> for the random code I'm more scared because of the platform dependence and other uglyness involved
 2532016-03-24T08:32:45  <sipa> wumpus: agree
 2542016-03-24T08:32:57  <gmaxwell> currently we don't care at all about fast AES. We should somewhat care that it doesn't have stupid sidechannels.  Later, if we care about fast AES for p2p encryption, it would be AES-GCM and end up with a CBC incompatible implementation in any case.
 2552016-03-24T08:32:57  *** Arnavion has joined #bitcoin-core-dev
 2562016-03-24T08:33:27  *** AtashiCon has quit IRC
 2572016-03-24T08:33:41  <wumpus> yes, for P2P we could use a different implementation of AES, or not AES at all (chacha20 or so :p)
 2582016-03-24T08:34:27  <gmaxwell> right or whatever CAESER selects, depending on what that happens.
 2592016-03-24T08:34:38  <wumpus> yes
 2602016-03-24T08:35:25  <sipa> gmaxwell: that's 2 years out
 2612016-03-24T08:35:46  <gmaxwell> (most of the CAESER candidates are AES based in any case, mostly addressing security complaints about GCM, and expecting hardware support to address the obnoxiousness of constant time AES)
 2622016-03-24T08:36:11  * wumpus really likes chacha20, it's so small, efficient, and seemingly simple
 2632016-03-24T08:36:51  *** AtashiCon has joined #bitcoin-core-dev
 2642016-03-24T08:37:56  <gmaxwell> it has a mixed following, it's apparently enough faster on ARM and without AES-NI that many in the web world have considered it urgent to support... but apparently a lot of the datacenter folks are less than enthused since it ends up much more power hungry than hardware AES.
 2652016-03-24T08:39:23  <wumpus> yes, right, it's specifically efficient in sw, if there is a hw implementation of AES it'll win out
 2662016-03-24T08:39:39  <jonasschnelli> What MAC AAD would make sense for the p2p encryption? hash of the pubkey&pubkey? IP/Port?
 2672016-03-24T08:40:44  <wumpus> let's just wait for hw chacha20 *ducks*
 2682016-03-24T08:41:03  <gmaxwell> wumpus: well, its not like they have anything better to do with the transistors...
 2692016-03-24T08:41:35  <wumpus> gmaxwell: agree, the dark silicon problem
 2702016-03-24T08:42:59  <wumpus> then again - for bitcoin nodes we need to worry about consumer hw, not so much big datacenters
 2712016-03-24T08:43:39  <wumpus> I think it's safe to say there won't be so many bitcoin nodes in datacenters to rival the number of https implementations, let alone be a power usage concern
 2722016-03-24T08:43:53  <gmaxwell> well a lot of 'consumer hardware' has hardware aes now too, just not phones so much. :)
 2732016-03-24T08:44:00  <gmaxwell> but indeed.
 2742016-03-24T08:44:41  <wumpus> power usage on the other hand is mostly a phone concern. But yeah.
 2752016-03-24T08:46:32  <gmaxwell> in any case both, given good implementations, should be faster the current 'checksum'
 2762016-03-24T08:48:16  <sipa> jonasschnelli: the additional data is sent in every message
 2772016-03-24T08:51:15  <sipa> jonasschnelli: you would include the message length there, for example
 2782016-03-24T08:51:21  <jonasschnelli> sipa: hmm.. so hash would be too expansive?
 2792016-03-24T08:51:39  <sipa> hash of what?
 2802016-03-24T08:52:06  <jonasschnelli> hash of the encryption pubkeys or could also be a hash of the plaintext message.
 2812016-03-24T08:54:29  <sipa> what are you trying to do?
 2822016-03-24T08:55:02  <sipa> the additional data is a feature provided by authenticated encryption schemes; you don't need to use it if you have nk need for it
 2832016-03-24T09:02:24  *** davec has quit IRC
 2842016-03-24T09:03:12  *** davec has joined #bitcoin-core-dev
 2852016-03-24T09:04:09  <jonasschnelli> sipa: So do I got this right? You could use something like the current protocol version "70012" as AAD. And during decryption, you feed the TAG (processed AAD) together with the AAD into the decryption method and get a true or false?
 2862016-03-24T09:04:18  <jonasschnelli> And thanks for you time to explain that to me. :)
 2872016-03-24T09:06:01  <gmaxwell> you could-- generally it's used for additional per message data that must be sent cleartext for protocol reasons, but which you'd still like included in the authentication.
 2882016-03-24T09:06:58  *** xiangfu has quit IRC
 2892016-03-24T09:07:37  <jonasschnelli> gmaxwell: Okay. Right. Message length would be ideal then.
 2902016-03-24T09:09:59  <gmaxwell> that would be the kind of thing; even that may not be necessary, if the length was wrong, the auth would fail in any case-- wrong amount of data in it.
 2912016-03-24T09:23:01  *** Alopex has quit IRC
 2922016-03-24T09:24:06  *** Alopex has joined #bitcoin-core-dev
 2932016-03-24T09:32:24  *** Arnavion has quit IRC
 2942016-03-24T09:35:32  *** MarcoFalke has joined #bitcoin-core-dev
 2952016-03-24T09:36:58  *** Arnavion has joined #bitcoin-core-dev
 2962016-03-24T09:37:19  *** Arnavion has quit IRC
 2972016-03-24T09:38:25  *** Arnavion has joined #bitcoin-core-dev
 2982016-03-24T09:38:34  *** molz has joined #bitcoin-core-dev
 2992016-03-24T09:41:41  *** molly has quit IRC
 3002016-03-24T09:46:27  *** cdecker has quit IRC
 3012016-03-24T09:55:02  *** p15 has joined #bitcoin-core-dev
 3022016-03-24T10:11:32  *** AaronvanW has joined #bitcoin-core-dev
 3032016-03-24T10:15:57  <sipa> jonasschnelli: i guess you can include the network magic to the aad
 3042016-03-24T10:16:44  <jonasschnelli> sipa: okay. Would that be much different then the encrypted message length?
 3052016-03-24T10:18:56  <sipa> jonasschnelli: i mean both... but there is no strong reason
 3062016-03-24T10:19:08  <jonasschnelli> okay.
 3072016-03-24T10:19:37  <sipa> essentially it guarantees that the length/magic came from someone with the session key
 3082016-03-24T10:20:39  <jonasschnelli> ok
 3092016-03-24T10:22:57  *** p15 has quit IRC
 3102016-03-24T10:36:40  <sipa> aes-gcm uses 96-bit IVs, but they only require being unique for a given key
 3112016-03-24T10:36:49  <sipa> and the key is establishes through ecdh
 3122016-03-24T10:37:23  <sipa> so they can just be the message number, and don't need to be transmitted
 3132016-03-24T11:14:44  *** MarcoFalke has quit IRC
 3142016-03-24T11:18:48  <GitHub140> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/597494f5a90c041945006b8f3eff8f7e482e0f2f
 3152016-03-24T11:18:48  <GitHub140> bitcoin/0.12 597494f Jonas Schnelli: Remove openssl info from init/log and from Qt debug window...
 3162016-03-24T11:20:01  <GitHub121> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/e5c35119e967...3ba07bdf7da4
 3172016-03-24T11:20:02  <GitHub121> bitcoin/master 146746b Alice Wonder: All files related to my RPM spec file project in one commit
 3182016-03-24T11:20:02  <GitHub121> bitcoin/master 0e4b50a Alice Wonder: Description of RPM directory
 3192016-03-24T11:20:03  <GitHub121> bitcoin/master 3ba07bd Wladimir J. van der Laan: Merge #7609: All files related to my RPM spec file project in one commit...
 3202016-03-24T11:20:08  <GitHub36> [bitcoin] laanwj closed pull request #7609: All files related to my RPM spec file project in one commit (master...master) https://github.com/bitcoin/bitcoin/pull/7609
 3212016-03-24T11:21:31  <GitHub199> [bitcoin] laanwj closed pull request #7650: Cache CBlockIndex::GetMedianTimePast (master...enhancement/cache-getmediantimepast) https://github.com/bitcoin/bitcoin/pull/7650
 3222016-03-24T11:28:48  <GitHub62> [bitcoin] laanwj closed pull request #6973: Compress Blocks before sending (master...compress) https://github.com/bitcoin/bitcoin/pull/6973
 3232016-03-24T11:32:34  <sipa> jonasschnelli: i now want to go implement gcm or poly1305...
 3242016-03-24T11:32:56  * sipa resists, and works on segwit instead
 3252016-03-24T11:50:56  *** fengling has quit IRC
 3262016-03-24T11:58:47  <jonasschnelli> sipa: hah. Yes. Work on segwit. The p2p encryption is not urgent.
 3272016-03-24T11:59:21  *** fengling has joined #bitcoin-core-dev
 3282016-03-24T12:02:26  <jonasschnelli> sipa: regarding GCM IV: «[...] For a fixed value of the key, each IV value must be distinct, but need not have equal lengths [...]»
 3292016-03-24T12:03:11  <jonasschnelli> 96 bit is ideal though.
 3302016-03-24T12:09:15  <sipa> jonasschnelli: you could get 64 bits of the iv out of ecdh, and then add a 32-bit message counter, but that shouldn't be needed
 3312016-03-24T12:15:38  *** laurentmt has joined #bitcoin-core-dev
 3322016-03-24T12:28:35  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 3332016-03-24T12:38:37  <Chris_Stewart_5> is there a BIP66 post mortem some where about the hard fork?
 3342016-03-24T12:39:17  <sipa> BIP66 was not a hard fork... there were some miners that mined invalid blocks, and kept going
 3352016-03-24T12:39:48  *** mihi has joined #bitcoin-core-dev
 3362016-03-24T12:40:32  <Chris_Stewart_5> sipa: Essentially the miners not enforcing this function right? https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L98
 3372016-03-24T12:52:59  <jonasschnelli> sipa: ecdh produces a 256bit secret? right? How would you take 64bits for the iv if you need to 256bits for the symmetric key?
 3382016-03-24T13:24:09  *** achow101 has joined #bitcoin-core-dev
 3392016-03-24T13:42:58  <jonasschnelli> sipa gmaxwell: I'm happy if you are interested in reviewing the authentication and the encryption BIP (https://github.com/jonasschnelli/bips/blob/7e8a9f6e6a06cec1e7842452c85a9dec3730771b/bip-undef-1.mediawiki   https://github.com/jonasschnelli/bips/blob/7e8a9f6e6a06cec1e7842452c85a9dec3730771b/bip-undef-0.mediawiki)
 3402016-03-24T13:43:45  <jonasschnelli> Don't review the language/orthography, will overhaul that once the technical details are clear.
 3412016-03-24T13:44:27  * jonasschnelli can't tell you how hard it is to talk and write english if its a complete foreign language
 3422016-03-24T13:56:29  *** supasonic has joined #bitcoin-core-dev
 3432016-03-24T13:58:59  *** mihi has quit IRC
 3442016-03-24T14:00:13  <GitHub55> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3ba07bdf7da4...b88e0b0c610a
 3452016-03-24T14:00:13  <GitHub55> bitcoin/master d6cc6a1 João Barbosa: Use CCoinControl selection in CWallet::FundTransaction
 3462016-03-24T14:00:14  <GitHub55> bitcoin/master b88e0b0 Wladimir J. van der Laan: Merge #7506: Use CCoinControl selection in CWallet::FundTransaction...
 3472016-03-24T14:00:18  <GitHub17> [bitcoin] laanwj closed pull request #7506: Use CCoinControl selection in CWallet::FundTransaction (master...enhancement/use-coin-control-selection) https://github.com/bitcoin/bitcoin/pull/7506
 3482016-03-24T14:18:11  <sipa> jonasschnelli: if you want to use a single negotiation to intialize encryption for both directions, you need to make sure that the IVs in bith directions are different, or you break the assumption that an attacker cannot see two different messages with the same key and iv
 3492016-03-24T14:20:27  <sipa> jonasschnelli: my suggestion here would be to have an encinit ("if you want, you can start sending encrypted messages to me and this is my key") and an encack ("yeah, every message after this one will be encrypted, and this is my key"), which you both send in both directions
 3502016-03-24T14:21:24  <sipa> jonasschnelli: that way, both directions have independent ECDH negotiations, and you also have no problems with synchronization
 3512016-03-24T14:22:18  <sipa> jonasschnelli: an alternative is thinking hard about synchronization, and making sure that the initiator and accepter of the encryption connection use separate IVs
 3522016-03-24T14:22:45  <sipa> jonasschnelli: reusing the key as IV is not very useful, i think
 3532016-03-24T14:24:50  <sipa> jonasschnelli: you probably want to include some text about how this interacts with version/verack (does encinit have to be the first message entirely? can it be done at any time?
 3542016-03-24T14:31:31  <sipa> jonasschnelli: your encrypted message protocol spec could make the authentication tag explicit (it's 16 bytes for aes-gcm, though we could decide to truncate it)
 3552016-03-24T14:33:09  *** davec has quit IRC
 3562016-03-24T14:38:16  *** fengling has quit IRC
 3572016-03-24T14:40:49  <jonasschnelli> sipa: thanks for the review! Will "process" your points and come back to you with feedback.
 3582016-03-24T14:44:11  *** lysobit has joined #bitcoin-core-dev
 3592016-03-24T14:44:41  *** musalbas has joined #bitcoin-core-dev
 3602016-03-24T14:54:26  *** BCBot_ has joined #bitcoin-core-dev
 3612016-03-24T15:02:43  *** davec has joined #bitcoin-core-dev
 3622016-03-24T15:05:32  *** fengling has joined #bitcoin-core-dev
 3632016-03-24T15:08:10  *** goregrin1 has joined #bitcoin-core-dev
 3642016-03-24T15:08:34  *** gmaxwell_ has joined #bitcoin-core-dev
 3652016-03-24T15:08:58  *** gmaxwell_ is now known as Guest18913
 3662016-03-24T15:11:43  *** CyrusV` has joined #bitcoin-core-dev
 3672016-03-24T15:13:16  *** arubi has quit IRC
 3682016-03-24T15:13:16  *** ebfull has quit IRC
 3692016-03-24T15:13:17  *** gmaxwell has quit IRC
 3702016-03-24T15:13:17  *** Cory has quit IRC
 3712016-03-24T15:13:18  *** goregrind has quit IRC
 3722016-03-24T15:13:18  *** CyrusV has quit IRC
 3732016-03-24T15:13:19  *** windsok has quit IRC
 3742016-03-24T15:29:39  *** ebfull has joined #bitcoin-core-dev
 3752016-03-24T15:29:39  *** Cory has joined #bitcoin-core-dev
 3762016-03-24T15:29:39  *** windsok has joined #bitcoin-core-dev
 3772016-03-24T15:30:21  *** Cory has quit IRC
 3782016-03-24T15:31:36  *** Guyver2 has joined #bitcoin-core-dev
 3792016-03-24T15:33:38  *** Cory has joined #bitcoin-core-dev
 3802016-03-24T15:36:28  *** arubi has joined #bitcoin-core-dev
 3812016-03-24T15:38:39  *** CyrusV` is now known as CyrusV
 3822016-03-24T16:03:57  *** arowser has quit IRC
 3832016-03-24T16:04:16  *** arowser has joined #bitcoin-core-dev
 3842016-03-24T16:51:24  *** Aleph0 has quit IRC
 3852016-03-24T16:54:07  *** Guest18913 has quit IRC
 3862016-03-24T16:54:08  *** Guest18913 has joined #bitcoin-core-dev
 3872016-03-24T16:54:18  *** Guest18913 is now known as gmaxwell
 3882016-03-24T17:01:53  *** Don_John has joined #bitcoin-core-dev
 3892016-03-24T17:03:51  *** Don_John has quit IRC
 3902016-03-24T17:04:21  *** Don_John has joined #bitcoin-core-dev
 3912016-03-24T17:09:53  *** laurentmt has quit IRC
 3922016-03-24T17:27:46  <sipa> jonasschnelli: this is interesting: https://github.com/jhcloos/openssh-chacha-poly1305/blob/master/PROTOCOL.chacha20poly1305
 3932016-03-24T17:28:10  <sipa> openssh's chacha20-poly1305 aead at a high level
 3942016-03-24T17:28:30  <sipa> they encrypt the packet sizes too, but with a separate key
 3952016-03-24T17:29:00  <sipa> and then authenticate using the second key which is used for authentication and encryption of the data
 3962016-03-24T17:29:28  <sipa> which means that an attacker can't even see the message sizes, except by traffic analysis
 3972016-03-24T17:30:02  <btcdrak> Clever
 3982016-03-24T17:48:53  <gmaxwell> a little sad to require 4 byte lenghts.
 3992016-03-24T17:50:38  *** jgarzik has joined #bitcoin-core-dev
 4002016-03-24T17:52:40  *** jcorgan has joined #bitcoin-core-dev
 4012016-03-24T17:58:52  *** moli has joined #bitcoin-core-dev
 4022016-03-24T17:59:30  <gmaxwell> sipa: for identified links, we could do a nice thing with rekeying to make them strong against ECC breaks.
 4032016-03-24T18:00:50  *** molz has quit IRC
 4042016-03-24T18:01:17  <sipa> cfields: if we'd want to use something like that, i guess we want an api that allows a client to tell the comnection manager "let me know when you have X bytes"
 4052016-03-24T18:01:50  <sipa> cfields: which is oerhaps a cleaner abstraction than hardcoding a particular stream protocol?
 4062016-03-24T18:04:00  *** t800 has joined #bitcoin-core-dev
 4072016-03-24T18:04:27  <sipa> jonasschnelli: another suggestion: i think we should drop the network magic... resynchronization after garbage hasn't been supported in a long time, it wastes (a tiny bit of) bandwidth, and makes the encrypted stream look decidedly identifiable (though if encrypted connections bootstrap as unencrypted ones, that isn't solvable)
 4082016-03-24T18:05:05  <gmaxwell> Alice connects to Bob, Alice sends,  Nonce_a, and alice ephemeral pubky (AEP), bob sends nonce_b and BEP.   ECDH runs,  then a KDF that takes the two nonces, and ecdh output, and spits out a session id and two keys (one for each direction.   Then Alice proposes, "I think you have identity X=H(bob pubkey, session_id)", if this is true, bob responds with "I am rekeying with IDX"  and takes his cur
 4092016-03-24T18:05:11  <gmaxwell> rent encryption key and changes it to H(bob pubkey|enckey).  When alice sees that message she updates her keys for the stream. Then also does the same herselve (sends a message, updates key). Then bob sends a signature with his pubkey.     Then this authentication procedure can just be repeated every so many bytes in order to rotate the stream cipher keys.
 4102016-03-24T18:05:45  <gmaxwell> If the 'public keys' are kept private, the link has at least symmetric keyed integrity and confidentality, even with an ECC break.
 4112016-03-24T18:08:06  *** kx has joined #bitcoin-core-dev
 4122016-03-24T18:14:45  *** kx has quit IRC
 4132016-03-24T18:17:10  *** kangx has joined #bitcoin-core-dev
 4142016-03-24T18:23:29  *** treehug88 has joined #bitcoin-core-dev
 4152016-03-24T18:25:38  *** MarcoFalke has joined #bitcoin-core-dev
 4162016-03-24T18:27:12  *** frankenmint has quit IRC
 4172016-03-24T18:28:29  <wumpus> <sipa> which means that an attacker can't even see the message sizes, except by traffic analysis <- I think that would be quite essential for bitcoin, as so much can already be identified from the packet sizes
 4182016-03-24T18:28:56  <wumpus> certain blocks, certain transactions, etc
 4192016-03-24T18:30:39  *** belcher has joined #bitcoin-core-dev
 4202016-03-24T18:31:25  <gmaxwell> I had assumed that the encrypted transport would encapsulate one or more length coded messages inside it... so mostly you'd just see the 'send units'.  Doing this would alow avoiding that extra layering, however... and thus decrease overhead.
 4212016-03-24T18:32:31  <gmaxwell> though perhaps we should make the encoding support padding, so that messages can be padded up to particular size multiplies, to reduce the traffic analysis sidechannel.
 4222016-03-24T18:34:09  <CodeShark> is there a meeting today?
 4232016-03-24T18:34:42  <gmaxwell> yes, in about 20 minutes.
 4242016-03-24T18:35:01  <gmaxwell> You've suffered DST. :)
 4252016-03-24T18:36:04  <btcdrak> we dont suffer it over this side of the Atlantic until Sunday
 4262016-03-24T18:37:59  *** moli has quit IRC
 4272016-03-24T18:38:30  <btcdrak> gmaxwell: is it maybe worth adding a version byte to the encryption scheme to allow upgrades in the future?
 4282016-03-24T18:38:31  <jonasschnelli> gmaxwell: +1 for padding to reduce sidechannel identification.
 4292016-03-24T18:39:03  <jonasschnelli> sipa: do you mean dropping the network magic for all post-encryption-init messages?
 4302016-03-24T18:39:27  <sipa> also +1 to allowing multiple messages in a packet
 4312016-03-24T18:39:49  <jonasschnelli> You mean one header with a message count int followed by n payloads?
 4322016-03-24T18:39:51  <sipa> jonasschnelli: yes, it serves no purpose other than detecting accidental connection to the wrong network
 4332016-03-24T18:40:05  *** moli has joined #bitcoin-core-dev
 4342016-03-24T18:40:07  <sipa> jonasschnelli: or just a concatenation of payloads even
 4352016-03-24T18:40:36  <jonasschnelli> Okay. size + payload + size + payload, etc. That makes sense.
 4362016-03-24T18:41:44  <sipa> jonasschnelli: inside the aead
 4372016-03-24T18:42:03  <gmaxwell> that cuts down on the authentication overhead.
 4382016-03-24T18:42:08  <jonasschnelli> sipa: But I still try to understand your AES GCM 96bit IV solution. How would you create distinctable IV  without transmitting in in the message?
 4392016-03-24T18:43:15  <sipa> jonasschnelli: use 0000 + counter for one side, use 1111 + counter for the other?
 4402016-03-24T18:43:39  <jonasschnelli> sipa: Are predictable IVs not a problem?
 4412016-03-24T18:43:45  <sipa> jonasschnelli: no, just no reuse
 4422016-03-24T18:43:49  <sipa> ivs are nkt secret
 4432016-03-24T18:43:52  <sipa> *not
 4442016-03-24T18:44:12  <jonasschnelli> Okay. I see.
 4452016-03-24T18:44:23  <gmaxwell> sipa: AEAD packet size does imply memory usage implications though.
 4462016-03-24T18:45:16  <gmaxwell> but its maximum could just be the same as the largest message.
 4472016-03-24T18:45:20  <sipa> right
 4482016-03-24T18:45:44  <jonasschnelli> So. Changes i'll work on a) ecdh exchange for both direction, two sym. keys, fix synchro. b) use chacha20 poly1305 instead of AES GCM c) optimize message structure (drop magic, allow multiple payloads in the AEAD part)
 4492016-03-24T18:46:17  <sipa> jonasschnelli: no need to fix synchronization if both directions are independent
 4502016-03-24T18:46:58  <jonasschnelli> yes. I mean fix syncho by using two keypair, one per direction.
 4512016-03-24T18:47:04  *** jcorgan has quit IRC
 4522016-03-24T18:47:59  <gmaxwell> great.
 4532016-03-24T18:47:59  <jonasschnelli> Has chacha20-poly1305 been proven to be stable? Reasonable amount of cryptanalysis? Or do we threat stuff out of DJB's kitchen as stable anyways? :)
 4542016-03-24T18:48:05  *** gevs has quit IRC
 4552016-03-24T18:48:18  <gmaxwell> jonasschnelli: it's part of TLS and SSH.
 4562016-03-24T18:48:28  <gmaxwell> I think it's sutiable for our purposes.
 4572016-03-24T18:48:31  <wumpus> it's geting a lot of testing and review in ssh at least
 4582016-03-24T18:49:00  <jonasschnelli> Okay. And how complex is an implementation? Or would this lead to a revival of openssl in core?
 4592016-03-24T18:49:12  <gmaxwell> it's very simple.
 4602016-03-24T18:49:26  <wumpus> their implementation is very short, that's one thing that is so cool about it
 4612016-03-24T18:49:29  <jonasschnelli> what speaks again unsing openssl (p2p layer enc is non consensus)?
 4622016-03-24T18:49:47  <jonasschnelli> *against
 4632016-03-24T18:50:07  <gmaxwell> we're not taking 400,000+ lines of unreviewable, frequently CVEed code for this; I'd rather not have the functionality.
 4642016-03-24T18:50:11  <wumpus> attack surface, making it impossible to get rid of openssl dependency
 4652016-03-24T18:50:44  <wumpus> and you can already do it using stunnel
 4662016-03-24T18:51:22  <gmaxwell> I hope that this works so that even if we don't care about the confidentiality improvement, it's just simply faster than the existing transport.
 4672016-03-24T18:52:06  <jonasschnelli> Right. I think dropping the sha256 per message could be an improvement in performance.
 4682016-03-24T18:52:24  <jonasschnelli> combining multiple messages into one encrypted message also
 4692016-03-24T18:53:03  <jonasschnelli> The question is, how to "bundle" messages (example: bundle invs). When releasing them, etc. I mean for the implementation.
 4702016-03-24T18:53:13  <gmaxwell> https://www.imperialviolet.org/2014/02/27/tlssymmetriccrypto.html has some benchmarks vs AES-GCM fwiw.
 4712016-03-24T18:53:17  <sipa> jonasschnelli: we already do that
 4722016-03-24T18:53:38  <sipa> jonasschnelli: SendMessages is a function in main that computes a block of messages to send at once
 4732016-03-24T18:53:44  <jonasschnelli> Ah. But they get sent individually (one inv per msg)?
 4742016-03-24T18:54:02  <sipa> jonasschnelli: maybe you should look into how our current protocol works :)
 4752016-03-24T18:54:08  <jonasschnelli> haha. True.
 4762016-03-24T18:54:23  <sipa> no, an inv messages can contain up to 5000 invs
 4772016-03-24T18:55:00  <jonasschnelli> Okay. So where would we benefit from bundling messages in one encrypted message?
 4782016-03-24T18:55:20  <wumpus> it makes sense to make it possible, it benefits privacy
 4792016-03-24T18:55:58  *** achow101 has quit IRC
 4802016-03-24T18:55:59  <sipa> well, if you take as baseline an implementation that encrypts message sizes, there is no privacy benefit to bundling
 4812016-03-24T18:56:04  <wumpus> even if your first implementation doesn't actually use it, imo the protocol should support it
 4822016-03-24T18:56:16  <gmaxwell> phantomcircuit had a patch to add corking and uncoarking in the message handling loop, I presume something similar would be done for message batching.
 4832016-03-24T18:56:18  <jonasschnelli> agree
 4842016-03-24T18:56:20  *** achow101 has joined #bitcoin-core-dev
 4852016-03-24T18:56:37  <wumpus> ok, yes, no privacy benefit to bundling makes it less important to me
 4862016-03-24T18:56:39  <sipa> as they would be sent back-to-back, so traffic analysis would not let you discover independent messages
 4872016-03-24T18:56:51  <gmaxwell> e.g. we'd cork both the crypto and tcp, do all the message handling for a peer, then uncork both.
 4882016-03-24T18:57:32  <sipa> however, bundling does give a small bandwidth improvement, as you'd share a authentication tag acorss muktiple messages
 4892016-03-24T18:57:50  <wumpus> well, let's not micro-optimize before we have anything
 4902016-03-24T18:57:56  <jonasschnelli> tag is 12bytes. Right.
 4912016-03-24T18:58:05  <gmaxwell> not necessarily all that small, the auth is relatively large to our average message size.
 4922016-03-24T18:58:38  <wumpus> let's first make sure it is secure, then wonder about bandwidth optimization
 4932016-03-24T18:59:11  <jonasschnelli> Right. I'll try to update the Bips with what we have discusses and ask for another review.
 4942016-03-24T18:59:20  <jonasschnelli> *discussed
 4952016-03-24T18:59:23  <wumpus> trying to tackle everything at once is going to make this go nowhere, I'm afraid
 4962016-03-24T18:59:31  <gmaxwell> the bundling also makes padding easy, you just define a new message type "pad" that is just discarded on the other end. :)
 4972016-03-24T18:59:37  <sipa> jo	16 if you want 128 bit security
 4982016-03-24T18:59:40  *** molz has joined #bitcoin-core-dev
 4992016-03-24T18:59:44  <gmaxwell> yea, iteration is fine.
 5002016-03-24T18:59:46  <jonasschnelli> should i say now: "perfect is the enemy of good". Yes? No. *duck*
 5012016-03-24T19:00:04  <MarcoFalke> meeting time?
 5022016-03-24T19:00:10  <gmaxwell> Meeting time.
 5032016-03-24T19:00:10  <wumpus> jonasschnelli: more like: keep your goal focused. Your goal  is security and privacy, first. Make no dumb mistakes there ;)
 5042016-03-24T19:00:24  <jonasschnelli> RIght!
 5052016-03-24T19:00:25  <wumpus> #startmeeting
 5062016-03-24T19:00:25  <lightningbot> Meeting started Thu Mar 24 19:00:25 2016 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
 5072016-03-24T19:00:25  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
 5082016-03-24T19:00:47  <wumpus> jonasschnelli: scope creep is a very bitcoin-y thing
 5092016-03-24T19:00:52  <wumpus> ok, topic proposals?
 5102016-03-24T19:01:05  <MarcoFalke> python 3 and rpc tests?
 5112016-03-24T19:01:14  <wumpus> from last meeitng we have ACTION: merge #7575 (wumpus, 19:04:04)  ... that was done
 5122016-03-24T19:01:19  <btcdrak> softfork #7648 status
 5132016-03-24T19:01:31  <wumpus> next was ACTION: review #7648 after #7575 is merged (wumpus, 19:09:39)
 5142016-03-24T19:01:32  <Luke-Jr> topic proposal: v0.11.3 and v0.10.5
 5152016-03-24T19:01:34  *** moli has quit IRC
 5162016-03-24T19:01:42  <wumpus> how is that going?
 5172016-03-24T19:01:53  <wumpus> #topic softfork #7648 status
 5182016-03-24T19:02:08  <wumpus> MarcoFalke: will get to python3 at the end, I think the softfork stuff is most important right now
 5192016-03-24T19:02:12  <btcdrak> #7648 has received some tested ACKs.
 5202016-03-24T19:02:37  <MarcoFalke> sure
 5212016-03-24T19:03:08  <wumpus> okay, good
 5222016-03-24T19:03:46  <wumpus> I think it needs more review still
 5232016-03-24T19:04:15  <btcdrak> time to crack the whip.
 5242016-03-24T19:04:24  <wumpus> for a softfork the bar is somewhat higher than for random pull #1234
 5252016-03-24T19:04:39  <wumpus> yes
 5262016-03-24T19:05:03  <jonasschnelli> 7648 looks good and will also review it in details soon (during weekend)
 5272016-03-24T19:05:08  <wumpus> I mean I've seen some acks but very little comment on the actual code, that could be a sign everything is perfect, or people need to look better
 5282016-03-24T19:05:29  <wumpus> thanks jonasschnelli
 5292016-03-24T19:05:40  <btcdrak> MarcFalke: can you look as well please?
 5302016-03-24T19:05:42  <wumpus> #action review more at https://github.com/bitcoin/bitcoin/pull/7648
 5312016-03-24T19:05:47  <btcdrak> and CodeShark
 5322016-03-24T19:05:52  <wumpus> I'll take a more in-depth look at it as well
 5332016-03-24T19:05:55  <btcdrak> ty
 5342016-03-24T19:05:57  <MarcoFalke> sure
 5352016-03-24T19:06:08  <gmaxwell> has anyone looked to see if there are MTP violations still being accepted on the network?
 5362016-03-24T19:06:39  <gmaxwell> I wouldn't consider them a hard blocker to 113 deployment, but if any are, we should make a concerted effort to extinguish them first.
 5372016-03-24T19:06:59  <btcdrak> gmaxwell: there should be 100% coverage of 113 since the upgrade to 0.11.2
 5382016-03-24T19:07:16  <gmaxwell> btcdrak: there may be miners with order or modified code, however.
 5392016-03-24T19:07:28  <gmaxwell> er, s/order/older/
 5402016-03-24T19:07:51  <sipa> s/may be/are/
 5412016-03-24T19:08:25  <gmaxwell> if there are any, we should try to find them and get them to stop mining MTP violations.
 5422016-03-24T19:09:34  <wumpus> yes
 5432016-03-24T19:09:39  <gmaxwell> I can work on this.
 5442016-03-24T19:09:49  <cfields> sipa: missed you earlier. ack on allowing for chunked decrypt/decode.
 5452016-03-24T19:10:02  <btcdrak> cfields: can you put 7648 review on your tasklist too please?
 5462016-03-24T19:10:08  <wumpus> #action hunt down miners that mine MTP violations
 5472016-03-24T19:10:35  <cfields> btcdrak: yes
 5482016-03-24T19:11:34  <wumpus> ok
 5492016-03-24T19:11:54  <wumpus> next topic I guess
 5502016-03-24T19:12:01  <wumpus> #topic proposal: v0.11.3 and v0.10.5
 5512016-03-24T19:12:29  <sipa> we discussed what would go in last week; did anythkng chang
 5522016-03-24T19:12:30  <sipa> ?
 5532016-03-24T19:13:03  <wumpus> I don't know, luke-jr requested the topic
 5542016-03-24T19:13:10  <btcdrak> well we didnt discuss 0.10.5, we discussed 0.12.1
 5552016-03-24T19:13:12  <Luke-Jr> I'm going through https://github.com/bitcoin/bitcoin/pull/7047 again, to check everything included is in 0.12 and make it mergable again
 5562016-03-24T19:14:00  <wumpus> that's way too much for a softfork release at least
 5572016-03-24T19:14:01  <Luke-Jr> what isn't clear to me is what the plan is for v0.11.3 being released - I assume wumpus is doing that at some point? - and whether I'm taking over for v0.10.5, or if wumpus wanted to do one final 0.10 before passing it off
 5582016-03-24T19:14:27  <wumpus> for a normal minor release it'd make more sense
 5592016-03-24T19:14:52  <wumpus> yes I'll do the 0.11.x release
 5602016-03-24T19:14:58  <MarcoFalke> 7047 also includes a lot of "comment-fixed". I am not sure if they are useful in our branch
 5612016-03-24T19:15:01  <gmaxwell> whats deployment of 0.11.2 vs 0.12 look like right now?
 5622016-03-24T19:15:06  <MarcoFalke> I'd rather only have bugfixes
 5632016-03-24T19:15:08  <wumpus> I'll leave 0.10.x to you
 5642016-03-24T19:15:08  <Luke-Jr> are we doing a softfork release in the next month?
 5652016-03-24T19:15:21  <sipa> Luke-Jr: i say yes
 5662016-03-24T19:15:32  <Luke-Jr> ok, so then we should let these things wait until after that
 5672016-03-24T19:15:38  <wumpus> I think we should do a softfork release asap
 5682016-03-24T19:15:50  <Luke-Jr> or maybe I should go through it and make sure there's nothing important
 5692016-03-24T19:15:52  <wumpus> let's just get those stupid things reviewed and merged
 5702016-03-24T19:15:57  <jonasschnelli> As soon as #7648 has been reviewed more and merged
 5712016-03-24T19:16:10  <wumpus> jonasschnelli: and backports reviewed and merged ofc
 5722016-03-24T19:16:21  <wumpus> but yes 7648 is the first step
 5732016-03-24T19:16:31  <jonasschnelli> RIght
 5742016-03-24T19:16:32  <Luke-Jr> (also, depending on ease of backporting the softforks, I may decide to just give up on 0.10 support; tbd)
 5752016-03-24T19:16:42  <btcdrak>  /Satoshi:0.12.0/	1713
 5762016-03-24T19:16:43  <btcdrak>  /Satoshi:0.11.2/	1387
 5772016-03-24T19:16:50  <jonasschnelli> 0.10 can be given up IMO
 5782016-03-24T19:16:55  * jonasschnelli is checking his seeder data
 5792016-03-24T19:17:01  <gmaxwell> Luke-Jr: I think absent someone requesting otherwise (and perhaps funding your effort) you probably should.
 5802016-03-24T19:17:08  <Luke-Jr> jonasschnelli: it's currently Gentoo stable, so at the very least I will need to bump that
 5812016-03-24T19:17:21  <btcdrak> The problem with 0.10 is we all need to put review time into the backports and that's difficult when we have trouble even reviewing backports for 0.11
 5822016-03-24T19:17:37  <petertodd> btcdrak: +1
 5832016-03-24T19:17:56  <Luke-Jr> btcdrak: most backports are trivial, but for softforks I agree if they're not a clean backport
 5842016-03-24T19:18:04  <wumpus> right
 5852016-03-24T19:18:12  <btcdrak> I think if there is a 0.10.5 release it should be extremely minimal... and frankly, backporting BIP68 wont be fun or easy.
 5862016-03-24T19:18:14  <sipa> bip68/112/113 are quite a bit of code
 5872016-03-24T19:18:28  <wumpus> let's spend our energy on moving forward
 5882016-03-24T19:18:33  <jonasschnelli> 0.10 should be last priority. If someone offers to do that: fine.
 5892016-03-24T19:18:41  <gmaxwell> plus, a far backport which is done by one person and used by few more likely won't be materially more stable than a more recent release.
 5902016-03-24T19:18:44  <Luke-Jr> ok, so for 0.10 I'll see the complexity in the softfork backports, and if it's anything more than absolutely trivial, I'll just decide to drop it
 5912016-03-24T19:18:47  <btcdrak> Luke-Jr: even BIP68 backport to 0.11 was tricky, not a clean cherry-pick.
 5922016-03-24T19:18:58  <Luke-Jr> btcdrak: well, I'd be backporting to 0.10 *from* 0.11
 5932016-03-24T19:19:19  <jonasschnelli> at dnsseed.dump | grep "  100.00% 100.00%" | grep "/Satoshi:0.10." | wc -l       ---> 266
 5942016-03-24T19:19:25  <sipa> we can still do critical security updates for 0.10 without choosing to backport sotfork
 5952016-03-24T19:19:33  <wumpus> sure
 5962016-03-24T19:19:45  <jonasschnelli> 6.3% 0.10.x nodes
 5972016-03-24T19:19:49  <btcdrak> so on this topic, people need to also look at backport #7543
 5982016-03-24T19:19:58  <Luke-Jr> and for 0.11, sounds like the plan is to do a softfork-only release before any non-essential bugfix backports
 5992016-03-24T19:20:08  <CodeShark> +1 jonasschnelli
 6002016-03-24T19:20:09  <petertodd> sipa: and people who need a v0.10.x node for software compatibility reasons can generally run it behind a newer node
 6012016-03-24T19:20:18  <sipa> petertodd: also true
 6022016-03-24T19:20:31  <wumpus> #action people need to also look at backport #7543: [0.12] Backport BIP9, BIP68 and BIP112 with softfork
 6032016-03-24T19:20:36  <sipa> petertodd: specifically, behind a 0.11 or 0.12 pruned one :)
 6042016-03-24T19:20:45  <petertodd> sipa: yes! that too
 6052016-03-24T19:20:57  <Luke-Jr> #action 0.10: attempt to backport softforks, and if non-trivial, discontinue support
 6062016-03-24T19:21:12  <Luke-Jr> #action 0.11: no non-essential fix backports until softfork release
 6072016-03-24T19:21:26  <Luke-Jr> #action bump Gentoo stable to 0.11 (from 0.10)
 6082016-03-24T19:21:38  <sipa> Luke-Jr: support is already discontinued according to policy, only critical biffixes
 6092016-03-24T19:21:42  <sipa> *bugfixes
 6102016-03-24T19:21:51  <CodeShark> I'd say discontinue support for 0.10 unless someone wishes to provide resources for it
 6112016-03-24T19:21:59  <wumpus> well if luke-jr wants to try to backport the softfork that's up to him
 6122016-03-24T19:21:59  <Luke-Jr> sipa: policy only dictates guaranteed support, not guaranteed to end
 6132016-03-24T19:22:22  <sipa> wumpus: sure, but i don't think that's very relevant here
 6142016-03-24T19:22:27  <wumpus> no need to even discuss it here :p
 6152016-03-24T19:22:28  <wumpus> right
 6162016-03-24T19:22:44  <btcdrak> #action 0.11 review #7716 Backport BIP9 and softfork for BIP's 68,112,113
 6172016-03-24T19:22:58  <Luke-Jr> any more topics?
 6182016-03-24T19:23:32  <sipa> as soon as we have 0.12.1, i will rebase segwit on top, so segnet can benefit from its bip9/68/112/113 support
 6192016-03-24T19:23:37  <cfields> can give a quick update on the net refactor
 6202016-03-24T19:23:50  <wumpus> I'd say spend the rest of the meeting time reviewing #7648 :p
 6212016-03-24T19:23:53  <wumpus> sure cfields!
 6222016-03-24T19:24:00  <wumpus> #topic cfields net refactor
 6232016-03-24T19:24:34  <sipa> please, stick to the net refactor, not the gross one
 6242016-03-24T19:24:56  <cfields> i finally have a working full-featured branch. it's a mess code-wise, but i think it should be approachable in the next day or two after some code movement. I expect to be looking for concept review next week
 6252016-03-24T19:24:58  <gmaxwell> cfields: start taking, I can't take the wait.
 6262016-03-24T19:25:02  <wumpus> lol
 6272016-03-24T19:25:23  <wumpus> cfields: I've been testing it on a node btw for last week, seems to be stable and working
 6282016-03-24T19:25:48  <wumpus> haven't looked at details of performance etc but no crashes or insane log entries
 6292016-03-24T19:26:02  <btcdrak> sipa: you can cherry-pick from  #7543 in that case
 6302016-03-24T19:26:10  <cfields> wumpus: that's great to hear. lots has changed since then, but the core is the same
 6312016-03-24T19:26:35  <cfields> i think i have an idea of how to handle code review, but i'd like to get some concept acks first...
 6322016-03-24T19:26:50  <wumpus> you got my concept ack at the boston meeting
 6332016-03-24T19:26:56  <sipa> and mine
 6342016-03-24T19:27:09  <cfields> there's a huge mix of code movement, a new lib, abstraction, etc. I think i have a plan for getting it slotted in
 6352016-03-24T19:27:28  <wumpus> great
 6362016-03-24T19:27:29  <cfields> ok. i can start PRing chunks into core as i go, then
 6372016-03-24T19:27:44  <Luke-Jr> cfields: practical to add block streaming? ;)
 6382016-03-24T19:28:02  *** frankenmint has joined #bitcoin-core-dev
 6392016-03-24T19:28:02  <cfields> first step is getting the net stuff instanced. that's a lot of movement without much functional difference
 6402016-03-24T19:28:02  <gmaxwell> do not creep cfield's scope. :P
 6412016-03-24T19:28:06  <wumpus> please, let's replace the current things first, then add additional features
 6422016-03-24T19:28:08  <jonasschnelli> hah
 6432016-03-24T19:28:09  <wumpus> example gmaxwell
 6442016-03-24T19:28:12  <wumpus> exactly*
 6452016-03-24T19:28:13  <cfields> i think that can start going in parallel
 6462016-03-24T19:28:37  <cfields> gmaxwell: heh, i've adjusted scope so many times now that i'm certainly not budging again :p
 6472016-03-24T19:28:44  <jonasschnelli> The net refactor will be very hard to review already. Lets keep it as simple as possible.
 6482016-03-24T19:28:51  <wumpus> that's good, stand your ground , people here are awful :)
 6492016-03-24T19:28:57  <cfields> turns out the absolute minimum scope for net refactor is already enormous
 6502016-03-24T19:29:07  <jonasschnelli> Indeed.
 6512016-03-24T19:29:29  <wumpus> yes it'll already be challenging to get this done before 0.13 I think, but we have to try
 6522016-03-24T19:29:31  <Luke-Jr> gmaxwell: indeed, just hope we don't need to refactor it again after
 6532016-03-24T19:29:41  <sipa> Luke-Jr: we absolutely will
 6542016-03-24T19:29:48  <cfields> wumpus: yes, it's much later than I was hoping for. If it slips past 0.13, I'll understand.
 6552016-03-24T19:29:51  <sipa> but at different layers :)
 6562016-03-24T19:30:04  <wumpus> right, this is only the bottom layer
 6572016-03-24T19:30:19  <cfields> sipa: well the hope is that it's been split up into chunks that can be changed in the future much easier
 6582016-03-24T19:30:37  <sipa> cfields: yes, that's what abstractions are for
 6592016-03-24T19:31:27  <jonasschnelli> topic: Should we shorty touch how we should proceed with sipas CT AES implementation? Extract to library? Yes/No?
 6602016-03-24T19:31:40  <cfields> there are a few hacks that i'm not going to mess with for now. For ex, even though it's all instanced, there's still a g_connman that's used liberally as opposed to the clean way of passing the instance around
 6612016-03-24T19:31:42  <wumpus> meh
 6622016-03-24T19:31:55  <cfields> i'll start working on a list of todos so that some of the decisions are more clear
 6632016-03-24T19:32:24  <petertodd> jonasschnelli: extract is a great idea I think - don't have to be fancy, just enough to continue to set a better standard than "roll our own"
 6642016-03-24T19:32:36  <sipa> cfields: just PR it! i want to see the code :)
 6652016-03-24T19:32:47  *** frankenmint has quit IRC
 6662016-03-24T19:32:49  <jonasschnelli> sipa: code: https://github.com/theuni/bitcoin/tree/net-refactor8
 6672016-03-24T19:32:52  <cfields> sipa: i'm still frantically coding it :)
 6682016-03-24T19:32:58  <sipa> ah.
 6692016-03-24T19:33:07  <cfields> jonasschnelli: nooooooo. everyone shield your eyes from that :)
 6702016-03-24T19:33:07  <wumpus> the code is actually buildable and works
 6712016-03-24T19:33:14  <sipa> now i have a moral obligation to actually go look, right?
 6722016-03-24T19:33:28  <wumpus> well test at least :)
 6732016-03-24T19:33:33  <btcdrak> cfields: love that last commit message :)
 6742016-03-24T19:33:35  <cfields> sipa: with the understanding that it's basically my /tmp :)
 6752016-03-24T19:33:35  <jonasschnelli> cfields: We all know how code during implementation can look. No shame for it!
 6762016-03-24T19:34:09  <wumpus> yes :)
 6772016-03-24T19:34:31  <jonasschnelli> petertodd: I'm also for extracting.
 6782016-03-24T19:34:54  <jonasschnelli> Simple autotools setup, add as subtree place on bitcoin/libctaes (or similar)
 6792016-03-24T19:34:57  <wumpus> #topic CT AES library
 6802016-03-24T19:35:12  <Luke-Jr> what is "CT" for here?
 6812016-03-24T19:35:14  <wumpus> I don't think extracting it into a library has to affect our use of it
 6822016-03-24T19:35:14  <petertodd> jonasschnelli yup, subtree is fine
 6832016-03-24T19:35:17  <wumpus> Constant Time
 6842016-03-24T19:35:18  <jonasschnelli> constant time
 6852016-03-24T19:35:24  <Luke-Jr> ah
 6862016-03-24T19:35:45  <sipa> i feel like it's more a code snippet than a library
 6872016-03-24T19:35:46  <gmaxwell> constant time.
 6882016-03-24T19:35:53  <wumpus> yes, it's more like a snippet
 6892016-03-24T19:35:56  <gmaxwell> "Petertodd raised verification concerns for the internal AES implementation. I could suggest some strategies on that." is what I wrote earlier. :P
 6902016-03-24T19:35:59  <wumpus> crypto has a rich tradition of copy/paste
 6912016-03-24T19:36:11  <cfields> fwiw, I plan on doing the same with libbtcnet
 6922016-03-24T19:36:13  <jonasschnelli> wumpus: I think placing stuff in libraries follows a modular approach  which is desirable for Bitcoin IMO
 6932016-03-24T19:36:24  <Luke-Jr> +1 for libctaes
 6942016-03-24T19:36:29  *** jcorgan has joined #bitcoin-core-dev
 6952016-03-24T19:36:34  <cfields> i have a tiny/crappy makefile that i use to build it externally, but I plan to just copy/paste it into Core and keep it in sync
 6962016-03-24T19:36:35  <petertodd> sipa: well, to be clear, you did write that code snippet from scratch yourself right?
 6972016-03-24T19:36:35  <wumpus> you could stil include it in a library, but please don't compilicate the build
 6982016-03-24T19:36:40  <Luke-Jr> cfields: libbcnet I hope; what does the BTC unit size have to do with networking? :P
 6992016-03-24T19:36:49  <gmaxwell> in any case, I'm not convinced that the standard test vectors prove the code correct. Beyond soliciting outside review, which btcdrak has done.  I was going to suggest trying some mutation testing to see if some additional better test vectors can be constructed.
 7002016-03-24T19:36:54  <jonasschnelli> sipa: Its a snipped now, ... but we all know how snippets evolve over time.
 7012016-03-24T19:37:00  <cfields> Luke-Jr: when i googled "libp2p" some etherium thing came up :)
 7022016-03-24T19:37:01  <sipa> petertodd: yes, though the logic gates for the sbox come from a paper
 7032016-03-24T19:37:14  <wumpus> I think we should separate concerns here: getting the code into a library, and getting the damn code reviewed
 7042016-03-24T19:37:18  <wumpus> the first is easy, just work
 7052016-03-24T19:37:18  <Luke-Jr> cfields: oh great, let's just integrate that! :P
 7062016-03-24T19:37:37  <petertodd> sipa: right, so it's independent work, which means it's crypto we wrote, which means no matter how small it is, we should at least make a token effort to try to let independent use and review :)
 7072016-03-24T19:37:38  <Luke-Jr> wumpus: I don't feel competent to review crypto code :<
 7082016-03-24T19:37:49  <wumpus> Luke-Jr: me neither.
 7092016-03-24T19:38:01  <wumpus> Luke-Jr: so we'll have to find someone that can, btcdrak had some ideas.
 7102016-03-24T19:38:04  <gmaxwell> You're all competent to review if the code is constant time or not.
 7112016-03-24T19:38:16  <sipa> or that it has test coverage
 7122016-03-24T19:38:16  <jonasschnelli> What about apoelstra?
 7132016-03-24T19:38:32  <gmaxwell> And probably no one is competent to review if it's correct, fortunately AES is a permutation, which does make confidence building via testing easier.
 7142016-03-24T19:38:40  <wumpus> but not if it's correct AES, or has some weird edge cases, mathatical inconsistencies, or other wacky side channels
 7152016-03-24T19:38:55  <wumpus> right
 7162016-03-24T19:39:02  <sipa> and notice it has no tables or branches, so it would be exceedingly hard to make it broken in a very small number of cases
 7172016-03-24T19:39:04  <jonasschnelli> IMO that all speak for splitting it off into a own library.
 7182016-03-24T19:39:04  <petertodd> gmaxwell: the problem here, is I don't know enough about the problem to know if I'm actually competent to review that
 7192016-03-24T19:39:07  <gmaxwell> (by no one, I mean no human. :) )
 7202016-03-24T19:39:35  <btcdrak> petertodd: I asked isis to take a look on a paid basis.
 7212016-03-24T19:39:41  <petertodd> btcdrak: good!
 7222016-03-24T19:39:48  <gmaxwell> petertodd: the requirement is that there should be no leakage of the secret state (key or data) in the timing of the program on any hardware we support.
 7232016-03-24T19:39:53  <btcdrak> that's @isislovecruft from Tor for anyone that doesnt know.
 7242016-03-24T19:40:01  * Luke-Jr volunteers to do the splitting-off-into-library work if sipa doesn't want to
 7252016-03-24T19:40:18  <sipa> anyway, i'll turn it into a .c/.h; if someone else wants to actually do the work for autotools etc, go ahead.
 7262016-03-24T19:40:19  <jonasschnelli> Luke-Jr: I already have started that. :)
 7272016-03-24T19:40:28  <wumpus> Luke-Jr: this will again trigger the subtree-or-external discussion, I can't take that another time
 7282016-03-24T19:40:45  <wumpus> Luke-Jr: and linux distributions messing it up etc
 7292016-03-24T19:40:51  <gmaxwell> it will not be external.
 7302016-03-24T19:40:53  <Luke-Jr> wumpus: seems that discussion should be project-global, not per-lib
 7312016-03-24T19:40:56  <jonasschnelli> sipa: I have checked to code and it looks pretty Cish.
 7322016-03-24T19:41:05  <Luke-Jr> subtree-with-configure-option seems to work
 7332016-03-24T19:41:12  <sipa> imho, it can be a separate repository without being a library
 7342016-03-24T19:41:22  <sipa> just copy it into your project if you neeed it
 7352016-03-24T19:41:27  <wumpus> right.
 7362016-03-24T19:41:38  <sipa> it's a single file and has no intention to grow beyond that
 7372016-03-24T19:41:56  <gmaxwell> nor any utility to grow beyond that.
 7382016-03-24T19:41:59  <jonasschnelli> meh. Adding a build setup should be trivial and would hurt nobody?
 7392016-03-24T19:42:02  <cfields> sipa: is there enough non-standard stuff to necessitate a buildsystem? or would a simple makefile do?
 7402016-03-24T19:42:11  <wumpus> the API will never change, the implementation will hopefully also never change (excluding critical problems)
 7412016-03-24T19:42:22  <sipa> cfields: it will be a single C89 file with no dependencies beyond stdint
 7422016-03-24T19:42:36  <Luke-Jr> stdint isn't C89, though?
 7432016-03-24T19:42:36  <gmaxwell> wumpus: the only 'change' I could imagine for it is wrapping it with something that uses AESNI when available instead.
 7442016-03-24T19:42:48  <sipa> Luke-Jr: it indeed isn't, which is why i mention it :)
 7452016-03-24T19:42:50  <cfields> sipa: ah great
 7462016-03-24T19:42:56  <gmaxwell> well C89+stdint, or C99...
 7472016-03-24T19:43:19  <gmaxwell> stdint is almost universal even in terrible embedded complilers.
 7482016-03-24T19:43:28  <jonasschnelli> Are there no plans to extend the "lib" with more stuff? Like for a possible p2p encryption?
 7492016-03-24T19:43:36  <wumpus> and if not it's easy enough to define the types yourself
 7502016-03-24T19:43:53  <Luke-Jr> jonasschnelli: well, then the name libctaes would be wrong
 7512016-03-24T19:43:57  <sipa> jonasschnelli: i'd do that on another layer
 7522016-03-24T19:43:58  <jonasschnelli> Right.
 7532016-03-24T19:44:03  <wumpus> no, jonasschnelli, that should be something else yet again
 7542016-03-24T19:44:05  <sipa> jonasschnelli: it would still rely on aes primitives
 7552016-03-24T19:44:27  <gmaxwell> This pressure to constantly create "libraries" to dump things in is streesful for me. A well constructed library is a serious amount of work in and of itself.
 7562016-03-24T19:44:29  <wumpus> ctaes would be constant time aes :-) I don't think you want/need that for the p2p
 7572016-03-24T19:44:31  <petertodd> gmaxwell: I may have to use a climbing analogy to get my point accross here :) sure, I can easily read a book on knots and belay systems and feel like I _should_ know everything I need to know to do that stuff safely, but the consequences of failure are high enough - and the field unusual enough - that I'm going to insist I check things out with others generally accepted as experts first - who cares how irrational ...
 7582016-03-24T19:44:32  <jonasschnelli> Okay. Then we might agree in a subtree with just a .c/.h instead of a autotools setup.
 7592016-03-24T19:44:37  <petertodd> ... that feels. I think general standards for low-level crypto implementations tend in that direction.
 7602016-03-24T19:45:05  <wumpus> jonasschnelli: yes
 7612016-03-24T19:45:06  <petertodd> jonasschnelli: yes, just a .c/.h is fine! like I said, bare minimum to set a precedent that we're trying to invite outside review
 7622016-03-24T19:45:08  <Luke-Jr> subtree with just .c/.h until some future time when someone decides to lib-ify it seems ok
 7632016-03-24T19:45:11  <wumpus> jonasschnelli: that's very common for crypto
 7642016-03-24T19:45:18  <sipa> and maybe for performance reasons, it makes sense to use a more complicated faster implementation, and maybe also warrants more maintainance afterwards if it's going to be adopted by other prijects for the purpose of p2p encryption
 7652016-03-24T19:45:31  <jonasschnelli> okay.
 7662016-03-24T19:45:43  <gmaxwell> petertodd: considering that basically all AES-CBC software (not AESNI) out there is gratitiously timing vulnerable, and we were going to take such a patch ourselves before; I feel you're compent enough to review and increase confidence, if not alone to achieve perfection.
 7672016-03-24T19:45:47  <Luke-Jr> sipa: seems likely other wallets may use it
 7682016-03-24T19:45:47  <jonasschnelli> agree on bitcoin/libctaes?
 7692016-03-24T19:45:48  <wumpus> and just copy/subtree the thing into your project and include it into your build system, all the dependency micro management isnt' worth it for everything
 7702016-03-24T19:46:05  <sipa> Luke-Jr: they absolutely may!
 7712016-03-24T19:46:16  <jonasschnelli> Luke-Jr: right.  I have two projects where i will use it immediately.
 7722016-03-24T19:46:21  <wumpus> jonasschnelli: bitcoin-core/libctaes it would be then
 7732016-03-24T19:46:27  <gmaxwell> as I noted before, this implementation is not super fast. We have no need to make it super fast. It may be of idependant interest to others because it is quite small.
 7742016-03-24T19:46:29  <sipa> Luke-Jr: but for wallets, it has no performance requirements
 7752016-03-24T19:46:36  <jonasschnelli> wumpus: ah. Right. The moving.
 7762016-03-24T19:46:56  <sipa> Luke-Jr: which is why it's easy, and needs no dependencies on assemblers or instruction sets, or simd extensions, or ...
 7772016-03-24T19:47:13  * Luke-Jr not sure libctaes makes sense as bitcoin-core as opposed to just bitcoin, but meh
 7782016-03-24T19:47:38  <petertodd> gmaxwell: yeah, I probably am - I also learned how to do climbing/caving ropework by buying a book on it, some equipment, and going out to find some trees to practice on. Doesn't mean it was a good idea. :) just having a separate repo with a bare .c/.h goes a long way to making our intentions clear and having good precedents, even if pragmatically we need a solution right now.
 7792016-03-24T19:47:38  <wumpus> jonasschnelli: which reminds me, I think I still need to add you to that project
 7802016-03-24T19:47:49  <jonasschnelli> bitcoin/ should be consensus critical stuff
 7812016-03-24T19:47:52  <wumpus> jonasschnelli: we're not actually doing anything with it at the moment
 7822016-03-24T19:47:59  <wumpus> jonasschnelli: right
 7832016-03-24T19:48:09  <jonasschnelli> +1 ;-)
 7842016-03-24T19:48:20  <cfields> topic has strayed a bit :)
 7852016-03-24T19:48:35  <wumpus> yes
 7862016-03-24T19:48:55  <sipa> petertodd: ok, agreed
 7872016-03-24T19:49:09  <petertodd> sipa: thanks!
 7882016-03-24T19:49:28  <wumpus> #topic convert python RPC tests to python 3
 7892016-03-24T19:49:29  <gmaxwell> I am disappointed that all of this time is being wasted over faffing about libraries, for code that has very narrow independant interest and likely against the wishes of its author. .. and no one commented on further verification strategy.
 7902016-03-24T19:49:49  * gmaxwell sees topic is over.
 7912016-03-24T19:50:00  <MarcoFalke> I think it's cleaner to just switch to py3
 7922016-03-24T19:50:04  <petertodd> wumpus: concept ack!
 7932016-03-24T19:50:26  <MarcoFalke> but I'd like to hear any drawbacks first.
 7942016-03-24T19:50:30  <wumpus> gmaxwell: I agree, I think this will do just as well as a code snippet, but I don't think discussing this further is constructive. The people that want a library so bad can work on it I guess :)
 7952016-03-24T19:50:41  <Luke-Jr> py3 vs py2, I can think of no drawbacks to exclusively py3
 7962016-03-24T19:50:44  <wumpus> as you may have read in https://github.com/bitcoin/bitcoin/issues/7717, the next release of ubuntu, 16.04, xenial something will no longer ship python 2
 7972016-03-24T19:50:47  <btcdrak> MarcoFalke: agreed
 7982016-03-24T19:51:01  <jonasschnelli> Is  the reason for the py3 switch the deprecating of py2 or the missing of it in some of the distributions?
 7992016-03-24T19:51:09  <wumpus> I think this makes a nice moment to decide on a switch
 8002016-03-24T19:51:16  <MarcoFalke> ok, so I will close the "refactor py2 code" pull and just go py3 only
 8012016-03-24T19:51:22  <wumpus> we're modernizing the build environment anyway by starting to rely on c++11
 8022016-03-24T19:51:31  <wumpus> MarcoFalke: it's the first step right
 8032016-03-24T19:51:43  <jonasschnelli> ACK on py3... the RPC tests are not something we need "full portability".
 8042016-03-24T19:51:56  <wumpus> for the build system I have py2+py3 compatibility ready
 8052016-03-24T19:51:56  <Luke-Jr> actually, we should retain support py2 at least initially
 8062016-03-24T19:51:59  *** jcorgan has quit IRC
 8072016-03-24T19:52:00  <Luke-Jr> since we may need to backport
 8082016-03-24T19:52:08  <wumpus> for the RPC tests I don't think that is necessary
 8092016-03-24T19:52:09  <petertodd> I'm not sure there exist any build environments that we need to support that are py2 only
 8102016-03-24T19:52:23  <wumpus> agree petertodd
 8112016-03-24T19:52:28  <jonasschnelli> agree with petertodd
 8122016-03-24T19:52:30  <cfields> agreed
 8132016-03-24T19:52:32  <sipa> agree, the backport trees could also just switch to py3
 8142016-03-24T19:52:42  <Luke-Jr> if it's not much effort, moving to py2+py3, and then to py3 before 0.13 would be best
 8152016-03-24T19:52:52  <wumpus> Luke-Jr: that's a lot more work
 8162016-03-24T19:52:58  <Luke-Jr> if it's a lot more work, forget it
 8172016-03-24T19:53:05  <wumpus> I did that for scripts used by the build system itself
 8182016-03-24T19:53:10  <wumpus> but the test framework is huge
 8192016-03-24T19:53:21  <wumpus> and it's a lot of work for people to make sure it works for both python versions
 8202016-03-24T19:53:25  <MarcoFalke> considering that the current rpc test are full of bugs due to missing maintanance and review, doing a py2+3 coverage is impossible
 8212016-03-24T19:53:25  <wumpus> for every change
 8222016-03-24T19:53:37  <sipa> i think we should just outright switch to py3
 8232016-03-24T19:53:39  <jonasschnelli> Yes. No backward comp. for test script please.
 8242016-03-24T19:53:42  <petertodd> fwiw, I'm somewhat in favor of making my own python-bitcoinlib py3-only, simply because I personally never use it on py2, which makes me worried I'm missing bugs
 8252016-03-24T19:53:42  <wumpus> exactly, practical concerns kind of rule this out
 8262016-03-24T19:54:16  <btcdrak> I think we should switch to py3 outright also
 8272016-03-24T19:54:27  <wumpus> that's clear, then
 8282016-03-24T19:54:31  <jonasschnelli> And kudos to MarcoFalke for working on this!
 8292016-03-24T19:54:36  <wumpus> yes!
 8302016-03-24T19:54:55  <btcdrak> +1
 8312016-03-24T19:55:32  <MarcoFalke> You can pipe the files through 2to3 and then "only" check the diff.
 8322016-03-24T19:55:32  <wumpus> ok, that concludes the topic I guess, I'm happy we can finally agree on something :)
 8332016-03-24T19:55:39  <cfields> yes, it's much easier for us than most because it's all internal and we really have no downstreams to cater to
 8342016-03-24T19:55:41  <wumpus> more topics?
 8352016-03-24T19:55:58  <wumpus> 5 minutes to go
 8362016-03-24T19:56:26  <gmaxwell> Lets end early.
 8372016-03-24T19:56:29  <MarcoFalke> #action Close https://github.com/bitcoin/bitcoin/pull/7722 and switch to py3
 8382016-03-24T19:56:33  <wumpus> great
 8392016-03-24T19:56:39  <wumpus> #endmeeting
 8402016-03-24T19:56:39  <lightningbot> Meeting ended Thu Mar 24 19:56:39 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
 8412016-03-24T19:56:39  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.html
 8422016-03-24T19:56:39  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.txt
 8432016-03-24T19:56:39  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.log.html
 8442016-03-24T19:57:03  <GitHub184> [bitcoin] btcdrak opened pull request #7741: [0.12] Mark p2p alert system as deprecated (0.12...alert0) https://github.com/bitcoin/bitcoin/pull/7741
 8452016-03-24T19:57:21  <jonasschnelli> sipa: how would you support test if the ctaeslib is just a c./.h?
 8462016-03-24T19:57:29  <jonasschnelli> Add a test.c and a Makefile?
 8472016-03-24T19:57:41  <gmaxwell> with a define.
 8482016-03-24T19:57:47  <gmaxwell> #ifdef CTAES_TESTS  int main()...
 8492016-03-24T19:57:58  <sipa> i preferna test.c
 8502016-03-24T19:58:12  <sipa> as it's perfectly externally testable
 8512016-03-24T19:58:24  <Luke-Jr> in this case, it may be wise to have a function for tests that software can call at startup
 8522016-03-24T19:58:35  <gmaxwell> sipa: k.
 8532016-03-24T19:58:52  <Luke-Jr> ie, refuse to start if the AES code fails its own testws
 8542016-03-24T19:59:17  <gmaxwell> well a test harness for verification may be very different than built in selftests.
 8552016-03-24T19:59:24  <Luke-Jr> sure
 8562016-03-24T19:59:48  <petertodd> test harness for the constant-time functionality is probably going to be a lot less portable than the code itself for instance...
 8572016-03-24T20:00:44  <gmaxwell> well for example, a verification test harness would likely contain another whole implementation of AES. :)
 8582016-03-24T20:00:48  <cfields> sipa: just pushed the code-movement to net-refactor8 branch. Still lots of placeholders, todos, and hacks in there, but it now looks overall how i want it to
 8592016-03-24T20:02:13  *** mrkent has joined #bitcoin-core-dev
 8602016-03-24T20:03:34  <sipa> gmaxwell: hmm... about tests; the s-box implementation is the most obscure/unreviewable/largest part of the code... but it's trivial to.test exhaustivelg against another implementation, as it only deals with 32-bit inputs
 8612016-03-24T20:05:09  <sipa> ah, nvm
 8622016-03-24T20:05:14  <gmaxwell> sipa: right. an exhaustive sbox test, e.g. with forward sbox pulled out of another implemetation. and checking forward and reverse. And then I thought it might be interesting to see if the existing vectors are enough to catch any simple or or two byte mutations in the current implementation. And if not, we could add vectors until they are.
 8632016-03-24T20:05:29  <gmaxwell> why cant the sbox be exaustively tested?
 8642016-03-24T20:06:23  <sipa> gmaxwell: not as a black box; its inputs are 8 16-bit integers (and.it does 16 sbox applications in parallel)... you can't exhaustive test 128 bits (i hope)
 8652016-03-24T20:07:08  <gmaxwell> right. well it could test each lane one at a time.
 8662016-03-24T20:07:14  <sipa> if you can inspect the code and see that it's "obviously" doing the same thing for every bit position, it suffices to test just the 2^8 inputs
 8672016-03-24T20:08:07  <sipa> petertodd: how is the bip9 warning code broken?
 8682016-03-24T20:08:13  <sipa> (re: github pr)
 8692016-03-24T20:08:43  <petertodd> sipa: oh, I spoke to soon on that - but it was more confusing than it should have been, which I'll fix next week when I get back after the long weekend
 8702016-03-24T20:08:48  <sipa> petertodd: the warning logic uses bit positions; the trigger code uses DeplymentPos
 8712016-03-24T20:08:58  <sipa> (i agree DeploymentId is better)
 8722016-03-24T20:09:24  <petertodd> sipa: yeah, simple enough pull-req, and I've got a few more improvements re: adding comments that I'll add to it too
 8732016-03-24T20:09:33  <sipa> great!
 8742016-03-24T20:10:13  <petertodd> oh, one thing I forgot to ask: if I understand it correctly, right now changes to the "meta-version" - the top three bits - do *not* trigger a "network may have upgraded warning"
 8752016-03-24T20:10:36  <sipa> petertodd: they do if it's permanent
 8762016-03-24T20:10:53  <petertodd> sipa: ah, alright, I'll go look at that more then and verify
 8772016-03-24T20:10:57  *** gevs has joined #bitcoin-core-dev
 8782016-03-24T20:10:58  *** gevs has joined #bitcoin-core-dev
 8792016-03-24T20:11:05  <sipa> there is the 50% of last blocks habe unexpected version warning still
 8802016-03-24T20:11:14  <petertodd> sipa: ah, ok, good
 8812016-03-24T20:11:31  <sipa> in addition, there is warning logic for each bip9 bit position, which works retroactively
 8822016-03-24T20:11:31  *** hybridsole has quit IRC
 8832016-03-24T20:11:57  <petertodd> sipa: anyway, probably best I don't ask too many more questions to keep my mind uncontaminated with what you guys think the code should be doing :)
 8842016-03-24T20:12:23  <sipa> petertodd: good strategy
 8852016-03-24T20:12:50  <petertodd> bbl - have a good long weekend!
 8862016-03-24T20:13:57  <cfields> wumpus: https://github.com/travis-ci/travis-build/pull/680
 8872016-03-24T20:14:03  <cfields> one step closer, i hope
 8882016-03-24T20:16:33  <wumpus> cfields: wow! finally :)
 8892016-03-24T20:16:47  *** hybridsole has joined #bitcoin-core-dev
 8902016-03-24T20:17:26  <cfields> wumpus: i have some work on docker that may not have been in vain, though. It looks like a very possible gitian replacement
 8912016-03-24T20:18:31  <warren> cfields: for your own deterministic toolchain?
 8922016-03-24T20:18:35  <wumpus> cfields: interesting
 8932016-03-24T20:19:23  <cfields> (after all, Gitian basically is an early/simplified version of what Docker has come to be)
 8942016-03-24T20:20:03  <wumpus> in a way, yes. At least docker is very actively supported/maintained.
 8952016-03-24T20:20:33  <warren> docker still puts an OS into a chroot
 8962016-03-24T20:20:42  <cfields> warren: kinda. it just builds from a pre-determined sandbox and caches each step. as a bonus, each step can be shared as well
 8972016-03-24T20:21:20  <warren> IMHO it's worthwhile to switch away from gitian if we are at the stage of being fully in control of our own deterministic toolchain
 8982016-03-24T20:21:32  <cfields> warren: well you'll always need a reference environment
 8992016-03-24T20:21:44  *** d_t has joined #bitcoin-core-dev
 9002016-03-24T20:22:05  <warren> cfields: the reference environment could be deterministic as well
 9012016-03-24T20:22:09  <wumpus> that's a long way, you don't just need a reference toolchain, but there's also other tools that have an effect on the result
 9022016-03-24T20:22:24  <cfields> in this case, docker basically acts as that reference environment, containing only our reference toolchain and the bare minimum other tools
 9032016-03-24T20:22:41  <cfields> seems to me, docker is the perfect tool for us in this case
 9042016-03-24T20:22:48  <wumpus> warren: we need a deterministically buildable OS, isn't debian working on that :)
 9052016-03-24T20:23:03  <cfields> warren: sure. and you have to run it somewhere :)
 9062016-03-24T20:23:14  <warren> ANY OS > build stage A that probably isn't deterministic -> use that to build stage B
 9072016-03-24T20:24:31  <cfields> warren: regardless of the determinism of the base, you have to have some tool for actually building the thing. Short of handing out a rootfs.tar.gz and praying that it will run on some baremetal, ofc
 9082016-03-24T20:25:01  <wumpus> anyhow this is driftng off topic
 9092016-03-24T20:25:10  <wumpus> main thing I'd like is to have c++11 in travis
 9102016-03-24T20:25:18  <cfields> warren: btw, I believe I've seen some early deterministic Debian base images on dockerhub
 9112016-03-24T20:25:54  <wumpus> cfields: so deterministic debian doesn't only exist in theory anymore? that's good to hear there's progress
 9122016-03-24T20:25:56  <cfields> wumpus: yes, i'm committed to making that happen one way or another. The net code depends on it, so I'm kinda forced to :)
 9132016-03-24T20:26:19  <wumpus> cfields: it sounds like a project akin to building the pyramids
 9142016-03-24T20:27:01  <cfields> hmm, trying to re-find what i saw
 9152016-03-24T20:27:02  <wumpus> even getting the bitcoin dependencies deterministic was a hell of a lot of work, just imagine for a full OS
 9162016-03-24T20:27:21  <wumpus> although it gets easier, with better tooling, some things only had to be done onece
 9172016-03-24T20:28:55  *** frankenmint has joined #bitcoin-core-dev
 9182016-03-24T20:29:06  <cfields> wumpus: hmm, i might've made up the Debian part. I saw someone working on a deterministic base for Travis, my brain might've just lumped 'em together.
 9192016-03-24T20:29:16  <cfields> s/Travis/Docker/
 9202016-03-24T20:30:10  <wumpus> right - any deterministic base wouldb e fine, it doesn't need to be debian
 9212016-03-24T20:32:56  <wumpus> docker is based on lxc isn't it? that means it at least doesn't need deterministic kernel, init etc
 9222016-03-24T20:33:33  <cfields> based on cgroups, I don't believe it uses lxc itself
 9232016-03-24T20:34:02  *** frankenmint has quit IRC
 9242016-03-24T20:39:12  *** t800 has quit IRC
 9252016-03-24T20:41:49  <wumpus> right, the same underlying mechanism
 9262016-03-24T20:43:11  <kanzure> in addition to deterministic debian please also consider deterministic nixos things
 9272016-03-24T20:43:58  <kanzure> fwiw i recommend not using docker's build tool. it's a mess.
 9282016-03-24T20:44:28  <kanzure> using docker for running containers or something is probably fine (although coreos/rkt things should be considered, since they seem to be compatible with docker containers and have a specification and words written)
 9292016-03-24T20:45:05  <kanzure> "docker build" is a build tool crammed into a container management system. a more realistic build tool is something like bazel ( http://bazel.io/ ), which can spit out finished container images.
 9302016-03-24T20:45:25  *** t800 has joined #bitcoin-core-dev
 9312016-03-24T20:45:54  <GitHub178> [bitcoin] jonasschnelli opened pull request #7742: [Wallet][RPC] add missing abandon status documentation (master...2016/03/ab_doc) https://github.com/bitcoin/bitcoin/pull/7742
 9322016-03-24T20:46:25  *** musalbas has quit IRC
 9332016-03-24T20:46:40  <kanzure> (with "docker build" you lose out on things like build caching unless you always build only on one machine. this negatively impacts things like distributed continuous integration build runners.)
 9342016-03-24T20:46:41  <cfields> kanzure: docker would only be used as a way to launch our own build process inside of a pre-determined environment
 9352016-03-24T20:46:49  *** lysobit has quit IRC
 9362016-03-24T20:47:16  *** Thireus has quit IRC
 9372016-03-24T20:47:30  *** treehug88 has quit IRC
 9382016-03-24T20:48:31  *** t800 has joined #bitcoin-core-dev
 9392016-03-24T20:48:38  *** lysobit has joined #bitcoin-core-dev
 9402016-03-24T20:48:56  <warren> kanzure: coreos supports docker, not the other way around
 9412016-03-24T20:49:10  *** t800 has quit IRC
 9422016-03-24T20:49:12  <kanzure> i don't think i said otherwise.
 9432016-03-24T20:49:49  *** musalbas has joined #bitcoin-core-dev
 9442016-03-24T20:50:39  <kanzure> cfields: right but even the pre-determined environment has a build toolchain. and the default way when using docker is to use "docker build". which is what my above rant is about.
 9452016-03-24T20:51:27  <kanzure> oh i guess that is inconsistent with me mentioning rkt. my mention of rkt is unrelated to my complaints about "docker build".
 9462016-03-24T20:51:47  <kanzure> (rkt doesn't do container building)
 9472016-03-24T20:53:12  <cfields> kanzure: I'll read up on those things
 9482016-03-24T20:53:33  *** lysobit has quit IRC
 9492016-03-24T20:53:51  <kanzure> ok feel free to pester me. i have been reading docker source code and such.
 9502016-03-24T20:54:01  *** musalbas has quit IRC
 9512016-03-24T20:54:07  <cfields> great, thanks :)
 9522016-03-24T20:54:20  *** musalbas has joined #bitcoin-core-dev
 9532016-03-24T20:55:42  *** Thireus has joined #bitcoin-core-dev
 9542016-03-24T20:56:26  *** laurentmt has joined #bitcoin-core-dev
 9552016-03-24T20:56:31  *** laurentmt has quit IRC
 9562016-03-24T20:56:50  *** lysobit has joined #bitcoin-core-dev
 9572016-03-24T20:57:28  *** droark has quit IRC
 9582016-03-24T21:07:37  *** Guest27862 has quit IRC
 9592016-03-24T21:08:46  *** ChillazZ has joined #bitcoin-core-dev
 9602016-03-24T21:13:28  *** droark has joined #bitcoin-core-dev
 9612016-03-24T21:19:50  <GitHub139> [bitcoin] luke-jr closed pull request #7047: [WIP] Backports for 0.11.3 (updated to 93ca5a3) (0.11...backports-for-0.11.3) https://github.com/bitcoin/bitcoin/pull/7047
 9622016-03-24T21:22:16  *** fengling has quit IRC
 9632016-03-24T21:27:23  <GitHub149> [bitcoin] luke-jr opened pull request #7743: [0.11] Important backports for 0.11.3 (updated to v0.12.0) (0.11...backports-for-0.11.3) https://github.com/bitcoin/bitcoin/pull/7743
 9642016-03-24T21:29:44  *** frankenmint has joined #bitcoin-core-dev
 9652016-03-24T21:31:32  *** Hybrids0le has joined #bitcoin-core-dev
 9662016-03-24T21:31:32  *** hybridsole has quit IRC
 9672016-03-24T21:32:45  *** CodeShark has quit IRC
 9682016-03-24T21:33:02  *** NicolasDorier has quit IRC
 9692016-03-24T21:33:03  *** CodeShark has joined #bitcoin-core-dev
 9702016-03-24T21:33:56  *** NicolasDorier has joined #bitcoin-core-dev
 9712016-03-24T21:33:57  *** Hybrids0le has quit IRC
 9722016-03-24T21:34:01  *** droark has quit IRC
 9732016-03-24T21:34:25  *** lysobit has quit IRC
 9742016-03-24T21:34:26  *** musalbas has quit IRC
 9752016-03-24T21:34:49  *** lesderid has quit IRC
 9762016-03-24T21:36:17  *** frankenmint has quit IRC
 9772016-03-24T21:41:00  *** musalbas has joined #bitcoin-core-dev
 9782016-03-24T21:41:24  *** lysobit has joined #bitcoin-core-dev
 9792016-03-24T21:41:34  *** lesderid has joined #bitcoin-core-dev
 9802016-03-24T21:44:43  *** d_t has quit IRC
 9812016-03-24T21:44:44  *** ebfull has quit IRC
 9822016-03-24T21:44:44  *** windsok has quit IRC
 9832016-03-24T21:50:10  *** fengling has joined #bitcoin-core-dev
 9842016-03-24T22:00:40  *** d_t has joined #bitcoin-core-dev
 9852016-03-24T22:00:40  *** ebfull has joined #bitcoin-core-dev
 9862016-03-24T22:00:40  *** windsok has joined #bitcoin-core-dev
 9872016-03-24T22:03:42  *** ibrightly has quit IRC
 9882016-03-24T22:05:05  *** ibrightly has joined #bitcoin-core-dev
 9892016-03-24T22:07:46  *** justanotheruser has quit IRC
 9902016-03-24T22:18:01  *** supasonic has quit IRC
 9912016-03-24T22:19:01  *** supasonic has joined #bitcoin-core-dev
 9922016-03-24T22:32:46  *** frankenmint has joined #bitcoin-core-dev
 9932016-03-24T22:33:16  *** fengling has quit IRC
 9942016-03-24T22:33:32  *** Guyver2 has quit IRC
 9952016-03-24T22:38:19  *** frankenmint has quit IRC
 9962016-03-24T22:43:01  *** Hybrids0le has joined #bitcoin-core-dev
 9972016-03-24T22:44:57  *** Hybrids0le has quit IRC
 9982016-03-24T22:55:39  *** shesek has joined #bitcoin-core-dev
 9992016-03-24T22:59:44  *** wallet42 has joined #bitcoin-core-dev
10002016-03-24T23:02:29  *** mrkent_ has joined #bitcoin-core-dev
10012016-03-24T23:04:35  *** mrkent has quit IRC
10022016-03-24T23:06:30  *** hybridsole has joined #bitcoin-core-dev
10032016-03-24T23:10:06  *** fengling has joined #bitcoin-core-dev
10042016-03-24T23:20:29  *** justanotheruser has joined #bitcoin-core-dev
10052016-03-24T23:23:26  *** musalbas has quit IRC
10062016-03-24T23:23:35  *** musalbas- has joined #bitcoin-core-dev
10072016-03-24T23:23:46  *** musalbas- is now known as musalbas
10082016-03-24T23:27:49  *** PRab has quit IRC
10092016-03-24T23:50:10  *** justanotheruser has quit IRC
10102016-03-24T23:52:19  *** droark has joined #bitcoin-core-dev