1 2016-03-24T00:00:45  <gmaxwell> oh hmph. made some pgoress but stuck again. :(
   2 2016-03-24T00:01:17  <gmaxwell> now this looks like local corruption from running out of space. 2016-03-24 00:00:19 ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=-1, nPos=0)
   3 2016-03-24T00:01:50  <sipax> ugh!
   4 2016-03-24T00:02:04  <sipax> that means the block data wasn't written but the index entry for it was
   5 2016-03-24T00:02:20  <sipax> remind me to investigate the write order during flush in such a case
   6 2016-03-24T00:02:29  * sipax goes into standby mode
   7 2016-03-24T00:03:17  <gmaxwell> well it may have been out of space and the write could have failed. Our out of space handling is not so great always.
   8 2016-03-24T00:03:37  <sipax> it should not go write a database entry for a file it failed to create
   9 2016-03-24T00:03:46  <sipax> is it a pruned node?
  10 2016-03-24T00:03:50  <gmaxwell> no.
  11 2016-03-24T00:03:56  <sipax> in that case, certainly not
  12 2016-03-24T00:04:44  <gmaxwell> what the heck, even after logging that a bunch of times it's making progress again.
  13 2016-03-24T00:06:01  <gmaxwell> okay, actually the readblockfromdisk is being triggered by getblock rpcs being called by p2pool
  14 2016-03-24T00:06:40  <gmaxwell> so this might just be an artifact of running getblock on a block we have headers for but no data.
  15 2016-03-24T00:07:32  <sipax> gmaxwell: no, it shouldn't
  16 2016-03-24T00:07:45  <sipax> getblock RPC checks whether the BLOCK_HAVE_DATA flag is present for that block index entry
  17 2016-03-24T00:14:33  <gmaxwell> well my log is full of
  18 2016-03-24T00:14:33  <gmaxwell> 2016-03-23 23:58:18 Received a POST request for / from
  19 2016-03-24T00:14:34  <gmaxwell> 2016-03-23 23:58:18 ERROR: ReadBlockFromDisk: OpenBlockFile failed for CBlockDiskPos(nFile=-1, nPos=0)
  20 2016-03-24T00:14:36  <gmaxwell> 2016-03-23 23:58:18 ThreadRPCServer method=getblock
  21 2016-03-24T00:32:55  <gmaxwell> and now that it's caught up, no more..
  22 2016-03-24T00:34:32  *** afk11 has quit IRC
  23 2016-03-24T00:37:17  *** afk11 has joined #bitcoin-core-dev
  24 2016-03-24T01:09:49  *** murch has quit IRC
  25 2016-03-24T01:11:01  *** Ylbam has quit IRC
  26 2016-03-24T01:12:32  *** fengling has joined #bitcoin-core-dev
  27 2016-03-24T01:56:42  *** supasonic has quit IRC
  28 2016-03-24T01:57:10  *** supasonic has joined #bitcoin-core-dev
  29 2016-03-24T02:09:02  *** Alopex has quit IRC
  30 2016-03-24T02:10:07  *** Alopex has joined #bitcoin-core-dev
  31 2016-03-24T02:19:06  <morcos> gmaxwell: sipax: looks like thats just the error that happens, the RPC only checks the BLOCK_HAVE_DATA flag if you're pruning.  Ha!
  32 2016-03-24T02:20:23  *** baldur has quit IRC
  33 2016-03-24T02:20:41  *** baldur has joined #bitcoin-core-dev
  34 2016-03-24T02:21:10  *** afk11 has quit IRC
  35 2016-03-24T02:24:04  *** zooko has joined #bitcoin-core-dev
  36 2016-03-24T02:28:57  *** afk11 has joined #bitcoin-core-dev
  37 2016-03-24T02:52:51  *** Chris_Stewart_5 has quit IRC
  38 2016-03-24T02:53:59  *** AdrianG_ is now known as AdrianG
  39 2016-03-24T02:54:29  *** AdrianG is now known as Guest82537
  40 2016-03-24T02:54:43  *** Guest82537 is now known as AdrianG_
  41 2016-03-24T03:01:02  *** AdrianG_ has quit IRC
  42 2016-03-24T03:01:02  *** AdrianG_ has joined #bitcoin-core-dev
  43 2016-03-24T03:01:16  *** AdrianG_ is now known as Aleph0
  44 2016-03-24T03:08:34  *** afk11 has quit IRC
  45 2016-03-24T03:16:08  *** PRab has quit IRC
  46 2016-03-24T03:17:08  *** PRab has joined #bitcoin-core-dev
  47 2016-03-24T03:22:01  *** afk11 has joined #bitcoin-core-dev
  48 2016-03-24T03:26:56  *** zooko has quit IRC
  49 2016-03-24T03:29:10  *** xiangfu has joined #bitcoin-core-dev
  50 2016-03-24T03:31:54  *** molz has joined #bitcoin-core-dev
  51 2016-03-24T03:34:05  *** mrkent has quit IRC
  52 2016-03-24T03:34:51  *** moli has quit IRC
  53 2016-03-24T03:37:10  *** frankenmint has quit IRC
  54 2016-03-24T03:39:36  *** achow101 has quit IRC
  55 2016-03-24T03:41:35  *** supasonic has quit IRC
  56 2016-03-24T03:54:25  *** supasonic has joined #bitcoin-core-dev
  57 2016-03-24T03:56:47  *** mrkent has joined #bitcoin-core-dev
  58 2016-03-24T04:49:01  *** Alopex has quit IRC
  59 2016-03-24T04:50:06  *** Alopex has joined #bitcoin-core-dev
  60 2016-03-24T04:50:58  *** frankenmint has joined #bitcoin-core-dev
  61 2016-03-24T04:55:17  *** BCB has quit IRC
  62 2016-03-24T05:09:24  *** mrkent has quit IRC
  63 2016-03-24T05:11:46  *** mrkent has joined #bitcoin-core-dev
  64 2016-03-24T05:21:57  *** Don_John has quit IRC
  65 2016-03-24T05:23:49  *** mrkent has quit IRC
  66 2016-03-24T05:24:02  *** mrkent has joined #bitcoin-core-dev
  67 2016-03-24T05:44:15  *** schmidty has quit IRC
  68 2016-03-24T05:48:01  *** Alopex has quit IRC
  69 2016-03-24T05:49:06  *** Alopex has joined #bitcoin-core-dev
  70 2016-03-24T05:51:15  *** supasonic has quit IRC
  71 2016-03-24T05:52:25  *** d_t has joined #bitcoin-core-dev
  72 2016-03-24T05:55:02  *** d_t has quit IRC
  73 2016-03-24T05:56:25  *** mrkent has quit IRC
  74 2016-03-24T05:59:36  *** fengling has quit IRC
  75 2016-03-24T06:00:03  *** xiangfu has quit IRC
  76 2016-03-24T06:00:04  *** dermoth has quit IRC
  77 2016-03-24T06:00:48  *** dermoth has joined #bitcoin-core-dev
  78 2016-03-24T06:02:04  *** xiangfu has joined #bitcoin-core-dev
  79 2016-03-24T06:04:01  *** Alopex has quit IRC
  80 2016-03-24T06:05:06  *** Alopex has joined #bitcoin-core-dev
  81 2016-03-24T06:07:05  *** mrkent has joined #bitcoin-core-dev
  82 2016-03-24T06:08:23  *** fengling has joined #bitcoin-core-dev
  83 2016-03-24T06:17:34  *** molly has joined #bitcoin-core-dev
  84 2016-03-24T06:20:58  *** molz has quit IRC
  85 2016-03-24T06:45:23  <jonasschnelli> encryption: would it be stupid to use the identity keypair (used for Auth / MITM protection) in a ECDH scheme to encrypt a dh key-exchange for further encryption?
  86 2016-03-24T06:45:47  <jonasschnelli> I think we should not use the "static" identity key for encryption.
  87 2016-03-24T06:46:16  <jonasschnelli> It would expose the shared ecdh secred (AES256key) to known-plaintext-attacks.
  88 2016-03-24T06:46:59  <jonasschnelli> But not sure if using the identity key for a quick ECDH keyexchange (known-plaintext-attacks are pretty impossible) would make sense.
  89 2016-03-24T06:51:35  <gmaxwell> It would be stupid beyond all comprehension.
  90 2016-03-24T06:52:14  <gmaxwell> The encryption should be ephemerally keyed. There is no need for the key to outlive the session.
  91 2016-03-24T06:57:35  *** Ylbam has joined #bitcoin-core-dev
  92 2016-03-24T06:57:40  <gmaxwell> What I suggested earlier-- which is a pretty standard and well studied architecture is to use randomly generated keys to establish a secure channel (with authenticated encryption like AES-GCM or chacha20-poly1305); then inside if-- if some identity is in use-- you use that identity to authenticate the secure session you just established. E.g. by signing the hash of the shared session key.
  93 2016-03-24T07:00:44  <sipax> gmaxwell: that is MitMable?
  94 2016-03-24T07:01:02  <gmaxwell> ... No.
  95 2016-03-24T07:01:36  <sipax> oh, no!
  96 2016-03-24T07:01:37  <jonasschnelli> gmaxwell: how would you pass the random generated symmetric cipher key to "the other side"? ECDH channel with the identity key?
  97 2016-03-24T07:03:06  <gmaxwell> Please please stop trying to use ecdh with long lived keys. This is almost always a severe design mistake.
  98 2016-03-24T07:04:21  <gmaxwell> Each side sends a new, randomly generated public key to the other side. The session keys (there will be more than one, likely four-- auth and encrypt in each direction) are the output of ECDH with those keys, fed through a KDF.
  99 2016-03-24T07:06:16  <jonasschnelli> gmaxwell: Okay. How would you protect from MITM. By auth with the identity key AFTER the encryption channel is setup?
 100 2016-03-24T07:08:03  <gmaxwell> if identification is in use, inside the encrypted channel you send a signature of the session id (a hash of the ephemerial key from ecdh which also commits to all the channel parameters) using the relevant identity.
 101 2016-03-24T07:09:20  <sipax> what do you mean by channel parameters?
 102 2016-03-24T07:09:47  <jonasschnelli> gmaxwell: how would you protect from attacking(DOSing) the encryption request (key generation!) if the auth is after the enc?
 103 2016-03-24T07:10:02  <jonasschnelli> Also fingerprinting
 104 2016-03-24T07:10:28  <jonasschnelli> (the later is probably fine if the 4 key are generated randomly)
 105 2016-03-24T07:10:31  <gmaxwell> key generation is very fast, faster then verifying your digital signatures you propose for authentication.
 106 2016-03-24T07:11:01  <gmaxwell> Fingerprinting what?
 107 2016-03-24T07:11:02  * jonasschnelli always though key generation is much more CPU intense then ecdsa verify.
 108 2016-03-24T07:11:12  <sipax> jonasschnelli: if you always use freshly generated ecdh keys, there is no fongerprinting
 109 2016-03-24T07:11:14  <jonasschnelli> gmaxwell: fingerprinting the node.
 110 2016-03-24T07:11:44  <jonasschnelli> Well,.. right. We could ban a node after a failed encryption request.
 111 2016-03-24T07:12:11  <sipax> i don't think it can fail
 112 2016-03-24T07:12:24  <jonasschnelli> sipax: I mean the identity check afterwards
 113 2016-03-24T07:13:04  <sipax> jonasschnelli: it's much cheaper for them to go ask us a bunch of random blocks, with far higher costs
 114 2016-03-24T07:13:09  <sipax> if they want to dos
 115 2016-03-24T07:13:11  <gmaxwell> jonasschnelli: no, it's fundimentally easier to generate a new keypair than to verify a ecdh signature. They're pretty close in timings, because we use a constant time implementation for the former.
 116 2016-03-24T07:13:14  <jonasschnelli> But this would basically mean, every peer can request encrypted sessions... (which i'm not sure if its good or bad).
 117 2016-03-24T07:13:44  <gmaxwell> they should, and they should all be encrypted so as not to distingush the ones using authentication.
 118 2016-03-24T07:14:10  <gmaxwell> and this can be made faster than the existing transport (or at least not considerably worse).
 119 2016-03-24T07:14:38  <sipax> jonasschnelli: for ECDSA, verify > sign >>> keygen
 120 2016-03-24T07:14:59  <jonasschnelli> sipax: I see. Thanks!
 121 2016-03-24T07:15:17  <sipax> jonasschnelli: computing the pubkey for a generated private key is similar to signing
 122 2016-03-24T07:15:24  <gmaxwell> well if by keygen you mean generate a pubkey too, then sign == keygen, pretty much.
 123 2016-03-24T07:15:48  <gmaxwell> and this is faster than verify.
 124 2016-03-24T07:16:01  <sipax> same order of magnitude in any case
 125 2016-03-24T07:16:05  <jonasschnelli> gmaxwell: So. The peer would pass one pubkey for the dh secret calculation and the (static) identity pubkey (static) for auth to the other peer. Right?
 126 2016-03-24T07:16:52  <jonasschnelli> (doesn't the current[and most] ec keygen algos do a dummy sign/verify to ensure validity?)
 127 2016-03-24T07:17:05  <sipax> jonasschnelli: if by auth you mean the signing of the session key, indeed
 128 2016-03-24T07:17:12  <sipax> jonasschnelli: bitcoin does
 129 2016-03-24T07:17:14  <gmaxwell> the only software ever created that does that is bitcoin core.
 130 2016-03-24T07:17:19  <gmaxwell> as far as I can tell.
 131 2016-03-24T07:17:34  <gmaxwell> and it's only sensible to do that for payment addresses, it wouldn't be done here.
 132 2016-03-24T07:18:15  <gmaxwell> first an encrypted, authenticated channel would be brought up using a ephemeral pubkey sent in each direction.  Then _inside_ that channel, the participants could identify themselves by signing the session-id.
 133 2016-03-24T07:18:24  <jonasschnelli> only signing the session key would not allow the remote peer to identify which identity is signing. Wouldn't it require at least the hash of the identity pubkey?
 134 2016-03-24T07:19:45  <gmaxwell> Techincally the signature and message its signing is enough, you can just get the pubkey from that. Though it could send the pubkey its using.
 135 2016-03-24T07:20:14  <jonasschnelli> Ah. Right pubkey recover.
 136 2016-03-24T07:20:20  <gmaxwell> but there are better protocols for identifying than just sending a signature-- ones that do not leak the identity of at least one participant.
 137 2016-03-24T07:20:24  *** mrkent has quit IRC
 138 2016-03-24T07:20:35  <jonasschnelli> 0-knowlage?
 139 2016-03-24T07:21:32  *** mrkent has joined #bitcoin-core-dev
 140 2016-03-24T07:21:49  *** PaulCape_ has joined #bitcoin-core-dev
 141 2016-03-24T07:22:53  <jonasschnelli> What about allowing auth without enc? We probably presume encryption when authenticating.
 142 2016-03-24T07:23:12  * gmaxwell looks through logs.
 143 2016-03-24T07:23:14  <gmaxwell> 13:14 < gmaxwell> There are protocols that avoid the fingerprinting; e.g. (the server protectng mode of) https://www.ietf.org/proceedings/54/I-D/draft-ietf-ipsec-jfk-04.txt
 144 2016-03-24T07:23:39  <gmaxwell> jonasschnelli: I think there is no utility to identifying without encryption.
 145 2016-03-24T07:24:21  *** PaulCapestany has quit IRC
 146 2016-03-24T07:24:29  <jonasschnelli> Agree. The current auth proposal would also require signing (auth) each message (which is resource hungry).
 147 2016-03-24T07:24:42  <gmaxwell> thats unacceptably slow.
 148 2016-03-24T07:24:45  <jonasschnelli> encryption before authentication allows to keep a auth-session.
 149 2016-03-24T07:25:34  <sipax> jonasschnelli: note that authentication is used in two meanings here; there is identification (done by signing the session key) and MAC (which you still need, but ks covered by using an authenticated mode of encryption such as AES-GCM)
 150 2016-03-24T07:25:40  <gmaxwell> (and signining indivigual messages provides non-repudiation, which is usually not what you want; except in specific instances where you do want it.)
 151 2016-03-24T07:25:42  *** sipax is now known as sipa
 152 2016-03-24T07:26:14  <jonasschnelli> I see.
 153 2016-03-24T07:26:38  <sipa> jonasschnelli: so there is "auth" on each message, but it's done with a symmetric shared session key, rather than the identity key
 154 2016-03-24T07:26:48  <jonasschnelli> And I guess it would be acceptable if the remote peer can fingerprint the requesting peer because it reveals its identity after the encryption channel is established.
 155 2016-03-24T07:27:29  <gmaxwell> jonasschnelli: it's better than the other way around at least. preferably no one could fingerprint anyone; but it's not clear how possible that is.
 156 2016-03-24T07:27:32  <jonasschnelli> sipa: Right. But we just need to make sure the symmetric shared session key was shared between the right parties and not Mitmdled.
 157 2016-03-24T07:27:42  <sipa> jonasschnelli: i think you probably want a per-IP identity, or perhaps just randomly generated identities by default, and allow a peer to ask "hey, you're identity X, right? prove it"
 158 2016-03-24T07:27:52  <gmaxwell> Other than the "connect to my own server" case it's not clear to me exactly how the identification would be used.
 159 2016-03-24T07:28:03  <jonasschnelli> sipa: But that would not work for DHCP nodes (SPV)?
 160 2016-03-24T07:29:03  <sipa> jonasschnelli: so perhaps a separate private non-random configurable host key, which is only used if requested by the peer
 161 2016-03-24T07:29:29  <sipa> that means no TOFU, though
 162 2016-03-24T07:29:32  <gmaxwell> we don't want to create cases where nodes can be tracked easily, we've worked to remove identifying information from the protocol...  having a protocol where there is mutual auth, say between your own 'server' and 'client'  or between your own hosts,  having the requester leak something the allows them to be correlated, but only inside an encrypted channel which they think is to their trusted pe
 163 2016-03-24T07:29:38  <gmaxwell> er, would be minimally harmful.
 164 2016-03-24T07:30:40  <gmaxwell> sipa: I don't know what use tofu would have... and what happens when a node goes away (say, gets deleted) and a new one is at that address? now you need an expiration mechenism.
 165 2016-03-24T07:31:03  <jonasschnelli> gmaxwell: at least you could setup a honeypot-like peer that waits until other peers connect, request encryption and identify themself. You could than combine this data with other fingerprint possibilities (mempool and stuff like that).
 166 2016-03-24T07:32:13  <gmaxwell> jonasschnelli: the party initating authentication would be the only one to lose privacy, so a honey pot would not work; unless it also successfully mitmed hosts that clients were attempting to connect to.
 167 2016-03-24T07:32:33  <jonasschnelli> But somehow I think either you have a MITM attack possibility or you reveal ones identity. Maybe some zero knowledge prove method could be used here (not familiar with these)
 168 2016-03-24T07:32:52  <gmaxwell> There are probably multiple usecases which would identify the channel in different ways; but the same secure channel mechenism would work.
 169 2016-03-24T07:33:58  <gmaxwell> jonasschnelli: I linked a likely sutable identification protocol above.
 170 2016-03-24T07:34:20  * jonasschnelli is reading the JFK proposal. Thanks to gmaxwell 
 171 2016-03-24T07:35:28  <gmaxwell> I am not suggesting we specifically use that, as much as pointing out that it's possible to do better... but even a very simple protocol does what we want, I think.
 172 2016-03-24T07:35:42  <sipa> jonasschnelli: you could also avoid the problem, and by default not have identification, and only have a "prove to me you're X before continuing" message inside the encryptee channel
 173 2016-03-24T07:36:37  <sipa> perhaps even using symmetric crypto, so it only works with a preshared symmetric key
 174 2016-03-24T07:36:44  <jonasschnelli> sipa: Yes. Agreed. Auth would only be necessary or useful if the both peers know each other (preshared key, same owner, SPV<->node stuff).
 175 2016-03-24T07:37:31  <gmaxwell> identification, when you say auth it easily gets confused with the authentication part of encryption which is necessary and cannot be omitted when encryption is used. :)
 176 2016-03-24T07:37:32  *** PaulCapestany has joined #bitcoin-core-dev
 177 2016-03-24T07:37:48  <jonasschnelli> A preshared preshared symmetric key would also work. Right. IMO its just easier to share two pubkeys instead of one "symmetric key".
 178 2016-03-24T07:38:59  <jonasschnelli> gmaxwell: Can you explain the differences between the identity authentication and the encryption authentication? Why would the later be required if both parties stay anonyme to each other?
 179 2016-03-24T07:39:37  *** PaulCape_ has quit IRC
 180 2016-03-24T07:39:55  <jonasschnelli> Once you exchanged encryption ephemeral pubkeys, MITM should no longer be possible?
 181 2016-03-24T07:40:23  <gmaxwell> jonasschnelli: any use of encryption _must_ come with symmetric-key authentication, or otherwise the encryption will be mallible and an attacker can corrupt the channel in ways that leak information about the content.
 182 2016-03-24T07:41:35  <gmaxwell> e.g. they flip a bit, and instead of instantly hanging up, the application is fed some gibberish, and then based on exactly how it responseds/doesn't respond the attacker learns something about the state of the connection and the prior encrypted data.
 183 2016-03-24T07:42:26  <gmaxwell> so the encryption must comes with a message-authentication-code keyed at the same time as the encryption, so any corruption is detected and the connection terminated without revealing anything, ideally before even decrypting the data.
 184 2016-03-24T07:42:55  <jonasschnelli> gmaxwell: sharing the encryption ephemeral pubkeys, calculate symmetric secret and hash the complete communication (starting with both pubkey in deterministic order), compare the hashes after decrypt..
 185 2016-03-24T07:43:20  <gmaxwell> jonasschnelli: that isn't sufficient.
 186 2016-03-24T07:43:50  <sipa> jonasschnelli: encryption provides confidentiality, not integrity
 187 2016-03-24T07:44:31  <jonasschnelli> hmm....
 188 2016-03-24T07:44:55  * jonasschnelli is reading...
 189 2016-03-24T07:45:05  <Luke-Jr> jonasschnelli: it might or might not make sense to coordinate with the new payment protocol encryption stuff
 190 2016-03-24T07:45:20  <Luke-Jr> in BIP 75
 191 2016-03-24T07:47:22  <jonasschnelli> Luke-Jr: thanks. Will check. Agree on the p2p 16x num range.
 192 2016-03-24T07:49:18  <sipa> jonasschnelli, gmaxwell: the paper my const time aes is based on actually discusses aes-gcm (and easily has 4x the performance of the cbc case, as it can do 4 parallel encryption operations)
 193 2016-03-24T07:50:27  <jonasschnelli> gmaxwell sipa: hmm... (having a hard time to understand this correctly): so you would require to generate two keypairs at the same time, one for the encryption sym. cipher key, one for the MAC key. Then you sould sha256_hmac the complete message traffic on both sides and compare to validate the integrity?
 194 2016-03-24T07:50:52  <sipa> jonasschnelli: no, you use aes-gcm
 195 2016-03-24T07:51:03  <sipa> which does encryption and mac simultanelusly
 196 2016-03-24T07:51:47  <jonasschnelli> sipa: okay. I see. But the overall concept would be correct (just to understand the basics).
 197 2016-03-24T07:51:56  <jonasschnelli> s/./?
 198 2016-03-24T07:52:10  <sipa> though, conceptually, yes, you can use aes for encryption, and then do hmac-sha256 on the encrypted data for integrity
 199 2016-03-24T07:52:56  <gmaxwell> There should be two "keys" if using authenticated encryption (like AES-GCM, chacha20-poly, or things coming out of CAESAR) one key in each direction;  or four keys if constructing autenticated encryption from a cipher and hmac.
 200 2016-03-24T07:53:15  <jonasschnelli> What I have problems to understand is how attacks would be possible if you would take the encryption ephemeral publey calculated sym. key for the MAC...
 201 2016-03-24T07:53:49  <sipa> jonasschnelli: let's say you use aes-ctr
 202 2016-03-24T07:53:57  <sipa> you know ctr?
 203 2016-03-24T07:54:13  * jonasschnelli reading... got the essence. Yes.
 204 2016-03-24T07:54:18  <jonasschnelli> streaming.
 205 2016-03-24T07:55:26  <sipa> someone may not be able to decrypt without having the key, but if they can guess what the contents is of a particular message, they can modify it by xoring it with both what they think the decrypted data is, and xoring it with what they want the decrypted data to be
 206 2016-03-24T07:56:33  <sipa> there are more complex bad things you can do with other encryption schemes
 207 2016-03-24T07:56:52  <jonasschnelli> sipa: Got it! Thanks!
 208 2016-03-24T07:57:18  <sipa> but ctr is a nice example, because it perfectly covers confidentially, but provides almost nothing else
 209 2016-03-24T07:57:36  <sipa> so it clearly shows the distinction
 210 2016-03-24T07:58:35  * jonasschnelli is reading the details of AES-GCM
 211 2016-03-24T07:59:39  <jonasschnelli> sipa: But IIRC, your AES PR is CBC. So GCM would require another implementation while our codebase already supports SHA2 HMAC?
 212 2016-03-24T07:59:57  <sipa> jonasschnelli: GCM is easy :)
 213 2016-03-24T08:00:19  <jonasschnelli> sipa: for you or for all other lazy programmers.. :)
 214 2016-03-24T08:00:35  <gmaxwell> sha2 is slow.
 215 2016-03-24T08:01:05  <jonasschnelli> And I guess we could drop the message headers 4byte sha256 checksum once encrypted and MACed
 216 2016-03-24T08:01:49  <gmaxwell> (AES-GCM is also not astonishingly fast without hardware AES, but should be faster than sha2)
 217 2016-03-24T08:02:09  <gmaxwell> jonasschnelli: yes, and the result, with sutiable primitive selection would be _faster_ than the existing protocol, even though it was encrypted.
 218 2016-03-24T08:02:25  <jonasschnelli> hah. Thats an argument.
 219 2016-03-24T08:02:32  <gmaxwell> (faster in terms of cpu usage, it would likely have some more bandwidth overhead, but not a ton)
 220 2016-03-24T08:03:15  <jonasschnelli> But I'm mostly thinking "in implementations". So,... I could write a first implementation that drops the 4byte header sha and uses SHA2 HMAC for the MAC.. just to have an implementation basepoint.
 221 2016-03-24T08:03:37  <jonasschnelli> (in addition to the whole encryption keysharing stuff)
 222 2016-03-24T08:03:47  *** mrkent has quit IRC
 223 2016-03-24T08:04:58  <jonasschnelli> I could be the guy that works on the code frontend while I require analysis and conceptual support from gmaxwell/sipa and others. This could lead to a pratical p2p encryption IMO.
 224 2016-03-24T08:06:26  <sipa> gmaxwell: my current const aes needs 270 cycles/byte; making it do 4 in parallel is trivial and would make it need 72; researchers claim they can do ~10 using simd
 225 2016-03-24T08:06:37  <sipa> and ~3 with aes-ni
 226 2016-03-24T08:06:39  *** mrkent has joined #bitcoin-core-dev
 227 2016-03-24T08:06:41  <gmaxwell> sipa: I think with a.. right.
 228 2016-03-24T08:07:13  <gmaxwell> See also the motivationes behind the CAESAR competition.
 229 2016-03-24T08:11:16  *** asdfdsas has quit IRC
 230 2016-03-24T08:13:20  <gmaxwell> sipa: lol one of the CAESAR entries is based on keccak and is named "keyak". That totally isn't going to cause any confusion...
 231 2016-03-24T08:16:15  <GitHub117> [bitcoin] laanwj closed pull request #7694: Rename AcceptBlock/AcceptBlockHeader to StoreBlock/StoreBlockHeader (master...2016-03-15-naming) https://github.com/bitcoin/bitcoin/pull/7694
 232 2016-03-24T08:16:45  <sipa> gmaxwell: i was just reading the keyak v2 paper
 233 2016-03-24T08:18:03  <btcdrak> I've asked a cryptographer to look at #7689
 234 2016-03-24T08:22:59  <jonasschnelli> Could we ask apoelstra?
 235 2016-03-24T08:23:41  <jonasschnelli> I'm not sure if a standalone AES library (move sipas PR to a custom library) would make sense.
 236 2016-03-24T08:24:59  <jonasschnelli> Adding more non-consensus stuff there will very likely lead to a openssl clone?!
 237 2016-03-24T08:25:05  <jonasschnelli> And adding SHA512 but not SHA256?
 238 2016-03-24T08:25:27  *** cjcj has joined #bitcoin-core-dev
 239 2016-03-24T08:25:40  <wumpus> jonasschnelli: correct
 240 2016-03-24T08:26:06  <wumpus> I don't think petertodd's concern is so much about the library, but more about 'how much review doees this get'
 241 2016-03-24T08:26:22  *** mrkent has quit IRC
 242 2016-03-24T08:28:19  <jonasschnelli> wumpus: after thinking about it. maybe a simple C bases AES library could have a reason to exists.
 243 2016-03-24T08:28:22  <sipa> gmaxwell: what is the fastest authenticated encryption scheme implementation that you know? (assume no hardware support, but constant time)?
 244 2016-03-24T08:28:52  <wumpus> jonasschnelli: sure, it could, but this is specifically a constant-time AES for wallet encryption
 245 2016-03-24T08:29:20  *** Arnavion has quit IRC
 246 2016-03-24T08:29:38  <jonasschnelli> wumpus: Right. If the p2p encryption will be implemented once, it could be extended there (AES CGM).
 247 2016-03-24T08:29:46  <gmaxwell> sipa: people who want that are using chacha20-poly1305.  AES-GCM with hardware support is apparently somewhat faster (and much less power hungry); but without it the chacha20-poly1305 stuff is faster.
 248 2016-03-24T08:29:47  <wumpus> it's e.g. not the fastest way to implement AES, because we specifically don't need that. There may not be many projects that want to tuse it at all, and there's a long tradition of copy/pasting crypto code anyhow....
 249 2016-03-24T08:30:31  <gmaxwell> people who care about fast AES use hardware or use these parallel implementations which cannot be used for CBC mode, which compatiblity with the wallet encryption needs.
 250 2016-03-24T08:31:11  <wumpus> gmaxwell: yes, exactly
 251 2016-03-24T08:31:48  <wumpus> for the AES code I really have only one concern: is it correct
 252 2016-03-24T08:32:10  <wumpus> for the random code I'm more scared because of the platform dependence and other uglyness involved
 253 2016-03-24T08:32:45  <sipa> wumpus: agree
 254 2016-03-24T08:32:57  <gmaxwell> currently we don't care at all about fast AES. We should somewhat care that it doesn't have stupid sidechannels.  Later, if we care about fast AES for p2p encryption, it would be AES-GCM and end up with a CBC incompatible implementation in any case.
 255 2016-03-24T08:32:57  *** Arnavion has joined #bitcoin-core-dev
 256 2016-03-24T08:33:27  *** AtashiCon has quit IRC
 257 2016-03-24T08:33:41  <wumpus> yes, for P2P we could use a different implementation of AES, or not AES at all (chacha20 or so :p)
 258 2016-03-24T08:34:27  <gmaxwell> right or whatever CAESER selects, depending on what that happens.
 259 2016-03-24T08:34:38  <wumpus> yes
 260 2016-03-24T08:35:25  <sipa> gmaxwell: that's 2 years out
 261 2016-03-24T08:35:46  <gmaxwell> (most of the CAESER candidates are AES based in any case, mostly addressing security complaints about GCM, and expecting hardware support to address the obnoxiousness of constant time AES)
 262 2016-03-24T08:36:11  * wumpus really likes chacha20, it's so small, efficient, and seemingly simple
 263 2016-03-24T08:36:51  *** AtashiCon has joined #bitcoin-core-dev
 264 2016-03-24T08:37:56  <gmaxwell> it has a mixed following, it's apparently enough faster on ARM and without AES-NI that many in the web world have considered it urgent to support... but apparently a lot of the datacenter folks are less than enthused since it ends up much more power hungry than hardware AES.
 265 2016-03-24T08:39:23  <wumpus> yes, right, it's specifically efficient in sw, if there is a hw implementation of AES it'll win out
 266 2016-03-24T08:39:39  <jonasschnelli> What MAC AAD would make sense for the p2p encryption? hash of the pubkey&pubkey? IP/Port?
 267 2016-03-24T08:40:44  <wumpus> let's just wait for hw chacha20 *ducks*
 268 2016-03-24T08:41:03  <gmaxwell> wumpus: well, its not like they have anything better to do with the transistors...
 269 2016-03-24T08:41:35  <wumpus> gmaxwell: agree, the dark silicon problem
 270 2016-03-24T08:42:59  <wumpus> then again - for bitcoin nodes we need to worry about consumer hw, not so much big datacenters
 271 2016-03-24T08:43:39  <wumpus> I think it's safe to say there won't be so many bitcoin nodes in datacenters to rival the number of https implementations, let alone be a power usage concern
 272 2016-03-24T08:43:53  <gmaxwell> well a lot of 'consumer hardware' has hardware aes now too, just not phones so much. :)
 273 2016-03-24T08:44:00  <gmaxwell> but indeed.
 274 2016-03-24T08:44:41  <wumpus> power usage on the other hand is mostly a phone concern. But yeah.
 275 2016-03-24T08:46:32  <gmaxwell> in any case both, given good implementations, should be faster the current 'checksum'
 276 2016-03-24T08:48:16  <sipa> jonasschnelli: the additional data is sent in every message
 277 2016-03-24T08:51:15  <sipa> jonasschnelli: you would include the message length there, for example
 278 2016-03-24T08:51:21  <jonasschnelli> sipa: hmm.. so hash would be too expansive?
 279 2016-03-24T08:51:39  <sipa> hash of what?
 280 2016-03-24T08:52:06  <jonasschnelli> hash of the encryption pubkeys or could also be a hash of the plaintext message.
 281 2016-03-24T08:54:29  <sipa> what are you trying to do?
 282 2016-03-24T08:55:02  <sipa> the additional data is a feature provided by authenticated encryption schemes; you don't need to use it if you have nk need for it
 283 2016-03-24T09:02:24  *** davec has quit IRC
 284 2016-03-24T09:03:12  *** davec has joined #bitcoin-core-dev
 285 2016-03-24T09:04:09  <jonasschnelli> sipa: So do I got this right? You could use something like the current protocol version "70012" as AAD. And during decryption, you feed the TAG (processed AAD) together with the AAD into the decryption method and get a true or false?
 286 2016-03-24T09:04:18  <jonasschnelli> And thanks for you time to explain that to me. :)
 287 2016-03-24T09:06:01  <gmaxwell> you could-- generally it's used for additional per message data that must be sent cleartext for protocol reasons, but which you'd still like included in the authentication.
 288 2016-03-24T09:06:58  *** xiangfu has quit IRC
 289 2016-03-24T09:07:37  <jonasschnelli> gmaxwell: Okay. Right. Message length would be ideal then.
 290 2016-03-24T09:09:59  <gmaxwell> that would be the kind of thing; even that may not be necessary, if the length was wrong, the auth would fail in any case-- wrong amount of data in it.
 291 2016-03-24T09:23:01  *** Alopex has quit IRC
 292 2016-03-24T09:24:06  *** Alopex has joined #bitcoin-core-dev
 293 2016-03-24T09:32:24  *** Arnavion has quit IRC
 294 2016-03-24T09:35:32  *** MarcoFalke has joined #bitcoin-core-dev
 295 2016-03-24T09:36:58  *** Arnavion has joined #bitcoin-core-dev
 296 2016-03-24T09:37:19  *** Arnavion has quit IRC
 297 2016-03-24T09:38:25  *** Arnavion has joined #bitcoin-core-dev
 298 2016-03-24T09:38:34  *** molz has joined #bitcoin-core-dev
 299 2016-03-24T09:41:41  *** molly has quit IRC
 300 2016-03-24T09:46:27  *** cdecker has quit IRC
 301 2016-03-24T09:55:02  *** p15 has joined #bitcoin-core-dev
 302 2016-03-24T10:11:32  *** AaronvanW has joined #bitcoin-core-dev
 303 2016-03-24T10:15:57  <sipa> jonasschnelli: i guess you can include the network magic to the aad
 304 2016-03-24T10:16:44  <jonasschnelli> sipa: okay. Would that be much different then the encrypted message length?
 305 2016-03-24T10:18:56  <sipa> jonasschnelli: i mean both... but there is no strong reason
 306 2016-03-24T10:19:08  <jonasschnelli> okay.
 307 2016-03-24T10:19:37  <sipa> essentially it guarantees that the length/magic came from someone with the session key
 308 2016-03-24T10:20:39  <jonasschnelli> ok
 309 2016-03-24T10:22:57  *** p15 has quit IRC
 310 2016-03-24T10:36:40  <sipa> aes-gcm uses 96-bit IVs, but they only require being unique for a given key
 311 2016-03-24T10:36:49  <sipa> and the key is establishes through ecdh
 312 2016-03-24T10:37:23  <sipa> so they can just be the message number, and don't need to be transmitted
 313 2016-03-24T11:14:44  *** MarcoFalke has quit IRC
 314 2016-03-24T11:18:48  <GitHub140> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/597494f5a90c041945006b8f3eff8f7e482e0f2f
 315 2016-03-24T11:18:48  <GitHub140> bitcoin/0.12 597494f Jonas Schnelli: Remove openssl info from init/log and from Qt debug window...
 316 2016-03-24T11:20:01  <GitHub121> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/e5c35119e967...3ba07bdf7da4
 317 2016-03-24T11:20:02  <GitHub121> bitcoin/master 146746b Alice Wonder: All files related to my RPM spec file project in one commit
 318 2016-03-24T11:20:02  <GitHub121> bitcoin/master 0e4b50a Alice Wonder: Description of RPM directory
 319 2016-03-24T11:20:03  <GitHub121> bitcoin/master 3ba07bd Wladimir J. van der Laan: Merge #7609: All files related to my RPM spec file project in one commit...
 320 2016-03-24T11:20:08  <GitHub36> [bitcoin] laanwj closed pull request #7609: All files related to my RPM spec file project in one commit (master...master) https://github.com/bitcoin/bitcoin/pull/7609
 321 2016-03-24T11:21:31  <GitHub199> [bitcoin] laanwj closed pull request #7650: Cache CBlockIndex::GetMedianTimePast (master...enhancement/cache-getmediantimepast) https://github.com/bitcoin/bitcoin/pull/7650
 322 2016-03-24T11:28:48  <GitHub62> [bitcoin] laanwj closed pull request #6973: Compress Blocks before sending (master...compress) https://github.com/bitcoin/bitcoin/pull/6973
 323 2016-03-24T11:32:34  <sipa> jonasschnelli: i now want to go implement gcm or poly1305...
 324 2016-03-24T11:32:56  * sipa resists, and works on segwit instead
 325 2016-03-24T11:50:56  *** fengling has quit IRC
 326 2016-03-24T11:58:47  <jonasschnelli> sipa: hah. Yes. Work on segwit. The p2p encryption is not urgent.
 327 2016-03-24T11:59:21  *** fengling has joined #bitcoin-core-dev
 328 2016-03-24T12:02:26  <jonasschnelli> sipa: regarding GCM IV: «[...] For a fixed value of the key, each IV value must be distinct, but need not have equal lengths [...]»
 329 2016-03-24T12:03:11  <jonasschnelli> 96 bit is ideal though.
 330 2016-03-24T12:09:15  <sipa> jonasschnelli: you could get 64 bits of the iv out of ecdh, and then add a 32-bit message counter, but that shouldn't be needed
 331 2016-03-24T12:15:38  *** laurentmt has joined #bitcoin-core-dev
 332 2016-03-24T12:28:35  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 333 2016-03-24T12:38:37  <Chris_Stewart_5> is there a BIP66 post mortem some where about the hard fork?
 334 2016-03-24T12:39:17  <sipa> BIP66 was not a hard fork... there were some miners that mined invalid blocks, and kept going
 335 2016-03-24T12:39:48  *** mihi has joined #bitcoin-core-dev
 336 2016-03-24T12:40:32  <Chris_Stewart_5> sipa: Essentially the miners not enforcing this function right? https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L98
 337 2016-03-24T12:52:59  <jonasschnelli> sipa: ecdh produces a 256bit secret? right? How would you take 64bits for the iv if you need to 256bits for the symmetric key?
 338 2016-03-24T13:24:09  *** achow101 has joined #bitcoin-core-dev
 339 2016-03-24T13:42:58  <jonasschnelli> sipa gmaxwell: I'm happy if you are interested in reviewing the authentication and the encryption BIP (https://github.com/jonasschnelli/bips/blob/7e8a9f6e6a06cec1e7842452c85a9dec3730771b/bip-undef-1.mediawiki   https://github.com/jonasschnelli/bips/blob/7e8a9f6e6a06cec1e7842452c85a9dec3730771b/bip-undef-0.mediawiki)
 340 2016-03-24T13:43:45  <jonasschnelli> Don't review the language/orthography, will overhaul that once the technical details are clear.
 341 2016-03-24T13:44:27  * jonasschnelli can't tell you how hard it is to talk and write english if its a complete foreign language
 342 2016-03-24T13:56:29  *** supasonic has joined #bitcoin-core-dev
 343 2016-03-24T13:58:59  *** mihi has quit IRC
 344 2016-03-24T14:00:13  <GitHub55> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3ba07bdf7da4...b88e0b0c610a
 345 2016-03-24T14:00:13  <GitHub55> bitcoin/master d6cc6a1 João Barbosa: Use CCoinControl selection in CWallet::FundTransaction
 346 2016-03-24T14:00:14  <GitHub55> bitcoin/master b88e0b0 Wladimir J. van der Laan: Merge #7506: Use CCoinControl selection in CWallet::FundTransaction...
 347 2016-03-24T14:00:18  <GitHub17> [bitcoin] laanwj closed pull request #7506: Use CCoinControl selection in CWallet::FundTransaction (master...enhancement/use-coin-control-selection) https://github.com/bitcoin/bitcoin/pull/7506
 348 2016-03-24T14:18:11  <sipa> jonasschnelli: if you want to use a single negotiation to intialize encryption for both directions, you need to make sure that the IVs in bith directions are different, or you break the assumption that an attacker cannot see two different messages with the same key and iv
 349 2016-03-24T14:20:27  <sipa> jonasschnelli: my suggestion here would be to have an encinit ("if you want, you can start sending encrypted messages to me and this is my key") and an encack ("yeah, every message after this one will be encrypted, and this is my key"), which you both send in both directions
 350 2016-03-24T14:21:24  <sipa> jonasschnelli: that way, both directions have independent ECDH negotiations, and you also have no problems with synchronization
 351 2016-03-24T14:22:18  <sipa> jonasschnelli: an alternative is thinking hard about synchronization, and making sure that the initiator and accepter of the encryption connection use separate IVs
 352 2016-03-24T14:22:45  <sipa> jonasschnelli: reusing the key as IV is not very useful, i think
 353 2016-03-24T14:24:50  <sipa> jonasschnelli: you probably want to include some text about how this interacts with version/verack (does encinit have to be the first message entirely? can it be done at any time?
 354 2016-03-24T14:31:31  <sipa> jonasschnelli: your encrypted message protocol spec could make the authentication tag explicit (it's 16 bytes for aes-gcm, though we could decide to truncate it)
 355 2016-03-24T14:33:09  *** davec has quit IRC
 356 2016-03-24T14:38:16  *** fengling has quit IRC
 357 2016-03-24T14:40:49  <jonasschnelli> sipa: thanks for the review! Will "process" your points and come back to you with feedback.
 358 2016-03-24T14:44:11  *** lysobit has joined #bitcoin-core-dev
 359 2016-03-24T14:44:41  *** musalbas has joined #bitcoin-core-dev
 360 2016-03-24T14:54:26  *** BCBot_ has joined #bitcoin-core-dev
 361 2016-03-24T15:02:43  *** davec has joined #bitcoin-core-dev
 362 2016-03-24T15:05:32  *** fengling has joined #bitcoin-core-dev
 363 2016-03-24T15:08:10  *** goregrin1 has joined #bitcoin-core-dev
 364 2016-03-24T15:08:34  *** gmaxwell_ has joined #bitcoin-core-dev
 365 2016-03-24T15:08:58  *** gmaxwell_ is now known as Guest18913
 366 2016-03-24T15:11:43  *** CyrusV` has joined #bitcoin-core-dev
 367 2016-03-24T15:13:16  *** arubi has quit IRC
 368 2016-03-24T15:13:16  *** ebfull has quit IRC
 369 2016-03-24T15:13:17  *** gmaxwell has quit IRC
 370 2016-03-24T15:13:17  *** Cory has quit IRC
 371 2016-03-24T15:13:18  *** goregrind has quit IRC
 372 2016-03-24T15:13:18  *** CyrusV has quit IRC
 373 2016-03-24T15:13:19  *** windsok has quit IRC
 374 2016-03-24T15:29:39  *** ebfull has joined #bitcoin-core-dev
 375 2016-03-24T15:29:39  *** Cory has joined #bitcoin-core-dev
 376 2016-03-24T15:29:39  *** windsok has joined #bitcoin-core-dev
 377 2016-03-24T15:30:21  *** Cory has quit IRC
 378 2016-03-24T15:31:36  *** Guyver2 has joined #bitcoin-core-dev
 379 2016-03-24T15:33:38  *** Cory has joined #bitcoin-core-dev
 380 2016-03-24T15:36:28  *** arubi has joined #bitcoin-core-dev
 381 2016-03-24T15:38:39  *** CyrusV` is now known as CyrusV
 382 2016-03-24T16:03:57  *** arowser has quit IRC
 383 2016-03-24T16:04:16  *** arowser has joined #bitcoin-core-dev
 384 2016-03-24T16:51:24  *** Aleph0 has quit IRC
 385 2016-03-24T16:54:07  *** Guest18913 has quit IRC
 386 2016-03-24T16:54:08  *** Guest18913 has joined #bitcoin-core-dev
 387 2016-03-24T16:54:18  *** Guest18913 is now known as gmaxwell
 388 2016-03-24T17:01:53  *** Don_John has joined #bitcoin-core-dev
 389 2016-03-24T17:03:51  *** Don_John has quit IRC
 390 2016-03-24T17:04:21  *** Don_John has joined #bitcoin-core-dev
 391 2016-03-24T17:09:53  *** laurentmt has quit IRC
 392 2016-03-24T17:27:46  <sipa> jonasschnelli: this is interesting: https://github.com/jhcloos/openssh-chacha-poly1305/blob/master/PROTOCOL.chacha20poly1305
 393 2016-03-24T17:28:10  <sipa> openssh's chacha20-poly1305 aead at a high level
 394 2016-03-24T17:28:30  <sipa> they encrypt the packet sizes too, but with a separate key
 395 2016-03-24T17:29:00  <sipa> and then authenticate using the second key which is used for authentication and encryption of the data
 396 2016-03-24T17:29:28  <sipa> which means that an attacker can't even see the message sizes, except by traffic analysis
 397 2016-03-24T17:30:02  <btcdrak> Clever
 398 2016-03-24T17:48:53  <gmaxwell> a little sad to require 4 byte lenghts.
 399 2016-03-24T17:50:38  *** jgarzik has joined #bitcoin-core-dev
 400 2016-03-24T17:52:40  *** jcorgan has joined #bitcoin-core-dev
 401 2016-03-24T17:58:52  *** moli has joined #bitcoin-core-dev
 402 2016-03-24T17:59:30  <gmaxwell> sipa: for identified links, we could do a nice thing with rekeying to make them strong against ECC breaks.
 403 2016-03-24T18:00:50  *** molz has quit IRC
 404 2016-03-24T18:01:17  <sipa> cfields: if we'd want to use something like that, i guess we want an api that allows a client to tell the comnection manager "let me know when you have X bytes"
 405 2016-03-24T18:01:50  <sipa> cfields: which is oerhaps a cleaner abstraction than hardcoding a particular stream protocol?
 406 2016-03-24T18:04:00  *** t800 has joined #bitcoin-core-dev
 407 2016-03-24T18:04:27  <sipa> jonasschnelli: another suggestion: i think we should drop the network magic... resynchronization after garbage hasn't been supported in a long time, it wastes (a tiny bit of) bandwidth, and makes the encrypted stream look decidedly identifiable (though if encrypted connections bootstrap as unencrypted ones, that isn't solvable)
 408 2016-03-24T18:05:05  <gmaxwell> Alice connects to Bob, Alice sends,  Nonce_a, and alice ephemeral pubky (AEP), bob sends nonce_b and BEP.   ECDH runs,  then a KDF that takes the two nonces, and ecdh output, and spits out a session id and two keys (one for each direction.   Then Alice proposes, "I think you have identity X=H(bob pubkey, session_id)", if this is true, bob responds with "I am rekeying with IDX"  and takes his cur
 409 2016-03-24T18:05:11  <gmaxwell> rent encryption key and changes it to H(bob pubkey|enckey).  When alice sees that message she updates her keys for the stream. Then also does the same herselve (sends a message, updates key). Then bob sends a signature with his pubkey.     Then this authentication procedure can just be repeated every so many bytes in order to rotate the stream cipher keys.
 410 2016-03-24T18:05:45  <gmaxwell> If the 'public keys' are kept private, the link has at least symmetric keyed integrity and confidentality, even with an ECC break.
 411 2016-03-24T18:08:06  *** kx has joined #bitcoin-core-dev
 412 2016-03-24T18:14:45  *** kx has quit IRC
 413 2016-03-24T18:17:10  *** kangx has joined #bitcoin-core-dev
 414 2016-03-24T18:23:29  *** treehug88 has joined #bitcoin-core-dev
 415 2016-03-24T18:25:38  *** MarcoFalke has joined #bitcoin-core-dev
 416 2016-03-24T18:27:12  *** frankenmint has quit IRC
 417 2016-03-24T18:28:29  <wumpus> <sipa> which means that an attacker can't even see the message sizes, except by traffic analysis <- I think that would be quite essential for bitcoin, as so much can already be identified from the packet sizes
 418 2016-03-24T18:28:56  <wumpus> certain blocks, certain transactions, etc
 419 2016-03-24T18:30:39  *** belcher has joined #bitcoin-core-dev
 420 2016-03-24T18:31:25  <gmaxwell> I had assumed that the encrypted transport would encapsulate one or more length coded messages inside it... so mostly you'd just see the 'send units'.  Doing this would alow avoiding that extra layering, however... and thus decrease overhead.
 421 2016-03-24T18:32:31  <gmaxwell> though perhaps we should make the encoding support padding, so that messages can be padded up to particular size multiplies, to reduce the traffic analysis sidechannel.
 422 2016-03-24T18:34:09  <CodeShark> is there a meeting today?
 423 2016-03-24T18:34:42  <gmaxwell> yes, in about 20 minutes.
 424 2016-03-24T18:35:01  <gmaxwell> You've suffered DST. :)
 425 2016-03-24T18:36:04  <btcdrak> we dont suffer it over this side of the Atlantic until Sunday
 426 2016-03-24T18:37:59  *** moli has quit IRC
 427 2016-03-24T18:38:30  <btcdrak> gmaxwell: is it maybe worth adding a version byte to the encryption scheme to allow upgrades in the future?
 428 2016-03-24T18:38:31  <jonasschnelli> gmaxwell: +1 for padding to reduce sidechannel identification.
 429 2016-03-24T18:39:03  <jonasschnelli> sipa: do you mean dropping the network magic for all post-encryption-init messages?
 430 2016-03-24T18:39:27  <sipa> also +1 to allowing multiple messages in a packet
 431 2016-03-24T18:39:49  <jonasschnelli> You mean one header with a message count int followed by n payloads?
 432 2016-03-24T18:39:51  <sipa> jonasschnelli: yes, it serves no purpose other than detecting accidental connection to the wrong network
 433 2016-03-24T18:40:05  *** moli has joined #bitcoin-core-dev
 434 2016-03-24T18:40:07  <sipa> jonasschnelli: or just a concatenation of payloads even
 435 2016-03-24T18:40:36  <jonasschnelli> Okay. size + payload + size + payload, etc. That makes sense.
 436 2016-03-24T18:41:44  <sipa> jonasschnelli: inside the aead
 437 2016-03-24T18:42:03  <gmaxwell> that cuts down on the authentication overhead.
 438 2016-03-24T18:42:08  <jonasschnelli> sipa: But I still try to understand your AES GCM 96bit IV solution. How would you create distinctable IV  without transmitting in in the message?
 439 2016-03-24T18:43:15  <sipa> jonasschnelli: use 0000 + counter for one side, use 1111 + counter for the other?
 440 2016-03-24T18:43:39  <jonasschnelli> sipa: Are predictable IVs not a problem?
 441 2016-03-24T18:43:45  <sipa> jonasschnelli: no, just no reuse
 442 2016-03-24T18:43:49  <sipa> ivs are nkt secret
 443 2016-03-24T18:43:52  <sipa> *not
 444 2016-03-24T18:44:12  <jonasschnelli> Okay. I see.
 445 2016-03-24T18:44:23  <gmaxwell> sipa: AEAD packet size does imply memory usage implications though.
 446 2016-03-24T18:45:16  <gmaxwell> but its maximum could just be the same as the largest message.
 447 2016-03-24T18:45:20  <sipa> right
 448 2016-03-24T18:45:44  <jonasschnelli> So. Changes i'll work on a) ecdh exchange for both direction, two sym. keys, fix synchro. b) use chacha20 poly1305 instead of AES GCM c) optimize message structure (drop magic, allow multiple payloads in the AEAD part)
 449 2016-03-24T18:46:17  <sipa> jonasschnelli: no need to fix synchronization if both directions are independent
 450 2016-03-24T18:46:58  <jonasschnelli> yes. I mean fix syncho by using two keypair, one per direction.
 451 2016-03-24T18:47:04  *** jcorgan has quit IRC
 452 2016-03-24T18:47:59  <gmaxwell> great.
 453 2016-03-24T18:47:59  <jonasschnelli> Has chacha20-poly1305 been proven to be stable? Reasonable amount of cryptanalysis? Or do we threat stuff out of DJB's kitchen as stable anyways? :)
 454 2016-03-24T18:48:05  *** gevs has quit IRC
 455 2016-03-24T18:48:18  <gmaxwell> jonasschnelli: it's part of TLS and SSH.
 456 2016-03-24T18:48:28  <gmaxwell> I think it's sutiable for our purposes.
 457 2016-03-24T18:48:31  <wumpus> it's geting a lot of testing and review in ssh at least
 458 2016-03-24T18:49:00  <jonasschnelli> Okay. And how complex is an implementation? Or would this lead to a revival of openssl in core?
 459 2016-03-24T18:49:12  <gmaxwell> it's very simple.
 460 2016-03-24T18:49:26  <wumpus> their implementation is very short, that's one thing that is so cool about it
 461 2016-03-24T18:49:29  <jonasschnelli> what speaks again unsing openssl (p2p layer enc is non consensus)?
 462 2016-03-24T18:49:47  <jonasschnelli> *against
 463 2016-03-24T18:50:07  <gmaxwell> we're not taking 400,000+ lines of unreviewable, frequently CVEed code for this; I'd rather not have the functionality.
 464 2016-03-24T18:50:11  <wumpus> attack surface, making it impossible to get rid of openssl dependency
 465 2016-03-24T18:50:44  <wumpus> and you can already do it using stunnel
 466 2016-03-24T18:51:22  <gmaxwell> I hope that this works so that even if we don't care about the confidentiality improvement, it's just simply faster than the existing transport.
 467 2016-03-24T18:52:06  <jonasschnelli> Right. I think dropping the sha256 per message could be an improvement in performance.
 468 2016-03-24T18:52:24  <jonasschnelli> combining multiple messages into one encrypted message also
 469 2016-03-24T18:53:03  <jonasschnelli> The question is, how to "bundle" messages (example: bundle invs). When releasing them, etc. I mean for the implementation.
 470 2016-03-24T18:53:13  <gmaxwell> https://www.imperialviolet.org/2014/02/27/tlssymmetriccrypto.html has some benchmarks vs AES-GCM fwiw.
 471 2016-03-24T18:53:17  <sipa> jonasschnelli: we already do that
 472 2016-03-24T18:53:38  <sipa> jonasschnelli: SendMessages is a function in main that computes a block of messages to send at once
 473 2016-03-24T18:53:44  <jonasschnelli> Ah. But they get sent individually (one inv per msg)?
 474 2016-03-24T18:54:02  <sipa> jonasschnelli: maybe you should look into how our current protocol works :)
 475 2016-03-24T18:54:08  <jonasschnelli> haha. True.
 476 2016-03-24T18:54:23  <sipa> no, an inv messages can contain up to 5000 invs
 477 2016-03-24T18:55:00  <jonasschnelli> Okay. So where would we benefit from bundling messages in one encrypted message?
 478 2016-03-24T18:55:20  <wumpus> it makes sense to make it possible, it benefits privacy
 479 2016-03-24T18:55:58  *** achow101 has quit IRC
 480 2016-03-24T18:55:59  <sipa> well, if you take as baseline an implementation that encrypts message sizes, there is no privacy benefit to bundling
 481 2016-03-24T18:56:04  <wumpus> even if your first implementation doesn't actually use it, imo the protocol should support it
 482 2016-03-24T18:56:16  <gmaxwell> phantomcircuit had a patch to add corking and uncoarking in the message handling loop, I presume something similar would be done for message batching.
 483 2016-03-24T18:56:18  <jonasschnelli> agree
 484 2016-03-24T18:56:20  *** achow101 has joined #bitcoin-core-dev
 485 2016-03-24T18:56:37  <wumpus> ok, yes, no privacy benefit to bundling makes it less important to me
 486 2016-03-24T18:56:39  <sipa> as they would be sent back-to-back, so traffic analysis would not let you discover independent messages
 487 2016-03-24T18:56:51  <gmaxwell> e.g. we'd cork both the crypto and tcp, do all the message handling for a peer, then uncork both.
 488 2016-03-24T18:57:32  <sipa> however, bundling does give a small bandwidth improvement, as you'd share a authentication tag acorss muktiple messages
 489 2016-03-24T18:57:50  <wumpus> well, let's not micro-optimize before we have anything
 490 2016-03-24T18:57:56  <jonasschnelli> tag is 12bytes. Right.
 491 2016-03-24T18:58:05  <gmaxwell> not necessarily all that small, the auth is relatively large to our average message size.
 492 2016-03-24T18:58:38  <wumpus> let's first make sure it is secure, then wonder about bandwidth optimization
 493 2016-03-24T18:59:11  <jonasschnelli> Right. I'll try to update the Bips with what we have discusses and ask for another review.
 494 2016-03-24T18:59:20  <jonasschnelli> *discussed
 495 2016-03-24T18:59:23  <wumpus> trying to tackle everything at once is going to make this go nowhere, I'm afraid
 496 2016-03-24T18:59:31  <gmaxwell> the bundling also makes padding easy, you just define a new message type "pad" that is just discarded on the other end. :)
 497 2016-03-24T18:59:37  <sipa> jo	16 if you want 128 bit security
 498 2016-03-24T18:59:40  *** molz has joined #bitcoin-core-dev
 499 2016-03-24T18:59:44  <gmaxwell> yea, iteration is fine.
 500 2016-03-24T18:59:46  <jonasschnelli> should i say now: "perfect is the enemy of good". Yes? No. *duck*
 501 2016-03-24T19:00:04  <MarcoFalke> meeting time?
 502 2016-03-24T19:00:10  <gmaxwell> Meeting time.
 503 2016-03-24T19:00:10  <wumpus> jonasschnelli: more like: keep your goal focused. Your goal  is security and privacy, first. Make no dumb mistakes there ;)
 504 2016-03-24T19:00:24  <jonasschnelli> RIght!
 505 2016-03-24T19:00:25  <wumpus> #startmeeting
 506 2016-03-24T19:00:25  <lightningbot> Meeting started Thu Mar 24 19:00:25 2016 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
 507 2016-03-24T19:00:25  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
 508 2016-03-24T19:00:47  <wumpus> jonasschnelli: scope creep is a very bitcoin-y thing
 509 2016-03-24T19:00:52  <wumpus> ok, topic proposals?
 510 2016-03-24T19:01:05  <MarcoFalke> python 3 and rpc tests?
 511 2016-03-24T19:01:14  <wumpus> from last meeitng we have ACTION: merge #7575 (wumpus, 19:04:04)  ... that was done
 512 2016-03-24T19:01:19  <btcdrak> softfork #7648 status
 513 2016-03-24T19:01:31  <wumpus> next was ACTION: review #7648 after #7575 is merged (wumpus, 19:09:39)
 514 2016-03-24T19:01:32  <Luke-Jr> topic proposal: v0.11.3 and v0.10.5
 515 2016-03-24T19:01:34  *** moli has quit IRC
 516 2016-03-24T19:01:42  <wumpus> how is that going?
 517 2016-03-24T19:01:53  <wumpus> #topic softfork #7648 status
 518 2016-03-24T19:02:08  <wumpus> MarcoFalke: will get to python3 at the end, I think the softfork stuff is most important right now
 519 2016-03-24T19:02:12  <btcdrak> #7648 has received some tested ACKs.
 520 2016-03-24T19:02:37  <MarcoFalke> sure
 521 2016-03-24T19:03:08  <wumpus> okay, good
 522 2016-03-24T19:03:46  <wumpus> I think it needs more review still
 523 2016-03-24T19:04:15  <btcdrak> time to crack the whip.
 524 2016-03-24T19:04:24  <wumpus> for a softfork the bar is somewhat higher than for random pull #1234
 525 2016-03-24T19:04:39  <wumpus> yes
 526 2016-03-24T19:05:03  <jonasschnelli> 7648 looks good and will also review it in details soon (during weekend)
 527 2016-03-24T19:05:08  <wumpus> I mean I've seen some acks but very little comment on the actual code, that could be a sign everything is perfect, or people need to look better
 528 2016-03-24T19:05:29  <wumpus> thanks jonasschnelli
 529 2016-03-24T19:05:40  <btcdrak> MarcFalke: can you look as well please?
 530 2016-03-24T19:05:42  <wumpus> #action review more at https://github.com/bitcoin/bitcoin/pull/7648
 531 2016-03-24T19:05:47  <btcdrak> and CodeShark
 532 2016-03-24T19:05:52  <wumpus> I'll take a more in-depth look at it as well
 533 2016-03-24T19:05:55  <btcdrak> ty
 534 2016-03-24T19:05:57  <MarcoFalke> sure
 535 2016-03-24T19:06:08  <gmaxwell> has anyone looked to see if there are MTP violations still being accepted on the network?
 536 2016-03-24T19:06:39  <gmaxwell> I wouldn't consider them a hard blocker to 113 deployment, but if any are, we should make a concerted effort to extinguish them first.
 537 2016-03-24T19:06:59  <btcdrak> gmaxwell: there should be 100% coverage of 113 since the upgrade to 0.11.2
 538 2016-03-24T19:07:16  <gmaxwell> btcdrak: there may be miners with order or modified code, however.
 539 2016-03-24T19:07:28  <gmaxwell> er, s/order/older/
 540 2016-03-24T19:07:51  <sipa> s/may be/are/
 541 2016-03-24T19:08:25  <gmaxwell> if there are any, we should try to find them and get them to stop mining MTP violations.
 542 2016-03-24T19:09:34  <wumpus> yes
 543 2016-03-24T19:09:39  <gmaxwell> I can work on this.
 544 2016-03-24T19:09:49  <cfields> sipa: missed you earlier. ack on allowing for chunked decrypt/decode.
 545 2016-03-24T19:10:02  <btcdrak> cfields: can you put 7648 review on your tasklist too please?
 546 2016-03-24T19:10:08  <wumpus> #action hunt down miners that mine MTP violations
 547 2016-03-24T19:10:35  <cfields> btcdrak: yes
 548 2016-03-24T19:11:34  <wumpus> ok
 549 2016-03-24T19:11:54  <wumpus> next topic I guess
 550 2016-03-24T19:12:01  <wumpus> #topic proposal: v0.11.3 and v0.10.5
 551 2016-03-24T19:12:29  <sipa> we discussed what would go in last week; did anythkng chang
 552 2016-03-24T19:12:30  <sipa> ?
 553 2016-03-24T19:13:03  <wumpus> I don't know, luke-jr requested the topic
 554 2016-03-24T19:13:10  <btcdrak> well we didnt discuss 0.10.5, we discussed 0.12.1
 555 2016-03-24T19:13:12  <Luke-Jr> I'm going through https://github.com/bitcoin/bitcoin/pull/7047 again, to check everything included is in 0.12 and make it mergable again
 556 2016-03-24T19:14:00  <wumpus> that's way too much for a softfork release at least
 557 2016-03-24T19:14:01  <Luke-Jr> what isn't clear to me is what the plan is for v0.11.3 being released - I assume wumpus is doing that at some point? - and whether I'm taking over for v0.10.5, or if wumpus wanted to do one final 0.10 before passing it off
 558 2016-03-24T19:14:27  <wumpus> for a normal minor release it'd make more sense
 559 2016-03-24T19:14:52  <wumpus> yes I'll do the 0.11.x release
 560 2016-03-24T19:14:58  <MarcoFalke> 7047 also includes a lot of "comment-fixed". I am not sure if they are useful in our branch
 561 2016-03-24T19:15:01  <gmaxwell> whats deployment of 0.11.2 vs 0.12 look like right now?
 562 2016-03-24T19:15:06  <MarcoFalke> I'd rather only have bugfixes
 563 2016-03-24T19:15:08  <wumpus> I'll leave 0.10.x to you
 564 2016-03-24T19:15:08  <Luke-Jr> are we doing a softfork release in the next month?
 565 2016-03-24T19:15:21  <sipa> Luke-Jr: i say yes
 566 2016-03-24T19:15:32  <Luke-Jr> ok, so then we should let these things wait until after that
 567 2016-03-24T19:15:38  <wumpus> I think we should do a softfork release asap
 568 2016-03-24T19:15:50  <Luke-Jr> or maybe I should go through it and make sure there's nothing important
 569 2016-03-24T19:15:52  <wumpus> let's just get those stupid things reviewed and merged
 570 2016-03-24T19:15:57  <jonasschnelli> As soon as #7648 has been reviewed more and merged
 571 2016-03-24T19:16:10  <wumpus> jonasschnelli: and backports reviewed and merged ofc
 572 2016-03-24T19:16:21  <wumpus> but yes 7648 is the first step
 573 2016-03-24T19:16:31  <jonasschnelli> RIght
 574 2016-03-24T19:16:32  <Luke-Jr> (also, depending on ease of backporting the softforks, I may decide to just give up on 0.10 support; tbd)
 575 2016-03-24T19:16:42  <btcdrak>  /Satoshi:0.12.0/	1713
 576 2016-03-24T19:16:43  <btcdrak>  /Satoshi:0.11.2/	1387
 577 2016-03-24T19:16:50  <jonasschnelli> 0.10 can be given up IMO
 578 2016-03-24T19:16:55  * jonasschnelli is checking his seeder data
 579 2016-03-24T19:17:01  <gmaxwell> Luke-Jr: I think absent someone requesting otherwise (and perhaps funding your effort) you probably should.
 580 2016-03-24T19:17:08  <Luke-Jr> jonasschnelli: it's currently Gentoo stable, so at the very least I will need to bump that
 581 2016-03-24T19:17:21  <btcdrak> The problem with 0.10 is we all need to put review time into the backports and that's difficult when we have trouble even reviewing backports for 0.11
 582 2016-03-24T19:17:37  <petertodd> btcdrak: +1
 583 2016-03-24T19:17:56  <Luke-Jr> btcdrak: most backports are trivial, but for softforks I agree if they're not a clean backport
 584 2016-03-24T19:18:04  <wumpus> right
 585 2016-03-24T19:18:12  <btcdrak> I think if there is a 0.10.5 release it should be extremely minimal... and frankly, backporting BIP68 wont be fun or easy.
 586 2016-03-24T19:18:14  <sipa> bip68/112/113 are quite a bit of code
 587 2016-03-24T19:18:28  <wumpus> let's spend our energy on moving forward
 588 2016-03-24T19:18:33  <jonasschnelli> 0.10 should be last priority. If someone offers to do that: fine.
 589 2016-03-24T19:18:41  <gmaxwell> plus, a far backport which is done by one person and used by few more likely won't be materially more stable than a more recent release.
 590 2016-03-24T19:18:44  <Luke-Jr> ok, so for 0.10 I'll see the complexity in the softfork backports, and if it's anything more than absolutely trivial, I'll just decide to drop it
 591 2016-03-24T19:18:47  <btcdrak> Luke-Jr: even BIP68 backport to 0.11 was tricky, not a clean cherry-pick.
 592 2016-03-24T19:18:58  <Luke-Jr> btcdrak: well, I'd be backporting to 0.10 *from* 0.11
 593 2016-03-24T19:19:19  <jonasschnelli> at dnsseed.dump | grep "  100.00% 100.00%" | grep "/Satoshi:0.10." | wc -l       ---> 266
 594 2016-03-24T19:19:25  <sipa> we can still do critical security updates for 0.10 without choosing to backport sotfork
 595 2016-03-24T19:19:33  <wumpus> sure
 596 2016-03-24T19:19:45  <jonasschnelli> 6.3% 0.10.x nodes
 597 2016-03-24T19:19:49  <btcdrak> so on this topic, people need to also look at backport #7543
 598 2016-03-24T19:19:58  <Luke-Jr> and for 0.11, sounds like the plan is to do a softfork-only release before any non-essential bugfix backports
 599 2016-03-24T19:20:08  <CodeShark> +1 jonasschnelli
 600 2016-03-24T19:20:09  <petertodd> sipa: and people who need a v0.10.x node for software compatibility reasons can generally run it behind a newer node
 601 2016-03-24T19:20:18  <sipa> petertodd: also true
 602 2016-03-24T19:20:31  <wumpus> #action people need to also look at backport #7543: [0.12] Backport BIP9, BIP68 and BIP112 with softfork
 603 2016-03-24T19:20:36  <sipa> petertodd: specifically, behind a 0.11 or 0.12 pruned one :)
 604 2016-03-24T19:20:45  <petertodd> sipa: yes! that too
 605 2016-03-24T19:20:57  <Luke-Jr> #action 0.10: attempt to backport softforks, and if non-trivial, discontinue support
 606 2016-03-24T19:21:12  <Luke-Jr> #action 0.11: no non-essential fix backports until softfork release
 607 2016-03-24T19:21:26  <Luke-Jr> #action bump Gentoo stable to 0.11 (from 0.10)
 608 2016-03-24T19:21:38  <sipa> Luke-Jr: support is already discontinued according to policy, only critical biffixes
 609 2016-03-24T19:21:42  <sipa> *bugfixes
 610 2016-03-24T19:21:51  <CodeShark> I'd say discontinue support for 0.10 unless someone wishes to provide resources for it
 611 2016-03-24T19:21:59  <wumpus> well if luke-jr wants to try to backport the softfork that's up to him
 612 2016-03-24T19:21:59  <Luke-Jr> sipa: policy only dictates guaranteed support, not guaranteed to end
 613 2016-03-24T19:22:22  <sipa> wumpus: sure, but i don't think that's very relevant here
 614 2016-03-24T19:22:27  <wumpus> no need to even discuss it here :p
 615 2016-03-24T19:22:28  <wumpus> right
 616 2016-03-24T19:22:44  <btcdrak> #action 0.11 review #7716 Backport BIP9 and softfork for BIP's 68,112,113
 617 2016-03-24T19:22:58  <Luke-Jr> any more topics?
 618 2016-03-24T19:23:32  <sipa> as soon as we have 0.12.1, i will rebase segwit on top, so segnet can benefit from its bip9/68/112/113 support
 619 2016-03-24T19:23:37  <cfields> can give a quick update on the net refactor
 620 2016-03-24T19:23:50  <wumpus> I'd say spend the rest of the meeting time reviewing #7648 :p
 621 2016-03-24T19:23:53  <wumpus> sure cfields!
 622 2016-03-24T19:24:00  <wumpus> #topic cfields net refactor
 623 2016-03-24T19:24:34  <sipa> please, stick to the net refactor, not the gross one
 624 2016-03-24T19:24:56  <cfields> i finally have a working full-featured branch. it's a mess code-wise, but i think it should be approachable in the next day or two after some code movement. I expect to be looking for concept review next week
 625 2016-03-24T19:24:58  <gmaxwell> cfields: start taking, I can't take the wait.
 626 2016-03-24T19:25:02  <wumpus> lol
 627 2016-03-24T19:25:23  <wumpus> cfields: I've been testing it on a node btw for last week, seems to be stable and working
 628 2016-03-24T19:25:48  <wumpus> haven't looked at details of performance etc but no crashes or insane log entries
 629 2016-03-24T19:26:02  <btcdrak> sipa: you can cherry-pick from  #7543 in that case
 630 2016-03-24T19:26:10  <cfields> wumpus: that's great to hear. lots has changed since then, but the core is the same
 631 2016-03-24T19:26:35  <cfields> i think i have an idea of how to handle code review, but i'd like to get some concept acks first...
 632 2016-03-24T19:26:50  <wumpus> you got my concept ack at the boston meeting
 633 2016-03-24T19:26:56  <sipa> and mine
 634 2016-03-24T19:27:09  <cfields> there's a huge mix of code movement, a new lib, abstraction, etc. I think i have a plan for getting it slotted in
 635 2016-03-24T19:27:28  <wumpus> great
 636 2016-03-24T19:27:29  <cfields> ok. i can start PRing chunks into core as i go, then
 637 2016-03-24T19:27:44  <Luke-Jr> cfields: practical to add block streaming? ;)
 638 2016-03-24T19:28:02  *** frankenmint has joined #bitcoin-core-dev
 639 2016-03-24T19:28:02  <cfields> first step is getting the net stuff instanced. that's a lot of movement without much functional difference
 640 2016-03-24T19:28:02  <gmaxwell> do not creep cfield's scope. :P
 641 2016-03-24T19:28:06  <wumpus> please, let's replace the current things first, then add additional features
 642 2016-03-24T19:28:08  <jonasschnelli> hah
 643 2016-03-24T19:28:09  <wumpus> example gmaxwell
 644 2016-03-24T19:28:12  <wumpus> exactly*
 645 2016-03-24T19:28:13  <cfields> i think that can start going in parallel
 646 2016-03-24T19:28:37  <cfields> gmaxwell: heh, i've adjusted scope so many times now that i'm certainly not budging again :p
 647 2016-03-24T19:28:44  <jonasschnelli> The net refactor will be very hard to review already. Lets keep it as simple as possible.
 648 2016-03-24T19:28:51  <wumpus> that's good, stand your ground , people here are awful :)
 649 2016-03-24T19:28:57  <cfields> turns out the absolute minimum scope for net refactor is already enormous
 650 2016-03-24T19:29:07  <jonasschnelli> Indeed.
 651 2016-03-24T19:29:29  <wumpus> yes it'll already be challenging to get this done before 0.13 I think, but we have to try
 652 2016-03-24T19:29:31  <Luke-Jr> gmaxwell: indeed, just hope we don't need to refactor it again after
 653 2016-03-24T19:29:41  <sipa> Luke-Jr: we absolutely will
 654 2016-03-24T19:29:48  <cfields> wumpus: yes, it's much later than I was hoping for. If it slips past 0.13, I'll understand.
 655 2016-03-24T19:29:51  <sipa> but at different layers :)
 656 2016-03-24T19:30:04  <wumpus> right, this is only the bottom layer
 657 2016-03-24T19:30:19  <cfields> sipa: well the hope is that it's been split up into chunks that can be changed in the future much easier
 658 2016-03-24T19:30:37  <sipa> cfields: yes, that's what abstractions are for
 659 2016-03-24T19:31:27  <jonasschnelli> topic: Should we shorty touch how we should proceed with sipas CT AES implementation? Extract to library? Yes/No?
 660 2016-03-24T19:31:40  <cfields> there are a few hacks that i'm not going to mess with for now. For ex, even though it's all instanced, there's still a g_connman that's used liberally as opposed to the clean way of passing the instance around
 661 2016-03-24T19:31:42  <wumpus> meh
 662 2016-03-24T19:31:55  <cfields> i'll start working on a list of todos so that some of the decisions are more clear
 663 2016-03-24T19:32:24  <petertodd> jonasschnelli: extract is a great idea I think - don't have to be fancy, just enough to continue to set a better standard than "roll our own"
 664 2016-03-24T19:32:36  <sipa> cfields: just PR it! i want to see the code :)
 665 2016-03-24T19:32:47  *** frankenmint has quit IRC
 666 2016-03-24T19:32:49  <jonasschnelli> sipa: code: https://github.com/theuni/bitcoin/tree/net-refactor8
 667 2016-03-24T19:32:52  <cfields> sipa: i'm still frantically coding it :)
 668 2016-03-24T19:32:58  <sipa> ah.
 669 2016-03-24T19:33:07  <cfields> jonasschnelli: nooooooo. everyone shield your eyes from that :)
 670 2016-03-24T19:33:07  <wumpus> the code is actually buildable and works
 671 2016-03-24T19:33:14  <sipa> now i have a moral obligation to actually go look, right?
 672 2016-03-24T19:33:28  <wumpus> well test at least :)
 673 2016-03-24T19:33:33  <btcdrak> cfields: love that last commit message :)
 674 2016-03-24T19:33:35  <cfields> sipa: with the understanding that it's basically my /tmp :)
 675 2016-03-24T19:33:35  <jonasschnelli> cfields: We all know how code during implementation can look. No shame for it!
 676 2016-03-24T19:34:09  <wumpus> yes :)
 677 2016-03-24T19:34:31  <jonasschnelli> petertodd: I'm also for extracting.
 678 2016-03-24T19:34:54  <jonasschnelli> Simple autotools setup, add as subtree place on bitcoin/libctaes (or similar)
 679 2016-03-24T19:34:57  <wumpus> #topic CT AES library
 680 2016-03-24T19:35:12  <Luke-Jr> what is "CT" for here?
 681 2016-03-24T19:35:14  <wumpus> I don't think extracting it into a library has to affect our use of it
 682 2016-03-24T19:35:14  <petertodd> jonasschnelli yup, subtree is fine
 683 2016-03-24T19:35:17  <wumpus> Constant Time
 684 2016-03-24T19:35:18  <jonasschnelli> constant time
 685 2016-03-24T19:35:24  <Luke-Jr> ah
 686 2016-03-24T19:35:45  <sipa> i feel like it's more a code snippet than a library
 687 2016-03-24T19:35:46  <gmaxwell> constant time.
 688 2016-03-24T19:35:53  <wumpus> yes, it's more like a snippet
 689 2016-03-24T19:35:56  <gmaxwell> "Petertodd raised verification concerns for the internal AES implementation. I could suggest some strategies on that." is what I wrote earlier. :P
 690 2016-03-24T19:35:59  <wumpus> crypto has a rich tradition of copy/paste
 691 2016-03-24T19:36:11  <cfields> fwiw, I plan on doing the same with libbtcnet
 692 2016-03-24T19:36:13  <jonasschnelli> wumpus: I think placing stuff in libraries follows a modular approach  which is desirable for Bitcoin IMO
 693 2016-03-24T19:36:24  <Luke-Jr> +1 for libctaes
 694 2016-03-24T19:36:29  *** jcorgan has joined #bitcoin-core-dev
 695 2016-03-24T19:36:34  <cfields> i have a tiny/crappy makefile that i use to build it externally, but I plan to just copy/paste it into Core and keep it in sync
 696 2016-03-24T19:36:35  <petertodd> sipa: well, to be clear, you did write that code snippet from scratch yourself right?
 697 2016-03-24T19:36:35  <wumpus> you could stil include it in a library, but please don't compilicate the build
 698 2016-03-24T19:36:40  <Luke-Jr> cfields: libbcnet I hope; what does the BTC unit size have to do with networking? :P
 699 2016-03-24T19:36:49  <gmaxwell> in any case, I'm not convinced that the standard test vectors prove the code correct. Beyond soliciting outside review, which btcdrak has done.  I was going to suggest trying some mutation testing to see if some additional better test vectors can be constructed.
 700 2016-03-24T19:36:54  <jonasschnelli> sipa: Its a snipped now, ... but we all know how snippets evolve over time.
 701 2016-03-24T19:37:00  <cfields> Luke-Jr: when i googled "libp2p" some etherium thing came up :)
 702 2016-03-24T19:37:01  <sipa> petertodd: yes, though the logic gates for the sbox come from a paper
 703 2016-03-24T19:37:14  <wumpus> I think we should separate concerns here: getting the code into a library, and getting the damn code reviewed
 704 2016-03-24T19:37:18  <wumpus> the first is easy, just work
 705 2016-03-24T19:37:18  <Luke-Jr> cfields: oh great, let's just integrate that! :P
 706 2016-03-24T19:37:37  <petertodd> sipa: right, so it's independent work, which means it's crypto we wrote, which means no matter how small it is, we should at least make a token effort to try to let independent use and review :)
 707 2016-03-24T19:37:38  <Luke-Jr> wumpus: I don't feel competent to review crypto code :<
 708 2016-03-24T19:37:49  <wumpus> Luke-Jr: me neither.
 709 2016-03-24T19:38:01  <wumpus> Luke-Jr: so we'll have to find someone that can, btcdrak had some ideas.
 710 2016-03-24T19:38:04  <gmaxwell> You're all competent to review if the code is constant time or not.
 711 2016-03-24T19:38:16  <sipa> or that it has test coverage
 712 2016-03-24T19:38:16  <jonasschnelli> What about apoelstra?
 713 2016-03-24T19:38:32  <gmaxwell> And probably no one is competent to review if it's correct, fortunately AES is a permutation, which does make confidence building via testing easier.
 714 2016-03-24T19:38:40  <wumpus> but not if it's correct AES, or has some weird edge cases, mathatical inconsistencies, or other wacky side channels
 715 2016-03-24T19:38:55  <wumpus> right
 716 2016-03-24T19:39:02  <sipa> and notice it has no tables or branches, so it would be exceedingly hard to make it broken in a very small number of cases
 717 2016-03-24T19:39:04  <jonasschnelli> IMO that all speak for splitting it off into a own library.
 718 2016-03-24T19:39:04  <petertodd> gmaxwell: the problem here, is I don't know enough about the problem to know if I'm actually competent to review that
 719 2016-03-24T19:39:07  <gmaxwell> (by no one, I mean no human. :) )
 720 2016-03-24T19:39:35  <btcdrak> petertodd: I asked isis to take a look on a paid basis.
 721 2016-03-24T19:39:41  <petertodd> btcdrak: good!
 722 2016-03-24T19:39:48  <gmaxwell> petertodd: the requirement is that there should be no leakage of the secret state (key or data) in the timing of the program on any hardware we support.
 723 2016-03-24T19:39:53  <btcdrak> that's @isislovecruft from Tor for anyone that doesnt know.
 724 2016-03-24T19:40:01  * Luke-Jr volunteers to do the splitting-off-into-library work if sipa doesn't want to
 725 2016-03-24T19:40:18  <sipa> anyway, i'll turn it into a .c/.h; if someone else wants to actually do the work for autotools etc, go ahead.
 726 2016-03-24T19:40:19  <jonasschnelli> Luke-Jr: I already have started that. :)
 727 2016-03-24T19:40:28  <wumpus> Luke-Jr: this will again trigger the subtree-or-external discussion, I can't take that another time
 728 2016-03-24T19:40:45  <wumpus> Luke-Jr: and linux distributions messing it up etc
 729 2016-03-24T19:40:51  <gmaxwell> it will not be external.
 730 2016-03-24T19:40:53  <Luke-Jr> wumpus: seems that discussion should be project-global, not per-lib
 731 2016-03-24T19:40:56  <jonasschnelli> sipa: I have checked to code and it looks pretty Cish.
 732 2016-03-24T19:41:05  <Luke-Jr> subtree-with-configure-option seems to work
 733 2016-03-24T19:41:12  <sipa> imho, it can be a separate repository without being a library
 734 2016-03-24T19:41:22  <sipa> just copy it into your project if you neeed it
 735 2016-03-24T19:41:27  <wumpus> right.
 736 2016-03-24T19:41:38  <sipa> it's a single file and has no intention to grow beyond that
 737 2016-03-24T19:41:56  <gmaxwell> nor any utility to grow beyond that.
 738 2016-03-24T19:41:59  <jonasschnelli> meh. Adding a build setup should be trivial and would hurt nobody?
 739 2016-03-24T19:42:02  <cfields> sipa: is there enough non-standard stuff to necessitate a buildsystem? or would a simple makefile do?
 740 2016-03-24T19:42:11  <wumpus> the API will never change, the implementation will hopefully also never change (excluding critical problems)
 741 2016-03-24T19:42:22  <sipa> cfields: it will be a single C89 file with no dependencies beyond stdint
 742 2016-03-24T19:42:36  <Luke-Jr> stdint isn't C89, though?
 743 2016-03-24T19:42:36  <gmaxwell> wumpus: the only 'change' I could imagine for it is wrapping it with something that uses AESNI when available instead.
 744 2016-03-24T19:42:48  <sipa> Luke-Jr: it indeed isn't, which is why i mention it :)
 745 2016-03-24T19:42:50  <cfields> sipa: ah great
 746 2016-03-24T19:42:56  <gmaxwell> well C89+stdint, or C99...
 747 2016-03-24T19:43:19  <gmaxwell> stdint is almost universal even in terrible embedded complilers.
 748 2016-03-24T19:43:28  <jonasschnelli> Are there no plans to extend the "lib" with more stuff? Like for a possible p2p encryption?
 749 2016-03-24T19:43:36  <wumpus> and if not it's easy enough to define the types yourself
 750 2016-03-24T19:43:53  <Luke-Jr> jonasschnelli: well, then the name libctaes would be wrong
 751 2016-03-24T19:43:57  <sipa> jonasschnelli: i'd do that on another layer
 752 2016-03-24T19:43:58  <jonasschnelli> Right.
 753 2016-03-24T19:44:03  <wumpus> no, jonasschnelli, that should be something else yet again
 754 2016-03-24T19:44:05  <sipa> jonasschnelli: it would still rely on aes primitives
 755 2016-03-24T19:44:27  <gmaxwell> This pressure to constantly create "libraries" to dump things in is streesful for me. A well constructed library is a serious amount of work in and of itself.
 756 2016-03-24T19:44:29  <wumpus> ctaes would be constant time aes :-) I don't think you want/need that for the p2p
 757 2016-03-24T19:44:31  <petertodd> gmaxwell: I may have to use a climbing analogy to get my point accross here :) sure, I can easily read a book on knots and belay systems and feel like I _should_ know everything I need to know to do that stuff safely, but the consequences of failure are high enough - and the field unusual enough - that I'm going to insist I check things out with others generally accepted as experts first - who cares how irrational ...
 758 2016-03-24T19:44:32  <jonasschnelli> Okay. Then we might agree in a subtree with just a .c/.h instead of a autotools setup.
 759 2016-03-24T19:44:37  <petertodd> ... that feels. I think general standards for low-level crypto implementations tend in that direction.
 760 2016-03-24T19:45:05  <wumpus> jonasschnelli: yes
 761 2016-03-24T19:45:06  <petertodd> jonasschnelli: yes, just a .c/.h is fine! like I said, bare minimum to set a precedent that we're trying to invite outside review
 762 2016-03-24T19:45:08  <Luke-Jr> subtree with just .c/.h until some future time when someone decides to lib-ify it seems ok
 763 2016-03-24T19:45:11  <wumpus> jonasschnelli: that's very common for crypto
 764 2016-03-24T19:45:18  <sipa> and maybe for performance reasons, it makes sense to use a more complicated faster implementation, and maybe also warrants more maintainance afterwards if it's going to be adopted by other prijects for the purpose of p2p encryption
 765 2016-03-24T19:45:31  <jonasschnelli> okay.
 766 2016-03-24T19:45:43  <gmaxwell> petertodd: considering that basically all AES-CBC software (not AESNI) out there is gratitiously timing vulnerable, and we were going to take such a patch ourselves before; I feel you're compent enough to review and increase confidence, if not alone to achieve perfection.
 767 2016-03-24T19:45:47  <Luke-Jr> sipa: seems likely other wallets may use it
 768 2016-03-24T19:45:47  <jonasschnelli> agree on bitcoin/libctaes?
 769 2016-03-24T19:45:48  <wumpus> and just copy/subtree the thing into your project and include it into your build system, all the dependency micro management isnt' worth it for everything
 770 2016-03-24T19:46:05  <sipa> Luke-Jr: they absolutely may!
 771 2016-03-24T19:46:16  <jonasschnelli> Luke-Jr: right.  I have two projects where i will use it immediately.
 772 2016-03-24T19:46:21  <wumpus> jonasschnelli: bitcoin-core/libctaes it would be then
 773 2016-03-24T19:46:27  <gmaxwell> as I noted before, this implementation is not super fast. We have no need to make it super fast. It may be of idependant interest to others because it is quite small.
 774 2016-03-24T19:46:29  <sipa> Luke-Jr: but for wallets, it has no performance requirements
 775 2016-03-24T19:46:36  <jonasschnelli> wumpus: ah. Right. The moving.
 776 2016-03-24T19:46:56  <sipa> Luke-Jr: which is why it's easy, and needs no dependencies on assemblers or instruction sets, or simd extensions, or ...
 777 2016-03-24T19:47:13  * Luke-Jr not sure libctaes makes sense as bitcoin-core as opposed to just bitcoin, but meh
 778 2016-03-24T19:47:38  <petertodd> gmaxwell: yeah, I probably am - I also learned how to do climbing/caving ropework by buying a book on it, some equipment, and going out to find some trees to practice on. Doesn't mean it was a good idea. :) just having a separate repo with a bare .c/.h goes a long way to making our intentions clear and having good precedents, even if pragmatically we need a solution right now.
 779 2016-03-24T19:47:38  <wumpus> jonasschnelli: which reminds me, I think I still need to add you to that project
 780 2016-03-24T19:47:49  <jonasschnelli> bitcoin/ should be consensus critical stuff
 781 2016-03-24T19:47:52  <wumpus> jonasschnelli: we're not actually doing anything with it at the moment
 782 2016-03-24T19:47:59  <wumpus> jonasschnelli: right
 783 2016-03-24T19:48:09  <jonasschnelli> +1 ;-)
 784 2016-03-24T19:48:20  <cfields> topic has strayed a bit :)
 785 2016-03-24T19:48:35  <wumpus> yes
 786 2016-03-24T19:48:55  <sipa> petertodd: ok, agreed
 787 2016-03-24T19:49:09  <petertodd> sipa: thanks!
 788 2016-03-24T19:49:28  <wumpus> #topic convert python RPC tests to python 3
 789 2016-03-24T19:49:29  <gmaxwell> I am disappointed that all of this time is being wasted over faffing about libraries, for code that has very narrow independant interest and likely against the wishes of its author. .. and no one commented on further verification strategy.
 790 2016-03-24T19:49:49  * gmaxwell sees topic is over.
 791 2016-03-24T19:50:00  <MarcoFalke> I think it's cleaner to just switch to py3
 792 2016-03-24T19:50:04  <petertodd> wumpus: concept ack!
 793 2016-03-24T19:50:26  <MarcoFalke> but I'd like to hear any drawbacks first.
 794 2016-03-24T19:50:30  <wumpus> gmaxwell: I agree, I think this will do just as well as a code snippet, but I don't think discussing this further is constructive. The people that want a library so bad can work on it I guess :)
 795 2016-03-24T19:50:41  <Luke-Jr> py3 vs py2, I can think of no drawbacks to exclusively py3
 796 2016-03-24T19:50:44  <wumpus> as you may have read in https://github.com/bitcoin/bitcoin/issues/7717, the next release of ubuntu, 16.04, xenial something will no longer ship python 2
 797 2016-03-24T19:50:47  <btcdrak> MarcoFalke: agreed
 798 2016-03-24T19:51:01  <jonasschnelli> Is  the reason for the py3 switch the deprecating of py2 or the missing of it in some of the distributions?
 799 2016-03-24T19:51:09  <wumpus> I think this makes a nice moment to decide on a switch
 800 2016-03-24T19:51:16  <MarcoFalke> ok, so I will close the "refactor py2 code" pull and just go py3 only
 801 2016-03-24T19:51:22  <wumpus> we're modernizing the build environment anyway by starting to rely on c++11
 802 2016-03-24T19:51:31  <wumpus> MarcoFalke: it's the first step right
 803 2016-03-24T19:51:43  <jonasschnelli> ACK on py3... the RPC tests are not something we need "full portability".
 804 2016-03-24T19:51:56  <wumpus> for the build system I have py2+py3 compatibility ready
 805 2016-03-24T19:51:56  <Luke-Jr> actually, we should retain support py2 at least initially
 806 2016-03-24T19:51:59  *** jcorgan has quit IRC
 807 2016-03-24T19:52:00  <Luke-Jr> since we may need to backport
 808 2016-03-24T19:52:08  <wumpus> for the RPC tests I don't think that is necessary
 809 2016-03-24T19:52:09  <petertodd> I'm not sure there exist any build environments that we need to support that are py2 only
 810 2016-03-24T19:52:23  <wumpus> agree petertodd
 811 2016-03-24T19:52:28  <jonasschnelli> agree with petertodd
 812 2016-03-24T19:52:30  <cfields> agreed
 813 2016-03-24T19:52:32  <sipa> agree, the backport trees could also just switch to py3
 814 2016-03-24T19:52:42  <Luke-Jr> if it's not much effort, moving to py2+py3, and then to py3 before 0.13 would be best
 815 2016-03-24T19:52:52  <wumpus> Luke-Jr: that's a lot more work
 816 2016-03-24T19:52:58  <Luke-Jr> if it's a lot more work, forget it
 817 2016-03-24T19:53:05  <wumpus> I did that for scripts used by the build system itself
 818 2016-03-24T19:53:10  <wumpus> but the test framework is huge
 819 2016-03-24T19:53:21  <wumpus> and it's a lot of work for people to make sure it works for both python versions
 820 2016-03-24T19:53:25  <MarcoFalke> considering that the current rpc test are full of bugs due to missing maintanance and review, doing a py2+3 coverage is impossible
 821 2016-03-24T19:53:25  <wumpus> for every change
 822 2016-03-24T19:53:37  <sipa> i think we should just outright switch to py3
 823 2016-03-24T19:53:39  <jonasschnelli> Yes. No backward comp. for test script please.
 824 2016-03-24T19:53:42  <petertodd> fwiw, I'm somewhat in favor of making my own python-bitcoinlib py3-only, simply because I personally never use it on py2, which makes me worried I'm missing bugs
 825 2016-03-24T19:53:42  <wumpus> exactly, practical concerns kind of rule this out
 826 2016-03-24T19:54:16  <btcdrak> I think we should switch to py3 outright also
 827 2016-03-24T19:54:27  <wumpus> that's clear, then
 828 2016-03-24T19:54:31  <jonasschnelli> And kudos to MarcoFalke for working on this!
 829 2016-03-24T19:54:36  <wumpus> yes!
 830 2016-03-24T19:54:55  <btcdrak> +1
 831 2016-03-24T19:55:32  <MarcoFalke> You can pipe the files through 2to3 and then "only" check the diff.
 832 2016-03-24T19:55:32  <wumpus> ok, that concludes the topic I guess, I'm happy we can finally agree on something :)
 833 2016-03-24T19:55:39  <cfields> yes, it's much easier for us than most because it's all internal and we really have no downstreams to cater to
 834 2016-03-24T19:55:41  <wumpus> more topics?
 835 2016-03-24T19:55:58  <wumpus> 5 minutes to go
 836 2016-03-24T19:56:26  <gmaxwell> Lets end early.
 837 2016-03-24T19:56:29  <MarcoFalke> #action Close https://github.com/bitcoin/bitcoin/pull/7722 and switch to py3
 838 2016-03-24T19:56:33  <wumpus> great
 839 2016-03-24T19:56:39  <wumpus> #endmeeting
 840 2016-03-24T19:56:39  <lightningbot> Meeting ended Thu Mar 24 19:56:39 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
 841 2016-03-24T19:56:39  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.html
 842 2016-03-24T19:56:39  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.txt
 843 2016-03-24T19:56:39  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-03-24-19.00.log.html
 844 2016-03-24T19:57:03  <GitHub184> [bitcoin] btcdrak opened pull request #7741: [0.12] Mark p2p alert system as deprecated (0.12...alert0) https://github.com/bitcoin/bitcoin/pull/7741
 845 2016-03-24T19:57:21  <jonasschnelli> sipa: how would you support test if the ctaeslib is just a c./.h?
 846 2016-03-24T19:57:29  <jonasschnelli> Add a test.c and a Makefile?
 847 2016-03-24T19:57:41  <gmaxwell> with a define.
 848 2016-03-24T19:57:47  <gmaxwell> #ifdef CTAES_TESTS  int main()...
 849 2016-03-24T19:57:58  <sipa> i preferna test.c
 850 2016-03-24T19:58:12  <sipa> as it's perfectly externally testable
 851 2016-03-24T19:58:24  <Luke-Jr> in this case, it may be wise to have a function for tests that software can call at startup
 852 2016-03-24T19:58:35  <gmaxwell> sipa: k.
 853 2016-03-24T19:58:52  <Luke-Jr> ie, refuse to start if the AES code fails its own testws
 854 2016-03-24T19:59:17  <gmaxwell> well a test harness for verification may be very different than built in selftests.
 855 2016-03-24T19:59:24  <Luke-Jr> sure
 856 2016-03-24T19:59:48  <petertodd> test harness for the constant-time functionality is probably going to be a lot less portable than the code itself for instance...
 857 2016-03-24T20:00:44  <gmaxwell> well for example, a verification test harness would likely contain another whole implementation of AES. :)
 858 2016-03-24T20:00:48  <cfields> sipa: just pushed the code-movement to net-refactor8 branch. Still lots of placeholders, todos, and hacks in there, but it now looks overall how i want it to
 859 2016-03-24T20:02:13  *** mrkent has joined #bitcoin-core-dev
 860 2016-03-24T20:03:34  <sipa> gmaxwell: hmm... about tests; the s-box implementation is the most obscure/unreviewable/largest part of the code... but it's trivial to.test exhaustivelg against another implementation, as it only deals with 32-bit inputs
 861 2016-03-24T20:05:09  <sipa> ah, nvm
 862 2016-03-24T20:05:14  <gmaxwell> sipa: right. an exhaustive sbox test, e.g. with forward sbox pulled out of another implemetation. and checking forward and reverse. And then I thought it might be interesting to see if the existing vectors are enough to catch any simple or or two byte mutations in the current implementation. And if not, we could add vectors until they are.
 863 2016-03-24T20:05:29  <gmaxwell> why cant the sbox be exaustively tested?
 864 2016-03-24T20:06:23  <sipa> gmaxwell: not as a black box; its inputs are 8 16-bit integers (and.it does 16 sbox applications in parallel)... you can't exhaustive test 128 bits (i hope)
 865 2016-03-24T20:07:08  <gmaxwell> right. well it could test each lane one at a time.
 866 2016-03-24T20:07:14  <sipa> if you can inspect the code and see that it's "obviously" doing the same thing for every bit position, it suffices to test just the 2^8 inputs
 867 2016-03-24T20:08:07  <sipa> petertodd: how is the bip9 warning code broken?
 868 2016-03-24T20:08:13  <sipa> (re: github pr)
 869 2016-03-24T20:08:43  <petertodd> sipa: oh, I spoke to soon on that - but it was more confusing than it should have been, which I'll fix next week when I get back after the long weekend
 870 2016-03-24T20:08:48  <sipa> petertodd: the warning logic uses bit positions; the trigger code uses DeplymentPos
 871 2016-03-24T20:08:58  <sipa> (i agree DeploymentId is better)
 872 2016-03-24T20:09:24  <petertodd> sipa: yeah, simple enough pull-req, and I've got a few more improvements re: adding comments that I'll add to it too
 873 2016-03-24T20:09:33  <sipa> great!
 874 2016-03-24T20:10:13  <petertodd> oh, one thing I forgot to ask: if I understand it correctly, right now changes to the "meta-version" - the top three bits - do *not* trigger a "network may have upgraded warning"
 875 2016-03-24T20:10:36  <sipa> petertodd: they do if it's permanent
 876 2016-03-24T20:10:53  <petertodd> sipa: ah, alright, I'll go look at that more then and verify
 877 2016-03-24T20:10:57  *** gevs has joined #bitcoin-core-dev
 878 2016-03-24T20:10:58  *** gevs has joined #bitcoin-core-dev
 879 2016-03-24T20:11:05  <sipa> there is the 50% of last blocks habe unexpected version warning still
 880 2016-03-24T20:11:14  <petertodd> sipa: ah, ok, good
 881 2016-03-24T20:11:31  <sipa> in addition, there is warning logic for each bip9 bit position, which works retroactively
 882 2016-03-24T20:11:31  *** hybridsole has quit IRC
 883 2016-03-24T20:11:57  <petertodd> sipa: anyway, probably best I don't ask too many more questions to keep my mind uncontaminated with what you guys think the code should be doing :)
 884 2016-03-24T20:12:23  <sipa> petertodd: good strategy
 885 2016-03-24T20:12:50  <petertodd> bbl - have a good long weekend!
 886 2016-03-24T20:13:57  <cfields> wumpus: https://github.com/travis-ci/travis-build/pull/680
 887 2016-03-24T20:14:03  <cfields> one step closer, i hope
 888 2016-03-24T20:16:33  <wumpus> cfields: wow! finally :)
 889 2016-03-24T20:16:47  *** hybridsole has joined #bitcoin-core-dev
 890 2016-03-24T20:17:26  <cfields> wumpus: i have some work on docker that may not have been in vain, though. It looks like a very possible gitian replacement
 891 2016-03-24T20:18:31  <warren> cfields: for your own deterministic toolchain?
 892 2016-03-24T20:18:35  <wumpus> cfields: interesting
 893 2016-03-24T20:19:23  <cfields> (after all, Gitian basically is an early/simplified version of what Docker has come to be)
 894 2016-03-24T20:20:03  <wumpus> in a way, yes. At least docker is very actively supported/maintained.
 895 2016-03-24T20:20:33  <warren> docker still puts an OS into a chroot
 896 2016-03-24T20:20:42  <cfields> warren: kinda. it just builds from a pre-determined sandbox and caches each step. as a bonus, each step can be shared as well
 897 2016-03-24T20:21:20  <warren> IMHO it's worthwhile to switch away from gitian if we are at the stage of being fully in control of our own deterministic toolchain
 898 2016-03-24T20:21:32  <cfields> warren: well you'll always need a reference environment
 899 2016-03-24T20:21:44  *** d_t has joined #bitcoin-core-dev
 900 2016-03-24T20:22:05  <warren> cfields: the reference environment could be deterministic as well
 901 2016-03-24T20:22:09  <wumpus> that's a long way, you don't just need a reference toolchain, but there's also other tools that have an effect on the result
 902 2016-03-24T20:22:24  <cfields> in this case, docker basically acts as that reference environment, containing only our reference toolchain and the bare minimum other tools
 903 2016-03-24T20:22:41  <cfields> seems to me, docker is the perfect tool for us in this case
 904 2016-03-24T20:22:48  <wumpus> warren: we need a deterministically buildable OS, isn't debian working on that :)
 905 2016-03-24T20:23:03  <cfields> warren: sure. and you have to run it somewhere :)
 906 2016-03-24T20:23:14  <warren> ANY OS > build stage A that probably isn't deterministic -> use that to build stage B
 907 2016-03-24T20:24:31  <cfields> warren: regardless of the determinism of the base, you have to have some tool for actually building the thing. Short of handing out a rootfs.tar.gz and praying that it will run on some baremetal, ofc
 908 2016-03-24T20:25:01  <wumpus> anyhow this is driftng off topic
 909 2016-03-24T20:25:10  <wumpus> main thing I'd like is to have c++11 in travis
 910 2016-03-24T20:25:18  <cfields> warren: btw, I believe I've seen some early deterministic Debian base images on dockerhub
 911 2016-03-24T20:25:54  <wumpus> cfields: so deterministic debian doesn't only exist in theory anymore? that's good to hear there's progress
 912 2016-03-24T20:25:56  <cfields> wumpus: yes, i'm committed to making that happen one way or another. The net code depends on it, so I'm kinda forced to :)
 913 2016-03-24T20:26:19  <wumpus> cfields: it sounds like a project akin to building the pyramids
 914 2016-03-24T20:27:01  <cfields> hmm, trying to re-find what i saw
 915 2016-03-24T20:27:02  <wumpus> even getting the bitcoin dependencies deterministic was a hell of a lot of work, just imagine for a full OS
 916 2016-03-24T20:27:21  <wumpus> although it gets easier, with better tooling, some things only had to be done onece
 917 2016-03-24T20:28:55  *** frankenmint has joined #bitcoin-core-dev
 918 2016-03-24T20:29:06  <cfields> wumpus: hmm, i might've made up the Debian part. I saw someone working on a deterministic base for Travis, my brain might've just lumped 'em together.
 919 2016-03-24T20:29:16  <cfields> s/Travis/Docker/
 920 2016-03-24T20:30:10  <wumpus> right - any deterministic base wouldb e fine, it doesn't need to be debian
 921 2016-03-24T20:32:56  <wumpus> docker is based on lxc isn't it? that means it at least doesn't need deterministic kernel, init etc
 922 2016-03-24T20:33:33  <cfields> based on cgroups, I don't believe it uses lxc itself
 923 2016-03-24T20:34:02  *** frankenmint has quit IRC
 924 2016-03-24T20:39:12  *** t800 has quit IRC
 925 2016-03-24T20:41:49  <wumpus> right, the same underlying mechanism
 926 2016-03-24T20:43:11  <kanzure> in addition to deterministic debian please also consider deterministic nixos things
 927 2016-03-24T20:43:58  <kanzure> fwiw i recommend not using docker's build tool. it's a mess.
 928 2016-03-24T20:44:28  <kanzure> using docker for running containers or something is probably fine (although coreos/rkt things should be considered, since they seem to be compatible with docker containers and have a specification and words written)
 929 2016-03-24T20:45:05  <kanzure> "docker build" is a build tool crammed into a container management system. a more realistic build tool is something like bazel ( http://bazel.io/ ), which can spit out finished container images.
 930 2016-03-24T20:45:25  *** t800 has joined #bitcoin-core-dev
 931 2016-03-24T20:45:54  <GitHub178> [bitcoin] jonasschnelli opened pull request #7742: [Wallet][RPC] add missing abandon status documentation (master...2016/03/ab_doc) https://github.com/bitcoin/bitcoin/pull/7742
 932 2016-03-24T20:46:25  *** musalbas has quit IRC
 933 2016-03-24T20:46:40  <kanzure> (with "docker build" you lose out on things like build caching unless you always build only on one machine. this negatively impacts things like distributed continuous integration build runners.)
 934 2016-03-24T20:46:41  <cfields> kanzure: docker would only be used as a way to launch our own build process inside of a pre-determined environment
 935 2016-03-24T20:46:49  *** lysobit has quit IRC
 936 2016-03-24T20:47:16  *** Thireus has quit IRC
 937 2016-03-24T20:47:30  *** treehug88 has quit IRC
 938 2016-03-24T20:48:31  *** t800 has joined #bitcoin-core-dev
 939 2016-03-24T20:48:38  *** lysobit has joined #bitcoin-core-dev
 940 2016-03-24T20:48:56  <warren> kanzure: coreos supports docker, not the other way around
 941 2016-03-24T20:49:10  *** t800 has quit IRC
 942 2016-03-24T20:49:12  <kanzure> i don't think i said otherwise.
 943 2016-03-24T20:49:49  *** musalbas has joined #bitcoin-core-dev
 944 2016-03-24T20:50:39  <kanzure> cfields: right but even the pre-determined environment has a build toolchain. and the default way when using docker is to use "docker build". which is what my above rant is about.
 945 2016-03-24T20:51:27  <kanzure> oh i guess that is inconsistent with me mentioning rkt. my mention of rkt is unrelated to my complaints about "docker build".
 946 2016-03-24T20:51:47  <kanzure> (rkt doesn't do container building)
 947 2016-03-24T20:53:12  <cfields> kanzure: I'll read up on those things
 948 2016-03-24T20:53:33  *** lysobit has quit IRC
 949 2016-03-24T20:53:51  <kanzure> ok feel free to pester me. i have been reading docker source code and such.
 950 2016-03-24T20:54:01  *** musalbas has quit IRC
 951 2016-03-24T20:54:07  <cfields> great, thanks :)
 952 2016-03-24T20:54:20  *** musalbas has joined #bitcoin-core-dev
 953 2016-03-24T20:55:42  *** Thireus has joined #bitcoin-core-dev
 954 2016-03-24T20:56:26  *** laurentmt has joined #bitcoin-core-dev
 955 2016-03-24T20:56:31  *** laurentmt has quit IRC
 956 2016-03-24T20:56:50  *** lysobit has joined #bitcoin-core-dev
 957 2016-03-24T20:57:28  *** droark has quit IRC
 958 2016-03-24T21:07:37  *** Guest27862 has quit IRC
 959 2016-03-24T21:08:46  *** ChillazZ has joined #bitcoin-core-dev
 960 2016-03-24T21:13:28  *** droark has joined #bitcoin-core-dev
 961 2016-03-24T21:19:50  <GitHub139> [bitcoin] luke-jr closed pull request #7047: [WIP] Backports for 0.11.3 (updated to 93ca5a3) (0.11...backports-for-0.11.3) https://github.com/bitcoin/bitcoin/pull/7047
 962 2016-03-24T21:22:16  *** fengling has quit IRC
 963 2016-03-24T21:27:23  <GitHub149> [bitcoin] luke-jr opened pull request #7743: [0.11] Important backports for 0.11.3 (updated to v0.12.0) (0.11...backports-for-0.11.3) https://github.com/bitcoin/bitcoin/pull/7743
 964 2016-03-24T21:29:44  *** frankenmint has joined #bitcoin-core-dev
 965 2016-03-24T21:31:32  *** Hybrids0le has joined #bitcoin-core-dev
 966 2016-03-24T21:31:32  *** hybridsole has quit IRC
 967 2016-03-24T21:32:45  *** CodeShark has quit IRC
 968 2016-03-24T21:33:02  *** NicolasDorier has quit IRC
 969 2016-03-24T21:33:03  *** CodeShark has joined #bitcoin-core-dev
 970 2016-03-24T21:33:56  *** NicolasDorier has joined #bitcoin-core-dev
 971 2016-03-24T21:33:57  *** Hybrids0le has quit IRC
 972 2016-03-24T21:34:01  *** droark has quit IRC
 973 2016-03-24T21:34:25  *** lysobit has quit IRC
 974 2016-03-24T21:34:26  *** musalbas has quit IRC
 975 2016-03-24T21:34:49  *** lesderid has quit IRC
 976 2016-03-24T21:36:17  *** frankenmint has quit IRC
 977 2016-03-24T21:41:00  *** musalbas has joined #bitcoin-core-dev
 978 2016-03-24T21:41:24  *** lysobit has joined #bitcoin-core-dev
 979 2016-03-24T21:41:34  *** lesderid has joined #bitcoin-core-dev
 980 2016-03-24T21:44:43  *** d_t has quit IRC
 981 2016-03-24T21:44:44  *** ebfull has quit IRC
 982 2016-03-24T21:44:44  *** windsok has quit IRC
 983 2016-03-24T21:50:10  *** fengling has joined #bitcoin-core-dev
 984 2016-03-24T22:00:40  *** d_t has joined #bitcoin-core-dev
 985 2016-03-24T22:00:40  *** ebfull has joined #bitcoin-core-dev
 986 2016-03-24T22:00:40  *** windsok has joined #bitcoin-core-dev
 987 2016-03-24T22:03:42  *** ibrightly has quit IRC
 988 2016-03-24T22:05:05  *** ibrightly has joined #bitcoin-core-dev
 989 2016-03-24T22:07:46  *** justanotheruser has quit IRC
 990 2016-03-24T22:18:01  *** supasonic has quit IRC
 991 2016-03-24T22:19:01  *** supasonic has joined #bitcoin-core-dev
 992 2016-03-24T22:32:46  *** frankenmint has joined #bitcoin-core-dev
 993 2016-03-24T22:33:16  *** fengling has quit IRC
 994 2016-03-24T22:33:32  *** Guyver2 has quit IRC
 995 2016-03-24T22:38:19  *** frankenmint has quit IRC
 996 2016-03-24T22:43:01  *** Hybrids0le has joined #bitcoin-core-dev
 997 2016-03-24T22:44:57  *** Hybrids0le has quit IRC
 998 2016-03-24T22:55:39  *** shesek has joined #bitcoin-core-dev
 999 2016-03-24T22:59:44  *** wallet42 has joined #bitcoin-core-dev
1000 2016-03-24T23:02:29  *** mrkent_ has joined #bitcoin-core-dev
1001 2016-03-24T23:04:35  *** mrkent has quit IRC
1002 2016-03-24T23:06:30  *** hybridsole has joined #bitcoin-core-dev
1003 2016-03-24T23:10:06  *** fengling has joined #bitcoin-core-dev
1004 2016-03-24T23:20:29  *** justanotheruser has joined #bitcoin-core-dev
1005 2016-03-24T23:23:26  *** musalbas has quit IRC
1006 2016-03-24T23:23:35  *** musalbas- has joined #bitcoin-core-dev
1007 2016-03-24T23:23:46  *** musalbas- is now known as musalbas
1008 2016-03-24T23:27:49  *** PRab has quit IRC
1009 2016-03-24T23:50:10  *** justanotheruser has quit IRC
1010 2016-03-24T23:52:19  *** droark has joined #bitcoin-core-dev