1 2018-08-12T00:01:53  *** BashCo has quit IRC
  2 2018-08-12T00:04:24  *** BashCo has joined #bitcoin-core-dev
  3 2018-08-12T01:01:53  *** lnostdal has joined #bitcoin-core-dev
  4 2018-08-12T01:12:18  *** promag has quit IRC
  5 2018-08-12T01:32:47  *** AaronvanW has quit IRC
  6 2018-08-12T01:53:40  *** esotericnonsense has quit IRC
  7 2018-08-12T02:19:53  *** BashCo has quit IRC
  8 2018-08-12T02:22:27  *** BashCo has joined #bitcoin-core-dev
  9 2018-08-12T02:30:36  *** drexl has quit IRC
 10 2018-08-12T02:43:29  *** bytting has joined #bitcoin-core-dev
 11 2018-08-12T02:51:37  *** unholymachine has quit IRC
 12 2018-08-12T03:17:34  *** esotericnonsense has joined #bitcoin-core-dev
 13 2018-08-12T03:34:26  *** justanotheruser has joined #bitcoin-core-dev
 14 2018-08-12T04:37:25  *** Krellan has joined #bitcoin-core-dev
 15 2018-08-12T04:43:37  *** profmac has joined #bitcoin-core-dev
 16 2018-08-12T04:44:02  *** d9b4bef9 has quit IRC
 17 2018-08-12T04:44:55  *** AaronvanW has joined #bitcoin-core-dev
 18 2018-08-12T04:45:08  *** d9b4bef9 has joined #bitcoin-core-dev
 19 2018-08-12T04:49:14  *** AaronvanW has quit IRC
 20 2018-08-12T04:51:48  *** murrayn has quit IRC
 21 2018-08-12T04:59:34  *** vicenteH has quit IRC
 22 2018-08-12T05:04:27  *** vicenteH has joined #bitcoin-core-dev
 23 2018-08-12T05:39:49  *** no_input_found has quit IRC
 24 2018-08-12T05:45:37  *** ken2812221 has joined #bitcoin-core-dev
 25 2018-08-12T05:51:01  *** TheHoliestRoger has quit IRC
 26 2018-08-12T05:54:30  *** TheHoliestRoger has joined #bitcoin-core-dev
 27 2018-08-12T05:59:27  *** TheHoliestRoger has quit IRC
 28 2018-08-12T06:01:53  *** intcat has quit IRC
 29 2018-08-12T06:03:21  *** TheHoliestRoger has joined #bitcoin-core-dev
 30 2018-08-12T06:04:45  *** intcat has joined #bitcoin-core-dev
 31 2018-08-12T06:05:54  <jonasschnelli> gmaxwell: yes. rekeying is done after a fix amount of traffic in bytes. But re-hashing the secret would not change anything if ECDA of logged traffic can be broken?
 32 2018-08-12T06:08:22  *** BashCo has quit IRC
 33 2018-08-12T06:10:00  *** BashCo has joined #bitcoin-core-dev
 34 2018-08-12T06:33:42  *** murrayn has joined #bitcoin-core-dev
 35 2018-08-12T06:33:55  <gmaxwell> jonasschnelli: right, we should also perhaps consider rekeying once an hour. What rekeying accomplishes, assuming the old key is deleted, is that if a system is compromised you can't extract the keys from memory to decrypt traffic you logged before compromise.
 36 2018-08-12T06:38:54  <gmaxwell> The reason I suggest triggering on time is because if you have e.g. a SPV client, it might be days until it has transfered 1GB of traffic. which might make it interesting to try to go seize other nodes a target under observation was connected to in order to decrypt their traffic. Admittedly a really fring risk, but it should be ~free to avoid.
 37 2018-08-12T06:39:14  <gmaxwell> (basically a similar motivation to why we don't log IPs by default)
 38 2018-08-12T06:40:55  <sipa> the only reason not to rekey on every message is performance, right?
 39 2018-08-12T06:41:15  <gmaxwell> Right.
 40 2018-08-12T06:41:25  <gmaxwell> sha2 is slower than chacha.. :)
 41 2018-08-12T06:42:15  <gmaxwell> interestingly, I'm not aware of any well known cipher mode which natively has irreversable state.
 42 2018-08-12T06:42:53  <sipa> chacha takes a 256 bits key, and produces blobs of 512 bits of output
 43 2018-08-12T06:43:30  <sipa> why not say encrypt every message with the current encryption key, and then afterwards extract another 256 bits from the stream, which become the new encryption key?
 44 2018-08-12T06:43:35  <sipa> chacha has 0 initialization cost
 45 2018-08-12T06:44:20  <gmaxwell> Because thats not a well studied construct. it would also be 50% of the speed of using it normally.
 46 2018-08-12T06:44:57  *** BashCo has quit IRC
 47 2018-08-12T06:45:43  *** AaronvanW has joined #bitcoin-core-dev
 48 2018-08-12T06:46:15  <sipa> why would it be slower?
 49 2018-08-12T06:46:53  <sipa> ah, if you assume the messages are small i guess
 50 2018-08-12T06:47:14  *** BashCo has joined #bitcoin-core-dev
 51 2018-08-12T06:47:42  <gmaxwell> ah I thought you meant per 512 bits of output rather than per protocol message.
 52 2018-08-12T06:48:24  *** Victorsueca has quit IRC
 53 2018-08-12T06:48:51  <gmaxwell> we could also email djb to ask, might well be that someone has published on a mode that does this. Though I think elsewhere where this concern was addressed, it was always just addressed by rekeying from a higher level rather than at the block cipher level.
 54 2018-08-12T06:49:34  *** Victorsueca has joined #bitcoin-core-dev
 55 2018-08-12T06:50:05  <gmaxwell> Though as I was saying, I think it's kind of a fringe concern, if we want to do something complicated, I'd rather it be armoring against ECDH break then N-th level optimizations to how fast we forget keying material.
 56 2018-08-12T06:50:57  *** AaronvanW has quit IRC
 57 2018-08-12T06:51:47  <gmaxwell> (or even better, get the indistinguishable authentication protocol finished)
 58 2018-08-12T06:55:01  <sipa> right; i'm mostly wondering why "use a prng-based stream cipher, and after each message, read the next encrpytion key from the stream" isn't a common construction
 59 2018-08-12T06:55:58  <gmaxwell> Because almost everything has key init costs?
 60 2018-08-12T06:56:13  <gmaxwell> also because the whole reason you normally use a stream cipher is random access.
 61 2018-08-12T06:57:33  <gmaxwell> there have been some 'reuse resistant' quasi-stream-cipher proposals perhaps some of those get irreversability as a side effect. dunno.
 62 2018-08-12T07:03:07  *** dcousens has quit IRC
 63 2018-08-12T07:03:36  *** dcousens has joined #bitcoin-core-dev
 64 2018-08-12T07:11:01  *** bytting has quit IRC
 65 2018-08-12T07:55:47  *** itaseski has joined #bitcoin-core-dev
 66 2018-08-12T08:00:46  *** csknk has joined #bitcoin-core-dev
 67 2018-08-12T08:00:46  *** BashCo has quit IRC
 68 2018-08-12T08:02:25  *** BashCo has joined #bitcoin-core-dev
 69 2018-08-12T08:22:14  *** Rootsudo has joined #bitcoin-core-dev
 70 2018-08-12T08:29:32  *** BashCo has quit IRC
 71 2018-08-12T08:31:28  *** BashCo has joined #bitcoin-core-dev
 72 2018-08-12T08:46:01  *** d9b4bef9 has quit IRC
 73 2018-08-12T08:47:08  *** d9b4bef9 has joined #bitcoin-core-dev
 74 2018-08-12T08:54:35  *** BashCo has quit IRC
 75 2018-08-12T09:20:41  *** SopaXorzTaker has joined #bitcoin-core-dev
 76 2018-08-12T09:46:57  *** fanquake has joined #bitcoin-core-dev
 77 2018-08-12T09:51:34  <fanquake> wumpus 13938 should be ok to go in
 78 2018-08-12T09:57:59  <fanquake> Also 13808
 79 2018-08-12T09:59:15  <Varunram> is the bot dead?
 80 2018-08-12T10:10:57  *** hex17or has joined #bitcoin-core-dev
 81 2018-08-12T10:14:18  *** AaronvanW has joined #bitcoin-core-dev
 82 2018-08-12T10:17:26  *** hex17or has quit IRC
 83 2018-08-12T10:58:07  <jonasschnelli> gmaxwell, sipa: the new protocol does have encryption optional, therefore the question rises if detecting the key handshake versus a version message is sane
 84 2018-08-12T10:58:55  <jonasschnelli> I guess its acceptable to assume a version message (an not a key) when we detect a message magic and the rest of a legacy header
 85 2018-08-12T10:59:33  <jonasschnelli> I guess it's almost impossible to derive a pubkey with the network magic & version part of the header
 86 2018-08-12T10:59:33  *** Krellan has quit IRC
 87 2018-08-12T11:00:29  *** Krellan has joined #bitcoin-core-dev
 88 2018-08-12T11:02:01  *** AaronvanW has quit IRC
 89 2018-08-12T11:02:37  *** AaronvanW has joined #bitcoin-core-dev
 90 2018-08-12T11:03:39  *** AaronvanW has quit IRC
 91 2018-08-12T11:03:58  *** AaronvanW has joined #bitcoin-core-dev
 92 2018-08-12T11:21:44  *** nodweber has quit IRC
 93 2018-08-12T11:22:14  *** nodweber has joined #bitcoin-core-dev
 94 2018-08-12T11:30:27  *** nodweber has quit IRC
 95 2018-08-12T11:30:41  *** nodweber has joined #bitcoin-core-dev
 96 2018-08-12T11:37:47  *** Victorsueca has quit IRC
 97 2018-08-12T11:39:09  *** Victorsueca has joined #bitcoin-core-dev
 98 2018-08-12T12:06:25  *** esotericnonsense has quit IRC
 99 2018-08-12T12:06:25  *** vicenteH has quit IRC
100 2018-08-12T12:07:08  *** esotericnonsense has joined #bitcoin-core-dev
101 2018-08-12T12:29:57  *** goatpig has joined #bitcoin-core-dev
102 2018-08-12T12:32:59  *** lnostdal has quit IRC
103 2018-08-12T12:43:29  *** pluit has joined #bitcoin-core-dev
104 2018-08-12T12:48:02  *** d9b4bef9 has quit IRC
105 2018-08-12T12:49:08  *** d9b4bef9 has joined #bitcoin-core-dev
106 2018-08-12T13:06:48  *** Krellan has quit IRC
107 2018-08-12T13:08:01  *** Krellan has joined #bitcoin-core-dev
108 2018-08-12T13:11:16  *** reald0ff1 has joined #bitcoin-core-dev
109 2018-08-12T13:11:26  <reald0ff1> hi
110 2018-08-12T13:12:56  <reald0ff1> Can someone please provide me download stats (or at least share in %) of bitcoin core, regarding the different platform versions (Win, Linux, OSX, etc) ?
111 2018-08-12T13:13:17  <reald0ff1> that would be very helpful for my master thesis
112 2018-08-12T13:15:41  *** face has joined #bitcoin-core-dev
113 2018-08-12T13:17:12  *** AaronvanW has quit IRC
114 2018-08-12T13:20:08  <reald0ff1> would very appreciate it, if someone could help me with that question
115 2018-08-12T13:20:47  <harding> reald0ff1: I don't know if anyone has that information for BitcoinCore.org, sorry.  In addition, the binaries can also be downloaded from Bitcoin.org (maintained by a different team) or via a torrent (with optional magnet URI) that contains the binaries for all platforms.
116 2018-08-12T13:25:56  *** lnostdal has joined #bitcoin-core-dev
117 2018-08-12T13:26:31  *** SopaXorzTaker has quit IRC
118 2018-08-12T13:30:11  *** lnostdal has joined #bitcoin-core-dev
119 2018-08-12T13:33:08  *** pluit has quit IRC
120 2018-08-12T13:34:33  <reald0ff1> well thanks for the answer. I think I will try to contact bitcoincore.org via email and bitcoin.org also via email to the website maintainer. Maybe some of them could provide me stats
121 2018-08-12T13:35:58  <reald0ff1> I developing a security tool for cryptocurrency users and I selected windows as target plattform. I have the feeling that the most users use windows (I am not talking about devs, etc.)
122 2018-08-12T13:36:38  <reald0ff1> however, would be still nice to have some stats to prove that "feeling"
123 2018-08-12T13:38:32  *** pluit has joined #bitcoin-core-dev
124 2018-08-12T13:40:16  *** lnostdal has quit IRC
125 2018-08-12T13:43:08  *** lnostdal has joined #bitcoin-core-dev
126 2018-08-12T13:50:13  *** reald0ff1 has quit IRC
127 2018-08-12T13:57:22  *** fanquake has quit IRC
128 2018-08-12T13:59:43  *** Rootsudo has quit IRC
129 2018-08-12T14:00:04  *** Rootsudo has joined #bitcoin-core-dev
130 2018-08-12T14:00:29  *** Rootsudo has quit IRC
131 2018-08-12T14:01:03  *** Rootsudo has joined #bitcoin-core-dev
132 2018-08-12T14:01:27  *** Rootsudo has quit IRC
133 2018-08-12T14:01:50  *** Rootsudo has joined #bitcoin-core-dev
134 2018-08-12T14:02:13  *** Rootsudo has quit IRC
135 2018-08-12T14:02:38  *** devmob has joined #bitcoin-core-dev
136 2018-08-12T14:02:45  *** Rootsudo has joined #bitcoin-core-dev
137 2018-08-12T14:03:08  <devmob> hi, I'd really like to know how bitcoin does gossip, like how the gossip protocol is implemented
138 2018-08-12T14:03:15  <devmob> can someone point me somewhere ?
139 2018-08-12T14:03:28  *** Rootsudo has joined #bitcoin-core-dev
140 2018-08-12T14:03:47  *** Rootsudo has quit IRC
141 2018-08-12T14:39:15  *** csknk has quit IRC
142 2018-08-12T14:45:23  *** devmob has quit IRC
143 2018-08-12T14:53:23  *** promag has joined #bitcoin-core-dev
144 2018-08-12T15:08:45  *** plankers has joined #bitcoin-core-dev
145 2018-08-12T15:18:40  *** SopaXorzTaker has joined #bitcoin-core-dev
146 2018-08-12T15:25:02  *** promag has quit IRC
147 2018-08-12T15:47:11  <itaseski> devmob: https://en.wikipedia.org/wiki/Gossip_protocol
148 2018-08-12T15:47:15  <itaseski> is this helpful?
149 2018-08-12T15:48:00  <itaseski> it is general but it explains how gossip protocols work
150 2018-08-12T15:48:45  <itaseski> i wasn't able to find anything bitcoin specific ...
151 2018-08-12T15:59:14  <sipa> he left. also, https://bitcoin.stackexchange.com is your friend
152 2018-08-12T16:31:22  *** andytoshi has joined #bitcoin-core-dev
153 2018-08-12T16:38:42  *** SopaXorzTaker has quit IRC
154 2018-08-12T16:40:38  *** jb55 has joined #bitcoin-core-dev
155 2018-08-12T16:50:02  *** d9b4bef9 has quit IRC
156 2018-08-12T16:50:55  *** kelt has quit IRC
157 2018-08-12T16:51:08  *** d9b4bef9 has joined #bitcoin-core-dev
158 2018-08-12T17:00:05  *** ghost43 has quit IRC
159 2018-08-12T17:00:19  *** ghost43 has joined #bitcoin-core-dev
160 2018-08-12T17:02:51  *** vexbuy has joined #bitcoin-core-dev
161 2018-08-12T17:05:06  *** vexbuy has quit IRC
162 2018-08-12T17:06:01  *** vexbuy has joined #bitcoin-core-dev
163 2018-08-12T17:19:40  *** Giszmo has joined #bitcoin-core-dev
164 2018-08-12T17:19:59  *** Giszmo has joined #bitcoin-core-dev
165 2018-08-12T17:29:22  <gmaxwell> jonasschnelli: we could just discard keys that match the strictest version handshake pattern we can come up with, no biggie
166 2018-08-12T17:30:28  <gmaxwell> sipa: on the encryption subject, libsodium has avx2 and ssse3 chacha20 implementations: https://github.com/jedisct1/libsodium/tree/1.0.16/src/libsodium/crypto_stream/chacha20/dolbeau
167 2018-08-12T17:31:33  <gmaxwell> and sse2 poly1305 https://github.com/jedisct1/libsodium/tree/1.0.16/src/libsodium/crypto_onetimeauth/poly1305/sse2
168 2018-08-12T17:43:49  *** itaseski has quit IRC
169 2018-08-12T17:45:32  *** vexbuy has quit IRC
170 2018-08-12T17:46:29  *** vexbuy has joined #bitcoin-core-dev
171 2018-08-12T17:56:37  *** SopaXorzTaker has joined #bitcoin-core-dev
172 2018-08-12T17:58:42  *** vexbuy has quit IRC
173 2018-08-12T17:59:03  *** vexbuy has joined #bitcoin-core-dev
174 2018-08-12T18:07:34  *** luke-jr has quit IRC
175 2018-08-12T18:12:57  *** Emcy_ has quit IRC
176 2018-08-12T18:14:13  *** Emcy_ has joined #bitcoin-core-dev
177 2018-08-12T18:16:56  *** tryphe has quit IRC
178 2018-08-12T18:17:41  *** tryphe has joined #bitcoin-core-dev
179 2018-08-12T18:20:58  *** luke-jr has joined #bitcoin-core-dev
180 2018-08-12T18:28:50  *** Victorsueca has quit IRC
181 2018-08-12T18:30:08  *** Victorsueca has joined #bitcoin-core-dev
182 2018-08-12T18:35:37  <jonasschnelli> gmaxwell: about dolbeau's Chacha20 AVX/SSSE implementation: "Beware: those implementations are purely designed for speed on recent Intel architectures (mostly Haswell and newer), and ARMv8 (64 bits) with the crypto extension. They were not verified to be resistant to side channel attacks."
183 2018-08-12T18:36:29  <jonasschnelli> The later would probably require further analysis since the timing side channel attackes seems to be one of the big benefits of chacha20 (I may be wrong though)
184 2018-08-12T18:36:42  <jonasschnelli> *attack resistance
185 2018-08-12T18:40:12  *** csknk has joined #bitcoin-core-dev
186 2018-08-12T18:58:23  *** csknk has quit IRC
187 2018-08-12T19:01:14  *** luke-jr has quit IRC
188 2018-08-12T19:01:49  *** luke-jr has joined #bitcoin-core-dev
189 2018-08-12T19:19:25  *** bytting has joined #bitcoin-core-dev
190 2018-08-12T19:20:01  <MarcoFalke> sipa: Wouldn't the stempool make every mempool action half as fast (since everything would have to be done once for the mempool and then for the stempool)
191 2018-08-12T19:20:29  <MarcoFalke> Also I am not sure about the memory overhead of having the mempool duplicated
192 2018-08-12T19:20:44  <MarcoFalke> The transactions are shared ptrs, but still...
193 2018-08-12T19:20:49  <sipa> MarcoFalke: well, dandelion needs some way of dealing with unconfirmed dependencies
194 2018-08-12T19:22:52  <sipa> MarcoFalke: the reference code the authors posted included a stempool, though i commented on memory usage concerns
195 2018-08-12T19:23:08  <MarcoFalke> I know that the BIP mentions a stempool
196 2018-08-12T19:23:27  <MarcoFalke> Agree that we need to handle dependencies
197 2018-08-12T19:24:35  <MarcoFalke> I was hoping we could do a primitive cache for now and later replace that with https://github.com/bitcoin/bitcoin/pull/13804 (tx pool layer)
198 2018-08-12T19:24:39  <sipa> an alternative would be to have a 2-tier mempool, where each transaction has a flag whether it's public or not
199 2018-08-12T19:25:16  <sipa> and accepting a public tx ignores (and kicks out) any nonpublic conflicts
200 2018-08-12T19:26:48  <MarcoFalke> That sounds like every single line of txmempool.cpp had to be amended with an if(public) else ...
201 2018-08-12T19:27:03  <sipa> i doubt that, tbh
202 2018-08-12T19:27:20  <sipa> most of it is just data structurr maintenance which would be unaffected
203 2018-08-12T19:27:36  <sipa> but i don't think it'd be a trivial change either
204 2018-08-12T19:28:37  <sipa> the set of non-public transactions in general should be very small
205 2018-08-12T19:29:07  <sipa> as you expect every non-public tx to become a public one after some time (and the auto fluff after timeout essentially guarantees that after some timeout)
206 2018-08-12T19:30:05  <sipa> so perhaps the "stempool excluding mempool" can be small and have lower consistency requirements
207 2018-08-12T19:30:09  *** harrymm has quit IRC
208 2018-08-12T19:30:31  <sipa> like, we run ATMP to accept things into it, but don't require it is at all times consistent with the actual mempool
209 2018-08-12T19:30:55  <sipa> as things expire quickly from the extra set, it can have a tight memory limit and not much avenue for dos
210 2018-08-12T19:31:27  <MarcoFalke> sipa: The dos protection should happen per edge (peer) and not on the global mempool, no?
211 2018-08-12T19:32:04  <MarcoFalke> The stempool limit would only be a fallback limit
212 2018-08-12T19:32:21  <MarcoFalke> We wouldn't want one peer use up all the stempool capacity
213 2018-08-12T19:32:21  *** promag has joined #bitcoin-core-dev
214 2018-08-12T19:32:43  <sipa> right
215 2018-08-12T19:33:20  *** SopaXorzTaker has quit IRC
216 2018-08-12T19:33:32  <MarcoFalke> Also, I am certain that we leak information by using the global (shared among all peers) stempool
217 2018-08-12T19:33:33  <sipa> so perhaps it can even be a per-peer small set of unconfirmed dandelion txn, which you use to do dependency checks for dandelion txn coming from that peer
218 2018-08-12T19:34:08  <sipa> which has much clearer privacy and dos reasonong
219 2018-08-12T19:34:19  <MarcoFalke> You'd forward but then later discard dandelion txs
220 2018-08-12T19:34:46  <sipa> well the combined set of those extra txn is your set of to-fluff things
221 2018-08-12T19:35:09  <MarcoFalke> So if an attacker send the same dandelion tx twice with a rbf one on another route they can guess part of the route
222 2018-08-12T19:35:29  <sipa> how so?
223 2018-08-12T19:35:51  <MarcoFalke> (talking about the shared mempool) Not the per-peer set of txs
224 2018-08-12T19:36:05  <sipa> i like the per-peer set :)
225 2018-08-12T19:36:19  <sipa> i think you're right that there is risk in a global stempool
226 2018-08-12T19:36:53  <sipa> the per-peer set sounds like it wouldn't need much more than a way to pass in an entra map with txn to ATMP
227 2018-08-12T19:37:45  <MarcoFalke> That would have a compute overhead
228 2018-08-12T19:37:53  <sipa> hardly, i think
229 2018-08-12T19:38:11  <MarcoFalke> (re-calculating the set of dependencies for all txs)
230 2018-08-12T19:38:18  <MarcoFalke> just to check on tx
231 2018-08-12T19:38:20  <sipa> no no
232 2018-08-12T19:38:26  <MarcoFalke> why?
233 2018-08-12T19:39:24  <sipa> just something that feeds into the lookup of utxos being spent logic
234 2018-08-12T19:39:36  <sipa> "if not found in mempool or chainstate, also look here'
235 2018-08-12T19:40:10  <sipa> but you don't do complete conflict analysis or replacement or whatever in those extra sets
236 2018-08-12T19:40:44  <MarcoFalke> So you could send a tx that spends and output and the output that was used to create that output (assuming 1in-1out txs for now)?
237 2018-08-12T19:40:57  <MarcoFalke> s/and/an
238 2018-08-12T19:41:09  <sipa> right
239 2018-08-12T19:41:41  <sipa> perhaps you could even permit double spends inside the extra set
240 2018-08-12T19:41:47  <MarcoFalke> So a peer could drain your allowance
241 2018-08-12T19:41:54  <MarcoFalke> for free
242 2018-08-12T19:41:59  <sipa> what allowance?
243 2018-08-12T19:42:22  <MarcoFalke> "allowance" = txs your dandelion destinations accept
244 2018-08-12T19:42:30  <MarcoFalke> num tx/minute or whatever
245 2018-08-12T19:42:55  <sipa> i'm confused
246 2018-08-12T19:43:47  <MarcoFalke> I think we just concluded that the "cheap check" (pass in set of previous txs) can lead to thinking an invalid tx is valid
247 2018-08-12T19:43:55  <MarcoFalke> so we'd forward invalid txs
248 2018-08-12T19:44:00  <sipa> right
249 2018-08-12T19:44:08  <sipa> well, not invalid
250 2018-08-12T19:44:16  <sipa> but conflicting, yes
251 2018-08-12T19:44:20  <MarcoFalke> They'd never be accepted to a real mempool
252 2018-08-12T19:44:40  <MarcoFalke> never as is invalid consensus
253 2018-08-12T19:44:40  <sipa> that depends on the order those extra txn get added to people's mempool
254 2018-08-12T19:44:51  <sipa> no, you do full consensus validation
255 2018-08-12T19:45:18  <MarcoFalke> So you need to calculate all mempool dependencies and stuff
256 2018-08-12T19:45:23  <sipa> how so?
257 2018-08-12T19:46:12  <sipa> validity is just a) can we find the inputs b) are those inputs not yet spent by another mempool txn c) do scripts validate
258 2018-08-12T19:46:28  <sipa> i suggest skipping just (b) for dandelion relay
259 2018-08-12T19:47:20  <MarcoFalke> Though, for a) you use the set of {mempool inputs} OR {prev dandelion txs inputs}
260 2018-08-12T19:47:24  *** owowo has quit IRC
261 2018-08-12T19:47:26  <sipa> right
262 2018-08-12T19:47:33  <MarcoFalke> so if a dandelion txs spent mempool tx
263 2018-08-12T19:48:12  <sipa> but whenever the mempool changes, you don't update the extra sets, so they can grow inconsistent with eachother
264 2018-08-12T19:48:31  <sipa> but i don't think that's a problem; you'll notice when trying to fluff
265 2018-08-12T19:48:48  <MarcoFalke> hmm, give me a sec
266 2018-08-12T19:48:58  <MarcoFalke> How do I draw a picture in irc?
267 2018-08-12T19:49:01  <sipa> haha
268 2018-08-12T19:50:18  <sipa> we should discuss this on the ML though
269 2018-08-12T19:51:43  *** owowo has joined #bitcoin-core-dev
270 2018-08-12T19:54:01  <MarcoFalke> Assume mempool has one output: A. Assume dandelion tx spends this input A and creates output B. We send this dandelion tx. Assume another dandelion tx spends {A,B} and creates output C, which is valid, since we use the set of outputs in the mempool and previous dandelion txs, but the tx itself is consensus invalid. Send this tx. Repeat with {A,C}->D, {A,D}->E ... for free
271 2018-08-12T19:55:15  <sipa> i see your point.
272 2018-08-12T19:55:51  <MarcoFalke> I hope you prove me wrong, because I also like the per peer set
273 2018-08-12T19:58:19  <sipa> is there some rate limiting on dandelion txn per peer?
274 2018-08-12T19:58:49  <MarcoFalke> In my implementation, yes
275 2018-08-12T19:59:03  <sipa> is there in the BIP? (i haven't read the latest draft)
276 2018-08-12T19:59:14  <MarcoFalke> not explicitly mentioned
277 2018-08-12T19:59:18  *** bytting has quit IRC
278 2018-08-12T20:00:30  <MarcoFalke> Maybe there is in the appendix (reference implementation), haven't looked too closely at that, though
279 2018-08-12T20:08:28  <sipa> if you disallow replacement of dandelion txn, it becomes a lot easier
280 2018-08-12T20:08:49  <MarcoFalke> yeah, but we don't want to kill rbf for dandelion txs
281 2018-08-12T20:08:54  <sipa> and perhaps that's not crazy; you can replace, but first need to wait until the dandelion relay has settled into the mempool
282 2018-08-12T20:08:57  <MarcoFalke> I'd rather enforce rbf
283 2018-08-12T20:09:08  <MarcoFalke> (which is what my cache is effectively doing, I think)
284 2018-08-12T20:09:43  <sipa> but you don't support dependencies between dandelion txn, or do you?
285 2018-08-12T20:09:50  <MarcoFalke> nope
286 2018-08-12T20:10:51  <MarcoFalke> You'd have to use rbf to "eat up" all dependencies
287 2018-08-12T20:11:36  *** promag has quit IRC
288 2018-08-12T20:12:40  <sipa> replacement generally seems to be something that happens in the scale of hours, and certainly longer than interblock time
289 2018-08-12T20:12:54  <sipa> both in use cases and incentives
290 2018-08-12T20:13:06  <sipa> while dependent transactions can be in the scale of seconds
291 2018-08-12T20:13:16  <sipa> (blobs of interdependent txn)
292 2018-08-12T20:13:22  *** promag has joined #bitcoin-core-dev
293 2018-08-12T20:13:52  <MarcoFalke> What about the use case of "replacement to avoid a change output-round-trip"
294 2018-08-12T20:14:14  <MarcoFalke> i.e. avoid long chain of unconfirmed
295 2018-08-12T20:14:30  <sipa> if you're doing that in a scale of seconds-minutes you should probably just batch better
296 2018-08-12T20:16:47  <MarcoFalke> hmm, starting to like that idea
297 2018-08-12T20:19:24  *** bytting has joined #bitcoin-core-dev
298 2018-08-12T20:24:32  *** promag has quit IRC
299 2018-08-12T20:30:27  <gmaxwell> jonasschnelli: It's somewhat implausable to me that someone managed to make a sidechannel vulnerable chacha20 which was also fast. I'm happy to review them for it.
300 2018-08-12T20:31:13  <gmaxwell> sipa: two layer mempool sounds hard to now screw up and accidentally leak data.
301 2018-08-12T20:36:29  <gmaxwell> A per peer stempool (which of course shares the actual tx data itself across all peers) makes sense to me.
302 2018-08-12T20:37:08  <gmaxwell> it it requires you augment the protocol to route the dependencies along the same path as the parent.
303 2018-08-12T20:37:39  <gmaxwell> which might have privacy implications. .. I think none of the research on dandelion so far really considered chains of unconfirmed txn.
304 2018-08-12T20:38:46  <gmaxwell> (I'd _generally_ expect that routing children along the same path as parents would be privacy improving, but there may be factors like leaking out of the stem at different points that have bad effects like reducing the privacy of the whole chain to that of the weakest one)
305 2018-08-12T20:39:09  <sipa> gmaxwell: read on
306 2018-08-12T20:39:47  <sipa> ah, you already saw the per-peer idea
307 2018-08-12T20:41:01  *** d9b4bef9 has quit IRC
308 2018-08-12T20:42:14  *** d9b4bef9 has joined #bitcoin-core-dev
309 2018-08-12T20:42:54  <gmaxwell> I also agree that we don't need to care about stem transactions getting invalidated by mempool txn. But I think we do want to check them against each other. In particular I shouldn't be able to give you 100 distinct spends of the same coin and have you route them all out to the same peer. To send two of them to two different peers would be ducky.
310 2018-08-12T20:44:36  <sipa> gmaxwell: yeah, if you don't care about replacing txn while they are not in the mempool that sounds easy
311 2018-08-12T20:44:52  <sipa> it means you don't need the dependency tracking or replacement or whatever logic
312 2018-08-12T20:45:29  <sipa> just verify against the combined set of chainstate+mempool+ peer-specific set of unconfirmed dandelion txn
313 2018-08-12T20:46:49  <gmaxwell> right, but again, if dandelion parents are peer spectific, we must endeavor to route children along the same path as parents, or otherwise they'll propagate poorly.
314 2018-08-12T20:47:19  <sipa> dandelion already does that; it has a per-peer destination peer
315 2018-08-12T20:47:33  <sipa> so subsequent transactions will go to the same outgoing peer
316 2018-08-12T20:47:39  <gmaxwell> sipa: it has _two_.
317 2018-08-12T20:47:39  <sipa> unless there is a shuffle in between
318 2018-08-12T20:47:49  <sipa> only one per incoming peer
319 2018-08-12T20:47:52  <sipa> two globally
320 2018-08-12T20:47:56  <gmaxwell> oh right, okay.
321 2018-08-12T20:47:59  <MarcoFalke> What is the use case for tx chains of dandelion txs?
322 2018-08-12T20:48:16  <gmaxwell> MarcoFalke: uh, being able to spend your funds without waiting for a block.
323 2018-08-12T20:48:28  <sipa> MarcoFalke: what is the use cade for tx chains in general? :)
324 2018-08-12T20:48:30  <sipa> same answer
325 2018-08-12T20:48:33  <MarcoFalke> why block, you can wait for a fluff
326 2018-08-12T20:48:46  <sipa> that's ~minute or so?
327 2018-08-12T20:49:05  <gmaxwell> if someone pays you 1 BTC, you spend 0.1 ... now your wallet interface needs to randomly _fail_ and tell you that you can't spend again, until a fluff has happened?
328 2018-08-12T20:49:06  <sipa> you're right, waiting for a block is not relevant here
329 2018-08-12T20:49:25  <MarcoFalke> yeah, I mean if we don't allow replacement of dandelion txs, we might as well not allow chains
330 2018-08-12T20:49:34  <sipa> MarcoFalke: i disagree
331 2018-08-12T20:49:52  <MarcoFalke> and ask people to batch if the time between spends is ~1minute
332 2018-08-12T20:49:54  <sipa> there is a different timescale
333 2018-08-12T20:50:05  <gmaxwell> That would mean that we couldn't use dandelion as the standard way to announce transactions, if that were the decision I'd say we shouldn't bother implementing it at all.
334 2018-08-12T20:50:38  <sipa> as i said before, i think it's reasonable if replacement only works in a timescale of minutes/hours
335 2018-08-12T20:50:47  <gmaxwell> Ideally people sould batch, sure, but someone cannot guarentee that they won't need to make another payment 40 seconds after the last.
336 2018-08-12T20:50:47  <sipa> but dependencies need to work in seconds
337 2018-08-12T20:51:09  <gmaxwell> Why wouldn't we allow replacements?
338 2018-08-12T20:51:38  <MarcoFalke> Would be more expensive to check
339 2018-08-12T20:52:02  <MarcoFalke> potentially scales with the number of txs in this edges cache (stem)
340 2018-08-12T20:52:07  <gmaxwell> you don't actually 'replace' the transaction, but you can relay a transaction that conflicts with the peer's stemppool if it would otherwise pass the replacement criteria.
341 2018-08-12T20:52:26  <sipa> gmaxwell: i think that's an order of magnitude more complex to implement
342 2018-08-12T20:52:43  <gmaxwell> how so? you have a map of tx parents. It's just like the orphan pool.
343 2018-08-12T20:52:47  *** bytting has quit IRC
344 2018-08-12T20:53:33  <gmaxwell> in any case, I don't see a fundimental reason to not allow replacement... it would probably be fine to skip it for now due to complexity.
345 2018-08-12T20:53:58  <sipa> gmaxwell: the rules for replacement are a complex piece of policy.. that depends on relay fee, discard fee, mempool size, cyclic dependency checks, ...
346 2018-08-12T20:54:07  <MarcoFalke> ^
347 2018-08-12T20:54:17  <sipa> all of those don't really have a direct translation to multiple layers of mempool
348 2018-08-12T20:54:47  <gmaxwell> so uh, how would we handle a dandelion txn which would be a replacement for something in the mempool?
349 2018-08-12T20:55:02  <sipa> we shouldn't?
350 2018-08-12T20:55:08  <MarcoFalke> That works
351 2018-08-12T20:55:10  <gmaxwell> Then I think it's busted.
352 2018-08-12T20:55:14  <sipa> heh?
353 2018-08-12T20:55:25  <MarcoFalke> Of course you can replace mempool txs with dandelion txs
354 2018-08-12T20:55:26  <sipa> oh, ugh.
355 2018-08-12T20:55:37  <sipa> of course that needs to work
356 2018-08-12T20:55:49  <MarcoFalke> I mean, maybe only once, but it works
357 2018-08-12T20:55:59  <gmaxwell> again: I think we cannot make dandelion the standard way to announce txn we should not deploy it. And if it kills replacement of long ago announced txn, then we can't do that.
358 2018-08-12T20:56:10  <sipa> right
359 2018-08-12T20:56:14  <MarcoFalke> agree
360 2018-08-12T20:56:17  <sipa> i don't think that's an issue though
361 2018-08-12T20:56:22  <gmaxwell> It's simple in any case, see if ATMP would accept, and if so it's eligable for stem relay if not conflicted in the peers' stem cache.
362 2018-08-12T20:56:31  <sipa> dandelion tx validation operates on the sum of mempool + extra tzn
363 2018-08-12T20:56:41  <sipa> but it doesn't need to deal with replacements
364 2018-08-12T20:56:48  <sipa> just validation against that set
365 2018-08-12T20:57:18  <gmaxwell> also I think we can also 'support replacement'  by fluffing anything that passes ATMP but conflicts with our stem cache.
366 2018-08-12T20:57:38  <sipa> MarcoFalke gave an example above where that's busted
367 2018-08-12T20:57:57  <MarcoFalke> sipa: I said it works
368 2018-08-12T20:58:19  <MarcoFalke> [16:55] <MarcoFalke> That works
369 2018-08-12T20:58:36  <sipa> oh? what about your a/b, a/c, a/d example?
370 2018-08-12T20:59:19  <MarcoFalke> Well, that is what I meant with "I mean, maybe only once, but it works"
371 2018-08-12T20:59:30  <gmaxwell> I'm not following.
372 2018-08-12T20:59:48  <MarcoFalke> We fell back to the earlier discussion
373 2018-08-12T20:59:55  <gmaxwell> okay
374 2018-08-12T21:00:04  <MarcoFalke> [15:54] <MarcoFalke> Assume mempool has one output: A. Assume dandelion tx spends this input A and creates output B. We send this dandelion tx. Assume another dandelion tx spends {A,B} and creates output C, which is valid, since we use the set of outputs in the mempool and previous dandelion txs, but the tx itself is consensus invalid. Send this tx. Repeat with {A,C}->D, {A,D}->E ... for free
375 2018-08-12T21:00:06  <sipa> if ATMP needs to do complex replacement checks w.r.t things already in the extra set, it becomes hard
376 2018-08-12T21:00:43  <sipa> replacement checks against the mempool of the form "would this be accepted to the mempool" are easy
377 2018-08-12T21:01:41  <gmaxwell> the combination of replacement and chaning is cancer. :(
378 2018-08-12T21:01:48  <MarcoFalke> jup
379 2018-08-12T21:01:54  <MarcoFalke> So pick one
380 2018-08-12T21:02:06  <sipa> however, if replacement within the extra set is not allowed, it's easy enough - discard anything that conflicts with the extra set already
381 2018-08-12T21:02:14  <gmaxwell> Well we can support replacement for non-chained, and also support chaining.
382 2018-08-12T21:02:34  <sipa> otherwise, validate against the mempool with full policy check, getting utxos from the extra set as needed
383 2018-08-12T21:02:55  <gmaxwell> and for the kind of replacement we don't support, I think we could still queue the transaction and not propagate it but fluff it when it times out.
384 2018-08-12T21:02:56  <sipa> if accepted, put in the extra set (which is limited is size, and automatically ezpires through auto fluffing)
385 2018-08-12T21:03:31  <gmaxwell> so at least chained replacements work, they just might have worse privacy/propagation.
386 2018-08-12T21:03:33  <sipa> and fluffing is just implemented as adding to the local mempool... which means that stuff that has been invalidated by intermediate mempool action just gets ignored
387 2018-08-12T21:06:16  <gmaxwell> so the criteria for going into the extra-set is "doesn't need a parent in the extraset and passes ATMP  OR  it needs a parent in the extraset, doesn't conflict with the extra set and with the parent its consensus valid/standard"
388 2018-08-12T21:06:53  <gmaxwell> and if you get something that conflicts with the extraset, and doesn't pass ATMP, you throw it in the orphanmap.  It'll get connected once the parents get fluffed.
389 2018-08-12T21:07:41  <gmaxwell> Then: replacement works, chaining works, and chaining+replacement turns into orphans which still work after the parents fluff.
390 2018-08-12T21:10:01  <gmaxwell> I totally agree that wallets shoudl be batching and whatnot, but consider: we don't even have a friendly way to do that... There is no dohicky in bitcoin core where you can queue a payment, have it draft it, but not send it, waiting for either more payments it can be bached with, timeout, or shutdown trigger.
391 2018-08-12T21:14:22  <MarcoFalke> So fluffing a chained dandelion tx also fluffs its parents? (even though one of the parents might still be "traveling" on a stem)
392 2018-08-12T21:15:28  *** bytting has joined #bitcoin-core-dev
393 2018-08-12T21:16:22  <gmaxwell> thats why I was saying 'weakest in the chain' above. :(
394 2018-08-12T21:16:49  <MarcoFalke> Yeah, so the suggestion would be to avoid chaining, but support it
395 2018-08-12T21:17:51  <sipa> don't fluff things which have an unfluffed parent?
396 2018-08-12T21:22:56  <MarcoFalke> You'd be keeping them much longer in the cache/embargo (on average) and thus use more space for chained txs than unchained ones on avg
397 2018-08-12T21:23:34  <MarcoFalke> A child times out, but you couldn't fluff it because the parent's timeout is in the future
398 2018-08-12T21:26:00  <sipa> i feel like there should perhaps be something where a dependency in the extra set results in the two txn being merged into a packaga
399 2018-08-12T21:26:18  <sipa> and then have the timeout for the package become a weighted average of the inout timeouts or so
400 2018-08-12T21:26:33  <sipa> *input
401 2018-08-12T21:27:27  <sipa> but... complicated
402 2018-08-12T21:37:18  *** Victorsueca has quit IRC
403 2018-08-12T21:38:37  *** Victorsueca has joined #bitcoin-core-dev
404 2018-08-12T21:41:47  *** Krellan has quit IRC
405 2018-08-12T21:42:35  *** Krellan has joined #bitcoin-core-dev
406 2018-08-12T22:14:32  *** bytting has quit IRC
407 2018-08-12T22:14:47  *** bytting has joined #bitcoin-core-dev
408 2018-08-12T22:39:25  *** promag has joined #bitcoin-core-dev
409 2018-08-12T22:58:36  *** ken2812221 has quit IRC
410 2018-08-12T23:02:06  *** vexbuy has quit IRC
411 2018-08-12T23:13:37  *** AaronvanW has joined #bitcoin-core-dev
412 2018-08-12T23:19:41  *** Aaronvan_ has joined #bitcoin-core-dev
413 2018-08-12T23:23:14  *** AaronvanW has quit IRC
414 2018-08-12T23:28:52  *** plankers has joined #bitcoin-core-dev
415 2018-08-12T23:47:34  *** unholymachine has joined #bitcoin-core-dev
416 2018-08-12T23:51:51  *** Randolf has joined #bitcoin-core-dev
417 2018-08-12T23:52:08  *** Krellan has quit IRC
418 2018-08-12T23:53:04  *** Krellan has joined #bitcoin-core-dev