1 2018-12-26T00:00:30  *** fabianfabian has quit IRC
  2 2018-12-26T00:06:14  *** reardencode has joined #bitcoin-core-dev
  3 2018-12-26T00:08:29  *** jarthur has joined #bitcoin-core-dev
  4 2018-12-26T00:09:26  *** Zenton has quit IRC
  5 2018-12-26T00:12:17  *** CodeBlue1776 has quit IRC
  6 2018-12-26T00:12:35  *** CodeBlue1776 has joined #bitcoin-core-dev
  7 2018-12-26T00:13:07  <sipa> jimmysong: "block store" ?
  8 2018-12-26T00:18:11  <jimmysong> storing that data in the db for a full node so it can be given to a peer that requests it
  9 2018-12-26T00:18:14  <sipa> a bloomfilter (or gcd filter) can't store any data, it's just a probabilistic structure to let you query for set elements
 10 2018-12-26T00:18:29  <sipa> a full node has no need for that data
 11 2018-12-26T00:19:03  <jimmysong> a full node doesn't but it'd be useful for a light client wanting to verify a cfilter hash
 12 2018-12-26T00:19:07  <sipa> it is store on disk though, in the undo block data in bitcoin core, for as long as the block itself is stored
 13 2018-12-26T00:19:21  <sipa> ah, for verification it should just be committed to
 14 2018-12-26T00:20:09  <jimmysong> a coinbase commitment is better, but is that something that's got a proposal?
 15 2018-12-26T00:21:24  <sipa> it's pretty trivial to do, but mucb easier to get agreement on once bip157 itself is deployed and used
 16 2018-12-26T00:21:35  <sipa> i expect
 17 2018-12-26T00:21:38  <jimmysong> makes sense
 18 2018-12-26T00:22:00  <jimmysong> i'm asking because i'm trying to write a wallet that uses neutrino
 19 2018-12-26T00:22:10  <jimmysong> the annoying part is the verification
 20 2018-12-26T00:22:51  <jimmysong> but getting the filter, checking against my script pubkeys, etc is great
 21 2018-12-26T00:23:04  <jimmysong> much better for privacy
 22 2018-12-26T00:23:48  <sipa> the node giving you prevour scripts along with the filter doesn't let you verify anything
 23 2018-12-26T00:23:57  <sipa> they can forge anything
 24 2018-12-26T00:23:58  *** fabianfabian has joined #bitcoin-core-dev
 25 2018-12-26T00:24:11  <jimmysong> it lets you verify that the filter bytes are what they said
 26 2018-12-26T00:24:21  <jimmysong> and you can check that the scripts for the inputs verify
 27 2018-12-26T00:24:28  <sipa> no, because the prevout scripts can be forged
 28 2018-12-26T00:24:44  <sipa> there is nothing committing to them
 29 2018-12-26T00:24:51  <jimmysong> wait, if you have a signature, you can make a pubkey that works with it?
 30 2018-12-26T00:25:10  <jimmysong> err, rather, a pubkey, you can make a hash preimage?
 31 2018-12-26T00:25:39  <jimmysong> the scriptsig/witness for p2pkh/p2wpkh is pubkey, sig
 32 2018-12-26T00:25:47  <jimmysong> so you'd have to forge a preimage of the pubkey
 33 2018-12-26T00:25:56  <sipa> oh, you mean full script validation?
 34 2018-12-26T00:26:08  <jimmysong> yes
 35 2018-12-26T00:26:14  <sipa> that shouldn't be something a light client cares about
 36 2018-12-26T00:26:32  <jimmysong> if it helps verify the cfilter hash, why not?
 37 2018-12-26T00:26:49  <jimmysong> commitment is obviously preferable
 38 2018-12-26T00:26:56  <jimmysong> but if we don't have that?
 39 2018-12-26T00:27:08  <roasbeef> jimmysong: yeh there's a half proposal on the ML to add that, the prior versions had the outpoint so they were fully verifiable, as is now you can verify half of it
 40 2018-12-26T00:27:21  <sipa> what is your attack model?
 41 2018-12-26T00:27:38  <roasbeef> differntiate honest peers from dishonest peers
 42 2018-12-26T00:27:55  <sipa> you don't want to get the prevout script for every block, right?
 43 2018-12-26T00:28:01  <roasbeef> it's not full script validation, it's verifying the filter is correct
 44 2018-12-26T00:28:21  <sipa> sure, but you'd need a full script interpreter for doing so
 45 2018-12-26T00:28:22  <roasbeef> yeh you'd fetch the block with the prior scripts and value why not while you're at it, w/ value you can verify fees to a degree as well
 46 2018-12-26T00:28:54  <roasbeef> no need to verify the script, just to verify a filter actually fully matchs a given block
 47 2018-12-26T00:29:13  <jimmysong> i was thinking full script interpreter
 48 2018-12-26T00:29:29  <jimmysong> which isn't trivial
 49 2018-12-26T00:29:44  <sipa> roasbeef: sure, that makes sense; but jimmysong was suggesring script validation, as without it, the prevout data can be trivially forged
 50 2018-12-26T00:29:44  <roasbeef> txscript.VM ;)
 51 2018-12-26T00:30:14  <sipa> jimmysong: but for which blocks would you get this prevout data?
 52 2018-12-26T00:30:26  <jimmysong> the blocks you download when you search for your utxos
 53 2018-12-26T00:30:31  <roasbeef> blocks for which you get conflicting advertisements
 54 2018-12-26T00:30:38  <jimmysong> that too
 55 2018-12-26T00:30:51  <sipa> jimmysong: a peer can still lie and give you a filter with no matches at all
 56 2018-12-26T00:30:56  *** sipa has quit IRC
 57 2018-12-26T00:31:14  <roasbeef> you can fetch the block and verify the outputs are properly included, but not fully inputs
 58 2018-12-26T00:31:15  *** sipa has joined #bitcoin-core-dev
 59 2018-12-26T00:31:43  <sipa> yes, for confloct detection it makes sense, and you need no script validation etc
 60 2018-12-26T00:31:53  <roasbeef> main purpose if conflict detection
 61 2018-12-26T00:31:59  <roasbeef> is*
 62 2018-12-26T00:32:17  <roasbeef> harding: the prev outs are in the undo blocks
 63 2018-12-26T00:32:52  <roasbeef> this was a concern during the switch over (that it would be slower to construct our possibly impossible), but the data is still around
 64 2018-12-26T00:34:29  <jimmysong> what's the objection to writing a script interpreter?
 65 2018-12-26T00:34:34  <jimmysong> is it because of all the edge cases?
 66 2018-12-26T00:34:41  <roasbeef> many already exist
 67 2018-12-26T00:34:55  <jimmysong> i've written a partial one
 68 2018-12-26T00:35:04  <jimmysong> i still don't get op_codeseparator
 69 2018-12-26T00:35:20  <roasbeef> code sep should die, ppl should stop trying to save it lol
 70 2018-12-26T00:36:01  <jimmysong> btw, roasbeef, those test vectors for neutrino broke my script interpreter =)
 71 2018-12-26T00:36:36  <sipa> jimmysong: in my view script interpreting is so easy to get wrong, it should only be done by full nodes
 72 2018-12-26T00:37:03  <jimmysong> it is easy to get wrong, but isn't there value to at least interpreting some subset?
 73 2018-12-26T00:37:25  <jimmysong> p2wsh/p2sh stuff can get pretty complicated
 74 2018-12-26T00:37:32  <sipa> i don't think so
 75 2018-12-26T00:37:50  <sipa> a wallet can recognize its own outputs, bit that doesn't need interpretatiom
 76 2018-12-26T00:38:18  <roasbeef> you can't even verify the outputs fully exist, so script verification is w/e, main reason for getting the prevouts is filter verification and fee awareness
 77 2018-12-26T00:38:18  <sipa> for debugging purposes it can be useful of course, but imo that's it
 78 2018-12-26T00:39:24  <jimmysong> so wallets should stick to standard scripts, or whatever has been analyzed beforehand
 79 2018-12-26T00:39:49  <sipa> well they construct them themselves
 80 2018-12-26T00:40:26  <sipa> so you don't need an interpreter to find its semantics
 81 2018-12-26T00:40:44  <jimmysong> right, the ones analyzed beforehand by the programmer
 82 2018-12-26T00:40:47  <sipa> and you won't trigger any edge cases unless you go out of your way to trigger them
 83 2018-12-26T00:42:17  <jimmysong> in any case, if the outpoint data is in the undo blocks, that can be made available?
 84 2018-12-26T00:42:32  <roasbeef> yep, just add a new inv type, and message format
 85 2018-12-26T00:43:41  <jimmysong> sure, you need the full tx's to verify against the hashes, which as harding pointed out can get pathologically large in edge cases
 86 2018-12-26T00:44:09  <jimmysong> similar to jj's attack from breaking bitcoin last year
 87 2018-12-26T00:44:30  <sipa> jimmysong: needing the full txn is unrealistic
 88 2018-12-26T00:44:49  <sipa> that data isn't available in general, and expensive to index
 89 2018-12-26T00:44:49  <roasbeef> why do you care about the hashes? get the script+value, verify the sig+fee, reconstruct filter, dunzo
 90 2018-12-26T00:45:24  <roasbeef> ofc if the witness commitment was to a merkalized version of the transaction, you could fetch that proof as well, but mo bytes there, what we have available atm is good enough to achieve the goal of cross checking
 91 2018-12-26T00:46:02  <sipa> in general we should aim to only expose data that is verifiable or can be made verifiable
 92 2018-12-26T00:46:34  <sipa> and adding comitments to undo data seems overkill, when all that's needed is a commitmemt to the filters
 93 2018-12-26T00:46:53  <jimmysong> yes, that's the preferred solution
 94 2018-12-26T00:47:00  <roasbeef> yeh not undo, just filters eventually, mean as in if the wtxid was a merkle tree with leaves of the inputs/outputs etc
 95 2018-12-26T00:47:59  <roasbeef> would also let you have a more compact proof of spentnesss, especially if things are also mega super coin-joiny in the future
 96 2018-12-26T00:48:24  <jimmysong> wow, good point
 97 2018-12-26T00:49:05  <sipa> i think all these adhoc methods of verifying filters break down if you look at them in an adverserial setting... you can use heuristics and download from multiple peers, and detect conflicts, and spot check here and there... but those approaches all add complexity and bandwidth proportionl to the level of guarantee they give
 98 2018-12-26T00:49:30  <roasbeef> yep those other appraoches are just plain easier to mess up, i prefer dead simple methods
 99 2018-12-26T00:49:58  <sipa> the only foolproof, cheap, ... way is a filter comitment in blocks
100 2018-12-26T00:50:19  <jimmysong> yep, which costs something for a miner, but not very much. but it is a soft fork
101 2018-12-26T00:50:28  <jimmysong> any idea how that would be received?
102 2018-12-26T00:51:05  <gmaxwell> the bigger limit there is just that there needs to be a higher level of confidence in the design, since removing the requirement is a hardfork. Fortunately, as it is now has actually had a fair level of refinement.
103 2018-12-26T00:51:51  <roasbeef> jimmysong: filter commit in op return, message to fetch header along w/ path for coinbase transaction? i think the filter chain would prob still have some use in that case as well
104 2018-12-26T00:52:11  <gmaxwell> I've suggested in the past potentially doing a kind of rolling softfork, where the commitment is required to be valid only if its provided and only until some height.  And so long as people keep using it and it isn't replaced with something better, we just keep soft-forking in updated heights.. but if ever it gets replaced, it can just be allowed to expire.
105 2018-12-26T00:52:46  <jimmysong> that would be a nice escape clause
106 2018-12-26T00:52:46  <roasbeef> oooOOo, i guess the witness commitment is kinda like that for blocks w/o any witness data
107 2018-12-26T00:53:01  <sipa> jimmysong: i think it's necessary; without commitment to the filters you can basically only use it locally or with a trusted full node
108 2018-12-26T00:53:23  <sipa> which is pretty useful on itself, but nearly the same
109 2018-12-26T00:53:33  <sipa> but not nearly
110 2018-12-26T00:53:35  <gmaxwell> I'm very glad, e.g. that nothing about BIP37 ended up softforked in... that protocol turned out to be a lemon in a number of ways, but it took a couple years of use to realize that.
111 2018-12-26T00:54:07  <roasbeef> well you can still use it in the open net, you have the same "one honest peer" assumption going on, the scripts just help you to distinguish between zero and 1 honest peer
112 2018-12-26T00:55:39  <gmaxwell> roasbeef: even 1 honest peer requires you to have some kind of complex resolution to deal with disagreement, also-- because the p2p network is trivally sybable 'one honest peer' probably doesn't mean much unless you're doing something like manually configuring a trusted one.
113 2018-12-26T00:55:45  *** fanquake has quit IRC
114 2018-12-26T00:56:36  <gmaxwell> One should also consider the effect of incentiving varrious kinds of trouble making.  (like generally we've found that when we added vulnerablities like BIP37 attackers emerged that didn't exist previously, and then made more trouble even for people that didn't care about using BIP37)
115 2018-12-26T00:57:58  <roasbeef> gmaxwell: you'd just fetch the blocks+scripts and reconstruct the filter, the commitment to the filters (the filter chain) helps you notice if something is funky at a glanec
116 2018-12-26T00:58:19  <roasbeef> but then again, for smaller values you're prob not super concerned about this stuff
117 2018-12-26T00:59:16  <gmaxwell> roasbeef: matching the spent inputs too requires fetching the inputs, which as sipa pointed out above is intractable in the worst case (and also astronishingly expensive even in the normal case)
118 2018-12-26T00:59:49  <roasbeef> yeh the assumption is a new inv type to allow you to fetch the the input scripts, i guess i miss this worst case scenario?
119 2018-12-26T00:59:57  <roasbeef> fetch with the block*
120 2018-12-26T01:00:04  <gmaxwell> keeping also in mind that this stuff is saving you 30kbit/sec of bandwidth over downloading the whole blocks... in the ongoing case.
121 2018-12-26T01:00:40  <gmaxwell> roasbeef: we have no way to provide that, and even if we did it couldn't be validated without fetching potentially gigabytes of additional data (you actually have to fetch the whole transactions).
122 2018-12-26T01:03:48  *** fanquake has joined #bitcoin-core-dev
123 2018-12-26T01:07:11  <roasbeef> ahh ok yeh i was missing the outpoints in the equation
124 2018-12-26T01:07:18  <sipa> roasbeef: what if you receive two different filters, and request block/prevout data for both, and they are both correct? (and it turns out you receiced one true prevout data and one false prevout data, but with correct filters for that data)
125 2018-12-26T01:07:21  <roasbeef> well can still verify half of it today ;)
126 2018-12-26T01:07:43  <gmaxwell> roasbeef: yes, half indeed. and really the more useful half.
127 2018-12-26T01:07:52  <gmaxwell> but at the expense of fetching the whole block.
128 2018-12-26T01:08:09  <sipa> that's still detectable, but you're rediced to a majoroty vote kind of model, and at a possibly very high bandwidh cost
129 2018-12-26T01:08:45  <gmaxwell> which also means that it really only takes one clown with an aws instance to cause a lot of users to fetch the blocks anyways.  I mean, perhaps not totally useless.  But as I mentioned, vulnerablties seem to attract attackers.
130 2018-12-26T01:10:03  <gmaxwell> so then you're left with the work of implementing the validation, testing it, dealing with vulnerablities in it... and some clown spins up a bunch of sybils and everyone is downloading the entire blocks anyways. And-- the sybils themselves act as a lasting nussance.  I don't mean to argue against the non-commited usage, but considering the all in effects it's not the obvious big win that it
131 2018-12-26T01:10:04  <gmaxwell> might seem at a glance.
132 2018-12-26T01:11:11  <gmaxwell> and the historical rate of protocol additions introducing vulnerablties (either in implementation or design) is really high...
133 2018-12-26T01:12:22  <roasbeef> it's a win for full nodes at least, serving the filters is much less intensive (and also stateless) compared to serving bip37, you also can't trigger worst case matching behavior over the entire chain
134 2018-12-26T01:13:12  <gmaxwell> roasbeef: quite a few nodes just disable BIP37 completely (which seemed to stop the BIP37 based attacks)
135 2018-12-26T01:14:28  <roasbeef> yeh i ended up doing that on my testnet nodes, seemed somsone was practicing their attacks on testnet lol
136 2018-12-26T01:14:33  <gmaxwell> (I'm not disagreeing with your point though)
137 2018-12-26T01:15:36  <gmaxwell> roasbeef: they were doing it on mainnet for a while. Though they seemed to give up after even a small set of their targets started disabling them. To the extent that they might have been targeting miners to cause block orphaning, that makes sense, but otherwise it's not really clear why it stopped.
138 2018-12-26T01:16:00  <gmaxwell> Perhaps because they realized if they kept it up BIP37 was just going to end up removed and then they'd lose their toy. Who knows.
139 2018-12-26T01:31:02  *** jpe_ has joined #bitcoin-core-dev
140 2018-12-26T02:07:08  *** EagleTM has quit IRC
141 2018-12-26T02:13:06  *** e4xit has quit IRC
142 2018-12-26T02:19:57  *** jpe___ has joined #bitcoin-core-dev
143 2018-12-26T02:20:32  *** jpe___ is now known as jpe
144 2018-12-26T02:21:33  *** jpe_ has quit IRC
145 2018-12-26T02:24:04  *** fabianfabian has quit IRC
146 2018-12-26T02:24:49  *** morcos has quit IRC
147 2018-12-26T02:27:13  *** morcos has joined #bitcoin-core-dev
148 2018-12-26T02:29:21  *** ghost43 has quit IRC
149 2018-12-26T02:29:38  *** ghost43 has joined #bitcoin-core-dev
150 2018-12-26T02:30:51  *** peevsie has joined #bitcoin-core-dev
151 2018-12-26T02:34:22  *** fabianfabian has joined #bitcoin-core-dev
152 2018-12-26T02:54:42  *** mistergold has quit IRC
153 2018-12-26T03:24:49  *** booyah has quit IRC
154 2018-12-26T03:26:18  *** booyah has joined #bitcoin-core-dev
155 2018-12-26T03:29:01  *** rh0nj has quit IRC
156 2018-12-26T03:30:08  *** rh0nj has joined #bitcoin-core-dev
157 2018-12-26T04:12:12  *** fabianfabian has quit IRC
158 2018-12-26T04:49:33  *** hebasto has joined #bitcoin-core-dev
159 2018-12-26T04:56:28  *** jpe has quit IRC
160 2018-12-26T05:22:42  *** bitcoin-git has joined #bitcoin-core-dev
161 2018-12-26T05:22:43  <bitcoin-git> [bitcoin] markaw67 opened pull request #15036: Mwortham (master...patch-2) https://github.com/bitcoin/bitcoin/pull/15036
162 2018-12-26T05:22:43  *** bitcoin-git has left #bitcoin-core-dev
163 2018-12-26T05:33:39  *** justanotheruser has quit IRC
164 2018-12-26T05:49:03  *** peevsie has quit IRC
165 2018-12-26T05:55:02  <fanquake> Wondering if I should even bother in #15036. The guy has turned up to the repo, spammed in one PR, then opened 15036.
166 2018-12-26T05:55:03  <gribble> https://github.com/bitcoin/bitcoin/issues/15036 | Mwortham by markaw67 · Pull Request #15036 · bitcoin/bitcoin · GitHub
167 2018-12-26T05:55:39  <fanquake> Clearly not bothering to read my comment, or from what I can tell, anything related to actually contributing to the project.
168 2018-12-26T06:01:18  <gmaxwell> I'd recommend just closing and locking the PR. They aren't following the guidelines, and there is a good chance that its just trolling.
169 2018-12-26T06:01:26  <gmaxwell> no different than any other driveby pr
170 2018-12-26T06:07:37  *** notmandatory has joined #bitcoin-core-dev
171 2018-12-26T06:09:41  <gmaxwell> and especially due to the principle that the project isn't a place that we'll tolerate other people turning into their performance art or battleground.
172 2018-12-26T06:22:07  *** spinza has quit IRC
173 2018-12-26T06:34:00  *** spinza has joined #bitcoin-core-dev
174 2018-12-26T06:34:45  *** bitcoin-git has joined #bitcoin-core-dev
175 2018-12-26T06:34:46  <bitcoin-git> [bitcoin] fanquake closed pull request #15036: Mwortham (master...patch-2) https://github.com/bitcoin/bitcoin/pull/15036
176 2018-12-26T06:34:46  *** bitcoin-git has left #bitcoin-core-dev
177 2018-12-26T06:39:09  *** rex4539 has joined #bitcoin-core-dev
178 2018-12-26T06:40:23  *** jarthur has quit IRC
179 2018-12-26T06:53:57  <gwillen> fanquake: if you are able, it seems like preemptively locking might be a good idea.
180 2018-12-26T06:55:03  <fanquake> I was going to leave it for a reply, but have done it now.
181 2018-12-26T06:57:57  <gwillen> the quality of his previous comment doesn't make me sanguine
182 2018-12-26T06:58:09  <gwillen> also, the account that hit 'approve' looks like a sockpuppet
183 2018-12-26T07:47:16  *** harrymm has quit IRC
184 2018-12-26T08:04:59  *** harrymm has joined #bitcoin-core-dev
185 2018-12-26T08:32:27  *** tiagotrs has joined #bitcoin-core-dev
186 2018-12-26T08:37:36  *** harrymm has quit IRC
187 2018-12-26T08:50:31  *** harrymm has joined #bitcoin-core-dev
188 2018-12-26T09:01:43  *** elichai2 has joined #bitcoin-core-dev
189 2018-12-26T09:04:07  *** justanotheruser has joined #bitcoin-core-dev
190 2018-12-26T09:34:02  *** Zenton has joined #bitcoin-core-dev
191 2018-12-26T09:50:02  *** e4xit has joined #bitcoin-core-dev
192 2018-12-26T10:02:36  *** spinza has quit IRC
193 2018-12-26T10:47:02  *** spinza has joined #bitcoin-core-dev
194 2018-12-26T10:56:07  *** harrymm has quit IRC
195 2018-12-26T11:02:01  *** rh0nj has quit IRC
196 2018-12-26T11:03:10  *** rh0nj has joined #bitcoin-core-dev
197 2018-12-26T11:35:20  *** spinza has quit IRC
198 2018-12-26T11:40:47  *** spinza has joined #bitcoin-core-dev
199 2018-12-26T11:56:51  *** EagleTM has joined #bitcoin-core-dev
200 2018-12-26T12:06:25  *** ghost43 has quit IRC
201 2018-12-26T12:06:36  *** ghost43 has joined #bitcoin-core-dev
202 2018-12-26T12:14:09  *** Yaggi has joined #bitcoin-core-dev
203 2018-12-26T12:25:48  *** AaronvanW has joined #bitcoin-core-dev
204 2018-12-26T12:29:19  <fanquake> wumpus are you around tonight?
205 2018-12-26T12:36:19  *** Yaggi has quit IRC
206 2018-12-26T12:46:55  *** YAGGI has joined #bitcoin-core-dev
207 2018-12-26T13:10:54  *** AaronvanW has quit IRC
208 2018-12-26T13:17:07  *** YAGGI has quit IRC
209 2018-12-26T13:26:48  *** EagleTM has quit IRC
210 2018-12-26T13:30:46  *** bsm117532 has joined #bitcoin-core-dev
211 2018-12-26T13:37:54  *** spaced0ut has joined #bitcoin-core-dev
212 2018-12-26T13:52:01  *** mistergold has joined #bitcoin-core-dev
213 2018-12-26T13:54:27  *** cluelessperson has joined #bitcoin-core-dev
214 2018-12-26T13:54:52  *** cluelessperson is now known as Guest88093
215 2018-12-26T13:56:05  *** bitcoin-git has joined #bitcoin-core-dev
216 2018-12-26T13:56:05  <bitcoin-git> [bitcoin] hebasto opened pull request #15038: docs: Get more info about GUI-related issue on Linux (master...20181226-issue-template-gui-linux) https://github.com/bitcoin/bitcoin/pull/15038
217 2018-12-26T13:56:05  *** bitcoin-git has left #bitcoin-core-dev
218 2018-12-26T14:04:14  *** e4xit has quit IRC
219 2018-12-26T14:04:26  *** Guest88093 has quit IRC
220 2018-12-26T14:07:00  *** e4xit has joined #bitcoin-core-dev
221 2018-12-26T14:08:13  *** cluelessperson has joined #bitcoin-core-dev
222 2018-12-26T14:31:59  *** belcher has quit IRC
223 2018-12-26T14:54:12  *** jpe has joined #bitcoin-core-dev
224 2018-12-26T14:55:36  *** mistergold has quit IRC
225 2018-12-26T14:59:14  *** fabianfabian has joined #bitcoin-core-dev
226 2018-12-26T15:03:43  <wumpus> fanquake: maybe a bit
227 2018-12-26T15:04:01  *** rh0nj has quit IRC
228 2018-12-26T15:05:08  *** rh0nj has joined #bitcoin-core-dev
229 2018-12-26T15:05:56  *** flex has joined #bitcoin-core-dev
230 2018-12-26T15:07:56  *** bitcoin-git has joined #bitcoin-core-dev
231 2018-12-26T15:07:56  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #15039: wallet: Avoid leaking nLockTime fingerprint when anti-fee-sniping (master...Mf1812-walletLocktimeFingerprint) https://github.com/bitcoin/bitcoin/pull/15039
232 2018-12-26T15:07:56  *** bitcoin-git has left #bitcoin-core-dev
233 2018-12-26T15:08:14  <fanquake> wumpus np, bunch of PRs mergable, but can deal with em later
234 2018-12-26T15:13:17  <wumpus> PSA: there's no meeting today (there was confusion about this last week)
235 2018-12-26T15:15:58  <hebasto> wumpus: today is Wednesday. Do you mean tomorrow?
236 2018-12-26T15:18:25  <wumpus> oh sorry yes I mean tomorrow
237 2018-12-26T15:19:25  <fanquake> only Wednesday for another 41 minutes anyway
238 2018-12-26T15:20:53  <sipa> wumpus: no, you're right, no meeting today!
239 2018-12-26T15:21:07  *** ryanofsky has quit IRC
240 2018-12-26T15:24:10  *** ryanofsky has joined #bitcoin-core-dev
241 2018-12-26T15:28:21  *** peevsie has joined #bitcoin-core-dev
242 2018-12-26T15:39:36  *** tiagotrs has quit IRC
243 2018-12-26T15:53:43  *** fanquake has quit IRC
244 2018-12-26T15:59:59  *** kinlo has quit IRC
245 2018-12-26T16:03:55  *** tiagotrs has joined #bitcoin-core-dev
246 2018-12-26T16:06:25  *** kinlo has joined #bitcoin-core-dev
247 2018-12-26T16:07:05  *** jpe has quit IRC
248 2018-12-26T16:13:23  *** ddustin has joined #bitcoin-core-dev
249 2018-12-26T16:16:32  *** cyber55 has quit IRC
250 2018-12-26T16:34:18  *** mistergold has joined #bitcoin-core-dev
251 2018-12-26T16:47:01  *** Murch has joined #bitcoin-core-dev
252 2018-12-26T17:10:14  <andytoshi> are there any guarantees about the order of `getrawmempool` output? in particular do ancestors always precede descendants?
253 2018-12-26T17:12:20  *** grubles has quit IRC
254 2018-12-26T17:19:27  *** achow101_ has joined #bitcoin-core-dev
255 2018-12-26T17:19:40  <instagibbs> from my cursory reading they are indexed in 4 different ways, none of which are related to ancestors/decendants
256 2018-12-26T17:19:45  *** Murch has quit IRC
257 2018-12-26T17:20:13  <andytoshi> kk thanks, i won't rely on that then
258 2018-12-26T17:20:21  <instagibbs> salted txid, feerate(including desc), entry time, and sorted feerate(including ancestor
259 2018-12-26T17:20:30  *** achow101 has quit IRC
260 2018-12-26T17:20:46  <andytoshi> i mean, somehow when creating blocks they have to wind up in ancestor order ... is there an explicit sorting step then?
261 2018-12-26T17:21:07  <instagibbs> they grab "packages" as they sort via package feerate
262 2018-12-26T17:22:05  <instagibbs> there are internal links between the entries, just not exposed here afaik
263 2018-12-26T17:22:14  <andytoshi> ah, yep, that makes sense
264 2018-12-26T17:22:28  <andytoshi> my goal here is to recreate the packages from the output of getrawmempool
265 2018-12-26T17:22:34  *** achow101_ is now known as achow101
266 2018-12-26T17:22:41  *** Murch has joined #bitcoin-core-dev
267 2018-12-26T17:26:06  <sipa> andytoshi: i'm pretty sure they're sorted by increasing total number of (recursive) unconfirmed dependencies, and then by feerate
268 2018-12-26T17:26:17  <sipa> and then by txid as tiebreaker or so
269 2018-12-26T17:26:41  <sipa> which guarantees that dependencies always come before the dependendees (?)
270 2018-12-26T17:27:14  <sipa> that code is not shared with the block creation code, btw
271 2018-12-26T17:30:12  <instagibbs> andytoshi, also try getmempoolentry which comes with more details?
272 2018-12-26T17:32:26  <andytoshi> instagibbs: oh, yeah (or `getrawmempool true` which gives me the same details). will take a look at that to see if it's useful. i suspect not, i think i need to manually recreate a lot of this data in my code because i need a bunch of extra details, like the set of yet-ununspent inputs/outputs for the whole package
273 2018-12-26T17:32:43  <andytoshi> sipa: cool, thanks. but i guess that's an implementation detail and if i were to write production software that depended on it, you'd be annoyed :)
274 2018-12-26T17:35:07  <andytoshi> i wish there was some way i could signal core that i don't want certain outputs to be 0conf-spendable (or if they are, that i don't want cpfp rules to be applied)
275 2018-12-26T17:36:25  <gmaxwell> andytoshi: encountering the RBF pinning problem?
276 2018-12-26T17:37:32  <andytoshi> gmaxwell: roughly ... but i think it doesn't even need RBF to be a problem .. e.g. see russell's mailing list post https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016519.html
277 2018-12-26T17:38:11  <andytoshi> the issue is that if my software is making a package A of transactions and doing all sorts of CPFP logic to make that sensible, and meanwhile some customer of mine is creating a package B with a massive low-fee sweep or whatever
278 2018-12-26T17:38:19  *** Murch has quit IRC
279 2018-12-26T17:38:23  <andytoshi> and then that customer creates a tx spending an output of A alongside the output of B...
280 2018-12-26T17:38:33  <andytoshi> ...those packages become merged and suddenly my logic has been blindsided
281 2018-12-26T17:38:44  <gmaxwell> that isn't how the implementation works.
282 2018-12-26T17:39:04  <gmaxwell> I think Russell is like... reasoning from the feature's name.
283 2018-12-26T17:39:32  *** jarthur has joined #bitcoin-core-dev
284 2018-12-26T17:40:23  <andytoshi> well, the code is not super straightforward to someone uninvolved with it. the mempool logic related to the descendant limit looks kinda like it would do stuff like what i described
285 2018-12-26T17:41:10  <gmaxwell> The descndant limit stuff can do things along those lines, but requires actually hitting the descendant limit.
286 2018-12-26T17:45:52  <gmaxwell> If the limits are getting hit by ordnary usage we should look into fixing that. They were set so they were never hit when established (except for some obviously dumb floody crap), and only exist to prevent a bad computational blowup in the tracking code.
287 2018-12-26T17:48:32  <instagibbs> one thing to note is that a single ~100kvB sweep can hose the descendant size limit pretty much
288 2018-12-26T17:54:17  <gmaxwell> so fix it?
289 2018-12-26T17:55:30  <gmaxwell> IIRC the reason for the size limits in the tracking is just so it doesn't falsely credit parents for feerate coming from transactions that are never going to fit in the same block.
290 2018-12-26T18:00:00  *** peevsie has quit IRC
291 2018-12-26T18:00:23  <andytoshi> maybe i'm confused about cpfp. my understanding is that if i make a tx with outputs controlled by other people, those other people are able to grief me and undermine my ability to use cpfp
292 2018-12-26T18:00:31  <andytoshi> by extending the package such that i'd be hitting limits
293 2018-12-26T18:02:16  <gmaxwell> yes, though also at the expense of delaying their own transaction confirmation.
294 2018-12-26T18:02:34  <gmaxwell> I don't believe we've ever seen cpfp 'griefing' reported.
295 2018-12-26T18:02:58  <gmaxwell> RBF pinning for sure, because a common usage pattern immediately causes it.
296 2018-12-26T18:04:33  <gmaxwell> The limits exist only because there are computational overheads in the tracking, e.g. when removing a transaction its ancestors and descendants need to be walked to update their tracking.
297 2018-12-26T18:11:59  *** jarthur has quit IRC
298 2018-12-26T18:12:15  *** gelmutshmidt has joined #bitcoin-core-dev
299 2018-12-26T18:13:03  *** gelmutshmidt has quit IRC
300 2018-12-26T18:13:29  *** gelmutshmidt has joined #bitcoin-core-dev
301 2018-12-26T18:14:03  *** tiagotrs has quit IRC
302 2018-12-26T18:16:01  *** IGHOR has quit IRC
303 2018-12-26T18:16:25  <andytoshi> sorry for the dumb questions, but can you clarify - if i'm making cpfp packages, and one of my customers 0conf spends one of the outputs i create, will their transaction pull my effective feerate toward the feerate of that tx? if so, then i need logic to reason about that, and by nature that logic needs to know about the limits (even though in practice i don't expect anyone to pull me close to
304 2018-12-26T18:16:31  <andytoshi> them)
305 2018-12-26T18:16:40  <andytoshi> or is it safe if i just make my own transactions that chain off each others' change outputs, and ignore everything else?
306 2018-12-26T18:17:30  <andytoshi> maybe i should just pester sipa in person in a couple of weeks :)
307 2018-12-26T18:17:44  <andytoshi> and i will try to write down what i'm learning as i do this
308 2018-12-26T18:18:57  <gmaxwell> No, it will not.
309 2018-12-26T18:20:28  <gmaxwell> The parents effective feerate is the highest feerate you can construct with it.
310 2018-12-26T18:20:45  <andytoshi> ok, i think i've got it
311 2018-12-26T18:21:09  <gmaxwell> They can, if they spam out to the limits, prevent new descendants from being taken.
312 2018-12-26T18:21:23  <gmaxwell> But thats only in the case that the tracking limits are hit.
313 2018-12-26T18:21:47  <andytoshi> so is this a rough high-level view of cpfp in core?: (a) "packages" only exist during miners' transaction selection; in the case that a transaction might be in multiple packages, they're computed greedily to maximize feerate; (b) but when accepting to the mempool, Core checks whether a transaction might cause a limit-violating package to exist, and if it would, the tx is rejected
314 2018-12-26T18:26:06  <gmaxwell> close enough;  the limits aren't really limits on 'packages', they're more limits on the tracking datastructures used to create packages.
315 2018-12-26T18:26:55  <andytoshi> yep, that's what i meant
316 2018-12-26T18:29:17  *** IGHOR has joined #bitcoin-core-dev
317 2018-12-26T18:39:59  *** Guyver2 has joined #bitcoin-core-dev
318 2018-12-26T18:40:07  *** Zenton has quit IRC
319 2018-12-26T18:48:36  *** Guyver2_ has joined #bitcoin-core-dev
320 2018-12-26T18:51:25  *** peevsie has joined #bitcoin-core-dev
321 2018-12-26T18:51:40  *** Guyver2 has quit IRC
322 2018-12-26T19:06:02  *** rh0nj has quit IRC
323 2018-12-26T19:08:20  *** AaronvanW has joined #bitcoin-core-dev
324 2018-12-26T19:09:08  *** rh0nj has joined #bitcoin-core-dev
325 2018-12-26T19:35:57  *** mistergold has quit IRC
326 2018-12-26T19:49:21  *** tiagotrs has joined #bitcoin-core-dev
327 2018-12-26T19:56:30  *** tiagotrs has quit IRC
328 2018-12-26T20:00:22  *** gelmutshmidt has quit IRC
329 2018-12-26T20:06:30  *** IGHOR has quit IRC
330 2018-12-26T20:11:26  *** elichai2 has quit IRC
331 2018-12-26T20:21:56  *** fabianfabian has quit IRC
332 2018-12-26T20:24:37  *** IGHOR has joined #bitcoin-core-dev
333 2018-12-26T20:27:24  *** bitcoin-git has joined #bitcoin-core-dev
334 2018-12-26T20:27:24  <bitcoin-git> [bitcoin] hebasto opened pull request #15040: qt: Add workaround for QProgressDialog bug on macOS (master...20181226-fix-macos-qprogressdialog) https://github.com/bitcoin/bitcoin/pull/15040
335 2018-12-26T20:27:24  *** bitcoin-git has left #bitcoin-core-dev
336 2018-12-26T20:29:55  *** EagleTM has joined #bitcoin-core-dev
337 2018-12-26T20:46:06  *** bsm117532 has quit IRC
338 2018-12-26T20:47:11  *** bsm117532 has joined #bitcoin-core-dev
339 2018-12-26T20:55:57  *** Krellan has quit IRC
340 2018-12-26T21:03:03  *** hebasto has quit IRC
341 2018-12-26T21:28:49  *** Zenton has joined #bitcoin-core-dev
342 2018-12-26T21:36:59  *** Guyver2_ has quit IRC
343 2018-12-26T21:41:54  *** promag has joined #bitcoin-core-dev
344 2018-12-26T21:46:24  *** promag has quit IRC
345 2018-12-26T21:56:10  *** jcorgan has quit IRC
346 2018-12-26T21:56:41  *** jcorgan has joined #bitcoin-core-dev
347 2018-12-26T22:03:34  *** spinza has quit IRC
348 2018-12-26T22:10:00  *** spinza has joined #bitcoin-core-dev
349 2018-12-26T22:10:56  *** tiagotrs has joined #bitcoin-core-dev
350 2018-12-26T22:15:24  *** rex4539 has quit IRC
351 2018-12-26T22:16:57  *** tiagotrs has quit IRC
352 2018-12-26T22:42:23  *** promag has joined #bitcoin-core-dev
353 2018-12-26T23:10:01  *** rh0nj has quit IRC
354 2018-12-26T23:16:10  <jnewbery> andytoshi: I think that's mostly right. In (a), the miner is ordering by ancestor feerate (see BlockAssembler::addPackageTxs in miner.cpp). In (b), all of the {(ancestor|descedant) (count|size)} are taken into account (see CTxMemPool::CalculateMemPoolAncestors() in txmempool.cpp)
355 2018-12-26T23:18:11  <jnewbery> "and i will try to write down what i'm learning as i do this". Please consider contributing that to https://github.com/bitcoinops/scaling-book/blob/master/1.fee_bumping/fee_bumping.md if you think you can document it!
356 2018-12-26T23:31:33  <gmaxwell> jnewbery: your last statement sounds confusing, and plays into roconnor's misunderstanding.
357 2018-12-26T23:32:31  <gmaxwell> Basically what roconnor was thinking was that if there is an unconfirmed txn and then I add a gigantic low feerate child to it,  I lower the feerate of the txn because the "package" has a lower feerate.. And that is not how it works, because of the max operation in the combining.
358 2018-12-26T23:37:14  *** spinza has quit IRC
359 2018-12-26T23:42:24  *** mistergold has joined #bitcoin-core-dev
360 2018-12-26T23:44:32  <jnewbery> (b) is not looking at feerate. Just tx count and size
361 2018-12-26T23:48:37  *** spinza has joined #bitcoin-core-dev
362 2018-12-26T23:54:07  *** EagleTM has quit IRC