12015-11-18T00:00:47 <gmaxwell> I think davec's comment was about the difference between disconnect instantly vs the score? I don't think the score has much use, but most of the reason we bother disconnecting is to avoid resource wasting attacks, and a little bit of vioation is fine on those terms. ... and handwave handwave, maybe the network is a bit more robust if its a little tolerant there. I'm skeptical.
22015-11-18T00:01:40 <davec> indeed. We're extremely strict in btcd. For example, if you claim to be a protocol version after the nonce was added to ping/pong, but you don't include them, you aren't following the protocol, so you get disconnected, etc
32015-11-18T00:02:02 <gmaxwell> Thats good feedback.
42015-11-18T00:02:04 <davec> unless things have changed recently that I missed, core doesn't care
52015-11-18T00:02:20 <gmaxwell> davec: do you disconnect on unsolicited TX messages (no getdata first?)
62015-11-18T00:02:34 <davec> we used to, but BitcoinJ didn't work, so we allow them now
72015-11-18T00:04:56 <phantomcircuit> gmaxwell, hmm that's kind of nasty the only way to relay a transaction is to place it into the mempool
82015-11-18T00:05:02 <davec> Another thing we added which I don't believe core does is maximums for every protocol message type
92015-11-18T00:05:19 <gmaxwell> phantomcircuit: Thats fine, we'll only allow whitelisted peers to do that.
102015-11-18T00:05:22 <davec> so like a transaction can't possibly be < x or > y so it's disallowed
112015-11-18T00:06:15 <phantomcircuit> gmaxwell, could soft change mempoolexpirey to 1 so at least they're thrown away rapidly
122015-11-18T00:06:27 <phantomcircuit> or not
132015-11-18T00:06:37 <gmaxwell> phantomcircuit: nah, I don't see a reason to.
142015-11-18T00:06:59 <gmaxwell> they'll only come in from whitelisted peers. Don't whitelist (and allow relay for them) stuff you don't want adding things to your mempool.
152015-11-18T00:07:12 <phantomcircuit> ok comments added and that commit removed
162015-11-18T00:07:30 <gmaxwell> phantomcircuit: I understood this completely and was comfortable with it before merging your prior code, FWIW!
172015-11-18T00:09:17 <phantomcircuit> gmaxwell, heh
182015-11-18T00:09:49 <gmaxwell> davec: any idea if bitcoinj still sends unsolicited transactions if its connected to something that has set the relay flag to false?
192015-11-18T00:09:52 <phantomcircuit> yes i tried to "fix" something which after thinking about it more is basically not fixable unless you're willing to push the tx messages instead of inv
202015-11-18T00:10:03 <phantomcircuit> which isn't very nice
212015-11-18T00:14:01 <gmaxwell> phantomcircuit: your first commit is changing too much of the logic, e.g. bypassing nSendsize check.
222015-11-18T00:15:47 <phantomcircuit> gmaxwell, the nSendSize check still runs if we do something that could change it
232015-11-18T00:17:46 <gmaxwell> phantomcircuit: right now. Creates non-local action; I'd prefer you just make the minimial change and then elseif the other reasons, and emit the log entry.
242015-11-18T00:18:07 <phantomcircuit> ok
252015-11-18T00:20:02 <davec> gmaxwell: I don't watch it's dev to know if it has changed, but back when we were getting them to work together the answer is yes. It always sent unsolicited txns regardless.
262015-11-18T00:20:11 <davec> its*
272015-11-18T00:20:37 <gmaxwell> davec: thanks. Yea, thats all I needed to know (expected the answer too)
282015-11-18T00:20:50 <davec> I believe the reasoning was that if it authored the transaction they are 100% sure you don't have it, so no point in the round trip
292015-11-18T00:21:02 <davec> except it did it for more than that case
302015-11-18T00:21:22 <sipa> that's only true when relaying to just a single peer
312015-11-18T00:21:54 <gmaxwell> also only true if you'd never relay a third party txn and don't care about privacy at all.
322015-11-18T00:22:37 <davec> I agree, but that was the reasoning provided when we asked about it
332015-11-18T00:23:11 <davec> at any rate, we either had to loosen up and allow them or not work with BitcoinJ, so we chose to interop
342015-11-18T00:23:40 <tulip> I didn't think btcd implemented BIP37 anyway?
352015-11-18T00:24:02 <gmaxwell> davec: sure, I'll see if I can convince the new maintership to change.
362015-11-18T00:24:12 <davec> it does
372015-11-18T00:24:40 <davec> We allow it to be disabled via --nobloom, but it fully supports it
382015-11-18T00:24:43 <tulip> ok, crossed wires.
392015-11-18T00:24:45 <dcousens> gmaxwell: to change [it]
402015-11-18T00:24:54 <gmaxwell> Even if it doesn't ... we may start kicking when fRelay is off-- because it's in their interest if they're relaying to us and we're just going to drop it.
412015-11-18T00:25:26 <dcousens> :P
422015-11-18T00:28:27 <davec> oh, if it's of any interest to you, we do d/c on unsolicited blocks and have not seen anything that doesn't follow the inv/getdata model there
432015-11-18T00:29:44 <sipa> ah, that is good to know!
442015-11-18T00:32:03 <tulip> I don't think the relaynode software sends inv for blocks either, I'll have to check though.
452015-11-18T00:33:49 <tulip> that looks right.
462015-11-18T00:35:18 <phantomcircuit> gmaxwell, i can do it as if(inv.type != MSG_BLOCK) {if (fBlocksOnly) Log() else if (!fAlreadyHave && !fImporting && !fReindex) AskFor();}
472015-11-18T00:35:58 <phantomcircuit> i think that's the cleanest change
482015-11-18T00:36:07 <phantomcircuit> without it being slightly insane
492015-11-18T00:36:12 <davec> tulip: hmmm I don't recall seeing any of them hit that, but I can comb back through the logs. We might have to change that as well.
502015-11-18T00:38:48 <davec> Although it seems pretty wasteful to send a full block only to have it ignored by the other end because they already have it or have already reusted it from another peer and its in-flight, etc
512015-11-18T00:39:10 <gmaxwell> phantomcircuit: I think you can do it clear, take existing code, add the whitelist. Then add an else that checks everything except the whitelist and the blocks only)
522015-11-18T00:39:37 <gmaxwell> tulip: relaynetwork client purposefully does not, but it's local.
532015-11-18T00:39:47 <tulip> davec: the relaynode software is only meant to communicate with local nodes anyway.
542015-11-18T00:40:29 <GitHub144> [bitcoin] luke-jr opened pull request #7047: [WIP] Backports for 0.11.3 (updated to e8df8a5) (0.11...backports-for-0.11.3) https://github.com/bitcoin/bitcoin/pull/7047
552015-11-18T00:42:12 <davec> tulpi: Oh, right. I thought you were talking about the relay network (which I know does things quite a bit differently)
562015-11-18T00:42:26 <sipa> davec: he is, i think
572015-11-18T00:43:45 <tulip> davec: we're talking about the same thing, just different name
582015-11-18T00:43:53 <gmaxwell> he's talking about the relay network client local gateway.
592015-11-18T00:44:13 <gmaxwell> As far as I know the normal relay network bitcoin nodes do nothing special on the bitcoin p2p protocol.
602015-11-18T00:45:37 <davec> Yeah that makes sense. I would say that's probably a case for whitelisted nodes accepting them but not allowing random unknown p2p nodes to do it
612015-11-18T00:45:37 <phantomcircuit> gmaxwell, like this?
622015-11-18T00:45:39 <phantomcircuit> https://gist.github.com/pstratem/a8bc031b959cd2225ef1
632015-11-18T00:46:09 <phantomcircuit> that definitely does what we want but... well it's the kind of thing im going to see in six months and go "what idiot did this! oh right me"
642015-11-18T00:47:53 <gmaxwell> phantomcircuit: you can reorg the branches to eliminate the duplication but be functionally equal then.
652015-11-18T00:48:19 <gmaxwell> also why are you keeping up with the fBlocks only flag. If its only getting used once, no need to make a new flag.
662015-11-18T00:48:30 <davec> or even from local conns
672015-11-18T00:49:37 <phantomcircuit> gmaxwell, without it as a flag the expression is insanely difficult to understand
682015-11-18T00:52:55 <phantomcircuit> gmaxwell, to be pedantic fRelayTxes literally only prevents transactions from being relayed, so if we were to add a new inv type in the future that isn't MSG_BLOCK this logic would be wrong
692015-11-18T00:53:18 <phantomcircuit> it would still do the right thing except for the logging
702015-11-18T00:54:50 <davec> iirc there is already a MSG_FILTERED_BLOCK too
712015-11-18T00:55:50 <phantomcircuit> davec, yeah but that's not relevant for "inv"
722015-11-18T00:55:51 <sipa> davec: only in getdata, not in iv
732015-11-18T00:57:40 <GitHub150> [bitcoin] fanquake opened pull request #7048: [doc][trivial] Update miniupnpc version in build-unix (master...miniupnpc-build-unix) https://github.com/bitcoin/bitcoin/pull/7048
742015-11-18T00:57:51 <phantomcircuit> gmaxwell, oh and using the flag in "inv" prevents calling GetBoolArg in a potentially tight loop
752015-11-18T00:58:17 <gmaxwell> phantomcircuit: K leave the flag but ditch the continue.
762015-11-18T00:58:24 <davec> sipa/phantomcircuit: right. thanks -- been a while since I looked so forgot the particulars there
772015-11-18T00:58:28 <phantomcircuit> gmaxwell, done
782015-11-18T01:10:54 *** zooko has joined #bitcoin-core-dev
792015-11-18T01:23:07 *** randy-waterhouse has quit IRC
802015-11-18T01:24:27 *** Ylbam has quit IRC
812015-11-18T01:52:17 *** Guest16267 has quit IRC
822015-11-18T01:52:17 *** Guest16267 has joined #bitcoin-core-dev
832015-11-18T01:52:22 *** Guest16267 is now known as s1w
842015-11-18T02:06:37 <dcousens> gmaxwell: maybe I should clarify, my NACK on that was weak
852015-11-18T02:06:56 <dcousens> I just didn't see a need for it except in highly custom setups which IMHO don't fit the bill for most node operators, I may be wrong though
862015-11-18T02:07:01 <gmaxwell> dcousens: ya thats fine, if I sounded too eager there in responding I'm sorry; wasn't communicating well.
872015-11-18T02:08:04 <dcousens> On reviewing the code, its plain enough (less than 40-50 lines), the rest is tests
882015-11-18T02:08:08 <gmaxwell> Does the thought of removing the old mechenism make it more appealing to you?
892015-11-18T02:08:24 <dcousens> I just saw +240 and jumped the gun
902015-11-18T02:09:03 <dcousens> and yes
912015-11-18T02:09:20 <dcousens> If the alterior method is removed, then this is an easy win
922015-11-18T02:09:48 <gmaxwell> whew
932015-11-18T02:11:25 <gmaxwell> Okay, I confess some paranoia here-- I don't want this to be like the rpc hangs issue. Where we pigheadily avoid improvements for some silly footgun which is making people stop using bitcoin core and switch to hosted apis.
942015-11-18T02:11:38 <gmaxwell> no reason to let me ramrod the feature though.
952015-11-18T02:11:48 <gmaxwell> If it goes in it should be the best possible.
962015-11-18T02:14:31 <dcousens> IMHO, 1 day, walletd, bitcoind, indexd. 1 day. Probably 10 years haha
972015-11-18T02:15:09 <instagibbs> it's a beautiful dream :)
982015-11-18T02:15:40 <dcousens> We need some Shia to make it happen
992015-11-18T02:16:30 <gmaxwell> dcousens: agreed, though we can't halt incremental progress on the way to that vision; ... we'll never get there if we lose all the users who might help support that work. :)
1002015-11-18T02:16:37 <dcousens> haha, indeed
1012015-11-18T02:16:41 <instagibbs> Oh, I was thinking "Shia Muslim" and was quite confused...
1022015-11-18T02:16:50 <dcousens> instagibbs: haha, I meant Laboef
1032015-11-18T02:17:07 <dcousens> Labouef*
1042015-11-18T02:18:04 *** belcher has quit IRC
1052015-11-18T02:18:25 <gmaxwell> #include <culturally_insentive_joke_of_some_form.h>
1062015-11-18T03:00:26 *** neha has quit IRC
1072015-11-18T03:01:17 *** neha has joined #bitcoin-core-dev
1082015-11-18T03:01:26 *** helo has quit IRC
1092015-11-18T03:02:20 *** helo has joined #bitcoin-core-dev
1102015-11-18T03:05:53 <morcos> sipa: that "Leaving block file" showing 0's thing. There is no reason not to fix that right?
1112015-11-18T03:24:27 <morcos> nm, just did it.. :)
1122015-11-18T03:24:28 <GitHub85> [bitcoin] morcos opened pull request #7050: Fix debug log message for block files (master...nitfix) https://github.com/bitcoin/bitcoin/pull/7050
1132015-11-18T03:32:04 *** parazyd has joined #bitcoin-core-dev
1142015-11-18T03:59:53 <phantomcircuit> gmaxwell, ok i've stopped updating 7046 every ten minutes
1152015-11-18T04:12:46 <phantomcircuit> gmaxwell, i still really do not like having Get*Arg everywhere
1162015-11-18T04:12:59 <phantomcircuit> but alas my brain is currently jello so im not doing that tonight
1172015-11-18T04:14:23 *** Greybits has joined #bitcoin-core-dev
1182015-11-18T04:23:56 *** jgarzik has joined #bitcoin-core-dev
1192015-11-18T04:23:56 *** jgarzik has joined #bitcoin-core-dev
1202015-11-18T04:27:54 *** guest234234 has joined #bitcoin-core-dev
1212015-11-18T04:32:56 *** guest234234 has quit IRC
1222015-11-18T04:42:56 *** d_t has quit IRC
1232015-11-18T05:12:34 *** Greybits has quit IRC
1242015-11-18T05:39:58 *** dcousens has quit IRC
1252015-11-18T05:40:15 *** dcousens has joined #bitcoin-core-dev
1262015-11-18T05:40:44 *** d_t has joined #bitcoin-core-dev
1272015-11-18T05:51:04 *** tulip has quit IRC
1282015-11-18T05:53:42 *** randy-waterhouse has joined #bitcoin-core-dev
1292015-11-18T06:07:59 *** jtimon has quit IRC
1302015-11-18T06:12:25 *** tulip has joined #bitcoin-core-dev
1312015-11-18T06:22:14 *** bsm117532 has quit IRC
1322015-11-18T06:23:39 *** bsm117532 has joined #bitcoin-core-dev
1332015-11-18T07:14:06 *** Ylbam has joined #bitcoin-core-dev
1342015-11-18T07:28:05 *** dcousens has quit IRC
1352015-11-18T07:29:04 *** baldur has quit IRC
1362015-11-18T07:29:27 *** baldur has joined #bitcoin-core-dev
1372015-11-18T08:10:37 <GitHub23> [bitcoin] laanwj opened pull request #7051: ui: Add "Copy raw transaction data" to transaction list context menu (master...2015_11_transaction_hex2) https://github.com/bitcoin/bitcoin/pull/7051
1382015-11-18T08:11:30 *** jcorgan has quit IRC
1392015-11-18T08:11:42 *** jcorgan has joined #bitcoin-core-dev
1402015-11-18T08:11:42 *** jcorgan has joined #bitcoin-core-dev
1412015-11-18T08:20:28 <GitHub147> [bitcoin] gmaxwell pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e8df8a5077df...f3ea48ad8b85
1422015-11-18T08:20:28 <GitHub147> bitcoin/master e855b01 Alex Morcos: Fix debug log message for block files
1432015-11-18T08:20:29 <GitHub147> bitcoin/master f3ea48a Gregory Maxwell: Merge pull request #7050...
1442015-11-18T08:20:38 <GitHub197> [bitcoin] gmaxwell closed pull request #7050: Fix debug log message for block files (master...nitfix) https://github.com/bitcoin/bitcoin/pull/7050
1452015-11-18T08:50:17 <GitHub3> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f3ea48ad8b85...7f8e90da335e
1462015-11-18T08:50:17 <GitHub3> bitcoin/master 013a364 MarcoFalke: [contrib] Delete test-patches
1472015-11-18T08:50:18 <GitHub3> bitcoin/master 7f8e90d Wladimir J. van der Laan: Merge pull request #7030...
1482015-11-18T08:50:26 <GitHub125> [bitcoin] laanwj closed pull request #7030: [contrib] Delete test-patches (master...MarcoFalke-2015-contribC) https://github.com/bitcoin/bitcoin/pull/7030
1492015-11-18T08:53:28 *** ParadoxSpiral has joined #bitcoin-core-dev
1502015-11-18T08:58:05 *** Guyver2 has joined #bitcoin-core-dev
1512015-11-18T09:27:28 *** d_t has quit IRC
1522015-11-18T09:40:12 *** cjcj has joined #bitcoin-core-dev
1532015-11-18T09:45:44 *** rubensayshi has joined #bitcoin-core-dev
1542015-11-18T10:08:49 *** jtimon has joined #bitcoin-core-dev
1552015-11-18T10:26:42 *** cjcj has quit IRC
1562015-11-18T10:31:41 *** randy-waterhouse has quit IRC
1572015-11-18T11:15:48 <GitHub184> [bitcoin] MarcoFalke opened pull request #7052: [qa] python-bitcoinrpc is no longer a subtree (master...MarcoFalke-2015-qaSubtree) https://github.com/bitcoin/bitcoin/pull/7052
1582015-11-18T11:16:06 *** MarcoFalke has joined #bitcoin-core-dev
1592015-11-18T11:51:42 <GitHub111> [bitcoin] jtimon opened pull request #7053: Globals: Remove a bunch of Params() from main.cpp before 0.12 (master...globals-chainparams-main) https://github.com/bitcoin/bitcoin/pull/7053
1602015-11-18T11:54:00 <GitHub25> [bitcoin] jtimon closed pull request #5970: DEPENDENT: Globals: Avoid calling Params() (master...chainparams_cleanup) https://github.com/bitcoin/bitcoin/pull/5970
1612015-11-18T12:33:46 *** guest234234 has joined #bitcoin-core-dev
1622015-11-18T12:47:35 *** rav3n_pl has quit IRC
1632015-11-18T13:21:38 *** MarcoFalke has quit IRC
1642015-11-18T13:23:14 *** dcousens has joined #bitcoin-core-dev
1652015-11-18T13:23:57 *** sdaftuar has joined #bitcoin-core-dev
1662015-11-18T13:28:47 <GitHub151> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7f8e90da335e...03403d8c0f3b
1672015-11-18T13:28:47 <GitHub151> bitcoin/master 513686d MarcoFalke: [qt] Use maxTxFee instead of 10000000
1682015-11-18T13:28:48 <GitHub151> bitcoin/master 03403d8 Jonas Schnelli: Merge pull request #6951...
1692015-11-18T13:28:52 <GitHub190> [bitcoin] jonasschnelli closed pull request #6951: [qt] Use maxTxFee instead of 10000000 (master...MarcoFalke-2015-qtMaxFee) https://github.com/bitcoin/bitcoin/pull/6951
1702015-11-18T13:46:23 *** zooko has quit IRC
1712015-11-18T13:54:07 <dcousens> out of interest, why username:password vs just token for the RPC?
1722015-11-18T13:54:13 <dcousens> they're both sent in the clear right?
1732015-11-18T13:56:25 *** Sanjay has joined #bitcoin-core-dev
1742015-11-18T14:01:00 <dcousens> ping gmaxwell: thoughts on https://github.com/bitcoin/bitcoin/pull/7044#issuecomment-157719965
1752015-11-18T14:01:45 *** zooko has joined #bitcoin-core-dev
1762015-11-18T14:02:17 *** MarcoFalke has joined #bitcoin-core-dev
1772015-11-18T14:06:40 <sipa> dcousens: if it was just a token, you wouldn't know which one to try
1782015-11-18T14:06:53 <sipa> dcousens: which is needed if you want strengthening
1792015-11-18T14:07:28 *** MarcoFalke has quit IRC
1802015-11-18T14:15:03 <jonasschnelli> dcousens: The hashing improves the attack vectors a little bit (tiny, i agree). Isn't the hashing is a little step towards a http-auth-digest likes authentication?
1812015-11-18T14:20:19 *** guest234234 has quit IRC
1822015-11-18T14:22:18 <gmaxwell> dcousens: username is useful because you can log it. also it fits into the http auth scheme.
1832015-11-18T14:22:31 <gmaxwell> of course, you could set all your usernames to x and just use passwords.
1842015-11-18T14:22:54 <gmaxwell> oh yea, sipas point too.
1852015-11-18T14:30:57 <jgarzik> jonasschnelli, http digest auth has been brought up quite a few times...
1862015-11-18T14:33:23 <jonasschnelli> jgarzik: Yes. I once looked at a possible implementation and decided that it's not worth doing it.
1872015-11-18T14:34:46 <gmaxwell> It's also not compatible with a surprising amount of stuff.
1882015-11-18T14:34:56 <jgarzik> the question really comes down to SSL, IMO. We are sending plaintext passwords. Is that a problem on localhost? no. Is that a problem on WAN? Yes. Should you be sending unencrypted RPC over WAN? No, use stunnel. So now you're back to not needing digest.
1892015-11-18T14:35:47 <gmaxwell> right.
1902015-11-18T14:40:00 <jonasschnelli> agreed
1912015-11-18T14:40:14 *** jgarzik has quit IRC
1922015-11-18T14:41:57 <wumpus> right: if you want secure RPC, tunnel it over something secure. Digest auth would be pretty pointless. Yes, you protect the auth, but your walletpassphrase will still go over the line unencrypted.
1932015-11-18T14:43:41 <wumpus> a challenge/response authentication system with the key storage/signer would be nice, as it'd also solve the 'walletpassphrase leaks into process memory' problem, but then again, you'd still be leaking actual commands over the line, which can (at the least) hurt privacy
1942015-11-18T14:43:59 <jonasschnelli> Right. #7044 (multiple RPC users) is interesting and it would allow to make permissions levels
1952015-11-18T14:44:02 <wumpus> so no, I think digest auth (or seemingly more secure) RPC auth is optimziing the wrong thing
1962015-11-18T14:44:44 <wumpus> but we want to get rid of the account stuff; what would be the use of multiple RPC accounts?
1972015-11-18T14:45:45 <wumpus> if you have multiple, completely separate wallets it could make sense I guess
1982015-11-18T14:46:28 <dcousens> gmaxwell: sure, but he could have been showing you a curl script with the rpcpassword in it
1992015-11-18T14:46:29 <jonasschnelli> companies that run services based on bitcoind could have multiple users, one that only accessed public data (like blocks / transaction), another user could have permissions to change things, like ban, etc.
2002015-11-18T14:46:37 <dcousens> the use case just isn't relevant imho, since, I doubt he minded lol
2012015-11-18T14:47:08 <gmaxwell> wumpus: not for seperate wallets, but access list management purposes; e.g. you with to withdraw one host, and add a new one. Otherwise you must disrupt everything tfor a flag-day change, and take an outage.
2022015-11-18T14:47:08 <wumpus> I don't want a complex, complete authentication and permissions system in bitcoin core
2032015-11-18T14:47:12 <wumpus> we're not buliding an OS here
2042015-11-18T14:47:29 <wumpus> it has to stop somewhere with importing all kind of ancillary concens
2052015-11-18T14:47:44 <dcousens> gmaxwell: I'm just not sure if its worth adding, since it'll give people impression that RPC is safer across a non-localized env? Weren't we discouraging that?
2062015-11-18T14:47:45 <jonasschnelli> Right. This could make things really complicate.
2072015-11-18T14:47:47 <wumpus> do you want access control? build an RPC proxy and have that check user permissions
2082015-11-18T14:48:02 <gmaxwell> Sure, but this is basic access management that any other network facing application provides (and basically any web api provides)
2092015-11-18T14:48:08 <wumpus> sigh...
2102015-11-18T14:48:08 <jonasschnelli> Yes. Thats a better approach.
2112015-11-18T14:48:09 *** wumpus has left #bitcoin-core-dev
2122015-11-18T14:48:09 <dcousens> (crap, just saw all the text above, notifications daemon died, reading now)
2132015-11-18T14:48:40 <dcousens> gmaxwell: isn't the point that this doesn't face the network?
2142015-11-18T14:48:55 <dcousens> All the docs have warnings to avoid that
2152015-11-18T14:49:10 <gmaxwell> It most certantly does-- not public networks, but even localhost is a network. Stunnel to it is network access.
2162015-11-18T14:49:19 <gmaxwell> If it's not facing things it cannot be used at all.
2172015-11-18T14:49:31 <dcousens> Sure, in which case, your on the machine, and, unless its a multi-user env, the hashing is pointless?
2182015-11-18T14:49:41 <gmaxwell> The secondary use, not enabled yet, is so that logging can also record which account did it, which is useful for debugging and forensics.
2192015-11-18T14:49:43 <dcousens> by multi-user env, I mean, the syystem itself, not bitcoind
2202015-11-18T14:50:41 <gmaxwell> The fact that the credentials are hased is a basic security practice that prevents leaking them when handling the credentials themselves; it's also a checklist requirement on many security standards (and not an unreasonable one)
2212015-11-18T14:51:30 <sipa> a single sha256 would suffice for that, though
2222015-11-18T14:52:23 <gmaxwell> It would. and if the credential is properly machine generated, then nothing more is actually useful.
2232015-11-18T14:53:33 <dcousens> gmaxwell: for logging, you could even just use hash(token), not untraceable
2242015-11-18T14:53:57 <dcousens> but even then, fine, user:pass still if you want make that easier
2252015-11-18T14:54:08 <gmaxwell> dcousens: doesn't really fit how people use things, the http auth already sends a user/password in any case.
2262015-11-18T14:54:21 *** Greyboy has joined #bitcoin-core-dev
2272015-11-18T14:54:37 *** MarcoFalke has joined #bitcoin-core-dev
2282015-11-18T14:54:56 <dcousens> gmaxwell: it just seems to me like the hashed password is implying a different security model to what has been advised up until now
2292015-11-18T14:55:26 <dcousens> and, aside from the auditing aspect, the username:password scheme is also misleading since its entirely in the clear
2302015-11-18T14:55:43 <dcousens> so, I guess, are we changing the security model? or keeping it the same
2312015-11-18T14:56:02 <gmaxwell> dcousens: almost 40 years of system security says otherwise.
2322015-11-18T14:56:28 <dcousens> gmaxwell: I have *no* issue with hashing, if we're protecting against local breach
2332015-11-18T14:56:34 *** Greyboy has quit IRC
2342015-11-18T14:56:35 *** Greyboy has joined #bitcoin-core-dev
2352015-11-18T14:56:42 <dcousens> but, AFAIK, we haven't protected that to date
2362015-11-18T14:56:43 <gmaxwell> Seriously, people get _fired_ for keeping unhashed password databases. When a breach results from one it is an embarssment that results in litigation.
2372015-11-18T14:57:57 <GitHub96> [bitcoin] ptschip opened pull request #7055: MAX_PROTOCOL_MESSAGE_LENGTH description (master...maxmessage) https://github.com/bitcoin/bitcoin/pull/7055
2382015-11-18T14:58:37 <dcousens> gmaxwell: does that make sense? I'm asking if we're wanting to be more permissive in telling users "local breach isn't catastrophic"
2392015-11-18T14:58:39 *** wumpus has joined #bitcoin-core-dev
2402015-11-18T14:58:57 <dcousens> To date, that isn't the case
2412015-11-18T15:00:53 <Greyboy> I'm late to this convo, but like security. may i ask what passwords you are discussing?
2422015-11-18T15:01:16 * instagibbs reading backlog
2432015-11-18T15:03:19 <instagibbs> Greyboy, https://github.com/bitcoin/bitcoin/pull/7044
2442015-11-18T15:04:31 <Greyboy> Thank you.
2452015-11-18T15:04:46 <dcousens> instagibbs: good PR :), just noticed it changes the security model, so, just wanted that to be clear, because it'll likely mean we'll need to update docs which currently mention "dont ever expose the RPC", and, "dont use SSL", etc
2462015-11-18T15:04:57 <instagibbs> yeah I'm thinking through all this too
2472015-11-18T15:05:32 <dcousens> and everything being in the clear made that pretty obvious, whereas, now this seems half-half, in what was previously a blatantly unprotected env
2482015-11-18T15:10:50 <dcousens> wumpus: lol: https://en.bitcoin.it/wiki/Enabling_SSL_on_original_client_daemon
2492015-11-18T15:10:55 <Greyboy> instagibbs, i know i'm new and my opinion is relatively worthless, but i think it's a good idea. my question is what percent of people who run bitcoin-core actually need this solution?
2502015-11-18T15:11:28 <wumpus> dcousens: hah - the 0.12 release notes specify how to use stunnel as well as transparant proxying, would be more useful to add that
2512015-11-18T15:12:14 <sipa> Greyboy: or what percentage doesn't run it because it misses features like this? :)
2522015-11-18T15:12:29 <instagibbs> dcousens, I understand and agree with the concerns, but if we want some basic audit logging, I don't think we should avoid basic improvements
2532015-11-18T15:12:30 <dcousens> or more recently
2542015-11-18T15:12:31 <dcousens> https://bitcoin.stackexchange.com/questions/39117/why-is-json-rpc-over-ssl-strongly-discouraged
2552015-11-18T15:12:33 <instagibbs> keep the warnings, in general
2562015-11-18T15:12:44 <dcousens> though, thats a bit circular
2572015-11-18T15:12:48 <instagibbs> btw where are the warnings today
2582015-11-18T15:12:55 *** zooko has quit IRC
2592015-11-18T15:13:11 <dcousens> instagibbs: my main question doesn't stand in the way of doing logging
2602015-11-18T15:13:34 <dcousens> instagibbs: I'm just talking about the hashing, I'm ACK on everything else, even the hashing, just, clarifying that that is intended
2612015-11-18T15:13:36 <Greyboy> On a different note, I was wondering what bitcoin-core does that requires it to connect to dyndns?
2622015-11-18T15:13:48 <instagibbs> dcousens, would you feel better if it was a single hash :P
2632015-11-18T15:14:45 <gmaxwell> Well that would also drop any worry about where the pbkdf2 comes from.
2642015-11-18T15:14:52 <instagibbs> ^
2652015-11-18T15:15:20 <instagibbs> warn people the same things we warned before (strong random keys only)
2662015-11-18T15:15:25 <gmaxwell> I don't consider the iterated hash actually important, it only potentially saves misusers from themselves slightly.
2672015-11-18T15:16:59 <dcousens> instagibbs: sure, I mean, the key-stretching seemed a bit unnecessary if we encourage what is already encouraged, strong random keys etc
2682015-11-18T15:17:17 <sipa> Greyboy: what?
2692015-11-18T15:17:39 <Greyboy> sipa, i was wondering why bitcoin-core tries to connect to dyndns and alerts my IDS
2702015-11-18T15:17:44 <instagibbs> I was going for "why not", but if it makes it more palatable, and strictly superior to today, and drops openssl dep....
2712015-11-18T15:17:52 <sipa> Greyboy: what version?
2722015-11-18T15:18:27 <dcousens> anyway, its late here, hope I didn't stir up to much of a hornet nest haha. Just, recognizing this was a changed expectation. maybe its a better one, hard to tell
2732015-11-18T15:18:35 <Greyboy> I don't have it on this machine or i would check, but i've seen this behaviour in many versions including recent ones.
2742015-11-18T15:18:45 <instagibbs> dcousens, no this is really appreciated. My first Core PR getting tons of attention from smart folks is nice
2752015-11-18T15:21:19 <dcousens> instagibbs: jgarzik summed it up nicely
2762015-11-18T15:21:20 <dcousens> "the question really comes down to SSL, IMO. We are sending plaintext passwords. Is that a problem on localhost? no. Is that a problem on WAN? Yes. Should you be sending unencrypted RPC over WAN? No, use stunnel. So now you're back to not needing digest.
2772015-11-18T15:21:24 <dcousens> "
2782015-11-18T15:22:12 <instagibbs> I get the concern. Single hash at least gets the "peek over shoulder" security model.
2792015-11-18T15:22:24 <sipa> and sha256 we already have
2802015-11-18T15:22:39 <instagibbs> *the village rejoices*
2812015-11-18T15:22:48 <gmaxwell> dcousens: the digest thing is specifically referring to http digest auth, orthorgonal issue.
2822015-11-18T15:22:57 <dcousens> yeah just realised, was re-writing that
2832015-11-18T15:23:17 <dcousens> the other issue was, well, read up about 10 lines, night all
2842015-11-18T15:23:49 *** dcousens has quit IRC
2852015-11-18T15:24:29 *** dcousens has joined #bitcoin-core-dev
2862015-11-18T15:26:51 <instagibbs> dcousens, cheers
2872015-11-18T15:29:46 <sipa> if we want PBKDF2 in there, it's trivial to implement, but if we're going with just hashed passwords... even easier\
2882015-11-18T15:34:08 *** Guyver2 has quit IRC
2892015-11-18T15:35:12 <wumpus> right - I think we don't want something with a lot of key stretching, as it'd increase the latency for RPC calls
2902015-11-18T15:35:46 <wumpus> but hashing the passwords in the config file is a great idea, not sure why we've not been doing that yet
2912015-11-18T15:36:14 <sipa> we can even do salting by using HMAC-SHA256, but without stretching
2922015-11-18T15:36:24 <sipa> though i don't think that adds much
2932015-11-18T15:36:40 <sipa> except "certification"
2942015-11-18T15:36:50 <gmaxwell> if the key is randomly generated as it should be, the salting does nothing.
2952015-11-18T15:37:03 <gmaxwell> but yea, it might fail some checkbox.
2962015-11-18T15:37:22 <wumpus> salting is pretty standard for password checks, yes
2972015-11-18T15:37:23 <gmaxwell> I suggest the format be such that we could add different types in the future and if we find out its a problem we can just make a new password type.
2982015-11-18T15:37:33 <gmaxwell> or keep the salt, I don't care. :)
2992015-11-18T15:37:35 * wumpus wonders what Tor uses for torcontrolpassword
3002015-11-18T15:37:56 <instagibbs> I was just going to drop salt, but either way is fine
3012015-11-18T15:38:14 <instagibbs> we're assuming people aren't foot-gunning bad passwords already
3022015-11-18T15:38:55 <gmaxwell> only reason to keep it is certificational, which is an issue here. But it might be answerable with "the salt is _in_ the password" :)
3032015-11-18T15:39:22 <instagibbs> Excuse my noobishness, what is "certificational" about a salt
3042015-11-18T15:39:42 <sipa> instagibbs: rainbow tables!
3052015-11-18T15:39:49 <wumpus> tor apparently does salting, hashing 'bla' repeatedly gives 16:DD2D9F35664C69D7604862845D52D4238AD5E702E8C9DF771108459C00 16:EC737D2A880C18996066882BAE3BA73E32543DA0F74D308572288E38F0 16:35D59809F0AA181160F34D05D6504F89958E2A1D393A29959937C577A0
3062015-11-18T15:40:07 <gmaxwell> thats a +1 for salt. :P
3072015-11-18T15:40:14 <instagibbs> damnit
3082015-11-18T15:40:20 * instagibbs re-adds
3092015-11-18T15:40:22 <sipa> http://www.antitree.com/onion-porn-hashedcontrolpassword/
3102015-11-18T15:40:22 <instagibbs> :P
3112015-11-18T15:40:35 <gmaxwell> instagibbs: the certificational part is that there are security checklists that ask if you're using salted hashed passwords.
3122015-11-18T15:40:46 <instagibbs> ohhhh like actualy certificates. Got it.
3132015-11-18T15:41:14 <wumpus> ah, thanks sipa, so 16 is the algorithm identifier, then follows the salt and password concatenated
3142015-11-18T15:41:23 <sipa> wumpus: nope, the 16 means it's encoded in hex :)
3152015-11-18T15:41:32 <sipa> seriously
3162015-11-18T15:41:44 <sipa> it's also the only thing supported
3172015-11-18T15:41:54 <wumpus> huh. ok
3182015-11-18T15:42:00 <sipa> but ehm... interoperability with tor would be nice imho
3192015-11-18T15:42:14 <wumpus> I'm confused with $3$ or /etc/shadow etc
3202015-11-18T15:42:22 <sipa> yeah
3212015-11-18T15:42:47 <gmaxwell> oh interesting actually thinking of using the same scheme as tor? is it complex to implement?
3222015-11-18T15:43:13 <sipa> 47 lines of python
3232015-11-18T15:43:15 <wumpus> we can just steal there code I guess :p although they may lean on OpenSSL as well
3242015-11-18T15:43:23 <Greyboy> *their
3252015-11-18T15:43:26 <sipa> it's just sha1
3262015-11-18T15:43:29 <sipa> iterated
3272015-11-18T15:45:14 <GitHub43> [bitcoin] ptschip closed pull request #7055: MAX_PROTOCOL_MESSAGE_LENGTH description (master...maxmessage) https://github.com/bitcoin/bitcoin/pull/7055
3282015-11-18T15:45:29 <wumpus> there's one difference though: tor only authenticates on opening the connection, bitcoin http authenticates for every request. So they can use more key stretching without creating command latency issues.
3292015-11-18T15:45:35 <gmaxwell> well we already have a sha1 dependency.
3302015-11-18T15:45:58 <instagibbs> hmm
3312015-11-18T15:46:07 <instagibbs> link to related tor source?
3322015-11-18T15:46:52 *** Guyver2 has joined #bitcoin-core-dev
3332015-11-18T15:49:00 <wumpus> https://github.com/torproject/tor/blob/master/src/or/main.c#L3210
3342015-11-18T15:50:07 <wumpus> https://github.com/torproject/tor/blob/master/src/common/crypto_s2k.c
3352015-11-18T15:50:36 <wumpus> but the python script sipa linked is more easily readable
3362015-11-18T15:50:53 *** Greyboy has quit IRC
3372015-11-18T15:55:12 <sipa> oh, it has a 1-byte iteration count which uses exponential notation
3382015-11-18T15:55:21 <sipa> 4 bits of mantissa, and 4 bits of exponent
3392015-11-18T15:55:23 <wumpus> yes, the 'indicator'
3402015-11-18T15:55:40 <sipa> up to max 65M iterations
3412015-11-18T15:56:02 <sipa> and min 1024
3422015-11-18T15:56:16 <gmaxwell> meh. thats kinda sad.
3432015-11-18T15:56:34 <gmaxwell> Again I remind: so long as the format has a type field we could add other types in the future if needed.
3442015-11-18T15:57:27 <sipa> the tor thing actually doesn't (though it starts with an undefined "16:" prefix, which could be redefined into being a type, if necessary, i guess)
3452015-11-18T15:59:04 <wumpus> normally iteration is done by feeding the previous hash back into the hasher, not in this case
3462015-11-18T15:59:25 <wumpus> "hash the salty password as many times as the length of the password divides into the count value"
3472015-11-18T15:59:47 <wumpus> so it defines (roughly) how many bytes will be passed into the hash function
3482015-11-18T16:00:19 <wumpus> yes, the 16: in case of tor is kind of a red herring, I think anyone would have expected it to be an algorithm id
3492015-11-18T16:01:38 *** MarcoFalke has quit IRC
3502015-11-18T16:12:05 *** MarcoFalke has joined #bitcoin-core-dev
3512015-11-18T16:13:36 <gmaxwell> wumpus: what are your thoughts on trying to make the final push to get openssl out of bitcoind? I think all code needed for it has already been written, then abandoned at varrious points in the past. The two big blockers were rpcssl and ecc, and both are resolved now.
3522015-11-18T16:14:07 <wumpus> we should definitely do it, but for 0.13 imo
3532015-11-18T16:14:24 <gmaxwell> OKAY, yes, that seems more reasonable I guess.
3542015-11-18T16:14:38 <sipa> agree
3552015-11-18T16:14:46 <wumpus> the deadline for 0.12 is getting closer and I'm sure there is high-prio stuff that still needs to get in
3562015-11-18T16:14:56 <gmaxwell> RNG replacement isn't a small project or one we'd want to do poorly. I think if not for that we could do the rest.
3572015-11-18T16:14:58 <wumpus> but as soon as 0.12 is branched off let's kill openssl
3582015-11-18T16:15:26 <sipa> i'll try to find the time after branching to turn the fortuna/entropy gathering code into an external library
3592015-11-18T16:15:51 <wumpus> I also have that on my list sipa, but feel free to do it :)
3602015-11-18T16:15:55 <gmaxwell> I am so tired of spurrious valgrind errors on various systems because after all these years openssl can't be bothered to not avoid undefined behavior in integrity critical RNG code. :-/
3612015-11-18T16:16:40 <wumpus> yes - RNG is kind of critical in our case
3622015-11-18T16:18:21 <wumpus> foremost importance is that if no good entropy source can be found on a system, it's better to fail than continue with a lousy one, many wallets made that mistake
3632015-11-18T16:19:01 <gmaxwell> yes, if you have no good entropy and know it. Irritatingly this failure mode is not as common as it should be. :(
3642015-11-18T16:19:04 <sipa> "Oh, you're using Internet Explorer 3? I'll just go use { return 4; // decided by a fair dice roll } as entry!
3652015-11-18T16:19:25 <wumpus> exactly :D
3662015-11-18T16:19:36 <sipa> s/ry/ropy/
3672015-11-18T16:19:46 <gmaxwell> openssl and libressl both fail that test though, even if they can't get any strong OS rng input, they keep on trucking.
3682015-11-18T16:19:49 <wumpus> gmaxwell: well I mean if the OS APIs somehow fail
3692015-11-18T16:21:20 <gmaxwell> wumpus: yea. the design Sipa had done before was also targeting improving the strength when they silently fail (happened somewhat recently in several OSes, :( ).. but certantly if the strong input fails it should full stop.
3702015-11-18T16:21:26 <wumpus> sure, it may happen that the OS happily returns values which happen to be perfectly predictable because it's only based on the timestamp of VM startup and timings on synthethic devices
3712015-11-18T16:22:42 <gmaxwell> for the GUI you can always pop up a initial start friendly game of 2048 to record user timings from; but alas, the enviroments most likely to be entropy starved are not the ones with a UI.
3722015-11-18T16:22:53 <wumpus> lol!
3732015-11-18T16:23:26 <wumpus> blockchain 2048
3742015-11-18T16:24:12 <wumpus> "combine the blocks to double the block reward"
3752015-11-18T16:24:27 <sipa> freicoin can use that 2048 version whose pawn disappear after a certain time...
3762015-11-18T16:24:51 <wumpus> hehehe
3772015-11-18T16:26:08 <gmaxwell> I've wondered what simple games extract the most entropy from users in the least time.
3782015-11-18T16:26:34 <sipa> gmaxwell: things with many keypresses, i'm sure :)
3792015-11-18T16:27:36 <gmaxwell> maybe a flashcard typing game.
3802015-11-18T16:28:07 <gmaxwell> oh that shows captcha flashcards so you make mistakes. :P
3812015-11-18T16:32:35 <wumpus> it's a nice change from the usual 'bash on the keyboard and move the mouse around like you have parkinson disease'
3822015-11-18T16:33:09 <sipa> LOL
3832015-11-18T16:37:03 <gmaxwell> wumpus: for a procedure at work I want to just instruct people to roll dice. actual dice rolls produce less entropy than the timing from typing them in, but its fun to do.
3842015-11-18T16:40:28 <wumpus> or combine the ideas, add a game that involves throwing dice
3852015-11-18T16:43:03 <wumpus> shuffling a pack of cards may also work, but it's so much work it enter it all :)
3862015-11-18T16:47:43 <wumpus> so sneaky, would have been so easy to miss "This should not be called with the 2 historical coinbase duplicate pairs because the new coins are marked fresh, and in the event the duplicate coinbase was spent before a flush, the now pruned coins would not properly overwrite the first coinbase of the pair. Simultaneous modifications are not allowed."
3872015-11-18T16:48:01 <wumpus> (re: #6932)
3882015-11-18T16:50:14 *** treehug88 has joined #bitcoin-core-dev
3892015-11-18T16:51:09 *** zooko has joined #bitcoin-core-dev
3902015-11-18T16:53:52 <GitHub151> [bitcoin] laanwj pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/03403d8c0f3b...73fa5e604356
3912015-11-18T16:53:53 <GitHub151> bitcoin/master 14470f9 Alex Morcos: ModifyNewCoins saves database lookups...
3922015-11-18T16:53:53 <GitHub151> bitcoin/master 03c8282 Alex Morcos: Make CCoinsViewTest behave like CCoinsViewDB
3932015-11-18T16:53:54 <GitHub151> bitcoin/master 1cf3dd8 Alex Morcos: Add unit test for UpdateCoins
3942015-11-18T16:53:57 <GitHub31> [bitcoin] laanwj closed pull request #6932: ModifyNewCoins saves database lookups (master...newCoinsThinAir) https://github.com/bitcoin/bitcoin/pull/6932
3952015-11-18T16:55:28 <gmaxwell> \O/
3962015-11-18T16:56:36 <sipa> someone should put a logprint in CCoinsViewDb on read and write, and reindex with max size dbcache
3972015-11-18T16:56:57 <sipa> there should only be 1 db operation per block left (the coinsbase overwrite)
3982015-11-18T16:57:49 <gmaxwell> save the logging until we've gotten that one fixed too? :)
3992015-11-18T16:58:43 <sipa> that needs #5967
4002015-11-18T17:01:59 *** davec has quit IRC
4012015-11-18T17:06:42 *** davec has joined #bitcoin-core-dev
4022015-11-18T17:07:19 *** MarcoFalke has quit IRC
4032015-11-18T17:22:22 <morcos> here you go:
4042015-11-18T17:22:27 <GitHub106> [bitcoin] morcos opened pull request #7056: Save last db read (master...saveLastDBRead) https://github.com/bitcoin/bitcoin/pull/7056
4052015-11-18T17:27:37 <gmaxwell> bam. theres to all the people shed painting over database engines, so long as you never stop your node the database no longer matters. :P
4062015-11-18T17:28:22 <sipa> gmaxwell: and have infinite dbcache
4072015-11-18T17:28:46 <gmaxwell> doesn't everyone have 128GB ram?
4082015-11-18T17:28:49 *** d_t has joined #bitcoin-core-dev
4092015-11-18T17:30:04 <wumpus> gmaxwell: is the UTXO set that big now? :(
4102015-11-18T17:30:35 <sipa> wumpus: he runs 20 bitcoind in parallel, i'm sure
4112015-11-18T17:30:38 <gmaxwell> wumpus: it's about 6GB in ram right now. Less with sipa's sold script vector replacement thing.
4122015-11-18T17:30:40 <wumpus> hehhe
4132015-11-18T17:31:16 <gmaxwell> right now I have a host importing one of the satoshi dice addresses into the wallet...
4142015-11-18T17:31:44 <gmaxwell> this might not have been a good idea; and .. I picked one with impossibly low odds too.
4152015-11-18T17:31:55 *** rubensayshi has quit IRC
4162015-11-18T17:32:21 <gmaxwell> (does anyone have an address to sugget importing that has a lot of txn but not ALL the transactions?)
4172015-11-18T17:34:48 <btcdrak> gmaxwell: you can find a bunch here https://www.walletexplorer.com
4182015-11-18T18:00:14 *** Greyboy has joined #bitcoin-core-dev
4192015-11-18T18:07:50 <davec> gmaxwell: if you still need one I typically use 14WCrzmqVCD7Thf79vutWTB6y1hUc8pP19 for such stress tests since it has ~62k
4202015-11-18T18:09:21 <gmaxwell> davec: thanks.
4212015-11-18T18:13:26 *** paveljanik has joined #bitcoin-core-dev
4222015-11-18T18:13:26 *** paveljanik has joined #bitcoin-core-dev
4232015-11-18T18:16:46 <gmaxwell> hm. this might be a bit suboptimal; reindex with dbcach=8192. leave sitting for a day. start a wallet rescan and decide that I'd rather not, kill -9 the process. start back up with a deleted wallet.. chainstate is at height=174109
4242015-11-18T18:27:50 <wumpus> need more db flushes :)
4252015-11-18T18:28:16 <sipa> if we had flushes that didn't wipe everything, we could do them more frequently
4262015-11-18T18:28:49 <wumpus> I also once lost almost a whole sync in a kill -9, that's the cost of using -dbcache=8000 I guess. On the other hand it's great to be able to do a sync with almost no i/o.
4272015-11-18T18:30:23 <wumpus> with the standard dbcache setting it's not an issue
4282015-11-18T18:30:44 <gmaxwell> heh, yea, it wasn't a complaint so much as a "Oh, yea. thats ... not technically surprising."
4292015-11-18T18:32:34 <wumpus> does it still do a flush after IsInitialBlockDownload is complete?
4302015-11-18T18:33:10 <gmaxwell> I think it does, in my case I'd been reindexing and it doesn't on reindex complete AFAIK.
4312015-11-18T18:33:35 <wumpus> hmm maybe it should
4322015-11-18T18:42:14 <gmaxwell> I think so.
4332015-11-18T18:43:04 <wumpus> (I can't think of a reason why it'd be a bad idea to do a one-time flush after it completes indexing)
4342015-11-18T18:44:23 <gmaxwell> the only downside I can think of is that right now doing that will blow out the cache.
4352015-11-18T18:44:31 *** zooko has quit IRC
4362015-11-18T18:46:01 *** zooko has joined #bitcoin-core-dev
4372015-11-18T18:54:09 *** Greyboy has quit IRC
4382015-11-18T18:54:45 *** evoskuil has joined #bitcoin-core-dev
4392015-11-18T19:15:22 *** zooko has quit IRC
4402015-11-18T19:45:48 *** randy-waterhouse has joined #bitcoin-core-dev
4412015-11-18T20:05:48 *** trippysalmon has joined #bitcoin-core-dev
4422015-11-18T20:32:39 *** belcher has joined #bitcoin-core-dev
4432015-11-18T21:20:42 *** arubi has quit IRC
4442015-11-18T21:28:36 *** arubi has joined #bitcoin-core-dev
4452015-11-18T21:30:29 <instagibbs> For debugging core where does one put the flag to do different optimization levels?
4462015-11-18T21:47:04 <phantomcircuit> instagibbs, CFLAGS="" ./configure
4472015-11-18T21:47:16 <phantomcircuit> grep for CFLAGS in config.log to see what they are by default
4482015-11-18T21:48:20 <BlueMatt> morcos: so I'm kinda confused about the proposed "score index" to mempool
4492015-11-18T21:48:39 <BlueMatt> is there any case where we really want a feerate index and do not intend to use it as "priority to get into next block"
4502015-11-18T21:48:50 <phantomcircuit> Luke-Jr, posted performance numbers in 6851
4512015-11-18T21:48:52 <BlueMatt> it seems to me we shouldnt need to indexes for this
4522015-11-18T21:48:58 <morcos> BlueMatt: Well it probably wont' last as its current incarnation anyway
4532015-11-18T21:49:10 <morcos> Why 2 indices?
4542015-11-18T21:49:19 <morcos> we definitely need a mining index and an eviction index?
4552015-11-18T21:49:40 <morcos> are those the 2 you are referring to?
4562015-11-18T21:49:46 <phantomcircuit> yeah see this is what i was worried about
4572015-11-18T21:49:51 <phantomcircuit> "just add another index"
4582015-11-18T21:50:26 <BlueMatt> oh, this is the inverse index of the one we have? yea, nvm, ignore me
4592015-11-18T21:50:35 <morcos> phantomcircuit: we right now have 3 indices. time and descendant package fee rate for eviction. and hash for lookup.
4602015-11-18T21:50:48 <morcos> we have to have a 4th index for mining
4612015-11-18T21:51:03 <morcos> because descendant package fee rate is useless
4622015-11-18T21:51:22 <phantomcircuit> probably dont actually need an index for time if the time based eviction has 1 hour granularity
4632015-11-18T21:51:48 <morcos> phantomcircuit: agreed, we could probably skip that index, and just scan the whole mempool periodically, it should be super fast
4642015-11-18T21:52:07 <morcos> but since it doesn't change it probably also doesnt hurt
4652015-11-18T21:52:23 <phantomcircuit> memory usage?
4662015-11-18T21:53:02 <morcos> BlueMatt: but sdaftuar was trying to add feeDelta prioritisation to your eviction code and realized that the way I did score made it hard. because he doesn't want the modified fee rate. he wants the modified fee
4672015-11-18T21:53:44 <morcos> So I think he will change it around, but the index will still be sorting by the same thing for mining, the internal data structure might just be a bit different
4682015-11-18T21:53:51 <BlueMatt> morcos: yea, fair enough...I do wonder, however, if we're gonna do a modified-feerate thing for tx selection, we probably dont want any indexes based on feerate
4692015-11-18T21:53:55 <BlueMatt> just base everything on modified feerate
4702015-11-18T21:54:06 <morcos> phantomcircuit: sure, i suppose, its really rather minor memory usage though
4712015-11-18T21:54:14 <morcos> BlueMatt: yes, thats what will happen
4722015-11-18T21:54:19 <BlueMatt> even if its modified-feereate-with-package-tracking
4732015-11-18T21:54:20 <BlueMatt> yea, ok
4742015-11-18T21:54:32 <morcos> But we need to store the actual fee without modification as well (even though there is no index on it)
4752015-11-18T21:54:41 <morcos> because it is now required for block construction
4762015-11-18T21:54:50 <morcos> so it doesn't have to be recalculated
4772015-11-18T21:56:15 *** Guyver2 has quit IRC
4782015-11-18T21:56:17 <phantomcircuit> morcos, i assumed as much, but do we have actual numbers for it?
4792015-11-18T21:57:11 <morcos> phantomcircuit: memory usage? i think sipa was approximating 3 pointers per entry per index, but i'm not sure where he got that?
4802015-11-18T21:57:54 <morcos> sigh... just realized i forgot to fix that for my 4th index!
4812015-11-18T21:59:07 <morcos> sipa: is that right, was it 3
4822015-11-18T21:59:17 <morcos> 3 pointers per index, or did i just make that up?
4832015-11-18T22:04:40 *** paveljanik has quit IRC
4842015-11-18T22:09:02 *** pmienk has joined #bitcoin-core-dev
4852015-11-18T22:11:08 *** treehug88 has quit IRC
4862015-11-18T22:52:55 <phantomcircuit> morcos, wait why is the descendent fee index useless for mining?
4872015-11-18T22:53:22 <phantomcircuit> oh because you need to also track ancestor fees
4882015-11-18T22:53:24 <phantomcircuit> ha
4892015-11-18T23:20:25 *** trippysalmon has quit IRC
4902015-11-18T23:33:09 <dcousens> gmaxwell: I wonder if the auth aspect will play into the RPC v REST speed, RPC can batch commands/per request which makes me think it'll lead for a while yet
4912015-11-18T23:34:01 <dcousens> though, without benchmarks, its hard to tell whether univalue is a factor too
4922015-11-18T23:35:38 <phantomcircuit> hmm
4932015-11-18T23:36:03 <phantomcircuit> there's no way for us to have more than 1 handle open for the berkeley db in the wallet code
4942015-11-18T23:36:18 <gmaxwell> univalue is pretty quick, at least my impression from AFLing it.
4952015-11-18T23:36:42 <phantomcircuit> so Flush is really only used for durability, right?
4962015-11-18T23:39:23 <phantomcircuit> ha
4972015-11-18T23:39:41 <phantomcircuit> i was going to ask why nMinutes is only set in CDB::Flush when we're in readonly mode, but satoshi wrote that code
4982015-11-18T23:40:24 <phantomcircuit> why would we be flushing the database at all if we were in read only mode?
4992015-11-18T23:40:44 <phantomcircuit> damn it satoshi come back so i can ask mundane questions about code
5002015-11-18T23:41:23 *** andytoshi has quit IRC
5012015-11-18T23:41:24 *** andytoshi has joined #bitcoin-core-dev
5022015-11-18T23:50:46 *** ParadoxSpiral has quit IRC
5032015-11-18T23:59:37 <phantomcircuit> huh