12017-02-16T00:10:34 *** Chris_Stewart_5 has quit IRC
22017-02-16T00:13:00 *** Chris_Stewart_5 has joined #bitcoin-core-dev
32017-02-16T00:48:56 *** abpa has quit IRC
42017-02-16T01:21:05 <bitcoin-git> [bitcoin] jmcorgan opened pull request #9774: Enable host lookups for -proxy and -onion parameters (master...hostname-lookups) https://github.com/bitcoin/bitcoin/pull/9774
52017-02-16T01:27:37 *** MarcoFalke has joined #bitcoin-core-dev
62017-02-16T01:41:08 *** MarcoFalke has quit IRC
72017-02-16T01:54:00 *** Ylbam has quit IRC
82017-02-16T02:11:09 *** chjj has quit IRC
92017-02-16T03:10:40 *** jtimon has quit IRC
102017-02-16T03:11:05 *** Victor_sueca has joined #bitcoin-core-dev
112017-02-16T03:14:14 *** Victorsueca has quit IRC
122017-02-16T03:57:43 *** goksinen has quit IRC
132017-02-16T03:58:31 *** goksinen has joined #bitcoin-core-dev
142017-02-16T04:03:27 *** goksinen has quit IRC
152017-02-16T04:07:46 *** PRab has joined #bitcoin-core-dev
162017-02-16T04:16:17 *** PRab has quit IRC
172017-02-16T04:19:25 *** roasbeef_ is now known as roasbeef
182017-02-16T04:53:48 *** e4xit has quit IRC
192017-02-16T04:54:45 *** e4xit has joined #bitcoin-core-dev
202017-02-16T05:33:20 *** sanada has joined #bitcoin-core-dev
212017-02-16T05:47:57 *** justanotheruser has joined #bitcoin-core-dev
222017-02-16T05:58:56 *** Victor_sueca has quit IRC
232017-02-16T06:00:03 *** Victor_sueca has joined #bitcoin-core-dev
242017-02-16T06:23:09 *** Victor_sueca is now known as Victorsueca
252017-02-16T06:31:15 *** cannon-c has joined #bitcoin-core-dev
262017-02-16T06:42:44 *** Ylbam has joined #bitcoin-core-dev
272017-02-16T07:18:24 *** d9b4bef9 has quit IRC
282017-02-16T07:19:07 *** d9b4bef9 has joined #bitcoin-core-dev
292017-02-16T07:37:31 *** BashCo has quit IRC
302017-02-16T08:02:55 *** BashCo has joined #bitcoin-core-dev
312017-02-16T08:09:23 *** Guest33433 has joined #bitcoin-core-dev
322017-02-16T08:16:24 *** cannon-c has quit IRC
332017-02-16T08:22:41 *** cannon-c has joined #bitcoin-core-dev
342017-02-16T08:35:38 *** Guest33433 has quit IRC
352017-02-16T08:42:20 *** goksinen has joined #bitcoin-core-dev
362017-02-16T08:46:57 *** goksinen has quit IRC
372017-02-16T08:53:40 *** Giszmo has quit IRC
382017-02-16T09:13:15 *** goksinen has joined #bitcoin-core-dev
392017-02-16T09:13:52 *** BashCo_ has joined #bitcoin-core-dev
402017-02-16T09:17:38 *** BashCo has quit IRC
412017-02-16T09:17:57 *** goksinen has quit IRC
422017-02-16T09:21:00 *** Guyver2 has joined #bitcoin-core-dev
432017-02-16T09:23:09 <wumpus> I think today is a good day to wrap up 0.14
442017-02-16T09:24:08 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7a93af8340d9...1e92e041ddc8
452017-02-16T09:24:09 <bitcoin-git> bitcoin/master ba803ef Suhas Daftuar: Harden against mistakes handling invalid blocks...
462017-02-16T09:24:09 <bitcoin-git> bitcoin/master 1e92e04 Wladimir J. van der Laan: Merge #9765: Harden against mistakes handling invalid blocks...
472017-02-16T09:24:29 <bitcoin-git> [bitcoin] laanwj closed pull request #9765: Harden against mistakes handling invalid blocks (master...fix-checkblock-call) https://github.com/bitcoin/bitcoin/pull/9765
482017-02-16T09:24:46 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/1e92e041ddc8...f8af89a91820
492017-02-16T09:24:46 <bitcoin-git> bitcoin/master 6c5427d Wladimir J. van der Laan: wallet: Prevent "overrides a member function but is not marked 'override'" warnings...
502017-02-16T09:24:47 <bitcoin-git> bitcoin/master f8af89a Wladimir J. van der Laan: Merge #9764: wallet: Prevent "overrides a member function but is not marked 'override'" warnings...
512017-02-16T09:25:06 <bitcoin-git> [bitcoin] laanwj closed pull request #9764: wallet: Prevent "overrides a member function but is not marked 'override'" warnings (master...2017_02_wallet_inconsistent_missing_override) https://github.com/bitcoin/bitcoin/pull/9764
522017-02-16T09:30:54 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f8af89a91820...e43a58514dd3
532017-02-16T09:30:54 <bitcoin-git> bitcoin/master 07afcd6 Russell Yanofsky: Add missing cs_wallet lock that triggers new lock held assertion...
542017-02-16T09:30:55 <bitcoin-git> bitcoin/master e43a585 Wladimir J. van der Laan: Merge #9771: Add missing cs_wallet lock that triggers new lock held assertion...
552017-02-16T09:31:15 <bitcoin-git> [bitcoin] laanwj closed pull request #9771: Add missing cs_wallet lock that triggers new lock held assertion (master...pr/loadlock) https://github.com/bitcoin/bitcoin/pull/9771
562017-02-16T09:35:31 <luke-jr> should #9524 be marked 0.14?
572017-02-16T09:35:32 <gribble> https://github.com/bitcoin/bitcoin/issues/9524 | rpc: Dont FlushStateToDisk when pruneblockchain(0) by MarcoFalke · Pull Request #9524 · bitcoin/bitcoin · GitHub
582017-02-16T09:36:45 <wumpus> I don't know. I'd say let's not add more stuff now and try to get a rc1 wrapped up
592017-02-16T09:37:23 <wumpus> it's a release candidate to get more testing
602017-02-16T09:37:32 <wumpus> issues will always be found in RCs of major releases
612017-02-16T09:37:36 <luke-jr> sure
622017-02-16T09:37:42 <gmaxwell> 9524 looks harmless.
632017-02-16T09:37:49 <wumpus> we can keep pushing this forward indefinitely and just not do releases anymore :/
642017-02-16T09:37:56 <luke-jr> not saying it needs to get into rc1, just hoping to avoid thigns being overlooked before final
652017-02-16T09:37:58 <gmaxwell> wumpus: would solve so many problems!
662017-02-16T09:38:17 <gmaxwell> luke-jr: I don't think that should go into 0.14.0, unless it has some effect I'm missing.
672017-02-16T09:38:37 <wumpus> it's under-discussed and under-reviewed
682017-02-16T09:38:49 <gmaxwell> by harmless I mean don't do it.
692017-02-16T09:38:54 <gmaxwell> The question is "what bad expirence will the user or network have as a result of this not going in" if the answer is none we shouldn't even be remotely considering it now.
702017-02-16T09:39:10 <gmaxwell> I mean the thing it fixes does not meet the above test ^, sorry for being unclear.
712017-02-16T09:39:15 <wumpus> oh I agree then :)
722017-02-16T09:39:15 <luke-jr> pruneblockchain(0) would probably delete a lot of data the user doesn't intend
732017-02-16T09:39:29 <gmaxwell> okay, that would be bad.
742017-02-16T09:39:51 <gmaxwell> wait, the argument there is what, /me looks
752017-02-16T09:40:18 <luke-jr> at least as I understand it, the 0 indicates the node should automatically prune up to tip minus saved-amount
762017-02-16T09:40:23 <wumpus> your are misunderstanding it luke-jr
772017-02-16T09:40:28 <wumpus> argument 0 means *do nothing*
782017-02-16T09:40:35 <wumpus> it doesn't delete anythhing in that case
792017-02-16T09:40:37 <wumpus> it just flushes to disk
802017-02-16T09:40:42 <wumpus> who cares :)
812017-02-16T09:41:00 <gmaxwell> That was what I got out of the PR.
822017-02-16T09:41:14 <wumpus> that PR bypasses the flush, I didn't regard that as very important as it's a NOOP anyway
832017-02-16T09:41:19 <gmaxwell> If it did what luke said-- prune as hard as allowed--, that would be worth fixing before final.
842017-02-16T09:41:26 <luke-jr> what am I missing?
852017-02-16T09:41:57 <wumpus> what you are missing is that pruneblockchain(0) already does nothing besides a FlushToDIsk
862017-02-16T09:42:00 <luke-jr> FlushStateToDisk with 0 triggers FindFilesToPrune(setFilesToPrune, chainparams.PruneAfterHeight()); etc
872017-02-16T09:42:06 <wumpus> that PR takes away the FlushToDisk
882017-02-16T09:42:24 <wumpus> "It might cause unwanted bugs if someone refactors the code in the future."
892017-02-16T09:42:29 <wumpus> that's all
902017-02-16T09:43:51 <wumpus> which we can consider for 0.15, but there will be no refactors of that code in the future of 0.14, so it's pointless there
912017-02-16T09:43:52 * gmaxwell risks his laptop node.
922017-02-16T09:44:27 *** goksinen has joined #bitcoin-core-dev
932017-02-16T09:47:24 <gmaxwell> luke-jr: so just calling that with 0 throws cannot prune because not in prune mode.
942017-02-16T09:47:41 <luke-jr> what manual pruning is enabled?
952017-02-16T09:47:56 <gmaxwell> trying.
962017-02-16T09:48:35 <gmaxwell> pfft. it's yelling at me because of txindex. @#$@
972017-02-16T09:48:42 <luke-jr> >_<
982017-02-16T09:48:57 *** goksinen has quit IRC
992017-02-16T09:49:13 <gmaxwell> so to do that test I'd have to reindex which would probably take days on this thing. :-/ and my other nodes are busy now with important test.s
1002017-02-16T09:49:42 <gmaxwell> I will test before 0.14 release however, if no one else does.
1012017-02-16T09:49:55 <Victorsueca> need testing?
1022017-02-16T09:50:10 <gmaxwell> if you don't mind blowing away your blockchain if luke is right.
1032017-02-16T09:50:14 <wumpus> ideally *disabling* txindex would not require a rescan, but meh :) I'll test it
1042017-02-16T09:50:27 * wumpus well test, has about 50 copies of the blockchain anyhow
1052017-02-16T09:50:50 <gmaxwell> Start a node without txindex with prune=1, then run pruneblockchain 0 and verify all your blocks didn't just suddenly go away (e.g. freeing up 100 GB of disk space).
1062017-02-16T09:51:37 <Victorsueca> ummm, I'll make a backup of my current chain, it probably won't be in time to test this one, but i'll be ready for future chain-blowing PRs
1072017-02-16T09:52:33 *** JackH has joined #bitcoin-core-dev
1082017-02-16T09:52:41 <wumpus> possibly no need to even copy, would hard-linking all but the latest block file work?
1092017-02-16T09:52:47 <luke-jr> not sure we have a way to restore such a backup without the chainstate
1102017-02-16T09:52:55 <wumpus> okay, definitely not going to try that at the same time
1112017-02-16T09:53:36 <wumpus> luke-jr: why is it a problem in this case to copy the chainstate too?
1122017-02-16T09:53:55 <luke-jr> wumpus: it might not be, just saying in case copying it also wasn't obvious
1132017-02-16T09:53:55 <gmaxwell> FS with cow would be nice for some of these tests.
1142017-02-16T09:54:18 * luke-jr does have a CoW filesystem, but also has txindex enabled etc.
1152017-02-16T09:54:34 <wumpus> luke-jr: if you don't copy the chainstate, all it can do is a reindex
1162017-02-16T09:55:14 <luke-jr> I suppose a unit test could check this, but seems probably too bug-specific
1172017-02-16T09:55:24 <luke-jr> s/unit/rpc/
1182017-02-16T09:55:46 <wumpus> well if this turns out to be a regression then a regression test could be added for it, but if this never was a bug in the first place...
1192017-02-16T09:55:59 <luke-jr> 0.13 didn't support manual pruning afaik
1202017-02-16T09:56:01 <wumpus> then we're just chasing ghosts
1212017-02-16T09:57:12 <wumpus> though testing it in regtest instead of a live node is a very good idea
1222017-02-16T09:59:47 <Victorsueca> so, if I copy the blocks and chainstate folder it should be enough right?
1232017-02-16T09:59:59 <wumpus> yes
1242017-02-16T10:00:18 <luke-jr> Victorsueca: may need to be sure the node is stopped when you copy
1252017-02-16T10:00:31 <Victorsueca> ok
1262017-02-16T10:01:16 <wumpus> oh absolutely, don't do anything low-level with the data files without stopping, that shouldn't have to be said
1272017-02-16T10:02:07 <Victorsueca> yeah it's stopped, the windows that tells me to wait disappeared too
1282017-02-16T10:02:17 <wumpus> on windows, copying *from* the bitcoin folder while bitcoin core is running can even cause it to crash
1292017-02-16T10:02:30 <gmaxwell> stupid file locking
1302017-02-16T10:02:34 <Victorsueca> rlly? copying?
1312017-02-16T10:02:56 <wumpus> https://github.com/bitcoin/bitcoin/issues/8250
1322017-02-16T10:03:42 <wumpus> not sure why, I'm not aware ldb does anything special while opening the files
1332017-02-16T10:03:56 <Victorsueca> maybe something related with atomic locks?
1342017-02-16T10:04:33 * luke-jr found it curious to see a Windows user actively using a "file unlocker" program like it was normal
1352017-02-16T10:05:31 <Victorsueca> windows file locking is great at preventing programs from crashing due to files that where expected to be there and now are not
1362017-02-16T10:05:40 <Victorsueca> but it's stupid in most cases
1372017-02-16T10:05:50 <wumpus> yes, it's great at preventing programs from crashing... we see that :-)
1382017-02-16T10:05:59 <Victorsueca> lol
1392017-02-16T10:06:16 <Victorsueca> let's just say the remedy is worst than the ill
1402017-02-16T10:06:29 <Victorsueca> it could be way better implemented
1412017-02-16T10:06:58 <wumpus> it's probably not that bad if you take extensive time to study the APIs involved and use them as they are supposed to be, but who has time to do native development for every special snowflake platform :/
1422017-02-16T10:07:07 <midnightmagic> pure snapshotting would work just fine for this, if you could convince the software to lock all files prior to the snapshot, or guarantee a write barrier
1432017-02-16T10:08:15 <wumpus> the annoying thing with microsoft has always been that they take existing concepts then change them slightly, just enough to be incompatible and lock you in a bit
1442017-02-16T10:08:26 <wumpus> POSIX? forget it
1452017-02-16T10:10:04 <midnightmagic> windows file locking works fine for monolithic files that are only modified by the software modifying them, even in 10,000+ user database environments far busier than I suspect bitcoin's databases will ever be.
1462017-02-16T10:10:06 <luke-jr> I thought Windows was POSIX compatible?
1472017-02-16T10:10:58 <midnightmagic> Windows has POSIX emulation layers but they all operate differently than UNIX. For example, when a file is locked, it is a much harder lock than its POSIX-style equivalents on other systems.
1482017-02-16T10:11:16 <wumpus> it tries, a bit, the devil is in the details
1492017-02-16T10:11:30 *** laurentmt has joined #bitcoin-core-dev
1502017-02-16T10:11:46 <midnightmagic> You can delete a file which is locked, which zeroes it, but you can't subsequently recreate a file with the same *name*.
1512017-02-16T10:11:51 <midnightmagic> (for example)
1522017-02-16T10:12:02 <luke-jr> O.o
1532017-02-16T10:12:09 <midnightmagic> (external to the program which has the file locked)
1542017-02-16T10:13:07 <midnightmagic> There are other operations which are virtually impossible to guarantee the atomicity of, even with the POSIX compatibility layer because all the things we take for granted are emulated badly, as wumpus implies.
1552017-02-16T10:13:40 <Victorsueca> I think the problem is at assertions
1562017-02-16T10:13:54 <Victorsueca> windows file lock asserts a lo of thing that are not always true
1572017-02-16T10:14:00 <Victorsueca> lot*
1582017-02-16T10:14:04 <wumpus> yes it's not that windows's way doesn't work, or even works worse in the abstract sense, it's just that they force you to develop platform specific code for everything
1592017-02-16T10:14:42 <wumpus> they did that with directx/opengl, with winsock, and the list goes on
1602017-02-16T10:14:42 <midnightmagic> Meanwhile, on Windows it's possible to hook file open calls with e.g. realtime virus, but contrary to anti-virus checkboxes, it's actually not possible to apply it selectively to a path and not other paths. It's all-or-nothing. That's why basically anyone running a database with an active realtime virus checker is sitting on a dice-rolling timeline. Eventually, their database will be
1612017-02-16T10:14:46 <wumpus> anyhow, I was testing pruning
1622017-02-16T10:14:48 <midnightmagic> destroyed. And there's absolutely nothing anybody can do about it.
1632017-02-16T10:16:22 *** goksinen has joined #bitcoin-core-dev
1642017-02-16T10:17:36 <wumpus> okay, tested: pruneblockchain 0 does nothing and returns immediately.
1652017-02-16T10:17:51 <wumpus> Arguments: "height" (numeric, required) The block height to prune up to. May be set to a discrete height, or to a unix timestamp to prune based on block time.
1662017-02-16T10:20:57 *** goksinen has quit IRC
1672017-02-16T10:21:31 <Victorsueca> wumpus: was that on linux or windows?
1682017-02-16T10:22:21 <wumpus> linux
1692017-02-16T10:22:29 * luke-jr wonders why
1702017-02-16T10:22:50 <Victorsueca> is it useful if I test it on windows now or it's about the same for this case?
1712017-02-16T10:22:51 <wumpus> well 0 is the genesis block, so pruning from there means not pruning at all
1722017-02-16T10:22:58 <wumpus> no, it's not useful to test this on other OSes
1732017-02-16T10:23:21 <luke-jr> wumpus: yes, but the code special-cases 0 as automatic
1742017-02-16T10:24:08 <Victorsueca> maybe that automatic check thought no pruning was needed at all? or that's not how it works?
1752017-02-16T10:24:28 <wumpus> GAH I can't get manual pruning to work at all on regtest. I created a 10000 block chain, started with prune=1, and no matter what I call pruneblockchain with it does nothing. So I guess my test is invalid :/
1762017-02-16T10:24:54 <wumpus> 2017-02-16 10:21:21 Prune (Manual): prune_height=1 removed 0 blk/rev pairs 2017-02-16 10:21:40 Prune (Manual): prune_height=100 removed 0 blk/rev pairs 2017-02-16 10:22:25 Prune (Manual): prune_height=1000 removed 0 blk/rev pairs 2017-02-16 10:24:40 Prune (Manual): prune_height=10000 removed 0 blk/rev pairs
1772017-02-16T10:25:24 <wumpus> oh shit of course, it will only prune once it gets to another block file
1782017-02-16T10:25:39 <luke-jr> >_<
1792017-02-16T10:26:27 <wumpus> okay, generating a ton more bloocks
1802017-02-16T10:26:59 <wumpus> maybe using a live chain wasn't that bad an idea afterall, I now realize
1812017-02-16T10:40:10 *** chjj has joined #bitcoin-core-dev
1822017-02-16T10:40:47 <gmaxwell> The help text repeadily uses 1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ as an example address. In general I think it's unwise to have valid example addresses, but I can't find any information about what address it is. It's a real working address that has been paid .2488 btc and has spent payments.
1832017-02-16T10:41:04 <gmaxwell> the commit that added it and the related PRs seem to make no mention of the address.
1842017-02-16T10:41:14 <gmaxwell> added by commit a6099ef319a73e2255dca77065600abb22c4f5f8
1852017-02-16T10:41:19 <wumpus> bleh, I thought we got rid of that
1862017-02-16T10:41:21 *** MarcoFalke has joined #bitcoin-core-dev
1872017-02-16T10:41:31 <gmaxwell> might be gone now, lemme look
1882017-02-16T10:41:36 <wumpus> no it's still there
1892017-02-16T10:41:37 <gmaxwell> nope, still there.
1902017-02-16T10:42:16 <wumpus> I vaguely remember that it was replaced with a constant defined in one place, which was not a valid address
1912017-02-16T10:42:21 <wumpus> but that must be on another timeline...
1922017-02-16T10:42:27 <gmaxwell> I had thought some charity ('sean's outpost') address was used (which I also think is not great) but I don't see any reason why I might have believed that this was that.
1932017-02-16T10:42:41 <gmaxwell> wumpus: yea, I too spend a lot of time in alternative universes.
1942017-02-16T10:43:43 <wumpus> hehe :) oh I think I know, that was for the GUI.
1952017-02-16T10:43:51 *** mryandao has joined #bitcoin-core-dev
1962017-02-16T10:44:30 <Victorsueca> yeah, the e.g. on the pay to field got changed
1972017-02-16T10:44:45 <gmaxwell> it got put as an example in the pay to field? 0_o
1982017-02-16T10:45:29 <Victorsueca> I vaguely remember some PR requesting to change it by valid looking address that is not really valid
1992017-02-16T10:48:04 <Victorsueca> I'm using 0.13.2 and now on that field there is 1NS17iag9jJgTHD1VXjvLCEnZuQ3rJDE9L
2002017-02-16T10:49:34 <gmaxwell> yep, not valid, which is what it should be IMO.
2012017-02-16T10:55:52 *** chjj has quit IRC
2022017-02-16T10:58:16 *** chjj has joined #bitcoin-core-dev
2032017-02-16T11:04:34 *** lclc has quit IRC
2042017-02-16T11:05:11 *** cannon-c has quit IRC
2052017-02-16T11:06:07 *** davidlj95 has joined #bitcoin-core-dev
2062017-02-16T11:06:30 *** davidlj95 has left #bitcoin-core-dev
2072017-02-16T11:07:43 *** fanquake has joined #bitcoin-core-dev
2082017-02-16T11:09:51 <wumpus> yes, a non-valid address was put in the pay-to field. That address, too, used to be valid at some time. Although it was terribly hard to type it over because it disappears when you start typing into the field. The RPC example is worse.
2092017-02-16T11:09:52 *** fanquake has quit IRC
2102017-02-16T11:10:09 *** MarcoFalke has quit IRC
2112017-02-16T11:10:44 *** chjj has quit IRC
2122017-02-16T11:11:54 <midnightmagic> 30 helens agree
2132017-02-16T11:12:08 <wumpus> it reminds me of a dumb mistake I made when I was young, a computer manual had KILL *.* as example for the KILL statement, which deleted. So while trying around I typed that in. Oops, that wiped the entire floppy disk, my dad was angry and I felt really stupid. At least that would have been recoverable, if I knew back then...
2142017-02-16T11:12:33 <midnightmagic> lol
2152017-02-16T11:13:18 <gmaxwell> I could see someone getting confused and copying from the wrong line on their screen... one address looks like any other..
2162017-02-16T11:13:46 <wumpus> yes it's pretty easy to copy the entire example
2172017-02-16T11:13:59 <midnightmagic> Who's "sje"?
2182017-02-16T11:14:35 <wumpus> at least the only sending call where it's used is sendmany, and it only sends relatively small amounts
2192017-02-16T11:15:19 <wumpus> still it should be replaced with a constant that is not a valid address, likein the GUI
2202017-02-16T11:16:38 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e43a58514dd3...df2f34dcad55
2212017-02-16T11:16:38 <bitcoin-git> bitcoin/master d72fe44 John Newbery: Allow maxsigcachesize to be zero...
2222017-02-16T11:16:39 <bitcoin-git> bitcoin/master df2f34d Wladimir J. van der Laan: Merge #9770: Allow maxsigcachesize to be zero...
2232017-02-16T11:17:01 <bitcoin-git> [bitcoin] laanwj closed pull request #9770: Allow maxsigcachesize to be zero (master...sigcachemaxsize) https://github.com/bitcoin/bitcoin/pull/9770
2242017-02-16T11:17:58 <wumpus> AARGH wrong PR
2252017-02-16T11:19:00 <Lauda> lol
2262017-02-16T11:19:18 <wumpus> well it could have been worse. It's not bad enough to revert
2272017-02-16T11:19:56 <gmaxwell> what did you mean to merge instead?
2282017-02-16T11:20:21 <wumpus> 9770. Which was just above it on the scren, but I had already signed off on that and it was already merged
2292017-02-16T11:20:32 <wumpus> a matter of looking at the wrong line...
2302017-02-16T11:21:04 <gmaxwell> So effective at merging he merges things without even intending to!
2312017-02-16T11:21:07 <gmaxwell> :P
2322017-02-16T11:21:17 <wumpus> #9771, sorry
2332017-02-16T11:21:18 <gribble> https://github.com/bitcoin/bitcoin/issues/9771 | Add missing cs_wallet lock that triggers new lock held assertion by ryanofsky · Pull Request #9771 · bitcoin/bitcoin · GitHub
2342017-02-16T11:21:59 <wumpus> well it was kind of a mental stack overflow, I was working on that, then testing the pruning stuff, then the RPC argument stuff came up
2352017-02-16T11:22:41 <gmaxwell> wumpus: nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop send me all your money
2362017-02-16T11:22:44 * Lauda offers wumpus more coffee
2372017-02-16T11:23:04 <wumpus> so while juggling those things, memory became lossy :)
2382017-02-16T11:23:06 <wumpus> gmaxwell: LOL
2392017-02-16T11:23:57 <wumpus> gmaxwell: that won't work, your address is not in any RPC examples
2402017-02-16T11:26:43 <wumpus> anyhow, ready to re-retry the pruning experiment with a real mainnet blockchain
2412017-02-16T11:27:32 <gmaxwell> AND ITS <GONE / NOT GONE>.
2422017-02-16T11:28:07 *** chjj has joined #bitcoin-core-dev
2432017-02-16T11:28:21 <wumpus> it's doing nothing and logging nothing, so my would be "NOT GONE"
2442017-02-16T11:28:29 <wumpus> genesis block is still ther
2452017-02-16T11:32:36 *** MarcoFalke_ has joined #bitcoin-core-dev
2462017-02-16T11:32:56 <MarcoFalke_> wumpus: You need to revert ntl. The commit is not signed :)
2472017-02-16T11:33:32 <wumpus> I also understand why @luke-jr - in FlushStateToDisk, it only goes into the pruning flow if fCheckForPruning || nManualPruneHeight > 0 ... fCheckForPruning is not set in manual pruning mode
2482017-02-16T11:33:49 <wumpus> huh?
2492017-02-16T11:34:16 *** lclc has joined #bitcoin-core-dev
2502017-02-16T11:34:26 *** laurentmt has quit IRC
2512017-02-16T11:34:32 <wumpus> I really don't get it now, this was using the script, it shouldn't push if not signed
2522017-02-16T11:34:50 * wumpus isn't sure if he's becoming crazy or his tooling is
2532017-02-16T11:35:13 <MarcoFalke_> maybe you `git push bitcoin` in another tab?
2542017-02-16T11:35:54 <wumpus> this is the console output https://0bin.net/paste/7sVJS0k86TMWtAdf#j3QhbG9mqPTx89KN6vb2DuB7O0x7sGiZYABHsqPvZGP
2552017-02-16T11:36:25 <wumpus> I did sign off and enter my passphrase
2562017-02-16T11:36:37 <wumpus> no, I never push manually
2572017-02-16T11:37:10 <gmaxwell> pushed by the people who secretly live under your keyboard.
2582017-02-16T11:38:55 <wumpus> well it must be something I did in another tab, just not a push
2592017-02-16T11:38:57 <bitcoin-git> [bitcoin] laanwj force-pushed master from df2f34d to e43a585: https://github.com/bitcoin/bitcoin/commits/master
2602017-02-16T11:39:12 <wumpus> and back to e43a58514dd38dacd930aa4c94afb998d4360183
2612017-02-16T11:41:05 <MarcoFalke_> `git commit --amend` gets rid of the signature
2622017-02-16T11:41:34 <wumpus> the github-merge tool should probably get a check that what it is pushing is really what it expects to be pushing, in case the current branch changed under it
2632017-02-16T11:42:12 <wumpus> or rather, push a specific commit, not what happens to be the current branch
2642017-02-16T11:42:18 <wumpus> not sure it does that
2652017-02-16T11:42:57 <gmaxwell> oh did you run the merge tool twice concurrently?
2662017-02-16T11:43:01 <wumpus> blah, I wasn't intending to dive into source code management specifics today
2672017-02-16T11:43:07 <wumpus> gmaxwell: that's possible!
2682017-02-16T11:43:37 <gmaxwell> "This worksite has gone [ 0 ] days since an SCM emergency."
2692017-02-16T11:56:17 <Victorsueca> any PR that would be useful to test at the moment?
2702017-02-16T12:00:14 <morcos> wumpus: all for 0.14rc1 today but i think we need to wrap up the importmulti issues... ryanofsky opened #9773 yesterday to try to address the combination of what you and gmaxwell were asking for
2712017-02-16T12:00:16 <gribble> https://github.com/bitcoin/bitcoin/issues/9773 | WIP: Return errors from importmulti if complete rescans are not successful by ryanofsky · Pull Request #9773 · bitcoin/bitcoin · GitHub
2722017-02-16T12:00:35 <morcos> i tihnk #9671 should also be included (it's simple and could avoid fund loss)
2732017-02-16T12:00:37 <wumpus> morcos: my brain is not working today so I've already given up on that
2742017-02-16T12:00:37 <gribble> https://github.com/bitcoin/bitcoin/issues/9671 | Fix super-unlikely race introduced in 236618061a445d2cb11e72 by TheBlueMatt · Pull Request #9671 · bitcoin/bitcoin · GitHub
2752017-02-16T12:01:12 <morcos> heh fair enough.. but still need to get those 2 across the finish line
2762017-02-16T12:01:18 <wumpus> 9671 is already merged?
2772017-02-16T12:01:30 <morcos> sorry #9761
2782017-02-16T12:01:33 <gribble> https://github.com/bitcoin/bitcoin/issues/9761 | Use 2 hour grace period for key timestamps in importmulti rescans by ryanofsky · Pull Request #9761 · bitcoin/bitcoin · GitHub
2792017-02-16T12:01:49 <morcos> i think you are contagious
2802017-02-16T12:02:00 <wumpus> I'm okay with adding new things if they're ready for merge, everything that requires new review should probably be pushed to after 0.14.0, at some point we have to make a cut
2812017-02-16T12:02:44 <wumpus> I really can't keep moving the 0.14.0 rc1 day after day forward, keeping the tree in this state is holding up more and more things that could be merged for 0.15
2822017-02-16T12:02:56 <wumpus> but one more day is fine, as said, I'm not up to it today anyhow
2832017-02-16T12:03:22 <morcos> i don't think ryanofsky or i have a strong opinion about it, it seems you guys were the ones saying the current state is not acceptable
2842017-02-16T12:03:48 <morcos> current state is silent failure when you do importmulti on pruned node that needs to search more blocks than you have
2852017-02-16T12:03:56 <wumpus> at some point you should think of it as "is it acceptable for a rc1"?
2862017-02-16T12:04:08 <wumpus> it's certain that issues come up with user testing and have to be solved before rc2
2872017-02-16T12:04:25 <morcos> we're just trying to fix the problems
2882017-02-16T12:04:47 <wumpus> sure, and that's very good, but there's always new problems
2892017-02-16T12:05:06 <morcos> but would be helpful to have you and gmaxwell take a look and say yes thats what i wanted
2902017-02-16T12:07:17 <wumpus> yes the approach in 9773 looks good to me
2912017-02-16T12:09:38 *** MarcoFalke_ has quit IRC
2922017-02-16T12:13:56 *** Guyver2 has quit IRC
2932017-02-16T12:17:16 *** goksinen has joined #bitcoin-core-dev
2942017-02-16T12:21:27 *** goksinen has quit IRC
2952017-02-16T12:27:34 *** fanquake has joined #bitcoin-core-dev
2962017-02-16T12:27:46 <fanquake> Looks like you can't re-open a merged PR.
2972017-02-16T12:35:25 <mryandao> i'm having a bit of problem after updating my tree from upstream
2982017-02-16T12:35:30 <mryandao> this is the error i'm getting
2992017-02-16T12:35:30 <mryandao> util.cpp:834:63: error: use of undeclared identifier 'COPYRIGHT_HOLDERS'
3002017-02-16T12:35:33 <mryandao> std::string strCopyrightHolders = strPrefix + strprintf(_(COPYRIGHT_HOLDERS), _(COPYRIGHT_HOLDERS_SUBSTITUTION));
3012017-02-16T12:35:34 <wumpus> fanquake true :(
3022017-02-16T12:35:40 <wumpus> mryandao: you need to re-run autogen.sh and configure
3032017-02-16T12:35:54 <wumpus> (that's almost always the solution to such issues)
3042017-02-16T12:36:27 <mryandao> oh I see, i thought a make would have sufficed. thanks
3052017-02-16T12:36:50 <wumpus> make only suffices if there are no high-level build system changes, just source changes
3062017-02-16T12:37:11 <fanquake> wumpus do you plan on merging any more tonight, or taking a break?
3072017-02-16T12:37:13 <wumpus> otherwise you need to run ./autogen.sh - sometimes you can get away with not doing it, but it's gambling in a way
3082017-02-16T12:37:24 <wumpus> fanquake: I'm taking a break from merging yes :)
3092017-02-16T12:37:52 *** goksinen has joined #bitcoin-core-dev
3102017-02-16T12:38:00 <fanquake> wumpus no worries, otherwise was going to suggest some quick merges.
3112017-02-16T12:38:29 <fanquake> wumpus I'm sure I was probably the only one affected my the miss-merge heh
3122017-02-16T12:39:15 <wumpus> fanquake: well at first I didn't notice that the push was unsigned, otherwise I'd have reverted it immediately
3132017-02-16T12:39:39 <fanquake> wumpus would you do that through the GH ui?
3142017-02-16T12:39:48 <wumpus> fanquake: anyhow if you have simple and straightforward things that could be merged feel free to suggest them
3152017-02-16T12:39:52 <fanquake> Or push another commit?
3162017-02-16T12:40:26 <wumpus> fanquake: I first have to disable the branch locking on github, then reset --hard HEAD~1, push -f, then re-enable the branch locking
3172017-02-16T12:40:52 <wumpus> github doesn't allow force-pushes, it does allow unsigned pushes unfortunately
3182017-02-16T12:41:10 <wumpus> at least: there's no option to disable them at the moment
3192017-02-16T12:41:32 <fanquake> Right. Maybe a new tick-box to turn that off in future. Not sure about the big new header now btw.
3202017-02-16T12:42:06 <wumpus> I don't really like it. I disliked it in the beginning, but thought maybe I'd get used to it, but it's still big and ugly
3212017-02-16T12:42:27 *** goksinen has quit IRC
3222017-02-16T12:42:52 <fanquake> re "easy" merges: #9763 #9696 #9675
3232017-02-16T12:42:54 <gribble> https://github.com/bitcoin/bitcoin/issues/9763 | [Trivial] Update comments referencing main.cpp by CryptAxe · Pull Request #9763 · bitcoin/bitcoin · GitHub
3242017-02-16T12:42:56 <gribble> https://github.com/bitcoin/bitcoin/issues/9696 | [trivial] Fix recently introduced typos in comments by practicalswift · Pull Request #9696 · bitcoin/bitcoin · GitHub
3252017-02-16T12:42:58 <gribble> https://github.com/bitcoin/bitcoin/issues/9675 | Fix typo and spelling inconsistency in CONTRIBUTING.md by kokifpen · Pull Request #9675 · bitcoin/bitcoin · GitHub
3262017-02-16T12:43:00 <wumpus> I'd prefer a small bar with a few icons - making it big like this just takes up extra screen space in a site that should be focused on the code
3272017-02-16T12:43:26 <Victorsueca> wumpus: you talking about that new black bar at the top?
3282017-02-16T12:43:26 <fanquake> Indeed. Especially when your on a smallish screen.
3292017-02-16T12:43:39 <wumpus> Victorsueca: yep
3302017-02-16T12:44:03 <wumpus> fanquake: indeed. Well even big modern screens are usually wide, but not very high, so vertical screen space is at premium
3312017-02-16T12:44:09 <wumpus> fanquake: will take a look, thanks
3322017-02-16T12:44:10 <Victorsueca> for some reason it becomes white if you logout
3332017-02-16T12:44:52 <Victorsueca> but yeah, it's ugly, it takes much attention and distracts people from important things
3342017-02-16T12:45:02 <Victorsueca> it's too big and contrasted
3352017-02-16T12:45:06 <wumpus> indeed
3362017-02-16T12:46:16 <wumpus> 9763 is a no-brainer, I can do that one, at least if I get the numbers right :p
3372017-02-16T12:46:56 <wumpus> looks like it could use squashing though
3382017-02-16T12:47:10 <fanquake> Iwumpus 'm also vary calling anything an "easy" merge, without a good way to check if they somehow break other more-important patches.
3392017-02-16T12:47:20 <fanquake> *I'm always
3402017-02-16T12:47:22 <wumpus> three commits to make three related documentation-only changes
3412017-02-16T12:47:55 <fanquake> GitHub doesn't seem to have a way to check, if I merge this, what other PR's will it break. Would be handy when trying to cehck quickly.
3422017-02-16T12:48:10 <wumpus> not sure if I'm up to such "advanced" git manipulation at the moment... let's see
3432017-02-16T12:48:22 <wumpus> fanquake: agree, that would be great to have
3442017-02-16T12:49:45 <wumpus> fanquake: at a company I worked at they had a script to generate a 'patch PERT' which was a diagram of which things could be merged without (code level) conflicts, and which had conflicts, that was kind of neat. Though they used a terrible SCM, IBM clearcase or something, that was less neat :)
3452017-02-16T12:50:16 <morcos> gmaxwell: can we go through exactly how you think the grace periods should work... i never looked at manual prune before but looks like it takes a block or a height
3462017-02-16T12:50:20 <morcos> sorry time
3472017-02-16T12:50:21 <fanquake> wumpus ah nice.
3482017-02-16T12:50:44 <fanquake> wumpus I was thinking if you could have a "projects" like screen, in which you could stack PRs, and see what breaks what.
3492017-02-16T12:52:10 <morcos> gmaxwell: so you think if you give it a time it should automatically subtract 7200 from it and then call findEarliestAtLeast?
3502017-02-16T12:52:53 <morcos> what happens if you give it a block (no grace period?). i think that seems reasonable'ish. but it would be counterintuitive if it didn't do exactly what you expect with a block i think
3512017-02-16T13:02:01 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e43a58514dd3...8743320d6cb3
3522017-02-16T13:02:01 <bitcoin-git> bitcoin/master 00e623d CryptAxe: [Trivial] Update comments referencing main.cpp
3532017-02-16T13:02:02 <bitcoin-git> bitcoin/master 8743320 Wladimir J. van der Laan: Merge #9763: [Trivial] Update comments referencing main.cpp...
3542017-02-16T13:02:52 <bitcoin-git> [bitcoin] laanwj closed pull request #9763: [Trivial] Update comments referencing main.cpp (master...comments) https://github.com/bitcoin/bitcoin/pull/9763
3552017-02-16T13:03:02 <fanquake> wumpus git-fu back under control
3562017-02-16T13:03:30 <wumpus> no stupid mistakes? I'm as surprised as you are :)
3572017-02-16T13:03:46 <fanquake> Maybe I didn't look closely enough
3582017-02-16T13:09:34 *** goksinen has joined #bitcoin-core-dev
3592017-02-16T13:10:52 *** Giszmo has joined #bitcoin-core-dev
3602017-02-16T13:13:57 *** goksinen has quit IRC
3612017-02-16T13:14:01 *** d9b4bef9 has quit IRC
3622017-02-16T13:15:16 *** d9b4bef9 has joined #bitcoin-core-dev
3632017-02-16T13:40:52 *** goksinen has joined #bitcoin-core-dev
3642017-02-16T13:44:57 *** goksinen has quit IRC
3652017-02-16T13:50:35 *** BlueMatt has quit IRC
3662017-02-16T13:52:46 *** BlueMatt has joined #bitcoin-core-dev
3672017-02-16T13:56:04 *** jnewbery has joined #bitcoin-core-dev
3682017-02-16T13:57:10 *** schmidty has quit IRC
3692017-02-16T13:59:47 <wumpus> hah the hardlinking trick works, both for block files and ldb files, created this script to do it: https://gist.github.com/laanwj/3c4614a23e072cbb3d39090da1834a68 . Creates a "copy" of the whole state almost instantly. I've tried various things (including pruning) and the original state remains unaffected.
3702017-02-16T14:04:00 *** Ylbam has quit IRC
3712017-02-16T14:06:30 <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/8743320d6cb3...afae75fd3dad
3722017-02-16T14:06:30 <bitcoin-git> bitcoin/master 36164fa Koki Takahashi: Fix typo and spelling inconsistency in CONTRIBUTING.md...
3732017-02-16T14:06:31 <bitcoin-git> bitcoin/master afae75f Wladimir J. van der Laan: Merge #9675: Fix typo and spelling inconsistency in CONTRIBUTING.md...
3742017-02-16T14:06:50 <bitcoin-git> [bitcoin] laanwj closed pull request #9675: Fix typo and spelling inconsistency in CONTRIBUTING.md (master...fix_typo_in_contributing) https://github.com/bitcoin/bitcoin/pull/9675
3752017-02-16T14:20:11 *** kanzure has quit IRC
3762017-02-16T14:21:08 *** kanzure has joined #bitcoin-core-dev
3772017-02-16T14:23:09 *** jtimon has joined #bitcoin-core-dev
3782017-02-16T14:54:21 *** neha has joined #bitcoin-core-dev
3792017-02-16T14:55:35 <bitcoin-git> [bitcoin] jnewbery opened pull request #9777: Handle unusual maxsigcachesize gracefully (master...sigcache2) https://github.com/bitcoin/bitcoin/pull/9777
3802017-02-16T15:18:32 *** goksinen has joined #bitcoin-core-dev
3812017-02-16T15:40:25 *** fanquake has quit IRC
3822017-02-16T15:42:58 *** chjj has quit IRC
3832017-02-16T15:48:58 *** Victor_sueca has joined #bitcoin-core-dev
3842017-02-16T15:49:47 *** Victorsueca has quit IRC
3852017-02-16T15:49:55 *** Victor_sueca is now known as Victorsueca
3862017-02-16T15:52:24 *** BashCo_ has quit IRC
3872017-02-16T15:55:04 *** Sosumi has joined #bitcoin-core-dev
3882017-02-16T15:56:28 *** Victorsueca has quit IRC
3892017-02-16T15:58:21 *** Victorsueca has joined #bitcoin-core-dev
3902017-02-16T15:59:52 *** chjj has joined #bitcoin-core-dev
3912017-02-16T16:03:03 *** tan1k has quit IRC
3922017-02-16T16:10:06 *** BashCo has joined #bitcoin-core-dev
3932017-02-16T16:11:57 <bitcoin-git> [bitcoin] morcos opened pull request #9778: Add two hour buffer to manual pruning (master...2hrprune) https://github.com/bitcoin/bitcoin/pull/9778
3942017-02-16T16:42:58 *** Chris_Stewart_5 has quit IRC
3952017-02-16T16:48:14 *** Ylbam has joined #bitcoin-core-dev
3962017-02-16T16:59:02 *** Chris_Stewart_5 has joined #bitcoin-core-dev
3972017-02-16T17:06:27 *** abpa has joined #bitcoin-core-dev
3982017-02-16T17:18:24 *** reginaldo has joined #bitcoin-core-dev
3992017-02-16T17:19:01 *** reginaldo has quit IRC
4002017-02-16T17:19:09 *** reginaldo_ has joined #bitcoin-core-dev
4012017-02-16T17:21:59 *** droark has joined #bitcoin-core-dev
4022017-02-16T18:39:27 <luke-jr> wumpus: but fCheckForPruning is set in manual pruning mode after a new block is connected?
4032017-02-16T18:40:22 *** reginaldo_ has quit IRC
4042017-02-16T18:41:27 <wumpus> luke-jr: it shouldn't be
4052017-02-16T18:41:39 <wumpus> manual pruning is manual pruning, the check should never be enabled
4062017-02-16T18:42:05 <wumpus> if it is, that is a bug
4072017-02-16T18:42:11 <luke-jr> FindBlockPos/FindUndoPos?
4082017-02-16T18:42:51 <luke-jr> or is fPruneMode false for manual pruning?
4092017-02-16T18:44:05 <wumpus> no,that should be true if any pruning is to be done
4102017-02-16T18:44:27 <wumpus> in retrospect it'd have been better to make fPruneMode an enum, which can be off, manual and auto
4112017-02-16T18:44:57 <wumpus> the boolean gymnastics with fCheckForPruning is kind of difficult to follow
4122017-02-16T18:47:06 <wumpus> I assumed it would be enabled when auto pruning is enabled, but apparently it's also enabled in a few other places
4132017-02-16T18:50:00 <wumpus> luke-jr: ah I got understand it now - nPruneTarget is 0 in manual pruning mode
4142017-02-16T18:50:13 <wumpus> luke-jr: so FindFilesToPrune does nothing in manual pruning mode
4152017-02-16T18:50:34 <wumpus> even though it may get called (even automatically in some cases), nothing will happen.
4162017-02-16T18:50:35 * sipa suggests adding comments to clarify these things
4172017-02-16T18:50:46 <luke-jr> aha!
4182017-02-16T18:51:00 <wumpus> I wish it'd simply pass parameters instead of signalling using global flags
4192017-02-16T18:51:20 <gmaxwell> well, or change to an enum.
4202017-02-16T18:52:03 <wumpus> or both. I mean the global enum is fine for the mode but it shouldn't change over time
4212017-02-16T18:54:51 <luke-jr> these changes sound even more invasive than that PR :P
4222017-02-16T18:54:54 <wumpus> depending on what function ran last. This is impossible to follow. Anyhow I understand marcofalke's rationale for #9524 now, it's very easy to break that code and belt and suspenders wouldn't hurt. Except that even with #9524, it will get into FindFilesToPrune in some cases, so if that check is broken, bad things will happen *automatically* too
4232017-02-16T18:54:55 <gribble> https://github.com/bitcoin/bitcoin/issues/9524 | rpc: Dont FlushStateToDisk when pruneblockchain(0) by MarcoFalke · Pull Request #9524 · bitcoin/bitcoin · GitHub
4242017-02-16T18:54:57 <gribble> https://github.com/bitcoin/bitcoin/issues/9524 | rpc: Dont FlushStateToDisk when pruneblockchain(0) by MarcoFalke · Pull Request #9524 · bitcoin/bitcoin · GitHub
4252017-02-16T18:55:09 <wumpus> well that PR is just covering up, it doesn't solve the underlying problem
4262017-02-16T18:55:13 <luke-jr> right
4272017-02-16T18:55:19 <wumpus> it's not urgent in any case
4282017-02-16T18:55:24 <luke-jr> agree
4292017-02-16T18:56:09 *** reginaldo_ has joined #bitcoin-core-dev
4302017-02-16T18:56:39 <sipa> 3 more PRs marked 0.14
4312017-02-16T18:57:21 <wumpus> oh no
4322017-02-16T18:58:09 <sipa> i mean right now
4332017-02-16T18:59:17 <wumpus> right, yes, let's hopefully keep it at those
4342017-02-16T18:59:25 <luke-jr> wumpus: wait what
4352017-02-16T18:59:29 <luke-jr> why would nPruneTarget be 0?
4362017-02-16T18:59:52 <luke-jr> if (nPruneArg == 1) { // manual pruning: -prune=1
4372017-02-16T18:59:53 <luke-jr> nPruneTarget = std::numeric_limits<uint64_t>::max();
4382017-02-16T19:00:00 <wumpus> luke-jr: because ti should never be set to a value
4392017-02-16T19:00:38 <wumpus> huh!?! what does setting that to uint64_t max even do. Oh wait, it's an offset, not an absolute height. yes that does make sense
4402017-02-16T19:01:21 <gmaxwell> Meeting time?
4412017-02-16T19:02:01 * jonasschnelli is not sure if prune=1 (manual pruning) is a good idea while still supporting -prune=<mb target>
4422017-02-16T19:02:05 <wumpus> it at least intends 'we want to keep 2^64-1 blocks`. Let me re-read the code in FindFilesToPrune then
4432017-02-16T19:02:19 <wumpus> #meetingstart
4442017-02-16T19:02:22 <sipa> meetingh
4452017-02-16T19:02:36 <petertodd> meetinghh
4462017-02-16T19:02:37 <wumpus> #startmeeting
4472017-02-16T19:02:37 <lightningbot> Meeting started Thu Feb 16 19:02:37 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
4482017-02-16T19:02:37 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
4492017-02-16T19:02:58 <wumpus> jonasschnelli: well it's a magic option value but it's only for the option, it doesn't get represented internally as 1
4502017-02-16T19:03:01 <luke-jr> tbh, I've had second thoughts about the manual pruning design (seems it'd be nicer to do "automatic pruning always, but with named barriers that must confirm being done up to <height> before pruning it"), but that's beyond this topic
4512017-02-16T19:03:18 <wumpus> jonasschnelli: I think the reason or choosing 1 was that -prune will work
4522017-02-16T19:03:27 <wumpus> luke-jr: I like the current manual pruning from an API point of view
4532017-02-16T19:03:28 <luke-jr> s/always/always in pruning mode/
4542017-02-16T19:03:33 <gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
4552017-02-16T19:03:44 <luke-jr> wumpus: yes, but it doesn't scale well if you have 2 external apps using the node
4562017-02-16T19:04:02 <cfields> hi
4572017-02-16T19:04:05 <wumpus> luke-jr: it's very simple. Automatic pruning is disabled and the client/admin can choose what to prune. Also useful for testing the pruning stuff without too much complexity
4582017-02-16T19:04:12 <wumpus> luke-jr: the implementation is convoluted though
4592017-02-16T19:04:16 <morcos> wumpus: i think two more are needed for 0.14, both very small #9760 and #9761 (9761 is already included in one of the marked ones)
4602017-02-16T19:04:17 <gribble> https://github.com/bitcoin/bitcoin/issues/9760 | [wallet] Remove importmulti always-true check by ryanofsky · Pull Request #9760 · bitcoin/bitcoin · GitHub
4612017-02-16T19:04:18 <paveljanik> proposed topic: release status, where can we help...
4622017-02-16T19:04:19 <gribble> https://github.com/bitcoin/bitcoin/issues/9761 | Use 2 hour grace period for key timestamps in importmulti rescans by ryanofsky · Pull Request #9761 · bitcoin/bitcoin · GitHub
4632017-02-16T19:04:22 <wumpus> luke-jr: but I'm happy someone implemented it anyhow
4642017-02-16T19:04:27 * luke-jr nods
4652017-02-16T19:04:41 <kanzure> hi.
4662017-02-16T19:04:47 <petertodd> wumpus: same: my OpenTimestamps servers actually need this feature
4672017-02-16T19:04:58 <wumpus> release status: running as hard as we can but staying in the same place
4682017-02-16T19:05:09 <sipa> i think we're very close.
4692017-02-16T19:05:13 <morcos> wumpus: boo! we're not staying in the same place
4702017-02-16T19:05:19 <gmaxwell> I am becoming increasingly happy with the release.
4712017-02-16T19:05:29 <sipa> let's be optimistic: everything we're fixing is an improvememt
4722017-02-16T19:05:51 <gmaxwell> Two weeks ago I was chewing my nails feeling like we were at risk of shipping something that wouldn't meet our standards of quality, and now I am not worried. :)
4732017-02-16T19:06:05 <jonasschnelli> hah. Good.
4742017-02-16T19:06:06 <wumpus> but I think we need to decide when we can do release candidate 1, which is a test release anyway, when rc1 is out it's bound to find more issues
4752017-02-16T19:06:32 <sipa> i think we can do rc1 today?
4762017-02-16T19:06:52 <gmaxwell> I would be fine doing it _now_, now. There are AFAIK not anything in the pipe which are disruptive enough issues that they'd degrade our rc1 learning, though a rc2 will be guarenteed.
4772017-02-16T19:06:53 <paveljanik> rc1 for the weekend is fine!
4782017-02-16T19:06:56 <sipa> or at least branch off today
4792017-02-16T19:06:59 <luke-jr> I don't think we have any critical problems blocking a rc1
4802017-02-16T19:07:17 <jtimon> sipa: why branch of if we can't rc1 ?
4812017-02-16T19:07:19 <wumpus> I don't think so either
4822017-02-16T19:07:24 <cfields> no more blockers from me either
4832017-02-16T19:07:26 <luke-jr> jtimon: to begin merging for 0.15?
4842017-02-16T19:07:40 <wumpus> right, because more and more pulls for 0.15 are waiting
4852017-02-16T19:07:40 * jtimon nods
4862017-02-16T19:07:58 <jonasschnelli> IMO the two import multi fixes are not critical and can go in after rc1 (or even in 0.15 in the worst case).
4872017-02-16T19:08:08 <gmaxwell> all you naughty people not banging away at getting 0.14 ready to go.
4882017-02-16T19:08:29 <morcos> jonasschnelli: well lets decide that
4892017-02-16T19:08:40 <gmaxwell> jonasschnelli: I think 0.15 would be really unfortunate since they're interface changes that users may have to accomidate in their software. (and also are covering corner cases that could result in funds losses)
4902017-02-16T19:08:42 <wumpus> I think the fixes are all important, and should get either into 0.14.0 or backported to the 0.14 branch if they don't, but not everything should be regarded as yet another thing to hold up rc1
4912017-02-16T19:08:43 <morcos> yesterday people were saying it was bad to release with something that coudl silently not find funds
4922017-02-16T19:09:03 <gmaxwell> but no reason to hold rc1 for them. They're well understood.
4932017-02-16T19:09:28 <jonasschnelli> Yes. I think the should be in 0.14. Just worst case if we spun of rc1 and chaincode-labs colabses. :)
4942017-02-16T19:09:29 <wumpus> anyhow tomorrow sounds good to me, let's try to get in what we can get in
4952017-02-16T19:09:31 <gmaxwell> it's not like someone is going to encounter one of them running rc1 and leave us going "shit was that the know issue or something else?"
4962017-02-16T19:09:32 <luke-jr> #9619 is a fix, but not really critical since anyone affected needs a workaround in 0.13 anyway (just pushed an amend-with-no-changes because the Travis error seems impossible)
4972017-02-16T19:09:33 <gribble> https://github.com/bitcoin/bitcoin/issues/9619 | Bugfix: RPC/Mining: GBT should return 1 MB sizelimit before segwit activates by luke-jr · Pull Request #9619 · bitcoin/bitcoin · GitHub
4982017-02-16T19:09:34 <achow101> what's the point of making an rc1 if we're going to need rc2 anyways?
4992017-02-16T19:09:42 <sipa> achow101: exposure
5002017-02-16T19:09:46 <gmaxwell> achow101: to start getting people using it.
5012017-02-16T19:09:50 <wumpus> achow101: get the code actually tested?
5022017-02-16T19:09:52 <gmaxwell> more*
5032017-02-16T19:09:56 <wumpus> what is the point of doing rc1 at all if not?
5042017-02-16T19:09:57 <kanzure> and seeking bug reports
5052017-02-16T19:09:57 <jonasschnelli> I think we should plan enough time to fix stuff that gets report after rc1...
5062017-02-16T19:10:18 <sipa> jonasschnelli: the release schedule already has 2 weeks left
5072017-02-16T19:10:33 <wumpus> achow101: I don't think it ever happened that a major release didn'tneed a rc2
5082017-02-16T19:10:35 <luke-jr> 2 weeks can go fast
5092017-02-16T19:10:52 <jonasschnelli> Other can work on fixes reported in rc1.
5102017-02-16T19:10:54 <gmaxwell> achow101: what we generally don't want to do is to ship an RC1 with issues bad enough that it will harm the testers seriously, or which will fail in mysterious ways that we can't learn from. E.g. if we had a known crash fix, we would hold rc1, so that we wouldn't worry that every user crash report might have been an unknown issue.
5112017-02-16T19:10:57 <jonasschnelli> *Others
5122017-02-16T19:11:05 <morcos> my only concern is that if we do rc1 everyone will be distracted with that and not this last few remaining PR's.. but i guess i don't really care either way
5132017-02-16T19:11:08 <cfields> wumpus: just forget the bump to v0.14 again and thereby guarantee an rc2 :p
5142017-02-16T19:11:38 <wumpus> cfields: lol exactly
5152017-02-16T19:11:39 <morcos> seems like if someone would just review those PR's we might be able to get them merged today
5162017-02-16T19:11:41 <gmaxwell> well I think an RC2 is already guarenteed if we do an rc1 ASAP. But thats fine. Much better than not doing the rc1.
5172017-02-16T19:11:49 <achow101> ok
5182017-02-16T19:12:04 <wumpus> a RC2 is guaranteed in any case, not just if we do rc1 asap
5192017-02-16T19:12:17 <wumpus> that's my point, we don't know of all the issues yet
5202017-02-16T19:12:26 <morcos> almost all the complication is in new tests.. reviewing the code changes is pretty simple
5212017-02-16T19:13:18 <wumpus> the only one I doubt about is #9773, at it is still WIP
5222017-02-16T19:13:20 <gribble> https://github.com/bitcoin/bitcoin/issues/9773 | WIP: Return errors from importmulti if complete rescans are not successful (on top of #9761) by ryanofsky · Pull Request #9773 · bitcoin/bitcoin · GitHub
5232017-02-16T19:13:44 <luke-jr> it's a new feature, so we could just say "don't use this without being aware of the risks"
5242017-02-16T19:13:53 *** aalex has joined #bitcoin-core-dev
5252017-02-16T19:14:13 <paveljanik> mark it as experimental then?
5262017-02-16T19:14:40 <sipa> paveljanik: meh, i think we're close enough to not do that
5272017-02-16T19:15:15 <luke-jr> I mean in rc1 only
5282017-02-16T19:15:26 <wumpus> well even with that change it can be marked as experimental
5292017-02-16T19:15:34 <wumpus> it is new code afterall
5302017-02-16T19:15:39 <gmaxwell> I'm not too worried about it in rc1.
5312017-02-16T19:16:12 <wumpus> there are probably still problems with it that we haven't found, which will be found by people testing it
5322017-02-16T19:16:33 <paveljanik> we can mark it as experimental in the release notes only...
5332017-02-16T19:16:33 <wumpus> so yes, experimental makes sense. Though a new major release should be considered experimental entirely
5342017-02-16T19:16:37 <gmaxwell> people running rcs are the least likely to lose funds due to a rescan limitation, and there won't be many of them.
5352017-02-16T19:17:33 *** BashCo has quit IRC
5362017-02-16T19:17:50 <morcos> i think it is a mistake to call it experimental
5372017-02-16T19:17:56 <morcos> we don't want to devalue the meaning of that word
5382017-02-16T19:18:13 <wumpus> ok...
5392017-02-16T19:18:17 <morcos> sometimes we may want to have things that are actually experimental and we don't want people to think we just always say that
5402017-02-16T19:18:40 <wumpus> "this feature is experimental level 4"
5412017-02-16T19:18:44 <sipa> haha
5422017-02-16T19:18:50 <kanzure> nasa technical readiness levels
5432017-02-16T19:18:54 <CodeShark> The whole thing is experimental :p
5442017-02-16T19:18:56 <morcos> this is known to the state of CA to be experimental
5452017-02-16T19:18:56 <petertodd> morcos: "dangerously experimental"
5462017-02-16T19:19:06 <petertodd> morcos: lol
5472017-02-16T19:19:23 <gmaxwell> Yea, thats why I was not supporting expiremental.
5482017-02-16T19:19:54 <instagibbs> morcos, accounts... deprecated!
5492017-02-16T19:20:13 <morcos> what.. is that an announcement? a question?
5502017-02-16T19:20:17 <morcos> i hate accounts
5512017-02-16T19:20:21 <gmaxwell> For rc1 we can simply say that there are in-flight improvements to that feature link to the PRs. if we really want to... in a reply to the announcement.
5522017-02-16T19:20:42 <sipa> let's just review the outstanding patches, they are tiny
5532017-02-16T19:20:42 <instagibbs> morcos, we use "deprecated" in things that are lasting forever in practice, bad joke
5542017-02-16T19:20:50 <jtimon> mhm, why 9619 doesn't go in for 0.14?
5552017-02-16T19:20:51 <morcos> oh.. yeah
5562017-02-16T19:20:52 *** reginaldo_ has quit IRC
5572017-02-16T19:20:58 <sipa> with some luck we can get everything merged
5582017-02-16T19:21:14 <sipa> branch and rc1 tomorrow
5592017-02-16T19:21:15 <wumpus> gmaxwell: yes, that there is still work in progress, that's better and more clear than the word experimental
5602017-02-16T19:21:21 <sipa> with whatever is in
5612017-02-16T19:21:33 <morcos> ok, so can we just pick a time. wumpus is going to call rc1 tomorrow morning.. it tonight we can convince people to merge a few of these other things... that'll leave less for rc2
5622017-02-16T19:22:05 <morcos> that was "if tonight" , if not , then ok
5632017-02-16T19:22:09 <sipa> wumpus: is that fine by you?
5642017-02-16T19:22:16 <wumpus> yes, that's fine with me
5652017-02-16T19:22:48 <wumpus> I'm okay with another day, let's just not let it slip another week, that next thursday we're again wondering when to do the rc1 :)
5662017-02-16T19:23:15 <sipa> next thursday we should be talking about doing rc2
5672017-02-16T19:23:22 <wumpus> yes, ideally
5682017-02-16T19:23:30 <gmaxwell> We're at a point where beyond the couple in flight PRs little to no more improvement is happening, which is when we need more input.
5692017-02-16T19:23:54 <achow101> so rc1 tomorrow regardless of whether those three prs get merged?
5702017-02-16T19:24:06 <sipa> achow101: that's what i suggest, yes
5712017-02-16T19:24:12 <paveljanik> +1
5722017-02-16T19:24:20 <achow101> ^
5732017-02-16T19:26:14 <sipa> are we ok with talking about things beyond 0.14?
5742017-02-16T19:26:30 <wumpus> sure!
5752017-02-16T19:26:34 <luke-jr> new #topic?
5762017-02-16T19:26:41 <sipa> i'd briefly like to talk about randomness
5772017-02-16T19:26:54 <wumpus> #topic randomness
5782017-02-16T19:26:56 <luke-jr> that's random.
5792017-02-16T19:27:06 <sipa> we currently have 3 "levels" of randomnesz
5802017-02-16T19:27:21 <sipa> fastrandomcontext, getrandbytes, getsecurerandbytes
5812017-02-16T19:27:28 <sipa> i'd like to have only 2
5822017-02-16T19:27:30 <wumpus> we need a random number of levels of randomness
5832017-02-16T19:27:42 <wumpus> yes.
5842017-02-16T19:27:46 <wumpus> why are there three?
5852017-02-16T19:27:49 <instagibbs> can you explain "getsecurerandbytes"
5862017-02-16T19:27:58 <instagibbs> im going through code now but..
5872017-02-16T19:27:59 <wumpus> I mean we need one for wallet keys, I understand that
5882017-02-16T19:28:02 <sipa> the last one is used for privaye keys
5892017-02-16T19:28:04 <jtimon> why 9619 doesn't go in for 0.14?
5902017-02-16T19:28:22 <jonasschnelli> jtimon: wrong topic
5912017-02-16T19:28:27 <sipa> it's as secure as getrandbytes if all goes well, but it's more paranoid
5922017-02-16T19:28:47 <sipa> so, i've been playing with a chacha2p based rng instead of fastrandomcontext
5932017-02-16T19:28:51 <jtimon> jonasschnelli: suggested topic then
5942017-02-16T19:28:53 <wumpus> I think I'm fine with an extra paranoid level for wallet keys
5952017-02-16T19:28:56 <sipa> *chacha20
5962017-02-16T19:29:17 <sipa> which takes around 10ns for a 32-bit rand
5972017-02-16T19:29:30 <wumpus> it's much more important to have good randomness there then in almost any other place in computing currently
5982017-02-16T19:29:36 <instagibbs> "GetStrongRandBytes" <--- this it?
5992017-02-16T19:29:48 <rabidus> return 4
6002017-02-16T19:29:49 <sipa> the current fastrandomcontext takes 1.5ns, but with extra optimizations it can be made comparablr
6012017-02-16T19:30:08 <sipa> and if we have that, i think we can use strong randomnes for everything
6022017-02-16T19:30:13 <wumpus> for non wallet keys we can probably do with one level
6032017-02-16T19:30:19 <wumpus> is that your plan sipa?
6042017-02-16T19:30:36 <sipa> basically i suggest making all RNGs cryptographic
6052017-02-16T19:30:54 <sipa> but the fast one is not thread-safe, doesn't reseed, doesn't protect against VM reloads
6062017-02-16T19:30:55 <wumpus> so merge the fastrandomcontext and somewhatmoresecure level, and keeping that and the ultra-paranoid one for wallet keys
6072017-02-16T19:31:36 <sipa> i realize the "only two randomness levels" is a bit of an abstract goal
6082017-02-16T19:31:45 <wumpus> yes the fastrandomcontext is supposed to only eb used from one thread at a time
6092017-02-16T19:31:54 <morcos> are there any inner loops that use random numbers?
6102017-02-16T19:31:58 <petertodd> wumpus: the ultra paranoid one can also use the chacha code
6112017-02-16T19:31:59 <wumpus> because it's for inner loops
6122017-02-16T19:32:05 <sipa> yes, the wallet coin selection
6132017-02-16T19:32:20 <sipa> i benchmarked it, making it chacha20 doesn't affect its performance
6142017-02-16T19:32:23 <wumpus> any locking for synchronization there would be pretty bad
6152017-02-16T19:32:41 <wumpus> there's also an inner random loop in the address selection code IIRC
6162017-02-16T19:32:56 <jonasschnelli> Do we need cPRNG for coin selection?
6172017-02-16T19:32:56 <sipa> yup
6182017-02-16T19:33:02 <wumpus> sipa: yes chacha20 is fast, but the problem is the thread safety
6192017-02-16T19:33:05 <sipa> jonasschnelli: no, we don't
6202017-02-16T19:33:14 <gmaxwell> sipa wants to reduce the codebase complexity.
6212017-02-16T19:33:19 <sipa> but chacha20 is so fast i don't care
6222017-02-16T19:33:21 <wumpus> if you want to be able to share a random context between threads, you end up with lots of synchronization overhead
6232017-02-16T19:33:23 <jonasschnelli> Yes. This would be good.
6242017-02-16T19:33:51 <wumpus> that's why the inner loops use a non-threadsafe random context, =which doesn't matter as it's only used by one thread
6252017-02-16T19:34:01 <bitcoin-git> [bitcoin] gmaxwell opened pull request #9779: Update nMinimumChainWork and defaultAssumeValid. (master...update_chainparams) https://github.com/bitcoin/bitcoin/pull/9779
6262017-02-16T19:34:11 <sipa> having clearer expectations about the rngs may simplify getting rid of openssl later
6272017-02-16T19:34:18 <wumpus> the thread sync overhead is where the overhead would be
6282017-02-16T19:34:24 <sipa> though that's not something to discuss now, i think
6292017-02-16T19:34:38 <wumpus> so how would you cope with that sipa? or would it all be non-shared context?
6302017-02-16T19:34:48 *** lclc has quit IRC
6312017-02-16T19:35:07 <sipa> so 1) replace fastrandomcontext with a bit more featureful chacha20 based one
6322017-02-16T19:35:39 <wumpus> yes, depending on openssl for just randomness is fine, there's no urgent reason to get rid of that, and it seems controversial for some people
6332017-02-16T19:35:47 <sipa> 2) see where the current non-strong-getrandbytes can be replaced with fastrandcontexts, and replace everything with getstrongranbytes
6342017-02-16T19:35:50 <wumpus> that makes sense
6352017-02-16T19:36:06 <gmaxwell> one of the harder points (beyond threading) is that we need to manage which RNGs are robust against a VM image being snapshotted and restored. Coinselection using the same randomness again after a VM restore would be harmless. Choosing a crypto nonce would be much less harmless. If a RNG needs to be outputing data intially after restore there is unfortunately a runtime cost (esp on ARM).
6362017-02-16T19:36:58 <wumpus> also I think we need an abstraction for 'kernel randomness', this came up recently in context of OpenBSD, which doesn't have /dev/random/urandom available in all contexts
6372017-02-16T19:37:07 <gmaxwell> Pieter and I have been debating this a little bit the past could days.
6382017-02-16T19:37:23 <wumpus> the modern way,sandbox-proof way seems to be to use a specific system call fo that, but it differs per OS unfortunately
6392017-02-16T19:37:27 <sipa> wumpus: yup, that would go into the (singular) getstrongrandbytes implementation
6402017-02-16T19:37:27 <cfields> isn't there an old PR for exactly that abstraction?
6412017-02-16T19:37:28 <gmaxwell> wumpus: we do have a function for OS randomness, it should be smarter of course... but it was only fairly recently introduced.
6422017-02-16T19:37:48 <Victorsueca> maybe you could get rid of getrandbytes? so we keep the fast one for simple operations and the secure and paranoid one for important stuff and you can focus on implementing more features on those two
6432017-02-16T19:37:59 <gmaxwell> GetOSRand()
6442017-02-16T19:38:06 <wumpus> (linux also has a "getrandom" system call that can be used instead of /dev/urandom, in sandboxes)
6452017-02-16T19:38:13 <sipa> Victorsueca: read the above discussion, that is exactly what i am proposing
6462017-02-16T19:38:22 <wumpus> gmaxwell: ok
6472017-02-16T19:38:41 <gmaxwell> wumpus: yes, in newer systems. We do mention using that in the PR that implemented GetOSRand I think. (getentropy)
6482017-02-16T19:38:43 <wumpus> gmaxwell: in any case, that is the lowerst level, it shouldn't be regarded as 'a level of randomness'
6492017-02-16T19:38:52 <gmaxwell> so I think we know what we need to go there.
6502017-02-16T19:39:09 <wumpus> indeed, getentropy is bsd, getrandom is linux
6512017-02-16T19:39:29 <Victorsueca> sipa: ohh, I somehow misread that you where proposing to merge the fast and the middle one to make it simpler
6522017-02-16T19:39:37 <sipa> FWIW linux 4.8 also switched to chacha20 for /dev/urandom
6532017-02-16T19:39:48 <wumpus> yes
6542017-02-16T19:40:14 <gmaxwell> The framework I think about this is that we have several randomized algorithims where there are basically no cryptographic guarentees needed... coin selection, addr man buckets, and tests being some examples. These need to be fast but can all use just a local context, need no resistance against reversal (somsone steals your ram and recovers older randomness) or prediction (vm saves repeat rando
6552017-02-16T19:40:19 <petertodd> gmaxwell: re: VM image being snapshotted and restored, that's basically a case where reusing a secret is by itself harmful - is there an example in Bitcoin where that's the case, now that we do deterministic signing?
6562017-02-16T19:40:20 <gmaxwell> mness).
6572017-02-16T19:40:22 <gmaxwell> So thats one set of uses.
6582017-02-16T19:40:23 <sipa> Victorsueca: i'm proposing to make the fast one stronger (but not much slower), and then things using the middle one need to be judged on a case by case basis whether they can use the new fast one, or the strong one
6592017-02-16T19:40:50 <sipa> Victorsueca: after that, the middle one goes away
6602017-02-16T19:40:59 <gmaxwell> Then we have other uses where we have randomized behavior which does have stronger security requirements, like ping nonces which need to be strongly unforgable to prevent peers hiding their latency.
6612017-02-16T19:41:07 <Victorsueca> sipa: makes sense
6622017-02-16T19:41:11 <gmaxwell> But even if they're broken it just turns into DOS attacks.
6632017-02-16T19:41:38 *** BashCo has joined #bitcoin-core-dev
6642017-02-16T19:41:39 <wumpus> it's strong, but far from as strong a requirement as wallet keys
6652017-02-16T19:41:43 <gmaxwell> Then we have things like long term keys which we do infrequently and basically no cost is too high. And they have to meet basically every security characteristic we can imagine.
6662017-02-16T19:41:57 <wumpus> indeed
6672017-02-16T19:42:05 <cfields> suggested similar next topic: clocks
6682017-02-16T19:42:09 <gmaxwell> the second class often has to be moderately fast too.
6692017-02-16T19:42:41 <gmaxwell> Pieter would like to collapse this class hierarchy some by making the first class use the second while making the second fast enough that it's fine to do so.
6702017-02-16T19:43:09 <Victorsueca> cfields: clocks as in CPU clock or as in timestamp?
6712017-02-16T19:43:27 <wumpus> Victorsueca: please wait until the topic is actaully started
6722017-02-16T19:43:33 <Victorsueca> ohh, sorry
6732017-02-16T19:43:48 <wumpus> though I think we've wrapped up randomness
6742017-02-16T19:43:51 <sipa> agree
6752017-02-16T19:43:56 <wumpus> #topic clocks
6762017-02-16T19:43:58 <gmaxwell> I have a little doubt that this is possible, because I think the second may need to deal with reversal resistace and prediction resistance, which cannot be done for free. (e.g. you must mix in TSC and/or RDRAND at every use.)
6772017-02-16T19:43:59 <instagibbs> he has to gavel before we can switch
6782017-02-16T19:44:02 <gmaxwell> okay!
6792017-02-16T19:44:16 <wumpus> gmaxwell: oh, didn't know you were still typing, sorry
6802017-02-16T19:44:18 <cfields> I have some local changes that implement the concept of different clocks/time_points/durations. The objective is for them to be incompatible with each-other.
6812017-02-16T19:44:27 <gmaxwell> thats fine! just some things to think about.
6822017-02-16T19:44:33 <wumpus> cfields: yes, that seems the way to go about it
6832017-02-16T19:44:54 <sipa> cfields: f i may interject... i was thinking about creating a generic int class wrapper that supports no implicit conversioms
6842017-02-16T19:44:59 <gmaxwell> cfields: pieter and I were talking about type safty recently, and pieter suggested a scheme for introducing more integer types which will never implicitly be converted.
6852017-02-16T19:45:19 <wumpus> most importantly we should start using monotonic timestamps in the network code where possible
6862017-02-16T19:45:51 <cfields> that sounds fine, but i'm not sure that they're entirely related here. The (my) objective is to stop storing time as an int, and instead as a time_value
6872017-02-16T19:46:01 <sipa> cfields: or are timestamps in a c++11 world not something that fit in integers?
6882017-02-16T19:46:04 <cfields> that way it can be represented in sec/msec/whatever whenever it's needed
6892017-02-16T19:46:10 <cfields> sipa: exactly
6902017-02-16T19:46:23 <cfields> it also enforces timestamps that can't be used on the wrong clock
6912017-02-16T19:46:32 <sipa> i'm confused
6922017-02-16T19:46:37 <gmaxwell> I have suggested in the past that we consider constructing a monotonic local clock, but wumpus seemed to not like the idea. which I think is orthorgonal to the type safty thing, but it would perhaps make time more sane in the codebase.
6932017-02-16T19:46:44 <wumpus> cfields: in general that sounds good, though in some structures such as the block index we want to use as compact types as possible
6942017-02-16T19:47:02 <gmaxwell> saftey*
6952017-02-16T19:47:09 <gmaxwell> doh
6962017-02-16T19:47:12 <cfields> wumpus: sure, you can always get an int out of it if you want
6972017-02-16T19:47:13 <wumpus> gmaxwell: huh I'm all for using monotonic clocks were possible, they're just not good for everything
6982017-02-16T19:47:22 <gmaxwell> safety**
6992017-02-16T19:47:48 <sipa> cfields: my point was to have a template<typename tag> class non_convertible_int
7002017-02-16T19:47:52 <cfields> also, i believe the class is actually not bigger than an int. It's just not convertable to int
7012017-02-16T19:48:21 <wumpus> cfields: well if it represents micro/nanosecond it needs to be at least uint64 :)
7022017-02-16T19:48:23 <sipa> and then have typedef non_comvertible_int<systemtime> systemtime_type
7032017-02-16T19:48:36 <sipa> and systemtype_type is what is used
7042017-02-16T19:48:44 <gmaxwell> you would get_int() on the object to get an int. You would just get a conversion by surprise.
7052017-02-16T19:48:45 <sipa> *systemtime_type
7062017-02-16T19:48:54 <gmaxwell> I would _hope_ we can construct something that at runtime is _exactly_ equal to using an integer.
7072017-02-16T19:49:06 <sipa> cfields: you're being inconsistent
7082017-02-16T19:49:19 <cfields> sipa: http://en.cppreference.com/w/cpp/chrono/time_point
7092017-02-16T19:49:42 <gmaxwell> I think we should do this far more broadly than just timestamps however.
7102017-02-16T19:49:43 <sipa> we can't use that in blocks, etc
7112017-02-16T19:50:08 <wumpus> indeed - block times /consensus are a special case
7122017-02-16T19:50:25 <cfields> sipa: we don't use timeval in blocks either though, we convert from the clock
7132017-02-16T19:50:26 <sipa> but perhaps we should use those types in network state, measuring speed, ...
7142017-02-16T19:50:35 <cfields> sipa: i'll demonstrate with code, probably easier that way
7152017-02-16T19:50:53 <wumpus> right
7162017-02-16T19:51:13 <sipa> cfields: what i want to address is the fact that an int or int64 can now mean microseconds, milliseconds, or seconds, and either system time, or monotonous time, or network-adjusted timr
7172017-02-16T19:51:44 <sipa> it's fine that those are int-like, but they shouldn't be convertible from one into another
7182017-02-16T19:51:59 <cfields> sipa: and that's exactly what i've addressed. Each of those gets its own type, and they're not convertable to eachother. But you can do a duration_cast<std::chrono::seconds>(foo) and get an int64_t seconds value out
7192017-02-16T19:52:09 <sipa> anyway, for everything but consensus data structures, perhaps there are more c++ish ways
7202017-02-16T19:52:50 <sipa> cfields: let's discuss this outside of theeeting
7212017-02-16T19:52:51 <gmaxwell> This problem exists far beyond timestamps however. Use a node ID as a tx count? no problem. Use a vin index as a block number? no problem. Use a bytes sent as a relay bool? no problem.
7222017-02-16T19:53:16 <gmaxwell> We have multiple times had potentially serious bugs from the general issue of implicit conversions.
7232017-02-16T19:53:35 <cfields> gmaxwell: yes, agreed that the tag would be very useful
7242017-02-16T19:53:42 <gmaxwell> (or in the case of sighash single, an actual consensus behavior flaw)
7252017-02-16T19:53:50 <cfields> sipa: np
7262017-02-16T19:53:51 <sipa> jtimon had a topic as well
7272017-02-16T19:53:56 <wumpus> using enumerations instead of booleans would also go a long way
7282017-02-16T19:54:00 <jtimon> well, just a question really
7292017-02-16T19:54:03 <jonasschnelli> I also have a little proposal
7302017-02-16T19:54:04 <gmaxwell> all the data is there in the compile to prevent these mistakes, we're just not exposing it right. :)
7312017-02-16T19:54:13 <jtimon> I guess it can wait after the meeting
7322017-02-16T19:54:16 <wumpus> especially for functions that have tons of boolean arguments in succession, that's just crazy
7332017-02-16T19:54:17 <sipa> wumpus: yeah, generally useful
7342017-02-16T19:54:26 <gmaxwell> wumpus: yea, using bools, also using structs to get named parameters... lots of things we can do.
7352017-02-16T19:54:30 <cfields> wumpus: for sure
7362017-02-16T19:54:43 <jtimon> or answered fast enough that it doesn't need its own topic: "why 9619 doesn't go in for 0.14?"
7372017-02-16T19:54:48 <gmaxwell> true false true true false die die die. ... I hate counting aruments when changing things.
7382017-02-16T19:54:48 <morcos> jtimon: i think because there was an error reported and no explanation that it was fixed or wasn't really a problem. that's at least why i haven't looked at the PR.
7392017-02-16T19:54:53 <wumpus> (well with some IDEs you can see what gets assigned to what parameter because it parses the interface, but that's not something you can realy on that everyone has available)
7402017-02-16T19:55:00 <wumpus> gmaxwell: exactly
7412017-02-16T19:55:08 <wumpus> jtimon: what was your topic?
7422017-02-16T19:55:09 <instagibbs> gmaxwell, the fun really starts when you drop an arg with a default value at the end
7432017-02-16T19:55:21 <cfields> (or three)
7442017-02-16T19:55:28 <jtimon> morcos: that explains why is not merged, not why it's not labeled 0.14, but ok I guess
7452017-02-16T19:55:53 <wumpus> jtimon: let's reverse the question: why would it need to be added for 0.14?
7462017-02-16T19:55:55 <jtimon> wumpus: including 9619 in 0.14
7472017-02-16T19:56:11 <jtimon> it's a bugfix and is simple enough, why not?
7482017-02-16T19:56:41 <sipa> i think it can go in, it's trivial and very small in scope
7492017-02-16T19:56:45 <gmaxwell> basically without it a plausable downstream gbt user that changes the transaction set could construct an overlarge block, is that the concern there?
7502017-02-16T19:56:46 <wumpus> I'm fine with merging it before 0.14, if its sufficiently reviewed and teste
7512017-02-16T19:56:50 <wumpus> it will however not hold up rc1
7522017-02-16T19:57:08 <gmaxwell> it looks trivial and small in scope, and if my understanding is correct it should go in.. but sure, nothing should hold up rc1.
7532017-02-16T19:57:19 <gmaxwell> at least nothing that we know of now.
7542017-02-16T19:57:20 <sipa> fair
7552017-02-16T19:57:35 <jonasschnelli> Not sure if this makes sense, But I propose to rename the ./qa dir to ./test (or something that sorts after ./src). I think it would be better to have the RPC tests further down in the PRs diff.
7562017-02-16T19:57:36 <jtimon> sure, was simply surprised it didn't had the 0.14 label since it's not really that new
7572017-02-16T19:57:57 <Victorsueca> I see lots of utACKs but not any ACK, so trivial it doesn't need testing I assume?
7582017-02-16T19:57:57 <gmaxwell> short of some kind of crash eat money doom bug showing up before we manage to get it out (and even that shouldn't delay _constructing_ RC1; if doom shows up we can abort launch)
7592017-02-16T19:58:30 <wumpus> gmaxwell: sure, there are always serious enough problems possible that should postpone the release
7602017-02-16T19:58:57 <gmaxwell> jonasschnelli: On that I'd like to get into a state where make check is running those tests. They are most of our tests now, and the most valuable.. and really people building with compilers we've never seen before _REALLY_ ought to be running them.
7612017-02-16T19:59:18 <wumpus> jonasschnelli: don't really have an opinion on that, does it matter much how things are sorted?
7622017-02-16T19:59:35 <gmaxwell> Why would the sorting matter?
7632017-02-16T19:59:39 <jonasschnelli> As long as review is a bottleneck I think it matters
7642017-02-16T19:59:53 <jonasschnelli> But maybe I'm alone with that opinion.
7652017-02-16T20:00:05 <gmaxwell> OH so they show up lower on github.
7662017-02-16T20:00:21 <sipa> /zzztest
7672017-02-16T20:00:28 <wumpus> my biggest pet peeve is that they're called RPC tests, not 'functional tests' or such, they're testing much more than RPC these days :)
7682017-02-16T20:00:41 <jonasschnelli> ./test_functional
7692017-02-16T20:00:43 <gmaxwell> It is sometimes a bit confusing that the tests show up first... otoh it can be informative to read the test before the change in the cases where the test is especially good. :)
7702017-02-16T20:00:45 <wumpus> but not serious enough to actually go on renamind directories
7712017-02-16T20:00:49 <sipa> also, pull-tester should be merged in rpc-tests
7722017-02-16T20:00:51 <gmaxwell> wumpus: they are "system tests" rpc is incidental.
7732017-02-16T20:01:36 <sipa> we don't have pulltester anymore, and it's annoying to always change directory :)
7742017-02-16T20:01:44 <wumpus> gmaxwell: yes, it doesn't matter what interface they use, whether it's RPC or P2P or ZMQ or REST or command line arguments etc
7752017-02-16T20:01:44 <instagibbs> we're over time
7762017-02-16T20:01:46 <instagibbs> btw
7772017-02-16T20:01:52 <jonasschnelli> Okay. I'll do a cleanup PR once we branch off.
7782017-02-16T20:01:53 <wumpus> instagibbs: ah yes
7792017-02-16T20:01:55 <wumpus> #endmeeting
7802017-02-16T20:01:55 <lightningbot> Meeting ended Thu Feb 16 20:01:55 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
7812017-02-16T20:01:55 <lightningbot> Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-16-19.02.html
7822017-02-16T20:01:55 <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-16-19.02.txt
7832017-02-16T20:01:55 <lightningbot> Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-16-19.02.log.html
7842017-02-16T20:02:10 <jnewbery> wumpus: Agree, and can we name it something other than pull-tester?
7852017-02-16T20:02:17 <Chris_Stewart_5> sipa: gmaxwell So if I understand you guys, you want to create custom types for all time related structures inside of core?
7862017-02-16T20:02:23 <gmaxwell> petertodd: nice ACK on 9779.
7872017-02-16T20:02:33 <Chris_Stewart_5> even further, integer related structures?
7882017-02-16T20:02:49 <sipa> Chris_Stewart_5: well it seems c++11 has things like that already, and they're more featureful than just integers
7892017-02-16T20:02:51 <cfields> Chris_Stewart_5: i'm suggesting that we switch to std::chrono::duration and std::chrono::time_point
7902017-02-16T20:02:56 <wumpus> jnewbery: yes, pulltester is a weird name now, it's simply a launch framework for the tests
7912017-02-16T20:03:05 <Victorsueca> midnightmagic: "and really people building with compilers we've never seen before _REALLY_ ought to be running them." <--- I tried it once on windows but it looks like they're broken and only work on linux
7922017-02-16T20:03:12 <sipa> Chris_Stewart_5: but yes, making ints not convertible into other int-like-types can fix bugs
7932017-02-16T20:03:24 <jnewbery> test-launcher.py or test-runner.py seem fine
7942017-02-16T20:03:41 <gmaxwell> wumpus: re 9779, assuming that gets ACKs feel free to merge that along with the version bump. (the assumevalid was 6119 blocks behind.)
7952017-02-16T20:03:44 <wumpus> Victorsueca: that's simply not true! they work on at least openbsd, freebsd, linux, and windows (in wine at least)
7962017-02-16T20:03:49 <Chris_Stewart_5> sipa: Sure, no arguments with that. Just wasn't sure if that was _exactly_ what you guys were saying in the meeting.
7972017-02-16T20:03:53 <wumpus> Victorsueca: oh and OSX
7982017-02-16T20:04:05 <Victorsueca> wumpus: hmmm should try it again
7992017-02-16T20:04:07 <Chris_Stewart_5> sipa: But conversions will still be able to be done right? Just with explicit function calls instead of casting correct?
8002017-02-16T20:04:15 <wumpus> Victorsueca: our tests are very well maintained on many platforms
8012017-02-16T20:04:21 <sipa> Chris_Stewart_5: of course
8022017-02-16T20:04:35 <sipa> otherwise it's useless
8032017-02-16T20:04:39 <cfields> Chris_Stewart_5: only to a degree. You shouldn't be able to compare a system_clock time_point to a steady_clock one. Because that makes no sense.
8042017-02-16T20:04:49 <gmaxwell> They tests have extra dependencies so its quite plausable that on some systems that compile the code the tests fail.
8052017-02-16T20:05:17 <wumpus> cfields: or a duration to a absolute time value!
8062017-02-16T20:05:32 <cfields> wumpus: right
8072017-02-16T20:05:37 <Victorsueca> wumpus: which one should I be running to run all tests?
8082017-02-16T20:05:55 <Victorsueca> ahh I think I found it
8092017-02-16T20:05:57 <wumpus> gmaxwell: it should disable the tests in question in that case, e.g. if there's no zmq if doesn't run that test
8102017-02-16T20:06:01 <jonasschnelli> Victorsueca: ./qa/pull-tester/rpc-tests.py
8112017-02-16T20:06:12 <jtimon> wumpus: right, functions with many bools could use thier own flags or something perhaps (sorry, was catching up on the previous discussion)
8122017-02-16T20:06:26 <jonasschnelli> Victorsueca: mind the -win parameter
8132017-02-16T20:06:34 <wumpus> jtimon: yes, either bitfield or individual enums, everything is better than bool,bool,bool,bool
8142017-02-16T20:06:52 <cfields> so instead, we get the concept of a clock, which gives you time_point's with some meaning. a time_point, which is an absolute time (but not necessarily wall-time) on some clock, and a duration, which is the difference between 2 time_points
8152017-02-16T20:07:00 <sipa> wumpus: for cases where it's excessive you can emulate named arguments using a struct
8162017-02-16T20:07:10 <jnewbery> jonasschnelli: My preference would be to also move the bitcoin-util-test.py and bctest.py (and data for those tests) from src/test to whatever you rename qa to. Those tests seem much more like the integration tests in qa than the unit tests in src/test
8172017-02-16T20:07:12 <jtimon> and I really hate we have so many bools with default values
8182017-02-16T20:07:35 <sipa> wumpus: for example a struct ATMPArgs that takes in its constructor all 'required' fields for ATMP, and has methods to change optionals
8192017-02-16T20:07:46 <Chris_Stewart_5> cfields: Wouldn't that comparison have to be enforced through code review and not a compile time failure? Sorry not super familar with the data structures..
8202017-02-16T20:07:49 <jonasschnelli> jnewbery: Indeed. I'll try to move them.
8212017-02-16T20:07:58 <gmaxwell> I believe a struct for inputs can be nearly zero runtime overhead for functions with many arguments.
8222017-02-16T20:08:17 <sipa> Chris_Stewart_5: it will result in compile time failure; a timepoint for a steady clock would be a different data type
8232017-02-16T20:08:28 <wumpus> sipa: c99 designated struct initializers were always nice to use for emulating named arguments in C, too bad that never made it to C++
8242017-02-16T20:08:32 <cfields> Chris_Stewart_5: no, it's enforced by the compiler. A time_point knows what clock it belongs to
8252017-02-16T20:08:50 <sipa> So you'd call AcceptToMemoryPool(ATMPArgs{std::move(tx)}.max_fee(fee))
8262017-02-16T20:08:57 <gmaxwell> Chris_Stewart_5: enforcing things through code review that could be enforced by the machine is a losing strategy. Code review is essential but it should be focused on the bugs the machine cannot prevent.
8272017-02-16T20:08:59 <petertodd> gmaxwell: heh, yeah, I wanted to make a point of what the trust model was, particularly since -assumevalid was (partly) my idea! :)
8282017-02-16T20:09:10 <Victorsueca> jonasschnelli: wumpus: File "rpc-tests.py", line 283 print('.', end='', flush=True) SyntaxError: invalid syntax
8292017-02-16T20:09:15 <Victorsueca> uhmm shit hit the fan?
8302017-02-16T20:09:21 <wumpus> sipa: in any case, we also shouldn't go over the top with this, trying to emulate something in c++ that it is not :)
8312017-02-16T20:09:23 <jonasschnelli> Victorsueca: python3?
8322017-02-16T20:09:35 <Chris_Stewart_5> gmaxwell: No doubt. I'm all for type safety. Compilers are our friends :-)
8332017-02-16T20:09:37 <jonasschnelli> (requires p3)
8342017-02-16T20:09:40 <sipa> Victorsueca: looks like you're trying to run with python2
8352017-02-16T20:09:57 <Victorsueca> ahh forgot python stands for 2 and py stands for 3-2 hub
8362017-02-16T20:10:25 <sipa> Victorsueca: no, the first line of the file states the interpreter to use, but i guess that only works in unix-like systems
8372017-02-16T20:10:44 <wumpus> you should run all the bitcoin core scripts with python 3
8382017-02-16T20:10:49 <jtimon> gmaxwell: ack on adding the rpc tests on make check
8392017-02-16T20:11:07 <Victorsueca> here on windows if I run py it auto-detects whether 2 or 3 should be used
8402017-02-16T20:11:25 <Victorsueca> it's just i'm used to write python
8412017-02-16T20:11:46 <gmaxwell> wumpus: MISRA C (standard for safety critical C code targeting embedded systems) requires that no generic types be used, and certantly no sizeless generic types. And that is in C where you still get the implicit conversion and can't really avoid it, but the standard argues that static analysis tools can still catch your errors so long as you've exposed the type information. Libsecp256k1 is pre
8422017-02-16T20:11:52 <gmaxwell> tty close to this now.
8432017-02-16T20:12:31 <wumpus> well if the first line contains python3 it should be run with python 3, lacking that there is no general way to detect whether code is python2 or python3. But believe me, the bitcoin core python scripts need to be run with python 3.
8442017-02-16T20:13:43 <wumpus> gmaxwell: the most significant non-MISRA thing about secp256k1 is probably that it allocates a context on the heap?
8452017-02-16T20:15:07 <bitcoin-git> [bitcoin] jnewbery opened pull request #9780: Suppress noisy output from qa tests in Travis (master...travislogging) https://github.com/bitcoin/bitcoin/pull/9780
8462017-02-16T20:16:09 <gmaxwell> wumpus: Well MISRA doesn't actually forbid anything but requires that for every violation you write an exception document. So the context is one where the exception document is very easy to right: this allocation is pure initilization, not runtime.
8472017-02-16T20:17:17 <gmaxwell> What I've found harder to deal with is that it requires no function have multiple returns. As there are some cases where we do where fixing it feels like it would objectively worsen the code. But their argument is also pretty compelling
8482017-02-16T20:18:20 <wumpus> yes, there's something to be said for that, especially with regard to cleanup and such
8492017-02-16T20:18:37 <gmaxwell> It would also be very easy for us to make it possible to make libsecp256k1 mallocless with ifdefs, it's specifically structured to enable that.
8502017-02-16T20:18:38 <wumpus> though in some cases it can lead to terribly verbose code when combined with error handling
8512017-02-16T20:19:01 <wumpus> e.g. either deeply nested if() or some status flag that is checked every other statement
8522017-02-16T20:19:28 <cfields> wumpus: but those easily fixed with a goto ;)
8532017-02-16T20:19:28 <gmaxwell> (My goal though not a priority is to make libsecp256k1 MISRA conformant eventually)
8542017-02-16T20:19:47 <gmaxwell> cfields: forbidden by MISRA C. (though like anything else you can make exceptions)
8552017-02-16T20:19:55 <jtimon> jonasschnelli: regarding your rename, if rpc-tests/ and pull-tester/ in qa (as I think sipa suggests ), it may make sense to take the opportunity to rename qa as well
8562017-02-16T20:19:56 <wumpus> cfields: hah! I didn't expect those to be allowed if multiple returns are not
8572017-02-16T20:20:08 <cfields> gmaxwell: was just joking about trading one thing for another
8582017-02-16T20:20:17 <gmaxwell> Gotos that go backwards also forbidden by an extra rule, so at least error handling doesn't hit those.
8592017-02-16T20:20:19 <wumpus> cfields: I always tend to use goto's in C code for cleanups, kernel style
8602017-02-16T20:21:03 <gmaxwell> cfields: well I wasn't goto is actually an excellent way to handle error handling in C in some cases. It's very similar to multiple returns in its downsides, and yet few people care about peppering functions with 15 returns.
8612017-02-16T20:21:23 <cfields> wumpus: yes, they seem pretty necessary after using raii for long enough
8622017-02-16T20:22:32 <gmaxwell> The MISRA document is a really good read FWIW. It's pretty thoughtful in its recommendations.
8632017-02-16T20:22:50 <gmaxwell> though not so applicable to C++. :)
8642017-02-16T20:22:52 <cfields> gmaxwell: sure, agreed. I just used it as an example because it's often multiple-returns and gotos blindly spouted as "thou-shalt-not"s in c-code
8652017-02-16T20:24:42 <wumpus> I can't come up with a good reason to use backwards gotos though
8662017-02-16T20:24:44 <cfields> gmaxwell: well, no-multiple-returns carries some weight in c++ too as they inhibit rvo. Which can be very useful for structures that you assume move freely/cheaply
8672017-02-16T20:25:29 <Victorsueca> ok... now this is a mess.... I use WSL to cross-compile windows binaries, now all the tests have UNIX-like paths and can't run them on windows
8682017-02-16T20:27:10 <isle2983> gmaxwell: do you try any clang/LLVM plugins for checking MISRA? I am just wondering if it is worthwhile exploring
8692017-02-16T20:27:20 <wumpus> gcc computed goto is even more evil: arrays of labels, though it's used in e.g. the python bytecode dispatch loop for the last bits of speed optimization
8702017-02-16T20:27:52 <gmaxwell> isle2983: not clang/llvm ones, but I've used other static analysis tools that can check it, in particular pc-lint... They're of some value.
8712017-02-16T20:28:56 <wumpus> Victorsueca: strange - python code doesn't get compiled as such, it must be one of the generated py files that has a path in it. You could try explicitly setting the environment variable BITCOIND to the path of bitcoind, that will override the built-in guess
8722017-02-16T20:29:16 <gmaxwell> wumpus: some kinds of valid but complex flow control is just not expressable in C without a loss of efficiency. So you could imagine some kind of hyper optimized inner loop that uses backwards goto. I don't recall any case where I've been forced into that, though I have run into cases where performance required do{}while() and non-trivial code inside for expressions, and forwards goto out of a
8732017-02-16T20:29:22 <gmaxwell> loop that also had breaks and continues.
8742017-02-16T20:30:02 <wumpus> gmaxwell: yes, forwards goto out of a loop makes a lot of sense, if you want to break out of multiple levels
8752017-02-16T20:30:52 <gmaxwell> I wish C(++) had named continue/break.
8762017-02-16T20:31:08 <wumpus> gmaxwell: and these things matter, I mean if efficiency is not important you wouldn't be using c in the first place
8772017-02-16T20:31:30 <gmaxwell> (Rust has named breaks)
8782017-02-16T20:31:46 <gmaxwell> (also more important because it lacks the other options C has for that)
8792017-02-16T20:31:50 <wumpus> yes I remember quite some languages had them ,though I can't think of any right now
8802017-02-16T20:32:22 <wumpus> didn't know that rust did
8812017-02-16T20:33:22 <sipa> in c++ it's more complicated because you can't jump over variable initializations
8822017-02-16T20:34:07 <cfields> gmaxwell: i suspect that architects assumed throw/catch would be used for breaking out of multiple loops. But that's seen as poisonous to most, i think
8832017-02-16T20:34:18 <wumpus> ah, java has them apparently
8842017-02-16T20:34:25 <gmaxwell> cfields: youch. :P
8852017-02-16T20:34:46 <cfields> gmaxwell: my thought exactly :)
8862017-02-16T20:34:51 <wumpus> cfields: exceptions as a control flow statement? speak about shooting a fly with a thermonuclear missile :p
8872017-02-16T20:35:02 <gmaxwell> beyond the other issues with exception, they're slow... sooo.
8882017-02-16T20:35:44 <wumpus> right, they're slow exactly because they don't know where they're going to be catched, so they're a bad fit if you know exactly where you wantcontrol flow to go
8892017-02-16T20:36:17 <wumpus> the code for exceptions and stack unrolling is extremely complex, I looked at it some time there's even some cute VM involved that interprets DWARF debug info
8902017-02-16T20:37:05 <cfields> wumpus: heh, i once worked on some code that used it to parse some data, construct the right handler, and throw it. Then the catch was an abstract parent, and it would continue on with the newly-constructed handler
8912017-02-16T20:37:27 <wumpus> cfields: hahaha
8922017-02-16T20:37:28 <cfields> it was ugly, but surprisingly effective :)
8932017-02-16T20:37:32 <wumpus> cfields: anti-reverse-engineering?
8942017-02-16T20:38:00 <cfields> wumpus: heh, it would seem. no idea what the historical reasoning was
8952017-02-16T20:38:52 <gmaxwell> later you find out the guy that wrote it didn't know that function pointers or virtual methods were possible. :P
8962017-02-16T20:39:02 <wumpus> like anytiing so evil it sounds like a hell to debug
8972017-02-16T20:39:07 <cfields> gmaxwell: entirely possible :)
8982017-02-16T20:40:53 <wumpus> it reminds me of that saying that you should always expect that the person maintaining the code after you is a crazy axe murderer that knows where you live, this certainly classifies as inhuman cruelty to your successors
8992017-02-16T20:41:26 <jeremyrubin> sorry for misisng meeting! I was off by an hour :p
9002017-02-16T20:41:37 <jeremyrubin> I think w.r.t. entropy levels we should keep 3
9012017-02-16T20:41:51 <jeremyrubin> Deterministic, non-deterministic, secure
9022017-02-16T20:41:52 <cfields> std::chrono::off_by_one_clock
9032017-02-16T20:42:08 <jeremyrubin> deterministic is really useful for writing tests/benchmarks IMO
9042017-02-16T20:42:28 <wumpus> it's only useful for the tests and benchmarks, and could be in test_util.cpp
9052017-02-16T20:42:55 <jeremyrubin> That's fine to me!
9062017-02-16T20:42:56 <cfields> jeremyrubin: we have siphasher, which we use for deterministic randomness, and i don't think it really needs to care about the source of its seed?
9072017-02-16T20:43:27 <wumpus> but I agree it's useful to have fast deterministic random for some things, I manually rolled a LFSR in one place in the tests to quickly generate deterministic "random" numbers
9082017-02-16T20:44:03 <jeremyrubin> cfields: fine too -- just having something convenient to use for this purpose is all I care for!
9092017-02-16T20:45:03 <jeremyrubin> The cuckoocache tests test hit rate against deterministic data, so if the entropy changes then it *could* (unlikely) fail.
9102017-02-16T20:45:51 <jeremyrubin> but it depends on the variance of the hit rate... so it would be annoying if it fails say 1/100 times due to unlucky entropy
9112017-02-16T20:46:34 <wumpus> yes, that should ideally never happen
9122017-02-16T20:47:14 <wumpus> tests that randomly fail are extremely discouraging
9132017-02-16T20:48:18 <jeremyrubin> Indeed
9142017-02-16T20:48:38 <jeremyrubin> travis-gate was really a sad week for bitcoin ;)
9152017-02-16T20:52:54 <cfields> heh
9162017-02-16T20:59:16 <jeremyrubin> Speaking of tests -- any chance for putting #9495 (bugfix) and #9497 (tests) in 0.14? The CheckQueue is a pretty big piece of consensus code with almost no test coverage on it... Then again, there's no reason these tests can't wait for 0.15 either :)
9172017-02-16T20:59:18 <gribble> https://github.com/bitcoin/bitcoin/issues/9495 | Fix CCheckQueue IsIdle (potential) race condition by JeremyRubin · Pull Request #9495 · bitcoin/bitcoin · GitHub
9182017-02-16T20:59:20 <gribble> https://github.com/bitcoin/bitcoin/issues/9497 | CCheckQueue Unit Tests by JeremyRubin · Pull Request #9497 · bitcoin/bitcoin · GitHub
9192017-02-16T21:15:54 <jnewbery> jonasschnelli: I've got several PRs that are very disruptive to the qa directory: #9577 #9657 #9768 and the completion of #9707 (which I haven't yet PR'ed). Can you hold off your reorganzation of the qa directory until those are merged? Perhaps open an issue now for discussion but hold off on PR'ing.
9202017-02-16T21:15:56 <gribble> https://github.com/bitcoin/bitcoin/issues/9577 | Fix docstrings in qa tests by jnewbery · Pull Request #9577 · bitcoin/bitcoin · GitHub
9212017-02-16T21:15:58 <gribble> https://github.com/bitcoin/bitcoin/issues/9657 | Improve rpc-tests.py by jnewbery · Pull Request #9657 · bitcoin/bitcoin · GitHub
9222017-02-16T21:16:00 <gribble> https://github.com/bitcoin/bitcoin/issues/9768 | [qa] [WIP] Add logging to test_framework.py by jnewbery · Pull Request #9768 · bitcoin/bitcoin · GitHub
9232017-02-16T21:16:02 <gribble> https://github.com/bitcoin/bitcoin/issues/9707 | Fix RPC failure testing by jnewbery · Pull Request #9707 · bitcoin/bitcoin · GitHub
9242017-02-16T21:31:34 <jeremyrubin> jnewbery: git can handle renames fine iirc?
9252017-02-16T21:32:56 <jnewbery> jeremyrubin: I think you're right. If jonas is only going to rename files then there shouldn't be any conflicts.
9262017-02-16T21:33:38 <jeremyrubin> jonasschnelli: ^^^
9272017-02-16T21:33:55 *** Guyver2 has joined #bitcoin-core-dev
9282017-02-16T21:50:03 *** Guyver2 has quit IRC
9292017-02-16T21:59:37 *** aj_ is now known as aj
9302017-02-16T22:10:46 *** wvr has quit IRC
9312017-02-16T22:11:24 *** wvr has joined #bitcoin-core-dev
9322017-02-16T22:13:08 *** chjj has quit IRC
9332017-02-16T22:22:22 *** chjj has joined #bitcoin-core-dev
9342017-02-16T23:09:41 *** face has quit IRC
9352017-02-16T23:11:59 *** face has joined #bitcoin-core-dev
9362017-02-16T23:15:27 *** paracyst_ is now known as paracyst
9372017-02-16T23:18:40 *** jnewbery has quit IRC
9382017-02-16T23:18:54 *** jnewbery has joined #bitcoin-core-dev
9392017-02-16T23:24:57 *** laurentmt has joined #bitcoin-core-dev
9402017-02-16T23:26:30 *** laurentmt has quit IRC
9412017-02-16T23:36:27 *** chjj has quit IRC
9422017-02-16T23:44:24 *** aalex has quit IRC
9432017-02-16T23:50:05 *** Giszmo has quit IRC
9442017-02-16T23:56:37 *** Giszmo has joined #bitcoin-core-dev