1 2017-02-16T00:10:34  *** Chris_Stewart_5 has quit IRC
  2 2017-02-16T00:13:00  *** Chris_Stewart_5 has joined #bitcoin-core-dev
  3 2017-02-16T00:48:56  *** abpa has quit IRC
  4 2017-02-16T01:21:05  <bitcoin-git> [bitcoin] jmcorgan opened pull request #9774: Enable host lookups for -proxy and -onion parameters (master...hostname-lookups) https://github.com/bitcoin/bitcoin/pull/9774
  5 2017-02-16T01:27:37  *** MarcoFalke has joined #bitcoin-core-dev
  6 2017-02-16T01:41:08  *** MarcoFalke has quit IRC
  7 2017-02-16T01:54:00  *** Ylbam has quit IRC
  8 2017-02-16T02:11:09  *** chjj has quit IRC
  9 2017-02-16T03:10:40  *** jtimon has quit IRC
 10 2017-02-16T03:11:05  *** Victor_sueca has joined #bitcoin-core-dev
 11 2017-02-16T03:14:14  *** Victorsueca has quit IRC
 12 2017-02-16T03:57:43  *** goksinen has quit IRC
 13 2017-02-16T03:58:31  *** goksinen has joined #bitcoin-core-dev
 14 2017-02-16T04:03:27  *** goksinen has quit IRC
 15 2017-02-16T04:07:46  *** PRab has joined #bitcoin-core-dev
 16 2017-02-16T04:16:17  *** PRab has quit IRC
 17 2017-02-16T04:19:25  *** roasbeef_ is now known as roasbeef
 18 2017-02-16T04:53:48  *** e4xit has quit IRC
 19 2017-02-16T04:54:45  *** e4xit has joined #bitcoin-core-dev
 20 2017-02-16T05:33:20  *** sanada has joined #bitcoin-core-dev
 21 2017-02-16T05:47:57  *** justanotheruser has joined #bitcoin-core-dev
 22 2017-02-16T05:58:56  *** Victor_sueca has quit IRC
 23 2017-02-16T06:00:03  *** Victor_sueca has joined #bitcoin-core-dev
 24 2017-02-16T06:23:09  *** Victor_sueca is now known as Victorsueca
 25 2017-02-16T06:31:15  *** cannon-c has joined #bitcoin-core-dev
 26 2017-02-16T06:42:44  *** Ylbam has joined #bitcoin-core-dev
 27 2017-02-16T07:18:24  *** d9b4bef9 has quit IRC
 28 2017-02-16T07:19:07  *** d9b4bef9 has joined #bitcoin-core-dev
 29 2017-02-16T07:37:31  *** BashCo has quit IRC
 30 2017-02-16T08:02:55  *** BashCo has joined #bitcoin-core-dev
 31 2017-02-16T08:09:23  *** Guest33433 has joined #bitcoin-core-dev
 32 2017-02-16T08:16:24  *** cannon-c has quit IRC
 33 2017-02-16T08:22:41  *** cannon-c has joined #bitcoin-core-dev
 34 2017-02-16T08:35:38  *** Guest33433 has quit IRC
 35 2017-02-16T08:42:20  *** goksinen has joined #bitcoin-core-dev
 36 2017-02-16T08:46:57  *** goksinen has quit IRC
 37 2017-02-16T08:53:40  *** Giszmo has quit IRC
 38 2017-02-16T09:13:15  *** goksinen has joined #bitcoin-core-dev
 39 2017-02-16T09:13:52  *** BashCo_ has joined #bitcoin-core-dev
 40 2017-02-16T09:17:38  *** BashCo has quit IRC
 41 2017-02-16T09:17:57  *** goksinen has quit IRC
 42 2017-02-16T09:21:00  *** Guyver2 has joined #bitcoin-core-dev
 43 2017-02-16T09:23:09  <wumpus> I think today is a good day to wrap up 0.14
 44 2017-02-16T09:24:08  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7a93af8340d9...1e92e041ddc8
 45 2017-02-16T09:24:09  <bitcoin-git> bitcoin/master ba803ef Suhas Daftuar: Harden against mistakes handling invalid blocks...
 46 2017-02-16T09:24:09  <bitcoin-git> bitcoin/master 1e92e04 Wladimir J. van der Laan: Merge #9765: Harden against mistakes handling invalid blocks...
 47 2017-02-16T09:24:29  <bitcoin-git> [bitcoin] laanwj closed pull request #9765: Harden against mistakes handling invalid blocks (master...fix-checkblock-call) https://github.com/bitcoin/bitcoin/pull/9765
 48 2017-02-16T09:24:46  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/1e92e041ddc8...f8af89a91820
 49 2017-02-16T09:24:46  <bitcoin-git> bitcoin/master 6c5427d Wladimir J. van der Laan: wallet: Prevent "overrides a member function but is not marked 'override'" warnings...
 50 2017-02-16T09:24:47  <bitcoin-git> bitcoin/master f8af89a Wladimir J. van der Laan: Merge #9764: wallet: Prevent "overrides a member function but is not marked 'override'" warnings...
 51 2017-02-16T09:25:06  <bitcoin-git> [bitcoin] laanwj closed pull request #9764: wallet: Prevent "overrides a member function but is not marked 'override'" warnings (master...2017_02_wallet_inconsistent_missing_override) https://github.com/bitcoin/bitcoin/pull/9764
 52 2017-02-16T09:30:54  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f8af89a91820...e43a58514dd3
 53 2017-02-16T09:30:54  <bitcoin-git> bitcoin/master 07afcd6 Russell Yanofsky: Add missing cs_wallet lock that triggers new lock held assertion...
 54 2017-02-16T09:30:55  <bitcoin-git> bitcoin/master e43a585 Wladimir J. van der Laan: Merge #9771: Add missing cs_wallet lock that triggers new lock held assertion...
 55 2017-02-16T09:31:15  <bitcoin-git> [bitcoin] laanwj closed pull request #9771: Add missing cs_wallet lock that triggers new lock held assertion (master...pr/loadlock) https://github.com/bitcoin/bitcoin/pull/9771
 56 2017-02-16T09:35:31  <luke-jr> should #9524 be marked 0.14?
 57 2017-02-16T09:35:32  <gribble> https://github.com/bitcoin/bitcoin/issues/9524 | rpc: Dont FlushStateToDisk when pruneblockchain(0) by MarcoFalke · Pull Request #9524 · bitcoin/bitcoin · GitHub
 58 2017-02-16T09:36:45  <wumpus> I don't know. I'd say let's not add more stuff now and try to get a rc1 wrapped up
 59 2017-02-16T09:37:23  <wumpus> it's a release candidate to get more testing
 60 2017-02-16T09:37:32  <wumpus> issues will always be found in RCs of major releases
 61 2017-02-16T09:37:36  <luke-jr> sure
 62 2017-02-16T09:37:42  <gmaxwell> 9524 looks harmless.
 63 2017-02-16T09:37:49  <wumpus> we can keep pushing this forward indefinitely and just not do releases anymore :/
 64 2017-02-16T09:37:56  <luke-jr> not saying it needs to get into rc1, just hoping to avoid thigns being overlooked before final
 65 2017-02-16T09:37:58  <gmaxwell> wumpus: would solve so many problems!
 66 2017-02-16T09:38:17  <gmaxwell> luke-jr: I don't think that should go into 0.14.0, unless it has some effect I'm missing.
 67 2017-02-16T09:38:37  <wumpus> it's under-discussed and under-reviewed
 68 2017-02-16T09:38:49  <gmaxwell> by harmless I mean don't do it.
 69 2017-02-16T09:38:54  <gmaxwell> The question is "what bad expirence will the user or network have as a result of this not going in" if the answer is none we shouldn't even be remotely considering it now.
 70 2017-02-16T09:39:10  <gmaxwell> I mean the thing it fixes does not meet the above test ^, sorry for being unclear.
 71 2017-02-16T09:39:15  <wumpus> oh I agree then :)
 72 2017-02-16T09:39:15  <luke-jr> pruneblockchain(0) would probably delete a lot of data the user doesn't intend
 73 2017-02-16T09:39:29  <gmaxwell> okay, that would be bad.
 74 2017-02-16T09:39:51  <gmaxwell> wait, the argument there is what, /me looks
 75 2017-02-16T09:40:18  <luke-jr> at least as I understand it, the 0 indicates the node should automatically prune up to tip minus saved-amount
 76 2017-02-16T09:40:23  <wumpus> your are misunderstanding it luke-jr
 77 2017-02-16T09:40:28  <wumpus> argument 0 means *do nothing*
 78 2017-02-16T09:40:35  <wumpus> it doesn't delete anythhing in that case
 79 2017-02-16T09:40:37  <wumpus> it just flushes to disk
 80 2017-02-16T09:40:42  <wumpus> who cares :)
 81 2017-02-16T09:41:00  <gmaxwell> That was what I got out of the PR.
 82 2017-02-16T09:41:14  <wumpus> that PR bypasses the flush, I didn't regard that as very important as it's a NOOP anyway
 83 2017-02-16T09:41:19  <gmaxwell> If it did what luke said-- prune as hard as allowed--, that would be worth fixing before final.
 84 2017-02-16T09:41:26  <luke-jr> what am I missing?
 85 2017-02-16T09:41:57  <wumpus> what you are missing is that pruneblockchain(0) already does nothing besides a FlushToDIsk
 86 2017-02-16T09:42:00  <luke-jr> FlushStateToDisk with 0 triggers FindFilesToPrune(setFilesToPrune, chainparams.PruneAfterHeight()); etc
 87 2017-02-16T09:42:06  <wumpus> that PR takes away the FlushToDisk
 88 2017-02-16T09:42:24  <wumpus> "It might cause unwanted bugs if someone refactors the code in the future."
 89 2017-02-16T09:42:29  <wumpus> that's all
 90 2017-02-16T09:43:51  <wumpus> which we can consider for 0.15, but there will be no refactors of that code in the future of 0.14, so it's pointless there
 91 2017-02-16T09:43:52  * gmaxwell risks his laptop node.
 92 2017-02-16T09:44:27  *** goksinen has joined #bitcoin-core-dev
 93 2017-02-16T09:47:24  <gmaxwell> luke-jr: so just calling that with 0 throws cannot prune because not in prune mode.
 94 2017-02-16T09:47:41  <luke-jr> what manual pruning is enabled?
 95 2017-02-16T09:47:56  <gmaxwell> trying.
 96 2017-02-16T09:48:35  <gmaxwell> pfft. it's yelling at me because of txindex. @#$@
 97 2017-02-16T09:48:42  <luke-jr> >_<
 98 2017-02-16T09:48:57  *** goksinen has quit IRC
 99 2017-02-16T09:49:13  <gmaxwell> so to do that test I'd have to reindex which would probably take days on this thing. :-/ and my other nodes are busy now with important test.s
100 2017-02-16T09:49:42  <gmaxwell> I will test before 0.14 release however, if no one else does.
101 2017-02-16T09:49:55  <Victorsueca> need testing?
102 2017-02-16T09:50:10  <gmaxwell> if you don't mind blowing away your blockchain if luke is right.
103 2017-02-16T09:50:14  <wumpus> ideally *disabling* txindex would not require a rescan, but meh :) I'll test it
104 2017-02-16T09:50:27  * wumpus well test, has about 50 copies of the blockchain anyhow
105 2017-02-16T09:50:50  <gmaxwell> Start a node without txindex with prune=1, then run pruneblockchain 0 and verify all your blocks didn't just suddenly go away (e.g. freeing up 100 GB of disk space).
106 2017-02-16T09:51:37  <Victorsueca> ummm, I'll make a backup of my current chain, it probably won't be in time to test this one, but i'll be ready for future chain-blowing PRs
107 2017-02-16T09:52:33  *** JackH has joined #bitcoin-core-dev
108 2017-02-16T09:52:41  <wumpus> possibly no need to even copy, would hard-linking all but the latest block file work?
109 2017-02-16T09:52:47  <luke-jr> not sure we have a way to restore such a backup without the chainstate
110 2017-02-16T09:52:55  <wumpus> okay, definitely not going to try that at the same time
111 2017-02-16T09:53:36  <wumpus> luke-jr: why is it a problem in this case to copy the chainstate too?
112 2017-02-16T09:53:55  <luke-jr> wumpus: it might not be, just saying in case copying it also wasn't obvious
113 2017-02-16T09:53:55  <gmaxwell> FS with cow would be nice for some of these tests.
114 2017-02-16T09:54:18  * luke-jr does have a CoW filesystem, but also has txindex enabled etc.
115 2017-02-16T09:54:34  <wumpus> luke-jr: if you don't copy the chainstate, all it can do is a reindex
116 2017-02-16T09:55:14  <luke-jr> I suppose a unit test could check this, but seems probably too bug-specific
117 2017-02-16T09:55:24  <luke-jr> s/unit/rpc/
118 2017-02-16T09:55:46  <wumpus> well if this turns out to be a regression then a regression test could be added for it, but if this never was a bug in the first place...
119 2017-02-16T09:55:59  <luke-jr> 0.13 didn't support manual pruning afaik
120 2017-02-16T09:56:01  <wumpus> then we're just chasing ghosts
121 2017-02-16T09:57:12  <wumpus> though testing it in regtest instead of a live node is a very good idea
122 2017-02-16T09:59:47  <Victorsueca> so, if I copy the blocks and chainstate folder it should be enough right?
123 2017-02-16T09:59:59  <wumpus> yes
124 2017-02-16T10:00:18  <luke-jr> Victorsueca: may need to be sure the node is stopped when you copy
125 2017-02-16T10:00:31  <Victorsueca> ok
126 2017-02-16T10:01:16  <wumpus> oh absolutely, don't do anything low-level with the data files without stopping, that shouldn't have to be said
127 2017-02-16T10:02:07  <Victorsueca> yeah it's stopped, the windows that tells me to wait disappeared too
128 2017-02-16T10:02:17  <wumpus> on windows, copying *from* the bitcoin folder while bitcoin core is running can even cause it to crash
129 2017-02-16T10:02:30  <gmaxwell> stupid file locking
130 2017-02-16T10:02:34  <Victorsueca> rlly? copying?
131 2017-02-16T10:02:56  <wumpus> https://github.com/bitcoin/bitcoin/issues/8250
132 2017-02-16T10:03:42  <wumpus> not sure why, I'm not aware ldb does anything special while opening the files
133 2017-02-16T10:03:56  <Victorsueca> maybe something related with atomic locks?
134 2017-02-16T10:04:33  * luke-jr found it curious to see a Windows user actively using a "file unlocker" program like it was normal
135 2017-02-16T10:05:31  <Victorsueca> windows file locking is great at preventing programs from crashing due to files that where expected to be there and now are not
136 2017-02-16T10:05:40  <Victorsueca> but it's stupid in most cases
137 2017-02-16T10:05:50  <wumpus> yes, it's great at preventing programs from crashing... we see that :-)
138 2017-02-16T10:05:59  <Victorsueca> lol
139 2017-02-16T10:06:16  <Victorsueca> let's just say the remedy is worst than the ill
140 2017-02-16T10:06:29  <Victorsueca> it could be way better implemented
141 2017-02-16T10:06:58  <wumpus> it's probably not that bad if you take extensive time to study the APIs involved and use them as they are supposed to be, but who has time to do native development for every special snowflake platform :/
142 2017-02-16T10:07:07  <midnightmagic> pure snapshotting would work just fine for this, if you could convince the software to lock all files prior to the snapshot, or guarantee a write barrier
143 2017-02-16T10:08:15  <wumpus> the annoying thing with microsoft has always been that they take existing concepts then change them slightly, just enough to be incompatible and lock you in a bit
144 2017-02-16T10:08:26  <wumpus> POSIX? forget it
145 2017-02-16T10:10:04  <midnightmagic> windows file locking works fine for monolithic files that are only modified by the software modifying them, even in 10,000+ user database environments far busier than I suspect bitcoin's databases will ever be.
146 2017-02-16T10:10:06  <luke-jr> I thought Windows was POSIX compatible?
147 2017-02-16T10:10:58  <midnightmagic> Windows has POSIX emulation layers but they all operate differently than UNIX. For example, when a file is locked, it is a much harder lock than its POSIX-style equivalents on other systems.
148 2017-02-16T10:11:16  <wumpus> it tries, a bit, the devil is in the details
149 2017-02-16T10:11:30  *** laurentmt has joined #bitcoin-core-dev
150 2017-02-16T10:11:46  <midnightmagic> You can delete a file which is locked, which zeroes it, but you can't subsequently recreate a file with the same *name*.
151 2017-02-16T10:11:51  <midnightmagic> (for example)
152 2017-02-16T10:12:02  <luke-jr> O.o
153 2017-02-16T10:12:09  <midnightmagic> (external to the program which has the file locked)
154 2017-02-16T10:13:07  <midnightmagic> There are other operations which are virtually impossible to guarantee the atomicity of, even with the POSIX compatibility layer because all the things we take for granted are emulated badly, as wumpus implies.
155 2017-02-16T10:13:40  <Victorsueca> I think the problem is at assertions
156 2017-02-16T10:13:54  <Victorsueca> windows file lock asserts a lo of thing that are not always true
157 2017-02-16T10:14:00  <Victorsueca> lot*
158 2017-02-16T10:14:04  <wumpus> yes it's not that windows's way doesn't work, or even works worse in the abstract sense, it's just that they force you to develop platform specific code for everything
159 2017-02-16T10:14:42  <wumpus> they did that with directx/opengl, with winsock, and the list goes on
160 2017-02-16T10:14:42  <midnightmagic> Meanwhile, on Windows it's possible to hook file open calls with e.g. realtime virus, but contrary to anti-virus checkboxes, it's actually not possible to apply it selectively to a path and not other paths. It's all-or-nothing. That's why basically anyone running a database with an active realtime virus checker is sitting on a dice-rolling timeline. Eventually, their database will be
161 2017-02-16T10:14:46  <wumpus> anyhow, I was testing pruning
162 2017-02-16T10:14:48  <midnightmagic> destroyed. And there's absolutely nothing anybody can do about it.
163 2017-02-16T10:16:22  *** goksinen has joined #bitcoin-core-dev
164 2017-02-16T10:17:36  <wumpus> okay, tested: pruneblockchain 0 does nothing and returns immediately.
165 2017-02-16T10:17:51  <wumpus> Arguments: "height"       (numeric, required) The block height to prune up to. May be set to a discrete height, or to a unix timestamp to prune based on block time.
166 2017-02-16T10:20:57  *** goksinen has quit IRC
167 2017-02-16T10:21:31  <Victorsueca> wumpus: was that on linux or windows?
168 2017-02-16T10:22:21  <wumpus> linux
169 2017-02-16T10:22:29  * luke-jr wonders why
170 2017-02-16T10:22:50  <Victorsueca> is it useful if I test it on windows now or it's about the same for this case?
171 2017-02-16T10:22:51  <wumpus> well 0 is the genesis block, so pruning from there means not pruning at all
172 2017-02-16T10:22:58  <wumpus> no, it's not useful to test this on other OSes
173 2017-02-16T10:23:21  <luke-jr> wumpus: yes, but the code special-cases 0 as automatic
174 2017-02-16T10:24:08  <Victorsueca> maybe that automatic check thought no pruning was needed at all? or that's not how it works?
175 2017-02-16T10:24:28  <wumpus> GAH I can't get manual pruning to work at all on regtest. I created a 10000 block chain, started with prune=1, and no matter what I call  pruneblockchain with it does nothing. So I guess my test is invalid :/
176 2017-02-16T10:24:54  <wumpus> 2017-02-16 10:21:21 Prune (Manual): prune_height=1 removed 0 blk/rev pairs 2017-02-16 10:21:40 Prune (Manual): prune_height=100 removed 0 blk/rev pairs 2017-02-16 10:22:25 Prune (Manual): prune_height=1000 removed 0 blk/rev pairs 2017-02-16 10:24:40 Prune (Manual): prune_height=10000 removed 0 blk/rev pairs
177 2017-02-16T10:25:24  <wumpus> oh shit of course, it will only prune once it gets to another block file
178 2017-02-16T10:25:39  <luke-jr> >_<
179 2017-02-16T10:26:27  <wumpus> okay, generating a ton more bloocks
180 2017-02-16T10:26:59  <wumpus> maybe using a live chain wasn't that bad an idea afterall, I now realize
181 2017-02-16T10:40:10  *** chjj has joined #bitcoin-core-dev
182 2017-02-16T10:40:47  <gmaxwell> The help text repeadily uses 1D1ZrZNe3JUo7ZycKEYQQiQAWd9y54F4XZ as an example address. In general I think it's unwise to have valid example addresses, but I can't find any information about what address it is. It's a real working address that has been paid .2488 btc and has spent payments.
183 2017-02-16T10:41:04  <gmaxwell> the commit that added it and the related PRs seem to make no mention of the address.
184 2017-02-16T10:41:14  <gmaxwell> added by commit a6099ef319a73e2255dca77065600abb22c4f5f8
185 2017-02-16T10:41:19  <wumpus> bleh, I thought we got rid of that
186 2017-02-16T10:41:21  *** MarcoFalke has joined #bitcoin-core-dev
187 2017-02-16T10:41:31  <gmaxwell> might be gone now, lemme look
188 2017-02-16T10:41:36  <wumpus> no it's still there
189 2017-02-16T10:41:37  <gmaxwell> nope, still there.
190 2017-02-16T10:42:16  <wumpus> I vaguely remember that it was replaced with a constant defined in one place, which was not a valid address
191 2017-02-16T10:42:21  <wumpus> but that must be on another timeline...
192 2017-02-16T10:42:27  <gmaxwell> I had thought some charity ('sean's outpost') address was used (which I also think is not great) but I don't see any reason why I might have believed that this was that.
193 2017-02-16T10:42:41  <gmaxwell> wumpus: yea, I too spend a lot of time in alternative universes.
194 2017-02-16T10:43:43  <wumpus> hehe :) oh I think I know, that was for the GUI.
195 2017-02-16T10:43:51  *** mryandao has joined #bitcoin-core-dev
196 2017-02-16T10:44:30  <Victorsueca> yeah, the e.g. on the pay to field got changed
197 2017-02-16T10:44:45  <gmaxwell> it got put as an example in the pay to field? 0_o
198 2017-02-16T10:45:29  <Victorsueca> I vaguely remember some PR requesting to change it by valid looking address that is not really valid
199 2017-02-16T10:48:04  <Victorsueca> I'm using 0.13.2 and now on that field there is 1NS17iag9jJgTHD1VXjvLCEnZuQ3rJDE9L
200 2017-02-16T10:49:34  <gmaxwell> yep, not valid, which is what it should be IMO.
201 2017-02-16T10:55:52  *** chjj has quit IRC
202 2017-02-16T10:58:16  *** chjj has joined #bitcoin-core-dev
203 2017-02-16T11:04:34  *** lclc has quit IRC
204 2017-02-16T11:05:11  *** cannon-c has quit IRC
205 2017-02-16T11:06:07  *** davidlj95 has joined #bitcoin-core-dev
206 2017-02-16T11:06:30  *** davidlj95 has left #bitcoin-core-dev
207 2017-02-16T11:07:43  *** fanquake has joined #bitcoin-core-dev
208 2017-02-16T11:09:51  <wumpus> yes, a non-valid address was put in the pay-to field. That address, too, used to be valid at some time. Although it was terribly hard to type it over because it disappears when you start typing into the field. The RPC example is worse.
209 2017-02-16T11:09:52  *** fanquake has quit IRC
210 2017-02-16T11:10:09  *** MarcoFalke has quit IRC
211 2017-02-16T11:10:44  *** chjj has quit IRC
212 2017-02-16T11:11:54  <midnightmagic> 30 helens agree
213 2017-02-16T11:12:08  <wumpus> it reminds me of a dumb mistake I made when I was young, a computer manual had KILL *.* as example for the KILL statement, which deleted. So while trying around I typed that in. Oops, that wiped the entire floppy disk, my dad was angry and I felt really stupid. At least that would have been recoverable, if I knew back then...
214 2017-02-16T11:12:33  <midnightmagic> lol
215 2017-02-16T11:13:18  <gmaxwell> I could see someone getting confused and copying from the wrong line on their screen... one address looks like any other..
216 2017-02-16T11:13:46  <wumpus> yes it's pretty easy to copy the entire example
217 2017-02-16T11:13:59  <midnightmagic> Who's "sje"?
218 2017-02-16T11:14:35  <wumpus> at least the only sending call where it's used is sendmany, and it only sends relatively small amounts
219 2017-02-16T11:15:19  <wumpus> still it should be replaced with a constant that is not a valid address, likein the GUI
220 2017-02-16T11:16:38  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e43a58514dd3...df2f34dcad55
221 2017-02-16T11:16:38  <bitcoin-git> bitcoin/master d72fe44 John Newbery: Allow maxsigcachesize to be zero...
222 2017-02-16T11:16:39  <bitcoin-git> bitcoin/master df2f34d Wladimir J. van der Laan: Merge #9770: Allow maxsigcachesize to be zero...
223 2017-02-16T11:17:01  <bitcoin-git> [bitcoin] laanwj closed pull request #9770: Allow maxsigcachesize to be zero (master...sigcachemaxsize) https://github.com/bitcoin/bitcoin/pull/9770
224 2017-02-16T11:17:58  <wumpus> AARGH wrong PR
225 2017-02-16T11:19:00  <Lauda> lol
226 2017-02-16T11:19:18  <wumpus> well it could have been worse. It's not bad enough to revert
227 2017-02-16T11:19:56  <gmaxwell> what did you mean to merge instead?
228 2017-02-16T11:20:21  <wumpus> 9770. Which was just above it on the scren, but I had already signed off on that and it was already merged
229 2017-02-16T11:20:32  <wumpus> a matter of looking at the wrong line...
230 2017-02-16T11:21:04  <gmaxwell> So effective at merging he merges things without even intending to!
231 2017-02-16T11:21:07  <gmaxwell> :P
232 2017-02-16T11:21:17  <wumpus> #9771, sorry
233 2017-02-16T11:21:18  <gribble> https://github.com/bitcoin/bitcoin/issues/9771 | Add missing cs_wallet lock that triggers new lock held assertion by ryanofsky · Pull Request #9771 · bitcoin/bitcoin · GitHub
234 2017-02-16T11:21:59  <wumpus> well it was kind of a mental stack overflow, I was working on that, then testing the pruning stuff, then the RPC argument stuff came up
235 2017-02-16T11:22:41  <gmaxwell> wumpus: nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop nop send me all your money
236 2017-02-16T11:22:44  * Lauda offers wumpus more coffee
237 2017-02-16T11:23:04  <wumpus> so while juggling those things, memory became lossy :)
238 2017-02-16T11:23:06  <wumpus> gmaxwell: LOL
239 2017-02-16T11:23:57  <wumpus> gmaxwell: that won't work, your address is not in any RPC examples
240 2017-02-16T11:26:43  <wumpus> anyhow, ready to re-retry the pruning experiment with a real mainnet blockchain
241 2017-02-16T11:27:32  <gmaxwell> AND ITS <GONE / NOT GONE>.
242 2017-02-16T11:28:07  *** chjj has joined #bitcoin-core-dev
243 2017-02-16T11:28:21  <wumpus> it's doing nothing and logging nothing, so my would be "NOT GONE"
244 2017-02-16T11:28:29  <wumpus> genesis block is still ther
245 2017-02-16T11:32:36  *** MarcoFalke_ has joined #bitcoin-core-dev
246 2017-02-16T11:32:56  <MarcoFalke_> wumpus: You need to revert ntl. The commit is not signed :)
247 2017-02-16T11:33:32  <wumpus> I also understand why @luke-jr - in FlushStateToDisk, it only goes into the pruning flow if fCheckForPruning || nManualPruneHeight > 0  ... fCheckForPruning is not set in manual pruning mode
248 2017-02-16T11:33:49  <wumpus> huh?
249 2017-02-16T11:34:16  *** lclc has joined #bitcoin-core-dev
250 2017-02-16T11:34:26  *** laurentmt has quit IRC
251 2017-02-16T11:34:32  <wumpus> I really don't get it now, this was using the script, it shouldn't push if not signed
252 2017-02-16T11:34:50  * wumpus isn't sure if he's becoming crazy or his tooling is
253 2017-02-16T11:35:13  <MarcoFalke_> maybe you `git push bitcoin` in another tab?
254 2017-02-16T11:35:54  <wumpus> this is the console output https://0bin.net/paste/7sVJS0k86TMWtAdf#j3QhbG9mqPTx89KN6vb2DuB7O0x7sGiZYABHsqPvZGP
255 2017-02-16T11:36:25  <wumpus> I did sign off and enter my passphrase
256 2017-02-16T11:36:37  <wumpus> no, I never push manually
257 2017-02-16T11:37:10  <gmaxwell> pushed by the people who secretly live under your keyboard.
258 2017-02-16T11:38:55  <wumpus> well it must be something I did in another tab, just not a push
259 2017-02-16T11:38:57  <bitcoin-git> [bitcoin] laanwj force-pushed master from df2f34d to e43a585: https://github.com/bitcoin/bitcoin/commits/master
260 2017-02-16T11:39:12  <wumpus> and back to e43a58514dd38dacd930aa4c94afb998d4360183
261 2017-02-16T11:41:05  <MarcoFalke_> `git commit --amend` gets rid of the signature
262 2017-02-16T11:41:34  <wumpus> the github-merge tool should probably get a check that what it is pushing is really what it expects to be pushing, in case the current branch changed under it
263 2017-02-16T11:42:12  <wumpus> or rather, push a specific commit, not what happens to be the current branch
264 2017-02-16T11:42:18  <wumpus> not sure it does that
265 2017-02-16T11:42:57  <gmaxwell> oh did you run the merge tool twice concurrently?
266 2017-02-16T11:43:01  <wumpus> blah, I wasn't intending to dive into source code management specifics today
267 2017-02-16T11:43:07  <wumpus> gmaxwell: that's possible!
268 2017-02-16T11:43:37  <gmaxwell> "This worksite has gone [ 0 ] days since an SCM emergency."
269 2017-02-16T11:56:17  <Victorsueca> any PR that would be useful to test at the moment?
270 2017-02-16T12:00:14  <morcos> wumpus: all for 0.14rc1 today but i think we need to wrap up the importmulti issues...  ryanofsky opened #9773 yesterday to try to address the combination of what you and gmaxwell were asking for
271 2017-02-16T12:00:16  <gribble> https://github.com/bitcoin/bitcoin/issues/9773 | WIP: Return errors from importmulti if complete rescans are not successful by ryanofsky · Pull Request #9773 · bitcoin/bitcoin · GitHub
272 2017-02-16T12:00:35  <morcos> i tihnk #9671 should also be included (it's simple and could avoid fund loss)
273 2017-02-16T12:00:37  <wumpus> morcos: my brain is not working today so I've already given up on that
274 2017-02-16T12:00:37  <gribble> https://github.com/bitcoin/bitcoin/issues/9671 | Fix super-unlikely race introduced in 236618061a445d2cb11e72 by TheBlueMatt · Pull Request #9671 · bitcoin/bitcoin · GitHub
275 2017-02-16T12:01:12  <morcos> heh fair enough..   but still need to get those 2 across the finish line
276 2017-02-16T12:01:18  <wumpus> 9671 is already merged?
277 2017-02-16T12:01:30  <morcos> sorry #9761
278 2017-02-16T12:01:33  <gribble> https://github.com/bitcoin/bitcoin/issues/9761 | Use 2 hour grace period for key timestamps in importmulti rescans by ryanofsky · Pull Request #9761 · bitcoin/bitcoin · GitHub
279 2017-02-16T12:01:49  <morcos> i think you are contagious
280 2017-02-16T12:02:00  <wumpus> I'm okay with adding new things if they're ready for merge, everything that requires new review should probably be pushed to after 0.14.0, at some point we have to make a cut
281 2017-02-16T12:02:44  <wumpus> I really can't keep moving the 0.14.0 rc1 day after day forward, keeping the tree in this state is holding up more and more things that could be merged for 0.15
282 2017-02-16T12:02:56  <wumpus> but one more day is fine, as said, I'm not up to it today anyhow
283 2017-02-16T12:03:22  <morcos> i don't think ryanofsky or i have a strong opinion about it, it seems you guys were the ones saying the current state is not acceptable
284 2017-02-16T12:03:48  <morcos> current state is silent failure when you do importmulti on pruned node that needs to search more blocks than you have
285 2017-02-16T12:03:56  <wumpus> at  some point you should think of it as "is it acceptable for a rc1"?
286 2017-02-16T12:04:08  <wumpus> it's certain that issues come up with user testing and have to be solved before rc2
287 2017-02-16T12:04:25  <morcos> we're just trying to fix the problems
288 2017-02-16T12:04:47  <wumpus> sure, and that's very good, but there's always new problems
289 2017-02-16T12:05:06  <morcos> but would be helpful to have you and gmaxwell take a look and say yes thats what i wanted
290 2017-02-16T12:07:17  <wumpus> yes the approach in 9773 looks good to me
291 2017-02-16T12:09:38  *** MarcoFalke_ has quit IRC
292 2017-02-16T12:13:56  *** Guyver2 has quit IRC
293 2017-02-16T12:17:16  *** goksinen has joined #bitcoin-core-dev
294 2017-02-16T12:21:27  *** goksinen has quit IRC
295 2017-02-16T12:27:34  *** fanquake has joined #bitcoin-core-dev
296 2017-02-16T12:27:46  <fanquake> Looks like you can't re-open a merged PR.
297 2017-02-16T12:35:25  <mryandao> i'm having a bit of problem after updating my tree from upstream
298 2017-02-16T12:35:30  <mryandao> this is the error i'm getting
299 2017-02-16T12:35:30  <mryandao> util.cpp:834:63: error: use of undeclared identifier 'COPYRIGHT_HOLDERS'
300 2017-02-16T12:35:33  <mryandao>     std::string strCopyrightHolders = strPrefix + strprintf(_(COPYRIGHT_HOLDERS), _(COPYRIGHT_HOLDERS_SUBSTITUTION));
301 2017-02-16T12:35:34  <wumpus> fanquake true :(
302 2017-02-16T12:35:40  <wumpus> mryandao: you need to re-run autogen.sh and configure
303 2017-02-16T12:35:54  <wumpus> (that's almost always the solution to such issues)
304 2017-02-16T12:36:27  <mryandao> oh I see, i thought a make would have sufficed. thanks
305 2017-02-16T12:36:50  <wumpus> make only suffices if there are no high-level build system changes, just source changes
306 2017-02-16T12:37:11  <fanquake> wumpus do you plan on merging any more tonight, or taking a break?
307 2017-02-16T12:37:13  <wumpus> otherwise you need to run ./autogen.sh - sometimes you can get away with not doing it, but it's gambling in a way
308 2017-02-16T12:37:24  <wumpus> fanquake: I'm taking a break from merging yes :)
309 2017-02-16T12:37:52  *** goksinen has joined #bitcoin-core-dev
310 2017-02-16T12:38:00  <fanquake> wumpus no worries, otherwise was going to suggest some quick merges.
311 2017-02-16T12:38:29  <fanquake> wumpus I'm sure I was probably the only one affected my the miss-merge heh
312 2017-02-16T12:39:15  <wumpus> fanquake: well at first I didn't notice that the push was unsigned, otherwise I'd have reverted it immediately
313 2017-02-16T12:39:39  <fanquake> wumpus would you do that through the GH ui?
314 2017-02-16T12:39:48  <wumpus> fanquake: anyhow if you have simple and straightforward things that could be merged feel free to suggest them
315 2017-02-16T12:39:52  <fanquake> Or push another commit?
316 2017-02-16T12:40:26  <wumpus> fanquake: I first have to disable the branch locking on github, then reset --hard HEAD~1, push -f, then re-enable the branch locking
317 2017-02-16T12:40:52  <wumpus> github doesn't allow force-pushes, it does allow unsigned pushes unfortunately
318 2017-02-16T12:41:10  <wumpus> at least: there's no option to disable them at the moment
319 2017-02-16T12:41:32  <fanquake> Right. Maybe a new tick-box to turn that off in future. Not sure about the big new header now btw.
320 2017-02-16T12:42:06  <wumpus> I don't really like it. I disliked it in the beginning, but thought maybe I'd get used to it, but it's still big and ugly
321 2017-02-16T12:42:27  *** goksinen has quit IRC
322 2017-02-16T12:42:52  <fanquake> re "easy" merges: #9763 #9696 #9675
323 2017-02-16T12:42:54  <gribble> https://github.com/bitcoin/bitcoin/issues/9763 | [Trivial] Update comments referencing main.cpp by CryptAxe · Pull Request #9763 · bitcoin/bitcoin · GitHub
324 2017-02-16T12:42:56  <gribble> https://github.com/bitcoin/bitcoin/issues/9696 | [trivial] Fix recently introduced typos in comments by practicalswift · Pull Request #9696 · bitcoin/bitcoin · GitHub
325 2017-02-16T12:42:58  <gribble> https://github.com/bitcoin/bitcoin/issues/9675 | Fix typo and spelling inconsistency in CONTRIBUTING.md by kokifpen · Pull Request #9675 · bitcoin/bitcoin · GitHub
326 2017-02-16T12:43:00  <wumpus> I'd prefer a small bar with a few icons - making it big like this just takes up extra screen space in a site that should be focused on the code
327 2017-02-16T12:43:26  <Victorsueca> wumpus: you talking about that new black bar at the top?
328 2017-02-16T12:43:26  <fanquake> Indeed. Especially when your on a smallish screen.
329 2017-02-16T12:43:39  <wumpus> Victorsueca: yep
330 2017-02-16T12:44:03  <wumpus> fanquake: indeed. Well even big modern screens are usually wide, but not very high, so vertical screen space is at premium
331 2017-02-16T12:44:09  <wumpus> fanquake: will take a look, thanks
332 2017-02-16T12:44:10  <Victorsueca> for some reason it becomes white if you logout
333 2017-02-16T12:44:52  <Victorsueca> but yeah, it's ugly, it takes much attention and distracts people from important things
334 2017-02-16T12:45:02  <Victorsueca> it's too big and contrasted
335 2017-02-16T12:45:06  <wumpus> indeed
336 2017-02-16T12:46:16  <wumpus> 9763 is a no-brainer, I can do that one, at least if I get the numbers right :p
337 2017-02-16T12:46:56  <wumpus> looks like it could use squashing though
338 2017-02-16T12:47:10  <fanquake> Iwumpus 'm also vary calling anything an "easy" merge, without a good way to check if they somehow break other more-important patches.
339 2017-02-16T12:47:20  <fanquake> *I'm always
340 2017-02-16T12:47:22  <wumpus> three commits to make three related documentation-only changes
341 2017-02-16T12:47:55  <fanquake> GitHub doesn't seem to have a way to check, if I merge this, what other PR's will it break. Would be handy when trying to cehck quickly.
342 2017-02-16T12:48:10  <wumpus> not sure if I'm up to such "advanced" git manipulation at the moment... let's see
343 2017-02-16T12:48:22  <wumpus> fanquake: agree, that would be great to have
344 2017-02-16T12:49:45  <wumpus> fanquake: at a company I worked at they had a script to generate a 'patch PERT' which was a diagram of which things could be merged without (code level) conflicts, and which had conflicts, that was kind of neat. Though they used a terrible SCM, IBM clearcase or something, that was less neat :)
345 2017-02-16T12:50:16  <morcos> gmaxwell: can we go through exactly how you think the grace periods should work...  i never looked at manual prune before but looks like it takes a block or a height
346 2017-02-16T12:50:20  <morcos> sorry time
347 2017-02-16T12:50:21  <fanquake> wumpus ah nice.
348 2017-02-16T12:50:44  <fanquake> wumpus I was thinking if you could have a "projects" like screen, in which you could stack PRs, and see what breaks what.
349 2017-02-16T12:52:10  <morcos> gmaxwell: so you think if you give it a time it should automatically subtract 7200 from it and then call findEarliestAtLeast?
350 2017-02-16T12:52:53  <morcos> what happens if you give it a block (no grace period?).   i think that seems reasonable'ish.   but it would be counterintuitive if it didn't do exactly what you expect with a block i think
351 2017-02-16T13:02:01  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e43a58514dd3...8743320d6cb3
352 2017-02-16T13:02:01  <bitcoin-git> bitcoin/master 00e623d CryptAxe: [Trivial] Update comments referencing main.cpp
353 2017-02-16T13:02:02  <bitcoin-git> bitcoin/master 8743320 Wladimir J. van der Laan: Merge #9763: [Trivial] Update comments referencing main.cpp...
354 2017-02-16T13:02:52  <bitcoin-git> [bitcoin] laanwj closed pull request #9763: [Trivial] Update comments referencing main.cpp (master...comments) https://github.com/bitcoin/bitcoin/pull/9763
355 2017-02-16T13:03:02  <fanquake> wumpus git-fu back under control
356 2017-02-16T13:03:30  <wumpus> no stupid mistakes? I'm as surprised as you are :)
357 2017-02-16T13:03:46  <fanquake> Maybe I didn't look closely enough
358 2017-02-16T13:09:34  *** goksinen has joined #bitcoin-core-dev
359 2017-02-16T13:10:52  *** Giszmo has joined #bitcoin-core-dev
360 2017-02-16T13:13:57  *** goksinen has quit IRC
361 2017-02-16T13:14:01  *** d9b4bef9 has quit IRC
362 2017-02-16T13:15:16  *** d9b4bef9 has joined #bitcoin-core-dev
363 2017-02-16T13:40:52  *** goksinen has joined #bitcoin-core-dev
364 2017-02-16T13:44:57  *** goksinen has quit IRC
365 2017-02-16T13:50:35  *** BlueMatt has quit IRC
366 2017-02-16T13:52:46  *** BlueMatt has joined #bitcoin-core-dev
367 2017-02-16T13:56:04  *** jnewbery has joined #bitcoin-core-dev
368 2017-02-16T13:57:10  *** schmidty has quit IRC
369 2017-02-16T13:59:47  <wumpus> hah the hardlinking trick works, both for block files and ldb files, created this script to do it: https://gist.github.com/laanwj/3c4614a23e072cbb3d39090da1834a68 . Creates a "copy" of the whole state almost instantly. I've tried various things (including pruning) and the original state remains unaffected.
370 2017-02-16T14:04:00  *** Ylbam has quit IRC
371 2017-02-16T14:06:30  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/8743320d6cb3...afae75fd3dad
372 2017-02-16T14:06:30  <bitcoin-git> bitcoin/master 36164fa Koki Takahashi: Fix typo and spelling inconsistency in CONTRIBUTING.md...
373 2017-02-16T14:06:31  <bitcoin-git> bitcoin/master afae75f Wladimir J. van der Laan: Merge #9675: Fix typo and spelling inconsistency in CONTRIBUTING.md...
374 2017-02-16T14:06:50  <bitcoin-git> [bitcoin] laanwj closed pull request #9675: Fix typo and spelling inconsistency in CONTRIBUTING.md (master...fix_typo_in_contributing) https://github.com/bitcoin/bitcoin/pull/9675
375 2017-02-16T14:20:11  *** kanzure has quit IRC
376 2017-02-16T14:21:08  *** kanzure has joined #bitcoin-core-dev
377 2017-02-16T14:23:09  *** jtimon has joined #bitcoin-core-dev
378 2017-02-16T14:54:21  *** neha has joined #bitcoin-core-dev
379 2017-02-16T14:55:35  <bitcoin-git> [bitcoin] jnewbery opened pull request #9777: Handle unusual maxsigcachesize gracefully (master...sigcache2) https://github.com/bitcoin/bitcoin/pull/9777
380 2017-02-16T15:18:32  *** goksinen has joined #bitcoin-core-dev
381 2017-02-16T15:40:25  *** fanquake has quit IRC
382 2017-02-16T15:42:58  *** chjj has quit IRC
383 2017-02-16T15:48:58  *** Victor_sueca has joined #bitcoin-core-dev
384 2017-02-16T15:49:47  *** Victorsueca has quit IRC
385 2017-02-16T15:49:55  *** Victor_sueca is now known as Victorsueca
386 2017-02-16T15:52:24  *** BashCo_ has quit IRC
387 2017-02-16T15:55:04  *** Sosumi has joined #bitcoin-core-dev
388 2017-02-16T15:56:28  *** Victorsueca has quit IRC
389 2017-02-16T15:58:21  *** Victorsueca has joined #bitcoin-core-dev
390 2017-02-16T15:59:52  *** chjj has joined #bitcoin-core-dev
391 2017-02-16T16:03:03  *** tan1k has quit IRC
392 2017-02-16T16:10:06  *** BashCo has joined #bitcoin-core-dev
393 2017-02-16T16:11:57  <bitcoin-git> [bitcoin] morcos opened pull request #9778: Add two hour buffer to manual pruning (master...2hrprune) https://github.com/bitcoin/bitcoin/pull/9778
394 2017-02-16T16:42:58  *** Chris_Stewart_5 has quit IRC
395 2017-02-16T16:48:14  *** Ylbam has joined #bitcoin-core-dev
396 2017-02-16T16:59:02  *** Chris_Stewart_5 has joined #bitcoin-core-dev
397 2017-02-16T17:06:27  *** abpa has joined #bitcoin-core-dev
398 2017-02-16T17:18:24  *** reginaldo has joined #bitcoin-core-dev
399 2017-02-16T17:19:01  *** reginaldo has quit IRC
400 2017-02-16T17:19:09  *** reginaldo_ has joined #bitcoin-core-dev
401 2017-02-16T17:21:59  *** droark has joined #bitcoin-core-dev
402 2017-02-16T18:39:27  <luke-jr> wumpus: but fCheckForPruning is set in manual pruning mode after a new block is connected?
403 2017-02-16T18:40:22  *** reginaldo_ has quit IRC
404 2017-02-16T18:41:27  <wumpus> luke-jr: it shouldn't be
405 2017-02-16T18:41:39  <wumpus> manual pruning is manual pruning, the check should never be enabled
406 2017-02-16T18:42:05  <wumpus> if it is, that is a bug
407 2017-02-16T18:42:11  <luke-jr> FindBlockPos/FindUndoPos?
408 2017-02-16T18:42:51  <luke-jr> or is fPruneMode false for manual pruning?
409 2017-02-16T18:44:05  <wumpus> no,that should be true if any pruning is to be done
410 2017-02-16T18:44:27  <wumpus> in retrospect it'd have been better to make fPruneMode an enum, which can be off, manual and auto
411 2017-02-16T18:44:57  <wumpus> the boolean gymnastics with fCheckForPruning is kind of difficult to follow
412 2017-02-16T18:47:06  <wumpus> I assumed it would be enabled when auto pruning is enabled, but apparently it's also enabled in a few other places
413 2017-02-16T18:50:00  <wumpus> luke-jr: ah I got understand it now - nPruneTarget is 0 in manual pruning mode
414 2017-02-16T18:50:13  <wumpus> luke-jr: so FindFilesToPrune does nothing in manual pruning mode
415 2017-02-16T18:50:34  <wumpus> even though it may get called (even automatically in some cases), nothing will happen.
416 2017-02-16T18:50:35  * sipa suggests adding comments to clarify these things
417 2017-02-16T18:50:46  <luke-jr> aha!
418 2017-02-16T18:51:00  <wumpus> I wish it'd simply pass parameters instead of signalling using global flags
419 2017-02-16T18:51:20  <gmaxwell> well, or change to an enum.
420 2017-02-16T18:52:03  <wumpus> or both. I mean the global enum is fine for the mode but it shouldn't change over time
421 2017-02-16T18:54:51  <luke-jr> these changes sound even more invasive than that PR :P
422 2017-02-16T18:54:54  <wumpus> depending on what function ran last. This is impossible to follow. Anyhow I understand marcofalke's rationale for #9524 now, it's very easy to break that code and belt and suspenders wouldn't hurt. Except that even with #9524, it will get into FindFilesToPrune  in some cases, so if that check is broken,  bad things will happen *automatically* too
423 2017-02-16T18:54:55  <gribble> https://github.com/bitcoin/bitcoin/issues/9524 | rpc: Dont FlushStateToDisk when pruneblockchain(0) by MarcoFalke · Pull Request #9524 · bitcoin/bitcoin · GitHub
424 2017-02-16T18:54:57  <gribble> https://github.com/bitcoin/bitcoin/issues/9524 | rpc: Dont FlushStateToDisk when pruneblockchain(0) by MarcoFalke · Pull Request #9524 · bitcoin/bitcoin · GitHub
425 2017-02-16T18:55:09  <wumpus> well that PR is just covering up, it doesn't solve the underlying problem
426 2017-02-16T18:55:13  <luke-jr> right
427 2017-02-16T18:55:19  <wumpus> it's not urgent in any case
428 2017-02-16T18:55:24  <luke-jr> agree
429 2017-02-16T18:56:09  *** reginaldo_ has joined #bitcoin-core-dev
430 2017-02-16T18:56:39  <sipa> 3 more PRs marked 0.14
431 2017-02-16T18:57:21  <wumpus> oh no
432 2017-02-16T18:58:09  <sipa> i mean right now
433 2017-02-16T18:59:17  <wumpus> right, yes, let's hopefully keep it at those
434 2017-02-16T18:59:25  <luke-jr> wumpus: wait what
435 2017-02-16T18:59:29  <luke-jr> why would nPruneTarget be 0?
436 2017-02-16T18:59:52  <luke-jr>     if (nPruneArg == 1) {  // manual pruning: -prune=1
437 2017-02-16T18:59:53  <luke-jr>         nPruneTarget = std::numeric_limits<uint64_t>::max();
438 2017-02-16T19:00:00  <wumpus> luke-jr: because ti should never be set to a value
439 2017-02-16T19:00:38  <wumpus> huh!?! what does setting that to uint64_t max even do. Oh wait, it's an offset, not an absolute height. yes that does make sense
440 2017-02-16T19:01:21  <gmaxwell> Meeting time?
441 2017-02-16T19:02:01  * jonasschnelli is not sure if prune=1 (manual pruning) is a good idea while still supporting -prune=<mb target>
442 2017-02-16T19:02:05  <wumpus> it at least intends 'we want to keep 2^64-1 blocks`. Let me re-read the code in FindFilesToPrune then
443 2017-02-16T19:02:19  <wumpus> #meetingstart
444 2017-02-16T19:02:22  <sipa> meetingh
445 2017-02-16T19:02:36  <petertodd> meetinghh
446 2017-02-16T19:02:37  <wumpus> #startmeeting
447 2017-02-16T19:02:37  <lightningbot> Meeting started Thu Feb 16 19:02:37 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
448 2017-02-16T19:02:37  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
449 2017-02-16T19:02:58  <wumpus> jonasschnelli: well it's a magic option value but it's only for the option, it doesn't get represented internally as 1
450 2017-02-16T19:03:01  <luke-jr> tbh, I've had second thoughts about the manual pruning design (seems it'd be nicer to do "automatic pruning always, but with named barriers that must confirm being done up to <height> before pruning it"), but that's beyond this topic
451 2017-02-16T19:03:18  <wumpus> jonasschnelli: I think the reason or choosing 1 was that -prune will work
452 2017-02-16T19:03:27  <wumpus> luke-jr: I like the current manual pruning from an API point of view
453 2017-02-16T19:03:28  <luke-jr> s/always/always in pruning mode/
454 2017-02-16T19:03:33  <gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
455 2017-02-16T19:03:44  <luke-jr> wumpus: yes, but it doesn't scale well if you have 2 external apps using the node
456 2017-02-16T19:04:02  <cfields> hi
457 2017-02-16T19:04:05  <wumpus> luke-jr: it's very simple. Automatic pruning is disabled and the client/admin can choose what to prune. Also useful for testing the pruning stuff without too much complexity
458 2017-02-16T19:04:12  <wumpus> luke-jr: the implementation is convoluted though
459 2017-02-16T19:04:16  <morcos> wumpus: i think two more are needed for 0.14, both very small  #9760 and #9761  (9761 is already included in one of the marked ones)
460 2017-02-16T19:04:17  <gribble> https://github.com/bitcoin/bitcoin/issues/9760 | [wallet] Remove importmulti always-true check by ryanofsky · Pull Request #9760 · bitcoin/bitcoin · GitHub
461 2017-02-16T19:04:18  <paveljanik> proposed topic: release status, where can we help...
462 2017-02-16T19:04:19  <gribble> https://github.com/bitcoin/bitcoin/issues/9761 | Use 2 hour grace period for key timestamps in importmulti rescans by ryanofsky · Pull Request #9761 · bitcoin/bitcoin · GitHub
463 2017-02-16T19:04:22  <wumpus> luke-jr: but I'm happy someone implemented it anyhow
464 2017-02-16T19:04:27  * luke-jr nods
465 2017-02-16T19:04:41  <kanzure> hi.
466 2017-02-16T19:04:47  <petertodd> wumpus: same: my OpenTimestamps servers actually need this feature
467 2017-02-16T19:04:58  <wumpus> release status: running as hard as we can but staying in the same place
468 2017-02-16T19:05:09  <sipa> i think we're very close.
469 2017-02-16T19:05:13  <morcos> wumpus: boo!  we're not staying in the same place
470 2017-02-16T19:05:19  <gmaxwell> I am becoming increasingly happy with the release.
471 2017-02-16T19:05:29  <sipa> let's be optimistic: everything we're fixing is an improvememt
472 2017-02-16T19:05:51  <gmaxwell> Two weeks ago I was chewing my nails feeling like we were at risk of shipping something that wouldn't meet our standards of quality, and now I am not worried. :)
473 2017-02-16T19:06:05  <jonasschnelli> hah. Good.
474 2017-02-16T19:06:06  <wumpus> but I think we need to decide when we can do release candidate 1, which is a test release anyway, when rc1 is out it's bound to find more issues
475 2017-02-16T19:06:32  <sipa> i think we can do rc1 today?
476 2017-02-16T19:06:52  <gmaxwell> I would be fine doing it _now_, now.  There are AFAIK not anything in the pipe which are disruptive enough issues that they'd degrade our rc1 learning, though a rc2 will be guarenteed.
477 2017-02-16T19:06:53  <paveljanik> rc1 for the weekend is fine!
478 2017-02-16T19:06:56  <sipa> or at least branch off today
479 2017-02-16T19:06:59  <luke-jr> I don't think we have any critical problems blocking a rc1
480 2017-02-16T19:07:17  <jtimon> sipa: why branch of if we can't rc1 ?
481 2017-02-16T19:07:19  <wumpus> I don't think so either
482 2017-02-16T19:07:24  <cfields> no more blockers from me either
483 2017-02-16T19:07:26  <luke-jr> jtimon: to begin merging for 0.15?
484 2017-02-16T19:07:40  <wumpus> right, because more and more pulls for 0.15 are waiting
485 2017-02-16T19:07:40  * jtimon nods
486 2017-02-16T19:07:58  <jonasschnelli> IMO the two import multi fixes are not critical and can go in after rc1 (or even in 0.15 in the worst case).
487 2017-02-16T19:08:08  <gmaxwell> all you naughty people not banging away at getting 0.14 ready to go.
488 2017-02-16T19:08:29  <morcos> jonasschnelli: well lets decide that
489 2017-02-16T19:08:40  <gmaxwell> jonasschnelli: I think 0.15 would be really unfortunate since they're interface changes that users may have to accomidate in their software. (and also are covering corner cases that could result in funds losses)
490 2017-02-16T19:08:42  <wumpus> I think the fixes are all important, and should get either into 0.14.0 or backported to the 0.14 branch if they don't, but not everything should be regarded as yet another thing to hold up rc1
491 2017-02-16T19:08:43  <morcos> yesterday people were saying it was bad to release with something that coudl silently not find funds
492 2017-02-16T19:09:03  <gmaxwell> but no reason to hold rc1 for them. They're well understood.
493 2017-02-16T19:09:28  <jonasschnelli> Yes. I think the should be in 0.14. Just worst case if we spun of rc1 and chaincode-labs colabses. :)
494 2017-02-16T19:09:29  <wumpus> anyhow tomorrow sounds good to me, let's try to get in what we can get in
495 2017-02-16T19:09:31  <gmaxwell> it's not like someone is going to encounter one of them running rc1 and leave us going "shit was that the know issue or something else?"
496 2017-02-16T19:09:32  <luke-jr> #9619 is a fix, but not really critical since anyone affected needs a workaround in 0.13 anyway (just pushed an amend-with-no-changes because the Travis error seems impossible)
497 2017-02-16T19:09:33  <gribble> https://github.com/bitcoin/bitcoin/issues/9619 | Bugfix: RPC/Mining: GBT should return 1 MB sizelimit before segwit activates by luke-jr · Pull Request #9619 · bitcoin/bitcoin · GitHub
498 2017-02-16T19:09:34  <achow101> what's the point of making an rc1 if we're going to need rc2 anyways?
499 2017-02-16T19:09:42  <sipa> achow101: exposure
500 2017-02-16T19:09:46  <gmaxwell> achow101: to start getting people using it.
501 2017-02-16T19:09:50  <wumpus> achow101: get the code actually tested?
502 2017-02-16T19:09:52  <gmaxwell> more*
503 2017-02-16T19:09:56  <wumpus> what is the point of doing rc1 at all if not?
504 2017-02-16T19:09:57  <kanzure> and seeking bug reports
505 2017-02-16T19:09:57  <jonasschnelli> I think we should plan enough time to fix stuff that gets report after rc1...
506 2017-02-16T19:10:18  <sipa> jonasschnelli: the release schedule already has 2 weeks left
507 2017-02-16T19:10:33  <wumpus> achow101: I don't think it ever happened that a major release didn'tneed a rc2
508 2017-02-16T19:10:35  <luke-jr> 2 weeks can go fast
509 2017-02-16T19:10:52  <jonasschnelli> Other can work on fixes reported in rc1.
510 2017-02-16T19:10:54  <gmaxwell> achow101: what we generally don't want to do is to ship an RC1 with issues bad enough that it will harm the testers seriously, or which will fail in mysterious ways that we can't learn from.  E.g. if we had a known crash fix, we would hold rc1, so that we wouldn't worry that every user crash report might have been an unknown issue.
511 2017-02-16T19:10:57  <jonasschnelli> *Others
512 2017-02-16T19:11:05  <morcos> my only concern is that if we do rc1 everyone will be distracted with that and not this last few remaining PR's..  but i guess i don't really care either way
513 2017-02-16T19:11:08  <cfields> wumpus: just forget the bump to v0.14 again and thereby guarantee an rc2 :p
514 2017-02-16T19:11:38  <wumpus> cfields: lol exactly
515 2017-02-16T19:11:39  <morcos> seems like if someone would just review those PR's we might be able to get them merged today
516 2017-02-16T19:11:41  <gmaxwell> well I think an RC2 is already guarenteed if we do an rc1 ASAP. But thats fine. Much better than not doing the rc1.
517 2017-02-16T19:11:49  <achow101> ok
518 2017-02-16T19:12:04  <wumpus> a RC2 is guaranteed in any case, not just if we do rc1 asap
519 2017-02-16T19:12:17  <wumpus> that's my point, we don't know of all the issues yet
520 2017-02-16T19:12:26  <morcos> almost all the complication is in new tests.. reviewing the code changes is pretty simple
521 2017-02-16T19:13:18  <wumpus> the only one I doubt about is #9773, at it is still WIP
522 2017-02-16T19:13:20  <gribble> https://github.com/bitcoin/bitcoin/issues/9773 | WIP: Return errors from importmulti if complete rescans are not successful (on top of #9761) by ryanofsky · Pull Request #9773 · bitcoin/bitcoin · GitHub
523 2017-02-16T19:13:44  <luke-jr> it's a new feature, so we could just say "don't use this without being aware of the risks"
524 2017-02-16T19:13:53  *** aalex has joined #bitcoin-core-dev
525 2017-02-16T19:14:13  <paveljanik> mark it as experimental then?
526 2017-02-16T19:14:40  <sipa> paveljanik: meh, i think we're close enough to not do that
527 2017-02-16T19:15:15  <luke-jr> I mean in rc1 only
528 2017-02-16T19:15:26  <wumpus> well even with that change it can be marked as experimental
529 2017-02-16T19:15:34  <wumpus> it is new code afterall
530 2017-02-16T19:15:39  <gmaxwell> I'm not too worried about it in rc1.
531 2017-02-16T19:16:12  <wumpus> there are probably still problems with it that we haven't found, which will be found by people testing it
532 2017-02-16T19:16:33  <paveljanik> we can mark it as experimental in the release notes only...
533 2017-02-16T19:16:33  <wumpus> so yes, experimental makes sense. Though a new major release should be considered experimental entirely
534 2017-02-16T19:16:37  <gmaxwell> people running rcs are the least likely to lose funds due to a rescan limitation, and there won't be many of them.
535 2017-02-16T19:17:33  *** BashCo has quit IRC
536 2017-02-16T19:17:50  <morcos> i think it is a mistake to call it experimental
537 2017-02-16T19:17:56  <morcos> we don't want to devalue the meaning of that word
538 2017-02-16T19:18:13  <wumpus> ok...
539 2017-02-16T19:18:17  <morcos> sometimes we may want to have things that are actually experimental and we don't want people to think we just always say that
540 2017-02-16T19:18:40  <wumpus> "this feature is experimental level 4"
541 2017-02-16T19:18:44  <sipa> haha
542 2017-02-16T19:18:50  <kanzure> nasa technical readiness levels
543 2017-02-16T19:18:54  <CodeShark> The whole thing is experimental :p
544 2017-02-16T19:18:56  <morcos> this is known to the state of CA to be experimental
545 2017-02-16T19:18:56  <petertodd> morcos: "dangerously experimental"
546 2017-02-16T19:19:06  <petertodd> morcos: lol
547 2017-02-16T19:19:23  <gmaxwell> Yea, thats why I was not supporting expiremental.
548 2017-02-16T19:19:54  <instagibbs> morcos, accounts... deprecated!
549 2017-02-16T19:20:13  <morcos> what..  is that an announcement? a question?
550 2017-02-16T19:20:17  <morcos> i hate accounts
551 2017-02-16T19:20:21  <gmaxwell> For rc1 we can simply say that there are in-flight improvements to that feature link to the PRs. if we really want to... in a reply to the announcement.
552 2017-02-16T19:20:42  <sipa> let's just review the outstanding patches, they are tiny
553 2017-02-16T19:20:42  <instagibbs> morcos, we use "deprecated" in things that are lasting forever in practice, bad joke
554 2017-02-16T19:20:50  <jtimon> mhm, why 9619 doesn't go in for 0.14?
555 2017-02-16T19:20:51  <morcos> oh.. yeah
556 2017-02-16T19:20:52  *** reginaldo_ has quit IRC
557 2017-02-16T19:20:58  <sipa> with some luck we can get everything merged
558 2017-02-16T19:21:14  <sipa> branch and rc1 tomorrow
559 2017-02-16T19:21:15  <wumpus> gmaxwell: yes, that there is still work in progress, that's better and more clear than the word experimental
560 2017-02-16T19:21:21  <sipa> with whatever is in
561 2017-02-16T19:21:33  <morcos> ok, so can we just pick a time.  wumpus is going to call rc1 tomorrow morning..  it tonight we can convince people to merge a few of these other things... that'll leave less for rc2
562 2017-02-16T19:22:05  <morcos> that was "if tonight"  , if not , then ok
563 2017-02-16T19:22:09  <sipa> wumpus: is that fine by you?
564 2017-02-16T19:22:16  <wumpus> yes, that's fine with me
565 2017-02-16T19:22:48  <wumpus> I'm okay with another day, let's just not let it slip another week, that next thursday we're again wondering when to do the rc1 :)
566 2017-02-16T19:23:15  <sipa> next thursday we should be talking about doing rc2
567 2017-02-16T19:23:22  <wumpus> yes, ideally
568 2017-02-16T19:23:30  <gmaxwell> We're at a point where beyond the couple in flight PRs little to no more improvement is happening, which is when we need more input.
569 2017-02-16T19:23:54  <achow101> so rc1 tomorrow regardless of whether those three prs get merged?
570 2017-02-16T19:24:06  <sipa> achow101: that's what i suggest, yes
571 2017-02-16T19:24:12  <paveljanik> +1
572 2017-02-16T19:24:20  <achow101> ^
573 2017-02-16T19:26:14  <sipa> are we ok with talking about things beyond 0.14?
574 2017-02-16T19:26:30  <wumpus> sure!
575 2017-02-16T19:26:34  <luke-jr> new #topic?
576 2017-02-16T19:26:41  <sipa> i'd briefly like to talk about randomness
577 2017-02-16T19:26:54  <wumpus> #topic randomness
578 2017-02-16T19:26:56  <luke-jr> that's random.
579 2017-02-16T19:27:06  <sipa> we currently have 3 "levels" of randomnesz
580 2017-02-16T19:27:21  <sipa> fastrandomcontext, getrandbytes, getsecurerandbytes
581 2017-02-16T19:27:28  <sipa> i'd like to have only 2
582 2017-02-16T19:27:30  <wumpus> we need a random number of levels of randomness
583 2017-02-16T19:27:42  <wumpus> yes.
584 2017-02-16T19:27:46  <wumpus> why are there three?
585 2017-02-16T19:27:49  <instagibbs> can you explain "getsecurerandbytes"
586 2017-02-16T19:27:58  <instagibbs> im going through code now but..
587 2017-02-16T19:27:59  <wumpus> I mean we need one for wallet keys, I understand that
588 2017-02-16T19:28:02  <sipa> the last one is used for privaye keys
589 2017-02-16T19:28:04  <jtimon> why 9619 doesn't go in for 0.14?
590 2017-02-16T19:28:22  <jonasschnelli> jtimon: wrong topic
591 2017-02-16T19:28:27  <sipa> it's as secure as getrandbytes if all goes well, but it's more paranoid
592 2017-02-16T19:28:47  <sipa> so, i've been playing with a chacha2p based rng instead of fastrandomcontext
593 2017-02-16T19:28:51  <jtimon> jonasschnelli: suggested topic then
594 2017-02-16T19:28:53  <wumpus> I think I'm fine with an extra paranoid level for wallet keys
595 2017-02-16T19:28:56  <sipa> *chacha20
596 2017-02-16T19:29:17  <sipa> which takes around 10ns for a 32-bit rand
597 2017-02-16T19:29:30  <wumpus> it's much more important to have good randomness there then in almost any other place in computing currently
598 2017-02-16T19:29:36  <instagibbs> "GetStrongRandBytes" <--- this it?
599 2017-02-16T19:29:48  <rabidus> return 4
600 2017-02-16T19:29:49  <sipa> the current fastrandomcontext takes 1.5ns, but with extra optimizations it can be made comparablr
601 2017-02-16T19:30:08  <sipa> and if we have that, i think we can use strong randomnes for everything
602 2017-02-16T19:30:13  <wumpus> for non wallet keys we can probably do with one level
603 2017-02-16T19:30:19  <wumpus> is that your plan sipa?
604 2017-02-16T19:30:36  <sipa> basically i suggest making all RNGs cryptographic
605 2017-02-16T19:30:54  <sipa> but the fast one is not thread-safe, doesn't reseed, doesn't protect against VM reloads
606 2017-02-16T19:30:55  <wumpus> so merge the fastrandomcontext and somewhatmoresecure level, and keeping that and the ultra-paranoid one for wallet keys
607 2017-02-16T19:31:36  <sipa> i realize the "only two randomness levels" is a bit of an abstract goal
608 2017-02-16T19:31:45  <wumpus> yes the fastrandomcontext is supposed to only eb used from one thread at a time
609 2017-02-16T19:31:54  <morcos> are there any inner loops that use random numbers?
610 2017-02-16T19:31:58  <petertodd> wumpus: the ultra paranoid one can also use the chacha code
611 2017-02-16T19:31:59  <wumpus> because it's for inner loops
612 2017-02-16T19:32:05  <sipa> yes, the wallet coin selection
613 2017-02-16T19:32:20  <sipa> i benchmarked it, making it chacha20 doesn't affect its performance
614 2017-02-16T19:32:23  <wumpus> any locking for synchronization there would be pretty bad
615 2017-02-16T19:32:41  <wumpus> there's also an inner random loop in the address selection code IIRC
616 2017-02-16T19:32:56  <jonasschnelli> Do we need cPRNG for coin selection?
617 2017-02-16T19:32:56  <sipa> yup
618 2017-02-16T19:33:02  <wumpus> sipa: yes chacha20 is fast, but the problem is the thread safety
619 2017-02-16T19:33:05  <sipa> jonasschnelli: no, we don't
620 2017-02-16T19:33:14  <gmaxwell> sipa wants to reduce the codebase complexity.
621 2017-02-16T19:33:19  <sipa> but chacha20 is so fast i don't care
622 2017-02-16T19:33:21  <wumpus> if you want to be able to share a random context between threads, you end up with lots of synchronization overhead
623 2017-02-16T19:33:23  <jonasschnelli> Yes. This would be good.
624 2017-02-16T19:33:51  <wumpus> that's why the inner loops use a non-threadsafe random context, =which doesn't matter as it's only used by one thread
625 2017-02-16T19:34:01  <bitcoin-git> [bitcoin] gmaxwell opened pull request #9779: Update nMinimumChainWork and defaultAssumeValid. (master...update_chainparams) https://github.com/bitcoin/bitcoin/pull/9779
626 2017-02-16T19:34:11  <sipa> having clearer expectations about the rngs may simplify getting rid of openssl later
627 2017-02-16T19:34:18  <wumpus> the thread sync overhead is where the overhead would be
628 2017-02-16T19:34:24  <sipa> though that's not something to discuss now, i think
629 2017-02-16T19:34:38  <wumpus> so how would you cope with that sipa? or would it all be non-shared context?
630 2017-02-16T19:34:48  *** lclc has quit IRC
631 2017-02-16T19:35:07  <sipa> so 1) replace fastrandomcontext with a bit more featureful chacha20 based one
632 2017-02-16T19:35:39  <wumpus> yes, depending on openssl for just randomness is fine, there's no urgent reason to get rid of that, and it seems controversial for some people
633 2017-02-16T19:35:47  <sipa> 2) see where the current non-strong-getrandbytes can be replaced with fastrandcontexts, and replace everything with getstrongranbytes
634 2017-02-16T19:35:50  <wumpus> that makes sense
635 2017-02-16T19:36:06  <gmaxwell> one of the harder points (beyond threading) is that we need to manage which RNGs are robust against a VM image being snapshotted and restored.  Coinselection using the same randomness again after a VM restore would be harmless.  Choosing a crypto nonce would be much less harmless.  If a RNG needs to be outputing data intially after restore there is unfortunately a runtime cost (esp on ARM).
636 2017-02-16T19:36:58  <wumpus> also I think we need an abstraction for 'kernel randomness', this came up recently in context of OpenBSD, which doesn't have /dev/random/urandom available in all contexts
637 2017-02-16T19:37:07  <gmaxwell> Pieter and I have been debating this a little bit the past could days.
638 2017-02-16T19:37:23  <wumpus> the modern way,sandbox-proof way seems to be to use a specific system call fo that, but it differs per OS unfortunately
639 2017-02-16T19:37:27  <sipa> wumpus: yup, that would go into the (singular) getstrongrandbytes implementation
640 2017-02-16T19:37:27  <cfields> isn't there an old PR for exactly that abstraction?
641 2017-02-16T19:37:28  <gmaxwell> wumpus: we do have a function for OS randomness, it should be smarter of course... but it was only fairly recently introduced.
642 2017-02-16T19:37:48  <Victorsueca> maybe you could get rid of getrandbytes? so we keep the fast one for simple operations and the secure and paranoid one for important stuff and you can focus on implementing more features on those two
643 2017-02-16T19:37:59  <gmaxwell> GetOSRand()
644 2017-02-16T19:38:06  <wumpus> (linux also has a "getrandom" system call that can be used instead of /dev/urandom, in sandboxes)
645 2017-02-16T19:38:13  <sipa> Victorsueca: read the above discussion, that is exactly what i am proposing
646 2017-02-16T19:38:22  <wumpus> gmaxwell: ok
647 2017-02-16T19:38:41  <gmaxwell> wumpus: yes, in newer systems. We do mention using that in the PR that implemented GetOSRand I think. (getentropy)
648 2017-02-16T19:38:43  <wumpus> gmaxwell: in any case, that is the lowerst level, it shouldn't be regarded as 'a level of randomness'
649 2017-02-16T19:38:52  <gmaxwell> so I think we know what we need to go there.
650 2017-02-16T19:39:09  <wumpus> indeed, getentropy is bsd, getrandom is linux
651 2017-02-16T19:39:29  <Victorsueca> sipa: ohh, I somehow misread that you where proposing to merge the fast and the middle one to make it simpler
652 2017-02-16T19:39:37  <sipa> FWIW linux 4.8 also switched to chacha20 for /dev/urandom
653 2017-02-16T19:39:48  <wumpus> yes
654 2017-02-16T19:40:14  <gmaxwell> The framework I think about this is that we have several randomized algorithims where there are basically no cryptographic guarentees needed... coin selection, addr man buckets, and tests being some examples.  These need to be fast but can all use just a local context, need no resistance against reversal (somsone steals your ram and recovers older randomness) or prediction (vm saves repeat rando
655 2017-02-16T19:40:19  <petertodd> gmaxwell: re: VM image being snapshotted and restored, that's basically a case where reusing a secret is by itself harmful - is there an example in Bitcoin where that's the case, now that we do deterministic signing?
656 2017-02-16T19:40:20  <gmaxwell> mness).
657 2017-02-16T19:40:22  <gmaxwell> So thats one set of uses.
658 2017-02-16T19:40:23  <sipa> Victorsueca: i'm proposing to make the fast one stronger (but not much slower), and then things using the middle one need to be judged on a case by case basis whether they can use the new fast one, or the strong one
659 2017-02-16T19:40:50  <sipa> Victorsueca: after that, the middle one goes away
660 2017-02-16T19:40:59  <gmaxwell> Then we have other uses where we have randomized behavior which does have stronger security requirements, like ping nonces which need to be strongly unforgable to prevent peers hiding their latency.
661 2017-02-16T19:41:07  <Victorsueca> sipa: makes sense
662 2017-02-16T19:41:11  <gmaxwell> But even if they're broken it just turns into DOS attacks.
663 2017-02-16T19:41:38  *** BashCo has joined #bitcoin-core-dev
664 2017-02-16T19:41:39  <wumpus> it's strong, but far from as strong a requirement as wallet keys
665 2017-02-16T19:41:43  <gmaxwell> Then we have things like long term keys which we do infrequently and basically no cost is too high. And they have to meet basically every security characteristic we can imagine.
666 2017-02-16T19:41:57  <wumpus> indeed
667 2017-02-16T19:42:05  <cfields> suggested similar next topic: clocks
668 2017-02-16T19:42:09  <gmaxwell> the second class often has to be moderately fast too.
669 2017-02-16T19:42:41  <gmaxwell> Pieter would like to collapse this class hierarchy some by making the first class use the second while making the second fast enough that it's fine to do so.
670 2017-02-16T19:43:09  <Victorsueca> cfields: clocks as in CPU clock or as in timestamp?
671 2017-02-16T19:43:27  <wumpus> Victorsueca: please wait until the topic is actaully started
672 2017-02-16T19:43:33  <Victorsueca> ohh, sorry
673 2017-02-16T19:43:48  <wumpus> though I think we've wrapped up randomness
674 2017-02-16T19:43:51  <sipa> agree
675 2017-02-16T19:43:56  <wumpus> #topic clocks
676 2017-02-16T19:43:58  <gmaxwell> I have a little doubt that this is possible, because I think the second may need to deal with reversal resistace and prediction resistance, which cannot be done for free. (e.g. you must mix in TSC and/or RDRAND at every use.)
677 2017-02-16T19:43:59  <instagibbs> he has to gavel before we can switch
678 2017-02-16T19:44:02  <gmaxwell> okay!
679 2017-02-16T19:44:16  <wumpus> gmaxwell: oh, didn't know you were still typing, sorry
680 2017-02-16T19:44:18  <cfields> I have some local changes that implement the concept of different clocks/time_points/durations. The objective is for them to be incompatible with each-other.
681 2017-02-16T19:44:27  <gmaxwell> thats fine! just some things to think about.
682 2017-02-16T19:44:33  <wumpus> cfields: yes, that seems the way to go about it
683 2017-02-16T19:44:54  <sipa> cfields: f i may interject... i was thinking about creating a generic int class wrapper that supports no implicit conversioms
684 2017-02-16T19:44:59  <gmaxwell> cfields: pieter and I were talking about type safty recently, and pieter suggested a scheme for introducing more integer types which will never implicitly be converted.
685 2017-02-16T19:45:19  <wumpus> most importantly we should start using monotonic timestamps in the network code where possible
686 2017-02-16T19:45:51  <cfields> that sounds fine, but i'm not sure that they're entirely related here. The (my) objective is to stop storing time as an int, and instead as a time_value
687 2017-02-16T19:46:01  <sipa> cfields: or are timestamps in a c++11 world not something that fit in integers?
688 2017-02-16T19:46:04  <cfields> that way it can be represented in sec/msec/whatever whenever it's needed
689 2017-02-16T19:46:10  <cfields> sipa: exactly
690 2017-02-16T19:46:23  <cfields> it also enforces timestamps that can't be used on the wrong clock
691 2017-02-16T19:46:32  <sipa> i'm confused
692 2017-02-16T19:46:37  <gmaxwell> I have suggested in the past that we consider constructing a monotonic local clock, but wumpus seemed to not like the idea. which I think is orthorgonal to the type safty thing, but it would perhaps make time more sane in the codebase.
693 2017-02-16T19:46:44  <wumpus> cfields: in general that sounds good, though in some structures such as the block index we want to use as compact types as possible
694 2017-02-16T19:47:02  <gmaxwell> saftey*
695 2017-02-16T19:47:09  <gmaxwell> doh
696 2017-02-16T19:47:12  <cfields> wumpus: sure, you can always get an int out of it if you want
697 2017-02-16T19:47:13  <wumpus> gmaxwell: huh I'm all for using monotonic clocks were possible, they're just not good for everything
698 2017-02-16T19:47:22  <gmaxwell> safety**
699 2017-02-16T19:47:48  <sipa> cfields: my point was to have a template<typename tag> class non_convertible_int
700 2017-02-16T19:47:52  <cfields> also, i believe the class is actually not bigger than an int. It's just not convertable to int
701 2017-02-16T19:48:21  <wumpus> cfields: well if it represents micro/nanosecond it needs to be at least uint64 :)
702 2017-02-16T19:48:23  <sipa> and then have typedef non_comvertible_int<systemtime> systemtime_type
703 2017-02-16T19:48:36  <sipa> and systemtype_type is what is used
704 2017-02-16T19:48:44  <gmaxwell> you would get_int() on the object to get an int. You would just get a conversion by surprise.
705 2017-02-16T19:48:45  <sipa> *systemtime_type
706 2017-02-16T19:48:54  <gmaxwell> I would _hope_ we can construct something that at runtime is _exactly_ equal to using an integer.
707 2017-02-16T19:49:06  <sipa> cfields: you're being inconsistent
708 2017-02-16T19:49:19  <cfields> sipa: http://en.cppreference.com/w/cpp/chrono/time_point
709 2017-02-16T19:49:42  <gmaxwell> I think we should do this far more broadly than just timestamps however.
710 2017-02-16T19:49:43  <sipa> we can't use that in blocks, etc
711 2017-02-16T19:50:08  <wumpus> indeed - block times /consensus are a special case
712 2017-02-16T19:50:25  <cfields> sipa: we don't use timeval in blocks either though, we convert from the clock
713 2017-02-16T19:50:26  <sipa> but perhaps we should use those types in network state, measuring speed, ...
714 2017-02-16T19:50:35  <cfields> sipa: i'll demonstrate with code, probably easier that way
715 2017-02-16T19:50:53  <wumpus> right
716 2017-02-16T19:51:13  <sipa> cfields: what i want to address is the fact that an int or int64 can now mean microseconds, milliseconds, or seconds, and either system time, or monotonous time, or network-adjusted timr
717 2017-02-16T19:51:44  <sipa> it's fine that those are int-like, but they shouldn't be convertible from one into another
718 2017-02-16T19:51:59  <cfields> sipa: and that's exactly what i've addressed. Each of those gets its own type, and they're not convertable to eachother. But you can do a duration_cast<std::chrono::seconds>(foo) and get an int64_t seconds value out
719 2017-02-16T19:52:09  <sipa> anyway, for everything but consensus data structures, perhaps there are more c++ish ways
720 2017-02-16T19:52:50  <sipa> cfields: let's discuss this outside of theeeting
721 2017-02-16T19:52:51  <gmaxwell> This problem exists far beyond timestamps however. Use a node ID as a tx count? no problem.  Use a vin index as a block number? no problem. Use a bytes sent as a relay bool? no problem.
722 2017-02-16T19:53:16  <gmaxwell> We have multiple times had potentially serious bugs from the general issue of implicit conversions.
723 2017-02-16T19:53:35  <cfields> gmaxwell: yes, agreed that the tag would be very useful
724 2017-02-16T19:53:42  <gmaxwell> (or in the case of sighash single, an actual consensus behavior flaw)
725 2017-02-16T19:53:50  <cfields> sipa: np
726 2017-02-16T19:53:51  <sipa> jtimon had a topic as well
727 2017-02-16T19:53:56  <wumpus> using enumerations instead of booleans would also go a long way
728 2017-02-16T19:54:00  <jtimon> well, just a question really
729 2017-02-16T19:54:03  <jonasschnelli> I also have a little proposal
730 2017-02-16T19:54:04  <gmaxwell> all the data is there in the compile to prevent these mistakes, we're just not exposing it right. :)
731 2017-02-16T19:54:13  <jtimon> I guess it can wait after the meeting
732 2017-02-16T19:54:16  <wumpus> especially for functions that have tons of boolean arguments in succession, that's just crazy
733 2017-02-16T19:54:17  <sipa> wumpus: yeah, generally useful
734 2017-02-16T19:54:26  <gmaxwell> wumpus: yea, using bools, also using structs to get named parameters... lots of things we can do.
735 2017-02-16T19:54:30  <cfields> wumpus: for sure
736 2017-02-16T19:54:43  <jtimon> or answered fast enough that it doesn't need its own topic: "why 9619 doesn't go in for 0.14?"
737 2017-02-16T19:54:48  <gmaxwell> true false true true false die die die.  ... I hate counting aruments when changing things.
738 2017-02-16T19:54:48  <morcos> jtimon: i think because there was an error reported and no explanation that it was fixed or wasn't really a problem.  that's at least why i haven't looked at the PR.
739 2017-02-16T19:54:53  <wumpus> (well with some IDEs you can see what gets assigned to what parameter because it parses the interface, but that's not something you can realy on that everyone has available)
740 2017-02-16T19:55:00  <wumpus> gmaxwell: exactly
741 2017-02-16T19:55:08  <wumpus> jtimon: what was your topic?
742 2017-02-16T19:55:09  <instagibbs> gmaxwell, the fun really starts when you drop an arg with a default value at the end
743 2017-02-16T19:55:21  <cfields> (or three)
744 2017-02-16T19:55:28  <jtimon> morcos: that explains why is not merged, not why it's not labeled 0.14, but ok I guess
745 2017-02-16T19:55:53  <wumpus> jtimon: let's reverse the question: why would it need to be added for 0.14?
746 2017-02-16T19:55:55  <jtimon> wumpus: including 9619 in 0.14
747 2017-02-16T19:56:11  <jtimon> it's a bugfix and is simple enough, why not?
748 2017-02-16T19:56:41  <sipa> i think it can go in, it's trivial and very small in scope
749 2017-02-16T19:56:45  <gmaxwell> basically without it a plausable downstream gbt user that changes the transaction set could construct an overlarge block, is that the concern there?
750 2017-02-16T19:56:46  <wumpus> I'm fine with merging it before 0.14, if its sufficiently reviewed and teste
751 2017-02-16T19:56:50  <wumpus> it will however not hold up rc1
752 2017-02-16T19:57:08  <gmaxwell> it looks trivial and small in scope, and if my understanding is correct it should go in.. but sure, nothing should hold up rc1.
753 2017-02-16T19:57:19  <gmaxwell> at least nothing that we know of now.
754 2017-02-16T19:57:20  <sipa> fair
755 2017-02-16T19:57:35  <jonasschnelli> Not sure if this makes sense, But I propose to rename the ./qa dir to ./test (or something that sorts after ./src). I think it would be better to have the RPC tests further down in the PRs diff.
756 2017-02-16T19:57:36  <jtimon> sure, was simply surprised it didn't had the 0.14 label since it's not really that new
757 2017-02-16T19:57:57  <Victorsueca> I see lots of utACKs but not any ACK, so trivial it doesn't need testing I assume?
758 2017-02-16T19:57:57  <gmaxwell> short of some kind of crash eat money doom bug showing up before we manage to get it out (and even that shouldn't delay _constructing_ RC1; if doom shows up we can abort launch)
759 2017-02-16T19:58:30  <wumpus> gmaxwell: sure, there are always serious enough problems possible that should postpone the release
760 2017-02-16T19:58:57  <gmaxwell> jonasschnelli: On that I'd like to get into a state where make check is running those tests. They are most of our tests now, and the most valuable.. and really people building with compilers we've never seen before _REALLY_ ought to be running them.
761 2017-02-16T19:59:18  <wumpus> jonasschnelli: don't really have an opinion on that, does it matter much how things are sorted?
762 2017-02-16T19:59:35  <gmaxwell> Why would the sorting matter?
763 2017-02-16T19:59:39  <jonasschnelli> As long as review is a bottleneck I think it matters
764 2017-02-16T19:59:53  <jonasschnelli> But maybe I'm alone with that opinion.
765 2017-02-16T20:00:05  <gmaxwell> OH so they show up lower on github.
766 2017-02-16T20:00:21  <sipa> /zzztest
767 2017-02-16T20:00:28  <wumpus> my biggest pet peeve is that they're called RPC tests, not 'functional tests' or such, they're testing much more than RPC these days :)
768 2017-02-16T20:00:41  <jonasschnelli> ./test_functional
769 2017-02-16T20:00:43  <gmaxwell> It is sometimes a bit confusing that the tests show up first... otoh it can be informative to read the test before the change in the cases where the test is especially good. :)
770 2017-02-16T20:00:45  <wumpus> but not serious enough to actually go on renamind directories
771 2017-02-16T20:00:49  <sipa> also, pull-tester should be merged in rpc-tests
772 2017-02-16T20:00:51  <gmaxwell> wumpus: they are "system tests"  rpc is incidental.
773 2017-02-16T20:01:36  <sipa> we don't have pulltester anymore, and it's annoying to always change directory :)
774 2017-02-16T20:01:44  <wumpus> gmaxwell: yes, it doesn't matter what interface they use, whether it's RPC or P2P or ZMQ or REST or command line arguments etc
775 2017-02-16T20:01:44  <instagibbs> we're over time
776 2017-02-16T20:01:46  <instagibbs> btw
777 2017-02-16T20:01:52  <jonasschnelli> Okay. I'll do a cleanup PR once we branch off.
778 2017-02-16T20:01:53  <wumpus> instagibbs: ah yes
779 2017-02-16T20:01:55  <wumpus> #endmeeting
780 2017-02-16T20:01:55  <lightningbot> Meeting ended Thu Feb 16 20:01:55 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
781 2017-02-16T20:01:55  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-16-19.02.html
782 2017-02-16T20:01:55  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-16-19.02.txt
783 2017-02-16T20:01:55  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-02-16-19.02.log.html
784 2017-02-16T20:02:10  <jnewbery> wumpus: Agree, and can we name it something other than pull-tester?
785 2017-02-16T20:02:17  <Chris_Stewart_5> sipa: gmaxwell So if I understand you guys, you want to create custom types for all time related structures inside of core?
786 2017-02-16T20:02:23  <gmaxwell> petertodd: nice ACK on 9779.
787 2017-02-16T20:02:33  <Chris_Stewart_5> even further, integer related structures?
788 2017-02-16T20:02:49  <sipa> Chris_Stewart_5: well it seems c++11 has things like that already, and they're more featureful than just integers
789 2017-02-16T20:02:51  <cfields> Chris_Stewart_5: i'm suggesting that we switch to std::chrono::duration and std::chrono::time_point
790 2017-02-16T20:02:56  <wumpus> jnewbery: yes, pulltester is a weird name now, it's simply a launch framework for the tests
791 2017-02-16T20:03:05  <Victorsueca> midnightmagic: "and really people building with compilers we've never seen before _REALLY_ ought to be running them." <--- I tried it once on windows but it looks like they're broken and only work on linux
792 2017-02-16T20:03:12  <sipa> Chris_Stewart_5: but yes, making ints not convertible into other int-like-types can fix bugs
793 2017-02-16T20:03:24  <jnewbery> test-launcher.py or test-runner.py seem fine
794 2017-02-16T20:03:41  <gmaxwell> wumpus: re 9779, assuming that gets ACKs feel free to merge that along with the version bump. (the assumevalid was 6119 blocks behind.)
795 2017-02-16T20:03:44  <wumpus> Victorsueca: that's simply not true! they work on at least openbsd, freebsd, linux, and windows (in wine at least)
796 2017-02-16T20:03:49  <Chris_Stewart_5> sipa: Sure, no arguments with that. Just wasn't sure if that was _exactly_ what you guys were saying in the meeting.
797 2017-02-16T20:03:53  <wumpus> Victorsueca: oh and OSX
798 2017-02-16T20:04:05  <Victorsueca> wumpus: hmmm should try it again
799 2017-02-16T20:04:07  <Chris_Stewart_5> sipa: But conversions will still be able to be done right? Just with explicit function calls instead of casting correct?
800 2017-02-16T20:04:15  <wumpus> Victorsueca: our tests are very well maintained on many platforms
801 2017-02-16T20:04:21  <sipa> Chris_Stewart_5: of course
802 2017-02-16T20:04:35  <sipa> otherwise it's useless
803 2017-02-16T20:04:39  <cfields> Chris_Stewart_5: only to a degree. You shouldn't be able to compare a system_clock time_point to a steady_clock one. Because that makes no sense.
804 2017-02-16T20:04:49  <gmaxwell> They tests have extra dependencies so its quite plausable that on some systems that compile the code the tests fail.
805 2017-02-16T20:05:17  <wumpus> cfields: or a duration to a absolute time value!
806 2017-02-16T20:05:32  <cfields> wumpus: right
807 2017-02-16T20:05:37  <Victorsueca> wumpus: which one should I be running to run all tests?
808 2017-02-16T20:05:55  <Victorsueca> ahh I think I found it
809 2017-02-16T20:05:57  <wumpus> gmaxwell: it should disable the tests in question in that case, e.g. if there's no zmq if doesn't run that test
810 2017-02-16T20:06:01  <jonasschnelli> Victorsueca: ./qa/pull-tester/rpc-tests.py
811 2017-02-16T20:06:12  <jtimon> wumpus: right, functions with many bools could use thier own flags or something perhaps (sorry, was catching up on the previous discussion)
812 2017-02-16T20:06:26  <jonasschnelli> Victorsueca: mind the -win parameter
813 2017-02-16T20:06:34  <wumpus> jtimon: yes, either bitfield or individual enums, everything is better than bool,bool,bool,bool
814 2017-02-16T20:06:52  <cfields> so instead, we get the concept of a clock, which gives you time_point's with some meaning. a time_point, which is an absolute time (but not necessarily wall-time) on some clock, and a duration, which is the difference between 2 time_points
815 2017-02-16T20:07:00  <sipa> wumpus: for cases where it's excessive you can emulate named arguments using a struct
816 2017-02-16T20:07:10  <jnewbery> jonasschnelli: My preference would be to also move the bitcoin-util-test.py and bctest.py (and data for those tests) from src/test to whatever you rename qa to. Those tests seem much more like the integration tests in qa than the unit tests in src/test
817 2017-02-16T20:07:12  <jtimon> and I really hate we have so many bools with default values
818 2017-02-16T20:07:35  <sipa> wumpus: for example a struct ATMPArgs that takes in its constructor all 'required' fields for ATMP, and has methods to change optionals
819 2017-02-16T20:07:46  <Chris_Stewart_5> cfields: Wouldn't that comparison have to be enforced through code review and not a compile time failure? Sorry not super familar with the data structures..
820 2017-02-16T20:07:49  <jonasschnelli> jnewbery: Indeed. I'll try to move them.
821 2017-02-16T20:07:58  <gmaxwell> I believe a struct for inputs can be nearly zero runtime overhead for functions with many arguments.
822 2017-02-16T20:08:17  <sipa> Chris_Stewart_5: it will result in compile time failure; a timepoint for a steady clock would be a different data type
823 2017-02-16T20:08:28  <wumpus> sipa: c99 designated struct initializers were always nice to use for emulating named arguments in C, too bad that never made it to C++
824 2017-02-16T20:08:32  <cfields> Chris_Stewart_5: no, it's enforced by the compiler. A time_point knows what clock it belongs to
825 2017-02-16T20:08:50  <sipa> So you'd call AcceptToMemoryPool(ATMPArgs{std::move(tx)}.max_fee(fee))
826 2017-02-16T20:08:57  <gmaxwell> Chris_Stewart_5: enforcing things through code review that could be enforced by the machine is a losing strategy. Code review is essential but it should be focused on the bugs the machine cannot prevent.
827 2017-02-16T20:08:59  <petertodd> gmaxwell: heh, yeah, I wanted to make a point of what the trust model was, particularly since -assumevalid was (partly) my idea! :)
828 2017-02-16T20:09:10  <Victorsueca> jonasschnelli: wumpus: File "rpc-tests.py", line 283 print('.', end='', flush=True) SyntaxError: invalid syntax
829 2017-02-16T20:09:15  <Victorsueca> uhmm shit hit the fan?
830 2017-02-16T20:09:21  <wumpus> sipa: in any case, we also shouldn't go over the top with this, trying to emulate something in c++ that it is not :)
831 2017-02-16T20:09:23  <jonasschnelli> Victorsueca: python3?
832 2017-02-16T20:09:35  <Chris_Stewart_5> gmaxwell: No doubt. I'm all for type safety. Compilers are our friends :-)
833 2017-02-16T20:09:37  <jonasschnelli> (requires p3)
834 2017-02-16T20:09:40  <sipa> Victorsueca: looks like you're trying to run with python2
835 2017-02-16T20:09:57  <Victorsueca> ahh forgot python stands for 2 and py stands for 3-2 hub
836 2017-02-16T20:10:25  <sipa> Victorsueca: no, the first line of the file states the interpreter to use, but i guess that only works in unix-like systems
837 2017-02-16T20:10:44  <wumpus> you should run all the bitcoin core scripts with python 3
838 2017-02-16T20:10:49  <jtimon> gmaxwell: ack on adding the rpc tests on make check
839 2017-02-16T20:11:07  <Victorsueca> here on windows if I run py it auto-detects whether 2 or 3 should be used
840 2017-02-16T20:11:25  <Victorsueca> it's just i'm used to write python
841 2017-02-16T20:11:46  <gmaxwell> wumpus: MISRA C (standard for safety critical C code targeting embedded systems) requires that no generic types be used, and certantly no sizeless generic types.  And that is in C where you still get the implicit conversion and can't really avoid it, but the standard argues that static analysis tools can still catch your errors so long as you've exposed the type information.  Libsecp256k1 is pre
842 2017-02-16T20:11:52  <gmaxwell> tty close to this now.
843 2017-02-16T20:12:31  <wumpus> well if the first line contains python3 it should be run with python 3, lacking that there is no general way to detect whether code is python2 or python3. But believe me, the bitcoin core python scripts need to be run with python 3.
844 2017-02-16T20:13:43  <wumpus> gmaxwell: the most significant non-MISRA thing about secp256k1 is probably that it allocates a context on the heap?
845 2017-02-16T20:15:07  <bitcoin-git> [bitcoin] jnewbery opened pull request #9780: Suppress noisy output from qa tests in Travis (master...travislogging) https://github.com/bitcoin/bitcoin/pull/9780
846 2017-02-16T20:16:09  <gmaxwell> wumpus: Well MISRA doesn't actually forbid anything but requires that for every violation you write an exception document. So the context is one where the exception document is very easy to right: this allocation is pure initilization, not runtime.
847 2017-02-16T20:17:17  <gmaxwell> What I've found harder to deal with is that it requires no function have multiple returns. As there are some cases where we do where fixing it feels like it would objectively worsen the code. But their argument is also pretty compelling
848 2017-02-16T20:18:20  <wumpus> yes, there's something to be said for that, especially with regard to cleanup and such
849 2017-02-16T20:18:37  <gmaxwell> It would also be very easy for us to make it possible to make libsecp256k1 mallocless with ifdefs, it's specifically structured to enable that.
850 2017-02-16T20:18:38  <wumpus> though in some cases it can lead to terribly verbose code when combined with error handling
851 2017-02-16T20:19:01  <wumpus> e.g. either deeply nested if() or some status flag that is checked every other statement
852 2017-02-16T20:19:28  <cfields> wumpus: but those easily fixed with a goto ;)
853 2017-02-16T20:19:28  <gmaxwell> (My goal though not a priority is to make libsecp256k1 MISRA conformant eventually)
854 2017-02-16T20:19:47  <gmaxwell> cfields: forbidden by MISRA C. (though like anything else you can make exceptions)
855 2017-02-16T20:19:55  <jtimon> jonasschnelli: regarding your rename, if rpc-tests/ and pull-tester/ in qa (as I think sipa suggests ), it may make sense to take the opportunity to rename qa as well
856 2017-02-16T20:19:56  <wumpus> cfields: hah! I didn't expect those to be allowed if multiple returns are not
857 2017-02-16T20:20:08  <cfields> gmaxwell: was just joking about trading one thing for another
858 2017-02-16T20:20:17  <gmaxwell> Gotos that go backwards also forbidden by an extra rule, so at least error handling doesn't hit those.
859 2017-02-16T20:20:19  <wumpus> cfields: I always tend to use goto's in C code for cleanups, kernel style
860 2017-02-16T20:21:03  <gmaxwell> cfields: well I wasn't goto is actually an excellent way to handle error handling in C in some cases. It's very similar to multiple returns in its downsides, and yet few people care about peppering functions with 15 returns.
861 2017-02-16T20:21:23  <cfields> wumpus: yes, they seem pretty necessary after using raii for long enough
862 2017-02-16T20:22:32  <gmaxwell> The MISRA document is a really good read FWIW. It's pretty thoughtful in its recommendations.
863 2017-02-16T20:22:50  <gmaxwell> though not so applicable to C++. :)
864 2017-02-16T20:22:52  <cfields> gmaxwell: sure, agreed. I just used it as an example because it's often multiple-returns and gotos blindly spouted as "thou-shalt-not"s in c-code
865 2017-02-16T20:24:42  <wumpus> I can't come up with a good reason to use backwards gotos though
866 2017-02-16T20:24:44  <cfields> gmaxwell: well, no-multiple-returns carries some weight in c++ too as they inhibit rvo. Which can be very useful for structures that you assume move freely/cheaply
867 2017-02-16T20:25:29  <Victorsueca> ok... now this is a mess.... I use WSL to cross-compile windows binaries, now all the tests have UNIX-like paths and can't run them on windows
868 2017-02-16T20:27:10  <isle2983> gmaxwell: do you try any clang/LLVM plugins for checking MISRA? I am just wondering if it is worthwhile exploring
869 2017-02-16T20:27:20  <wumpus> gcc computed goto is even more evil: arrays of labels, though it's used in e.g. the python bytecode dispatch loop for the last bits of speed optimization
870 2017-02-16T20:27:52  <gmaxwell> isle2983: not clang/llvm ones, but I've used other static analysis tools that can check it, in particular pc-lint... They're of some value.
871 2017-02-16T20:28:56  <wumpus> Victorsueca: strange - python code doesn't get compiled as such, it must be one of the generated py files that has a path in it. You could try explicitly setting the environment variable BITCOIND to the path of bitcoind, that will override the built-in guess
872 2017-02-16T20:29:16  <gmaxwell> wumpus: some kinds of valid but complex flow control is just not expressable in C without a loss of efficiency.  So you could imagine some kind of hyper optimized inner loop that uses backwards goto.  I don't recall any case where I've been forced into that, though I have run into cases where performance required do{}while() and non-trivial code inside for expressions, and forwards goto out of a
873 2017-02-16T20:29:22  <gmaxwell>  loop that also had breaks and continues.
874 2017-02-16T20:30:02  <wumpus> gmaxwell: yes, forwards goto out of a loop makes a lot of sense, if you want to break out of multiple levels
875 2017-02-16T20:30:52  <gmaxwell> I wish C(++) had named continue/break.
876 2017-02-16T20:31:08  <wumpus> gmaxwell: and these things matter, I mean if efficiency is not important you wouldn't be using c in the first place
877 2017-02-16T20:31:30  <gmaxwell> (Rust has named breaks)
878 2017-02-16T20:31:46  <gmaxwell> (also more important because it lacks the other options C has for that)
879 2017-02-16T20:31:50  <wumpus> yes I remember quite some languages had them ,though I can't think of any right now
880 2017-02-16T20:32:22  <wumpus> didn't know that rust did
881 2017-02-16T20:33:22  <sipa> in c++ it's more complicated because you can't jump over variable initializations
882 2017-02-16T20:34:07  <cfields> gmaxwell: i suspect that architects assumed throw/catch would be used for breaking out of multiple loops. But that's seen as poisonous to most, i think
883 2017-02-16T20:34:18  <wumpus> ah, java has them apparently
884 2017-02-16T20:34:25  <gmaxwell> cfields: youch. :P
885 2017-02-16T20:34:46  <cfields> gmaxwell: my thought exactly :)
886 2017-02-16T20:34:51  <wumpus> cfields: exceptions as a control flow statement?  speak about shooting a fly with a thermonuclear missile :p
887 2017-02-16T20:35:02  <gmaxwell> beyond the other issues with exception, they're slow... sooo.
888 2017-02-16T20:35:44  <wumpus> right, they're slow exactly because they don't know where they're going to be catched, so they're a bad fit if you know exactly where you wantcontrol flow to go
889 2017-02-16T20:36:17  <wumpus> the code for exceptions and stack unrolling is extremely complex, I looked at it some time there's even some cute VM involved that interprets DWARF debug info
890 2017-02-16T20:37:05  <cfields> wumpus: heh, i once worked on some code that used it to parse some data, construct the right handler, and throw it. Then the catch was an abstract parent, and it would continue on with the newly-constructed handler
891 2017-02-16T20:37:27  <wumpus> cfields: hahaha
892 2017-02-16T20:37:28  <cfields> it was ugly, but surprisingly effective :)
893 2017-02-16T20:37:32  <wumpus> cfields: anti-reverse-engineering?
894 2017-02-16T20:38:00  <cfields> wumpus: heh, it would seem. no idea what the historical reasoning was
895 2017-02-16T20:38:52  <gmaxwell> later you find out the guy that wrote it didn't know that function pointers or virtual methods were possible. :P
896 2017-02-16T20:39:02  <wumpus> like anytiing so evil it sounds like a hell to debug
897 2017-02-16T20:39:07  <cfields> gmaxwell: entirely possible :)
898 2017-02-16T20:40:53  <wumpus> it reminds me of that saying that you should always expect that the person maintaining the code after you is a crazy axe murderer that knows where you live, this certainly classifies as inhuman cruelty to your successors
899 2017-02-16T20:41:26  <jeremyrubin> sorry for misisng meeting! I was off by an hour :p
900 2017-02-16T20:41:37  <jeremyrubin> I think w.r.t. entropy levels we should keep 3
901 2017-02-16T20:41:51  <jeremyrubin> Deterministic, non-deterministic, secure
902 2017-02-16T20:41:52  <cfields> std::chrono::off_by_one_clock
903 2017-02-16T20:42:08  <jeremyrubin> deterministic is really useful for writing tests/benchmarks IMO
904 2017-02-16T20:42:28  <wumpus> it's only useful for the tests and benchmarks, and could be in test_util.cpp
905 2017-02-16T20:42:55  <jeremyrubin> That's fine to me!
906 2017-02-16T20:42:56  <cfields> jeremyrubin: we have siphasher, which we use for deterministic randomness, and i don't think it really needs to care about the source of its seed?
907 2017-02-16T20:43:27  <wumpus> but I agree it's useful to have fast deterministic random for some things, I manually rolled a LFSR in one place in the tests to quickly generate deterministic "random" numbers
908 2017-02-16T20:44:03  <jeremyrubin> cfields: fine too -- just having something convenient to use for this purpose is all I care for!
909 2017-02-16T20:45:03  <jeremyrubin> The cuckoocache tests test hit rate against deterministic data, so if the entropy changes then it *could* (unlikely) fail.
910 2017-02-16T20:45:51  <jeremyrubin> but it depends on the variance of the hit rate... so it would be annoying if it fails say 1/100 times due to unlucky entropy
911 2017-02-16T20:46:34  <wumpus> yes, that should ideally never happen
912 2017-02-16T20:47:14  <wumpus> tests that randomly fail are extremely discouraging
913 2017-02-16T20:48:18  <jeremyrubin> Indeed
914 2017-02-16T20:48:38  <jeremyrubin> travis-gate was really a sad week for bitcoin ;)
915 2017-02-16T20:52:54  <cfields> heh
916 2017-02-16T20:59:16  <jeremyrubin> Speaking of tests -- any chance for putting #9495 (bugfix) and #9497 (tests) in 0.14? The CheckQueue is a pretty big piece of consensus code with almost no test coverage on it... Then again, there's no reason these tests can't wait for 0.15 either :)
917 2017-02-16T20:59:18  <gribble> https://github.com/bitcoin/bitcoin/issues/9495 | Fix CCheckQueue IsIdle (potential) race condition by JeremyRubin · Pull Request #9495 · bitcoin/bitcoin · GitHub
918 2017-02-16T20:59:20  <gribble> https://github.com/bitcoin/bitcoin/issues/9497 | CCheckQueue Unit Tests by JeremyRubin · Pull Request #9497 · bitcoin/bitcoin · GitHub
919 2017-02-16T21:15:54  <jnewbery> jonasschnelli: I've got several PRs that are very disruptive to the qa directory: #9577 #9657 #9768 and the completion of #9707 (which I haven't yet PR'ed). Can you hold off your reorganzation of the qa directory until those are merged? Perhaps open an issue now for discussion but hold off on PR'ing.
920 2017-02-16T21:15:56  <gribble> https://github.com/bitcoin/bitcoin/issues/9577 | Fix docstrings in qa tests by jnewbery · Pull Request #9577 · bitcoin/bitcoin · GitHub
921 2017-02-16T21:15:58  <gribble> https://github.com/bitcoin/bitcoin/issues/9657 | Improve rpc-tests.py by jnewbery · Pull Request #9657 · bitcoin/bitcoin · GitHub
922 2017-02-16T21:16:00  <gribble> https://github.com/bitcoin/bitcoin/issues/9768 | [qa] [WIP] Add logging to test_framework.py by jnewbery · Pull Request #9768 · bitcoin/bitcoin · GitHub
923 2017-02-16T21:16:02  <gribble> https://github.com/bitcoin/bitcoin/issues/9707 | Fix RPC failure testing by jnewbery · Pull Request #9707 · bitcoin/bitcoin · GitHub
924 2017-02-16T21:31:34  <jeremyrubin> jnewbery: git can handle renames fine iirc?
925 2017-02-16T21:32:56  <jnewbery> jeremyrubin: I think you're right. If jonas is only going to rename files then there shouldn't be any conflicts.
926 2017-02-16T21:33:38  <jeremyrubin> jonasschnelli: ^^^
927 2017-02-16T21:33:55  *** Guyver2 has joined #bitcoin-core-dev
928 2017-02-16T21:50:03  *** Guyver2 has quit IRC
929 2017-02-16T21:59:37  *** aj_ is now known as aj
930 2017-02-16T22:10:46  *** wvr has quit IRC
931 2017-02-16T22:11:24  *** wvr has joined #bitcoin-core-dev
932 2017-02-16T22:13:08  *** chjj has quit IRC
933 2017-02-16T22:22:22  *** chjj has joined #bitcoin-core-dev
934 2017-02-16T23:09:41  *** face has quit IRC
935 2017-02-16T23:11:59  *** face has joined #bitcoin-core-dev
936 2017-02-16T23:15:27  *** paracyst_ is now known as paracyst
937 2017-02-16T23:18:40  *** jnewbery has quit IRC
938 2017-02-16T23:18:54  *** jnewbery has joined #bitcoin-core-dev
939 2017-02-16T23:24:57  *** laurentmt has joined #bitcoin-core-dev
940 2017-02-16T23:26:30  *** laurentmt has quit IRC
941 2017-02-16T23:36:27  *** chjj has quit IRC
942 2017-02-16T23:44:24  *** aalex has quit IRC
943 2017-02-16T23:50:05  *** Giszmo has quit IRC
944 2017-02-16T23:56:37  *** Giszmo has joined #bitcoin-core-dev