1 2017-08-08T00:00:16  *** chjj has quit IRC
  2 2017-08-08T00:01:38  *** promag has joined #bitcoin-core-dev
  3 2017-08-08T00:12:32  *** chjj has joined #bitcoin-core-dev
  4 2017-08-08T00:22:12  *** Chris_Stewart_5 has joined #bitcoin-core-dev
  5 2017-08-08T00:27:35  *** AaronvanW has quit IRC
  6 2017-08-08T00:30:18  *** jimpo has joined #bitcoin-core-dev
  7 2017-08-08T00:32:38  <jimpo> What is HSNG? (mentioned in discussion of new addr message)
  8 2017-08-08T00:33:26  *** abpa has quit IRC
  9 2017-08-08T00:35:08  <gmaxwell> Hidden services NG. Tor HS addresses will be becoming 256 bits long.
 10 2017-08-08T00:35:43  <gmaxwell> NG = next gen, I assume
 11 2017-08-08T00:36:06  <jimpo> interesting. thx
 12 2017-08-08T00:38:27  *** veleiro has quit IRC
 13 2017-08-08T00:39:10  *** chjj has quit IRC
 14 2017-08-08T00:43:03  *** veleiro has joined #bitcoin-core-dev
 15 2017-08-08T00:45:20  *** AaronvanW has joined #bitcoin-core-dev
 16 2017-08-08T00:49:38  *** Cheeseo has quit IRC
 17 2017-08-08T00:52:19  *** chjj has joined #bitcoin-core-dev
 18 2017-08-08T01:00:10  *** chjj has quit IRC
 19 2017-08-08T01:05:58  *** jimpo has quit IRC
 20 2017-08-08T01:13:56  *** chjj has joined #bitcoin-core-dev
 21 2017-08-08T01:18:30  *** chjj has quit IRC
 22 2017-08-08T01:29:01  *** shaolinfry has quit IRC
 23 2017-08-08T01:29:17  *** dgenr8 has quit IRC
 24 2017-08-08T01:31:57  *** chjj has joined #bitcoin-core-dev
 25 2017-08-08T01:33:28  *** jimpo has joined #bitcoin-core-dev
 26 2017-08-08T01:34:55  <jimpo> is there anyone working/focussing on extracting libbitcoinconsensus?
 27 2017-08-08T01:36:07  <Chris_Stewart_5> jimpo: Last I knew jtimon was, not sure if he is still working on it though
 28 2017-08-08T01:39:02  <bitcoin-git> [bitcoin] eklitzke opened pull request #11005: Tests: Add a lint check for trailing whitespace (master...whitespace) https://github.com/bitcoin/bitcoin/pull/11005
 29 2017-08-08T02:04:38  *** Alina-malina has quit IRC
 30 2017-08-08T02:07:38  *** jimpo has quit IRC
 31 2017-08-08T02:08:38  *** Alina-malina has joined #bitcoin-core-dev
 32 2017-08-08T02:17:19  *** windsok has quit IRC
 33 2017-08-08T02:17:22  *** Alina-malina has quit IRC
 34 2017-08-08T02:18:37  *** windsok has joined #bitcoin-core-dev
 35 2017-08-08T02:29:48  *** Alina-malina has joined #bitcoin-core-dev
 36 2017-08-08T02:31:59  *** Murch has quit IRC
 37 2017-08-08T02:34:27  *** jamesob has quit IRC
 38 2017-08-08T02:35:56  *** Alina-malina has quit IRC
 39 2017-08-08T02:37:14  *** veleiro has quit IRC
 40 2017-08-08T02:38:30  *** Alina-malina has joined #bitcoin-core-dev
 41 2017-08-08T02:42:51  *** veleiro has joined #bitcoin-core-dev
 42 2017-08-08T02:44:27  *** Alina-malina has quit IRC
 43 2017-08-08T02:47:00  *** Alina-malina has joined #bitcoin-core-dev
 44 2017-08-08T02:53:37  *** jouke has quit IRC
 45 2017-08-08T02:55:27  *** Alina-malina has quit IRC
 46 2017-08-08T02:55:45  *** jouke has joined #bitcoin-core-dev
 47 2017-08-08T02:59:25  *** Alina-malina has joined #bitcoin-core-dev
 48 2017-08-08T03:02:40  <bitcoin-git> [bitcoin] promag opened pull request #11006: WIP: Improve shutdown process (master...201708-fast-shutdown) https://github.com/bitcoin/bitcoin/pull/11006
 49 2017-08-08T03:05:40  *** Alina-malina has quit IRC
 50 2017-08-08T03:06:22  *** Alina-malina has joined #bitcoin-core-dev
 51 2017-08-08T03:12:39  *** veleiro1 has joined #bitcoin-core-dev
 52 2017-08-08T03:12:51  *** veleiro has quit IRC
 53 2017-08-08T03:14:57  *** Alina-malina has quit IRC
 54 2017-08-08T03:26:42  *** Alina-malina has joined #bitcoin-core-dev
 55 2017-08-08T03:29:26  *** ula has quit IRC
 56 2017-08-08T03:40:37  *** promag has quit IRC
 57 2017-08-08T03:42:47  *** veleiro1 has left #bitcoin-core-dev
 58 2017-08-08T03:54:16  *** chjj has quit IRC
 59 2017-08-08T04:14:35  *** Chris_Stewart_5 has quit IRC
 60 2017-08-08T04:22:38  <luke-jr> gmaxwell: hmm, but it makes sense to show them up top in the question page :/
 61 2017-08-08T04:27:31  <gmaxwell> luke-jr: so make the order different for each, questions by least answers answers by most
 62 2017-08-08T04:35:56  *** Alina-malina has quit IRC
 63 2017-08-08T04:35:56  *** Alina-malina has joined #bitcoin-core-dev
 64 2017-08-08T04:52:28  *** d_t has quit IRC
 65 2017-08-08T05:20:35  *** dcousens has joined #bitcoin-core-dev
 66 2017-08-08T05:20:39  <luke-jr> gmaxwell: done
 67 2017-08-08T05:29:17  *** chjj has joined #bitcoin-core-dev
 68 2017-08-08T05:37:53  *** corinrose_ has joined #bitcoin-core-dev
 69 2017-08-08T06:12:00  *** miknotauro has joined #bitcoin-core-dev
 70 2017-08-08T06:12:00  *** Austindoggie has joined #bitcoin-core-dev
 71 2017-08-08T06:17:03  *** Austindoggie has left #bitcoin-core-dev
 72 2017-08-08T06:25:18  *** dermoth has quit IRC
 73 2017-08-08T06:25:38  *** dermoth has joined #bitcoin-core-dev
 74 2017-08-08T06:48:32  *** BashCo has quit IRC
 75 2017-08-08T06:49:06  *** BashCo has joined #bitcoin-core-dev
 76 2017-08-08T06:53:53  *** BashCo has quit IRC
 77 2017-08-08T07:12:10  *** BashCo has joined #bitcoin-core-dev
 78 2017-08-08T07:15:03  *** Deacydal has joined #bitcoin-core-dev
 79 2017-08-08T07:17:07  *** Deacyde has quit IRC
 80 2017-08-08T07:18:06  *** promag has joined #bitcoin-core-dev
 81 2017-08-08T07:20:15  *** miknotauro has quit IRC
 82 2017-08-08T07:45:02  <luke-jr> hm, it's a shame we're releasing 0.15 without bech32 sending :x
 83 2017-08-08T07:45:53  <gmaxwell> luke-jr: when does segwit activate?
 84 2017-08-08T07:46:17  <luke-jr> gmaxwell: Aug 21 it looks like
 85 2017-08-08T07:46:37  <gmaxwell> luke-jr: also it's just not that mature yet,... a serious problem was found in the test vectors of the spec and ref implementations just a week ago.
 86 2017-08-08T07:46:46  <luke-jr> :<
 87 2017-08-08T07:47:09  <gmaxwell> luke-jr: dunno if you read the minutes from the last meeting (or was it the one before last) but we were talkign about a short cycle release with segwit wallet support right after 15 in any case.
 88 2017-08-08T07:47:36  <luke-jr> I didn't, but newbery did mention it to me. I agree a rapid 0.16 would be good.
 89 2017-08-08T07:47:50  <gmaxwell> https://github.com/sipa/bech32/pull/21 also re: bech32 support.
 90 2017-08-08T07:49:04  <luke-jr> is there a reason to have a C++ implementation at all? seems to me the proper route to go would be to make a C libbech32, which can be used by Core…?
 91 2017-08-08T07:51:24  <gmaxwell> a library for 50 lines of code... what are you, a nodejs developer? lpad() ftw? :P  But seriously, a native C++ interface is typesafe. Wrappign a C implementation to give it a C++ friendly interface would probably end up being the same size as the library itself.
 92 2017-08-08T07:51:49  <gmaxwell> FWIW it's much simpler to implement bech32 than base58.
 93 2017-08-08T07:52:30  <luke-jr> hmm
 94 2017-08-08T08:23:06  *** SopaXorzTaker has joined #bitcoin-core-dev
 95 2017-08-08T08:25:16  <achow101> I suspect that #10982 is going to need to be locked soon due to brigading and trolling
 96 2017-08-08T08:25:20  <gribble> https://github.com/bitcoin/bitcoin/issues/10982 | Disconnect network service bits 6 and 8 until Aug 1, 2018 by TheBlueMatt · Pull Request #10982 · bitcoin/bitcoin · GitHub
 97 2017-08-08T08:25:26  <jonasschnelli> How do we deal with the service bits 6 and 8? Should NODE_NETWORK_LIMITED avoid bit6 because something else claimed it (although not via BIP process)?
 98 2017-08-08T08:27:18  <achow101> It was actually bits 5 and 7 that were claimed
 99 2017-08-08T08:30:09  <luke-jr> kidnapped*? :P
100 2017-08-08T08:33:08  <jonasschnelli> achow101: Bip 7 is SW2x, right? Whats 5?
101 2017-08-08T08:35:32  <luke-jr> jonasschnelli: BCash I assume
102 2017-08-08T08:37:59  *** AaronvanW has quit IRC
103 2017-08-08T08:41:56  *** promag has quit IRC
104 2017-08-08T08:42:46  *** Bartholome has joined #bitcoin-core-dev
105 2017-08-08T08:46:50  *** Bartholome has quit IRC
106 2017-08-08T08:51:29  <luke-jr> achow101: sigh, would have been nice to get the commit message fixed before merging :/
107 2017-08-08T08:54:22  <gmaxwell> unfortunately s2x people were spamming their slack with it and such the moment it went up.
108 2017-08-08T08:54:24  *** AaronvanW has joined #bitcoin-core-dev
109 2017-08-08T08:55:10  <luke-jr> it would also be nice if we didn't merge things because of trolls :x
110 2017-08-08T08:56:49  *** corinrose_ has quit IRC
111 2017-08-08T09:03:29  <luke-jr> jonasschnelli: perhaps: uint16_t archival_data_flag = (servicebits & NODE_NETWORK_FLAG_MASK); if (servicebits & NODE_NETWORK) { … } else if (archival_data_flag == NODE_NETWORK_LIMITED_HIGH) { … } else if (archival_data_flag == NODE_NETWORK_LIMITED_LOW) { … } would fit nicely
112 2017-08-08T09:08:03  *** timothy has joined #bitcoin-core-dev
113 2017-08-08T09:18:50  <MarcoFalke> Reminder to call gpg --keyserver pgp.mit.edu --refresh-keys F4316B9B
114 2017-08-08T09:21:06  <gmaxwell> achow101: depends on if you count the bits starting at 0 or 1
115 2017-08-08T09:25:22  <luke-jr> gmaxwell: which has always been counted from 0..
116 2017-08-08T09:26:13  <jnewbery> NODE_NONE must surely be zero?
117 2017-08-08T09:26:22  <gmaxwell> depends on your language,  "first bit" is 1<<0.
118 2017-08-08T09:28:34  <jonasschnelli> From what I read you usually start a 0
119 2017-08-08T09:28:36  <jonasschnelli> *at
120 2017-08-08T09:28:40  <bitcoin-git> [bitcoin] laanwj pushed 5 new commits to master: https://github.com/bitcoin/bitcoin/compare/fa8a0639f7b0...627c3c0e495b
121 2017-08-08T09:28:41  <bitcoin-git> bitcoin/master dac3782 Wladimir J. van der Laan: doc: Correct AmountFromValue/ValueFromAmount names
122 2017-08-08T09:28:41  <bitcoin-git> bitcoin/master 46347ad Wladimir J. van der Laan: rpc: Move ValueFromAmount to core_write...
123 2017-08-08T09:28:42  <bitcoin-git> bitcoin/master ec05c50 Wladimir J. van der Laan: rpc: Use ValueFromAmount instead of FormatMoney in TxToUniv...
124 2017-08-08T09:28:56  <bitcoin-git> [bitcoin] laanwj closed pull request #10999: Fix amounts formatting in `decoderawtransaction` (master...2017_08_decoderawtx_amount) https://github.com/bitcoin/bitcoin/pull/10999
125 2017-08-08T09:42:08  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/627c3c0e495b...4268426b4500
126 2017-08-08T09:42:09  <bitcoin-git> bitcoin/master 055d95f John Newbery: [wallet] return correct error code from resendwallettransaction
127 2017-08-08T09:42:09  <bitcoin-git> bitcoin/master 4268426 Wladimir J. van der Laan: Merge #11002: [wallet] return correct error code from resendwallettransaction...
128 2017-08-08T09:42:38  <bitcoin-git> [bitcoin] laanwj closed pull request #11002: [wallet] return correct error code from resendwallettransaction (master...resendwallettransaction_error_code) https://github.com/bitcoin/bitcoin/pull/11002
129 2017-08-08T09:58:54  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/4268426b4500...2507fd55568b
130 2017-08-08T09:58:54  <bitcoin-git> bitcoin/master 861f9a2 Matt Corallo: Skip remainder of init if upgrade is cancelled
131 2017-08-08T09:58:55  <bitcoin-git> bitcoin/master 2507fd5 Wladimir J. van der Laan: Merge #10998: Fix upgrade cancel warnings...
132 2017-08-08T09:59:44  <bitcoin-git> [bitcoin] laanwj closed pull request #10998: Fix upgrade cancel warnings (master...2017-08-fix-upgrade-cancel-warnings) https://github.com/bitcoin/bitcoin/pull/10998
133 2017-08-08T10:00:47  *** JackH has joined #bitcoin-core-dev
134 2017-08-08T10:10:29  *** johnpark_pj has quit IRC
135 2017-08-08T10:11:40  *** johnpark_pj has joined #bitcoin-core-dev
136 2017-08-08T10:17:17  *** jrayhawk has quit IRC
137 2017-08-08T10:17:43  *** jrayhawk has joined #bitcoin-core-dev
138 2017-08-08T10:18:11  *** comboy_ has quit IRC
139 2017-08-08T10:19:06  *** Guyver2 has joined #bitcoin-core-dev
140 2017-08-08T10:19:49  *** comboy has joined #bitcoin-core-dev
141 2017-08-08T10:29:57  <wumpus> doing last checks on 10882, after which I'll merge it. After that I'm planning to tag 0.15.0rc1. Any objections?
142 2017-08-08T10:30:55  <wumpus> (ah yes #10483 and #10607 too)
143 2017-08-08T10:30:57  <gribble> https://github.com/bitcoin/bitcoin/issues/10483 | scripted-diff: Use the C++11 keyword nullptr to denote the pointer literal instead of the macro NULL by practicalswift · Pull Request #10483 · bitcoin/bitcoin · GitHub
144 2017-08-08T10:30:58  <gribble> https://github.com/bitcoin/bitcoin/issues/10607 | scripted-diff: stop using the gArgs wrappers by benma · Pull Request #10607 · bitcoin/bitcoin · GitHub
145 2017-08-08T10:31:46  <wumpus> then branching off 0.15, bumping version numbers, then tagging the release
146 2017-08-08T10:32:51  <gmaxwell> I think 10882 is still in a state where if you run getnewaddress to the pont where it exhausts your keypool then recieve a transaction your node shuts off.
147 2017-08-08T10:33:45  <wumpus> can you coment jnewbery?
148 2017-08-08T10:33:51  <wumpus> if so, i'm going to bump it to 0.15.1 instead
149 2017-08-08T10:34:21  <jnewbery> Yes, that's the case - I reverted to the commit that had ACKs
150 2017-08-08T10:35:08  <wumpus> is it enough of an improvement to merge it like this?
151 2017-08-08T10:35:43  <jnewbery> I think it's an improvement
152 2017-08-08T10:35:54  <wumpus> (or does it make things worse? I don't know anymore)
153 2017-08-08T10:36:16  <gmaxwell> It's a critical improvement, but I think the sequence of events I described should block a release, just like absense of this PR.  Keep in mind that most existing already encrypted wallets have less than 500 keys in their keypool already.
154 2017-08-08T10:36:31  <wumpus> you want to block the release on this?
155 2017-08-08T10:37:11  <gmaxwell> I think we need to turn HD off without this. We currently have non-trivial funds loss exposure without this.
156 2017-08-08T10:37:21  <wumpus> is it a regression since 0.14?
157 2017-08-08T10:37:46  <gmaxwell> It was introduced in 0.14 with hdwallet, though we only realized over time all the ways it could fail.
158 2017-08-08T10:38:06  <wumpus> I'd really dislike blocking 0.15
159 2017-08-08T10:38:21  <wumpus> anyone else think we should delay 0.15?
160 2017-08-08T10:38:44  <wumpus> as I understand, the plan is to do a 0.15.1 fairly quick after it anyway
161 2017-08-08T10:39:01  <gmaxwell> Agreed! but I don't think it's acceptable to keep shipping stuff we know will basically make people lose funds, and yet I don't think we can ship something that will just fall over on most encrypted wallets in use right now.
162 2017-08-08T10:39:03  <wumpus> so every day we delay here delays that, too
163 2017-08-08T10:39:09  <gmaxwell> I know.
164 2017-08-08T10:39:27  <wumpus> and if it was already broken in 0.14, and still broken in 0.15, releaseing at least won't make things worse
165 2017-08-08T10:39:50  <wumpus> delaying just means people will keep using 0.14.x in which it is broken too
166 2017-08-08T10:40:23  <gmaxwell> whatever we do next might be fast but it's not going to be days.
167 2017-08-08T10:40:27  <wumpus> anyhow, going back to other things then, we can postpone for another day at least
168 2017-08-08T10:40:50  <gmaxwell> jnewbery: can you make a second PR with the remaining parts
169 2017-08-08T10:41:01  <gmaxwell> I don't understand why that fix was removed.. it's really pretty broken without it.
170 2017-08-08T10:41:05  <jnewbery> I plan to
171 2017-08-08T10:41:09  <gmaxwell> OK.
172 2017-08-08T10:41:39  <jnewbery> because it and the other changes broke a test, and it was suggested I revert to a commit that already had lots of ACKs
173 2017-08-08T10:41:52  <wumpus> jnewbery: agree that doing a new PR is better
174 2017-08-08T10:42:19  <wumpus> let's just merge 10882 before the review there turns even more into a mess, if some state of a PR is sufficiently accepted and reviewed ideally no new things are added
175 2017-08-08T10:42:27  <gmaxwell> (It's not that I don't think we can merge whats there, but I think we can't release with it because it will fail very quickly for basically anyone with a pre-existing encrypted wallet)
176 2017-08-08T10:42:45  <wumpus> it will fail for anyone?
177 2017-08-08T10:43:03  <wumpus> ok, then I'm not going to merge it, I already had half a hart attack with a 'corrupted' wallet yesterday :p
178 2017-08-08T10:43:12  <gmaxwell> wumpus: right now with that PR if you get a transaction while your keypool has under 500 addresses in it, you'll shut down.
179 2017-08-08T10:43:13  <jnewbery> yes, I realise now that the combination of 10882 and increasing the keypool defaults probably breaks this for pre-existing wallets
180 2017-08-08T10:43:41  <gmaxwell> jnewbery: well this was going to need a look ahead greater than 50 regardless.
181 2017-08-08T10:44:33  <wumpus> then why does everyone ack it if it creates a broken situation
182 2017-08-08T10:44:42  <gmaxwell> jnewbery: sorry for my earlier comments on it not making it adequately clear that this applies to almost everyone, I just thought that pointing out that it would go down after a getnewaddress loop was enough. :)
183 2017-08-08T10:46:24  <luke-jr> wumpus: my only possibly-not-ignored objection I think, is to breaking backward compatibility for RPC :/  (#10989)
184 2017-08-08T10:46:30  <gribble> https://github.com/bitcoin/bitcoin/issues/10989 | RPC: Restore backward compatibility, in multiwallet mode by luke-jr · Pull Request #10989 · bitcoin/bitcoin · GitHub
185 2017-08-08T10:47:16  <gmaxwell> luke-jr: I think that objection is acknoweldged and set aside. Making MW require the wallet was an intentional decision at least for now that basically everyone supported.
186 2017-08-08T10:47:28  <jnewbery> wumpus - I think because it's been open for a long time and gone through many iterations, during which time the keypool defaults were increased. I've only just realised the implications from what gmaxwell said just now
187 2017-08-08T10:48:47  <jnewbery> I can hopefully get a minimal PR on top of #10882 in the next hour that includes juse the getnewaddress fix. gmaxwell, will you be able to review today?
188 2017-08-08T10:48:50  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
189 2017-08-08T10:49:19  <luke-jr> so people will just be forced to switch to Knots to actually use MW I guess, whatever
190 2017-08-08T10:50:01  <luke-jr> I suppose that's the same as with previous versions
191 2017-08-08T10:50:55  <gmaxwell> luke-jr: come on dude, having to specify which wallet you want is not "having to use".  It's perhaps arguably less useful, but the feature is expiremental in this version.
192 2017-08-08T10:51:05  <wumpus> multiwallet is experimental
193 2017-08-08T10:51:18  <wumpus> even if it's completely useless in rc1, I'm not going to block the release on that
194 2017-08-08T10:51:40  <wumpus> (and I know for sure it's not compltely useless)
195 2017-08-08T10:51:58  <luke-jr> gmaxwell: no existing software uses the new API, and as you point out, the new API is experimental.. so there's basically no way to get the primary "just keep wallets in sync" benefit without using a new experimental API
196 2017-08-08T10:52:31  <gmaxwell> luke-jr: bitcoin cli works, and many other things are easy to get working...
197 2017-08-08T10:52:37  <luke-jr> anyway, I acknowledge that this isn't going to be fixed; I'm just not happy about it.
198 2017-08-08T10:52:45  <gmaxwell> I agree it dimishes the wallet warming usecase.
199 2017-08-08T10:54:13  <gmaxwell> luke-jr: the concern people have about accidentally interacting with the wrong wallet is a good one though. It's an obvious footgun and it's more conservative to avoid it for now.
200 2017-08-08T10:54:53  <gmaxwell> luke-jr: also, perhaps that walletwarming usecase would better be met with a seperate state e.g. a scanwallet=file  where there is no risk of accidentally using it beyond getting some status about it.
201 2017-08-08T10:55:09  <luke-jr> gmaxwell: yes, I agree that's a reasonable concern. I don't know a way to address it.
202 2017-08-08T10:55:25  <luke-jr> hopefully MW will be more than warming by 0.16 anyway
203 2017-08-08T10:55:54  <luke-jr> (I actually think it might have been better to release without *any* RPC MW support, than with what we have now)
204 2017-08-08T10:56:24  <luke-jr> (that'd solve the footgun issue, while not breaking the warming use case)
205 2017-08-08T10:56:28  <gmaxwell> luke-jr: fwiw, I think it would be more constructive on concerns like this if instead of just saying it's wrong, if you walked us through the specific case that you care about that it screws up.  It's a lot easier to sympatize with your concern if someone understands it, and they can't unless you explain it.  When you just state matter of factly that it's currently wrong or broken, it's random c
206 2017-08-08T10:56:34  <gmaxwell> hance if any given person will understand why you think that.
207 2017-08-08T10:56:59  <gmaxwell> luke-jr: For many people, updating to make the calls (e.g. cli users) is trivial.
208 2017-08-08T10:57:05  <luke-jr> gmaxwell: my understanding was that it's basically the only real use case 0.15 will support MW for
209 2017-08-08T10:57:08  <gmaxwell> and being able to actually use MW will be nice.
210 2017-08-08T10:57:45  <gmaxwell> luke-jr: that wasn't my understanding. Actually, I didn't even really know people were thinking that much about "warming but not using" until I read the release note text re: GUI.  Which I assume you wrote.
211 2017-08-08T10:58:25  <gmaxwell> Though I agree it's a useful thing to do, esp in a world with pruning.. (also rescan times are a seriously negative part of my life these days)
212 2017-08-08T10:59:01  <luke-jr> (FWIW, Newbery wrote it)
213 2017-08-08T10:59:04  <gmaxwell> I manage cold wallets for myself, for blockstream, for varrious projects... and it's not uncommon that I lose hours of waiting due to having to rescan something.
214 2017-08-08T10:59:08  <gmaxwell> ah okay!
215 2017-08-08T10:59:49  <jnewbery> yeah - I think for qt-only users warming is the only use in v0.15
216 2017-08-08T10:59:59  <luke-jr> I suppose a reasonable compromise is to have a command-line option to set a default wallet, and only support it in that case. But probably too late to get it into 0.15
217 2017-08-08T11:00:05  <jnewbery> but for RPC users, multiwallet is fully supported
218 2017-08-08T11:00:22  <luke-jr> hmm. can a Qt user even access their wallet via debug console now?
219 2017-08-08T11:01:45  <jnewbery> yes, just wallet 0
220 2017-08-08T11:01:55  <jnewbery> #10870
221 2017-08-08T11:01:56  <gribble> https://github.com/bitcoin/bitcoin/issues/10870 | [Qt] Use wallet 0 in rpc console if running with multiple wallets by jonasschnelli · Pull Request #10870 · bitcoin/bitcoin · GitHub
222 2017-08-08T11:03:42  <luke-jr> ah
223 2017-08-08T11:05:05  <jnewbery> gmaxwell - I don't think the getnewaddress changes you're asking for in 10882 address the encrypted-wallet-with-fewer-than-500-keys issue that you just raised
224 2017-08-08T11:05:29  <jnewbery> basically every HD wallet now has fewer than 500 keys
225 2017-08-08T11:06:20  <jnewbery> so on upgrade, everyone with an encrypted HD wallet will have their node shut down and be presented with the 'restart with bypasskeypoolcritical blah blah blah' error
226 2017-08-08T11:07:12  <jnewbery> it's not a loss of funds risk, but it's a terrible first user impression of v0.15
227 2017-08-08T11:08:52  *** Aaronvan_ has joined #bitcoin-core-dev
228 2017-08-08T11:08:52  <gmaxwell> the whole thing is a bit weird. When the wallet is current there is no need for the keypool to have lookahead.
229 2017-08-08T11:10:20  <gmaxwell> But I didn't want to get into dictating major changes to it.
230 2017-08-08T11:11:40  *** AaronvanW has quit IRC
231 2017-08-08T11:13:08  <wumpus> maybe it should only enlarge the keypool the first time the user unlocks the wallet
232 2017-08-08T11:13:09  <wumpus> after upgrading
233 2017-08-08T11:13:17  <Lauda> Does anyone know what the limiting factor for rescan speed is? I have been running a few imports on a fairly good machine, and I don't see everything being utilized whilst this is taking up a lot of time (single key 2+ hours). Disk I/O maybe?
234 2017-08-08T11:13:56  <Lauda> CPU <20% on average, 2 GB of RAM out of 32 (with dbcache 12GB).
235 2017-08-08T11:14:05  *** rjak has joined #bitcoin-core-dev
236 2017-08-08T11:14:37  <wumpus> rescan could be a lot faster if you'd not care about transaction history, then it'd only have to scan over the utxo set instead of every single block
237 2017-08-08T11:15:15  <Lauda> I don't, I just care about the final balance (in this case). Is there an option to ignore the TX history?
238 2017-08-08T11:15:48  <Lauda> Seems like rescan speed will keep getting much worse with time, no?
239 2017-08-08T11:16:35  <jnewbery> wumpus, the problem is that keypool_critical was changed to 500 during the iterations of this PR (when keypool was changed to 1000). We'd need a way for keypool_critical to be lowered to 50 on the first run after upgrading, or similar. I can't think of a good way to do that
240 2017-08-08T11:20:17  <Lauda> I should also note that it is taking over 2 hours on a VPS with an SSD. I wonder how much of a difference it would make with an HDD (% duration increase).
241 2017-08-08T11:25:25  *** Aaronvan_ has quit IRC
242 2017-08-08T11:31:54  *** AaronvanW has joined #bitcoin-core-dev
243 2017-08-08T11:33:22  *** Aaronvan_ has joined #bitcoin-core-dev
244 2017-08-08T11:34:42  <gmaxwell> the rescan is slow presumably because it deseralizes each block and each transaction and hashes each trasnaction and does dozens of memory allocations for each transactions.
245 2017-08-08T11:35:27  <gmaxwell> It could probably take your scriptpubkeys and txns as bytes and scan the raw block data without deseralizing things and be 500 times faster... but it would be a fair amount of work to implement that.
246 2017-08-08T11:36:08  *** AaronvanW has quit IRC
247 2017-08-08T11:36:23  <gmaxwell> jnewbery: keypool_criticial being 50 is not really reasonable in any case. it would miss transactions in many of my existing wallets, for example, and I doubt my usage is that atypical.
248 2017-08-08T11:36:46  <gmaxwell> jnewbery: I think the bigger point though is that when the wallet is current there is no reason to shut down.
249 2017-08-08T11:37:10  <gmaxwell> It doesn't matter if there is look ahead if the wallet hasn't fallen behind.
250 2017-08-08T11:37:12  <wumpus> yes, it will get slower with time, assuming you're donig a full rescan
251 2017-08-08T11:37:30  <wumpus> if you rescan a more or less fixed number of blocks it should stay at the same speed
252 2017-08-08T11:37:41  <wumpus> or at least not become much slower...
253 2017-08-08T11:38:10  *** promag has joined #bitcoin-core-dev
254 2017-08-08T11:38:44  <wumpus> and yes, rescan as it is could certainly be optimized, though I doubt anyone sees it as that much of a priority
255 2017-08-08T11:38:58  <jnewbery> "when the wallet is current there is no reason to shut down." - the getnewaddress change doesn't fix that
256 2017-08-08T11:39:04  *** berndj has quit IRC
257 2017-08-08T11:39:32  <Lauda> gmaxwell that sounds like a good optimization for the future. Thanks for the responses.
258 2017-08-08T11:39:53  <wumpus> you're welcome to give it a try of course
259 2017-08-08T11:40:56  <gmaxwell> jnewbery: I realize this. When you suggested that I thought it would be simpler than another solution.
260 2017-08-08T11:41:13  *** berndj has joined #bitcoin-core-dev
261 2017-08-08T11:44:08  <jnewbery> I must have misunderstood you in the issue. I suggested the getnewaddress change because I thought you were talking about running getnewaddress 500 times. I didn't realise there was an upgrade issue until you mentioned it today
262 2017-08-08T11:46:26  <gmaxwell> I was just trying to give an example where the keypool would be below the threshold. I don't really see them as different... end result is the same regardless of how you get there. Sorry.
263 2017-08-08T11:47:07  <gmaxwell> e.g. imagine that the crit threshold were still 50, you'd still have existing wallets shut down immediately, or after the user executes a dozen getnewaddresses. :)
264 2017-08-08T11:47:33  <gmaxwell> Just fewer of them.
265 2017-08-08T11:47:55  <gmaxwell> but they'd mostly all die eventually except for users that unlock frequently before running out.
266 2017-08-08T11:50:41  *** promag has quit IRC
267 2017-08-08T11:54:59  <jnewbery> ok, so how do we fix this?
268 2017-08-08T11:55:12  <jnewbery> Offline keypool topup would be *really* useful
269 2017-08-08T11:58:12  <gmaxwell> jnewbery: Can we distinguish the case where we are current vs rescanning? If so, then we just shouldn't perform the shutdown when we're current.
270 2017-08-08T12:04:49  <gmaxwell> I think it would still be good to stop newaddresses short of the limit, so that we don't immediately go into shutdown on any attempt to rescan when the pool is empty.. but that is a less criticial improvement.
271 2017-08-08T12:05:16  <jnewbery> We rescan from the wallet's best block on startup. So even with that change, we're going to barf on upgrade
272 2017-08-08T12:06:36  <jnewbery> (unless you shutdown v0.14, upgrade, start v0.15 before a block has been produced)
273 2017-08-08T12:07:00  *** dermoth has quit IRC
274 2017-08-08T12:07:01  <gmaxwell> hm. but we don't have those blocks at startup.
275 2017-08-08T12:07:13  *** arubi has quit IRC
276 2017-08-08T12:07:29  <gmaxwell> when we start, the best block should be the same... though yea, IIRC we do a rescan in any case. I think just as a belt and suspenders check.
277 2017-08-08T12:08:00  <jnewbery> oh, ok. Makes sense
278 2017-08-08T12:08:05  *** dermoth has joined #bitcoin-core-dev
279 2017-08-08T12:09:55  *** arubi has joined #bitcoin-core-dev
280 2017-08-08T12:11:42  <morcos> jnewbery: gmaxwell: i dont have all the code handy, but just brainstorming a quick fix for 0.15
281 2017-08-08T12:11:53  <morcos> wouldn't it be possible to take advantage of the two different limits
282 2017-08-08T12:12:18  <gmaxwell> I think that initial rescan can be disabled, but it's been so long since I've thought about it, there may be reasons that I am not remembering.
283 2017-08-08T12:12:24  <morcos> if we set keypoolmin to 500 and keypoolcritical to 25, then don't we not miss anything since our best block has stopped advancing?
284 2017-08-08T12:13:00  <gmaxwell> oh. that is an interesting point, but what about wallets with empty keypools
285 2017-08-08T12:13:09  <gmaxwell> (below 25)
286 2017-08-08T12:13:19  <morcos> then with the release notes we can recommend running walletpasphrase as soon as you startup  so your keypool will top up to the new bigger amounts
287 2017-08-08T12:13:42  <morcos> oh wait, that doesnt work exactly...
288 2017-08-08T12:13:44  <morcos> shoot
289 2017-08-08T12:14:05  <morcos> also it would be potentially a problem for pruned nodes
290 2017-08-08T12:14:17  <gmaxwell> need a third number, but then I think it does except for but what if there isn't even 25.. and pruned.
291 2017-08-08T12:15:09  <morcos> i need to look at the code, but right now i actually think it is a broken state to set keypoolmin different from keypoolcritical
292 2017-08-08T12:15:23  <gmaxwell> morcos: If instead we just make it so that the only time this applies is when the wallet has fallen behind the node tip (or is otherwise rescanned)-- then I think that would be fine.
293 2017-08-08T12:15:55  <gmaxwell> You'd still get forced into topping up anytime you imported keys, moved wallets between nodes, etc.. but normal operation wouldn't run into it.
294 2017-08-08T12:17:37  *** promag has joined #bitcoin-core-dev
295 2017-08-08T12:17:41  <gmaxwell> I believe the reason we do rescanning at tip even when we're in sync (assuming we still do it) was a concern about reorgs and or the wallet not getting flushed consistently with blocks in an unclean shutdown or something like that.
296 2017-08-08T12:17:58  <morcos> The scary thing to me is the way we turn off setting the best block, but then potentially turn it back on if we haven't rescanned (thereby missing scanning a bunch of blocks)
297 2017-08-08T12:18:20  <morcos> i haven't looked at the most recent code, and maybe ryanofsky's suggestion fixed that i don't remember
298 2017-08-08T12:18:59  <morcos> i'm going to stop giving out ideas until i look more closely at the code again
299 2017-08-08T12:19:23  <morcos> but for the record wumpus, my view is it would be better to delay 0.15 (even by a week or two) to get this right
300 2017-08-08T12:20:13  <morcos> i think we're getting pretty well into understanding all the issues now, so would be good to fix it while it's fresh, and it also seems like a big enough problem, that would be nice to fix before release
301 2017-08-08T12:24:34  <sdaftuar> i think #10982 should be locked
302 2017-08-08T12:24:36  <gribble> https://github.com/bitcoin/bitcoin/issues/10982 | Disconnect network service bits 6 and 8 until Aug 1, 2018 by TheBlueMatt · Pull Request #10982 · bitcoin/bitcoin · GitHub
303 2017-08-08T12:27:24  <gmaxwell> :(
304 2017-08-08T12:30:41  *** Aaronvan_ is now known as AaronvanW
305 2017-08-08T12:32:11  <jnewbery> morcos - I think ryanofsky's change is helpful. I tried including it but it broke tests, so I rolled back to a previous commit
306 2017-08-08T12:34:38  <jnewbery> gmaxwell - ShutdownIfKeypoolCritical() is only called in CreateWalletFromFile() and AddToWalletIfInvolvingMe(). I think your change would be to remove it from CreateWalletFromFile(), and only call it from AddToWalletIfInvolvingMe() if that function was called by SyncTransaction() (and not by ScanForWalletTransactions())
307 2017-08-08T12:34:46  *** Aaronvan_ has joined #bitcoin-core-dev
308 2017-08-08T12:35:50  <jnewbery> I think I know what I need to do - re-add the getnewaddress changes, re-add ryanofksy's changes about setting best block, and make the change above
309 2017-08-08T12:36:36  *** Aaronvan_ is now known as AaronvanW_
310 2017-08-08T12:36:38  *** AaronvanW has quit IRC
311 2017-08-08T12:40:26  *** AaronvanW_ is now known as AaronvanW
312 2017-08-08T12:41:41  <sdaftuar> MarcoFalke: thank you
313 2017-08-08T12:41:50  <wumpus> MarcoFalke: you just beat me to it :)
314 2017-08-08T12:42:06  <MarcoFalke> Too much mail spam :(
315 2017-08-08T12:43:31  <gmaxwell> :-( sorry that i even commented on it at all, it just made it worse
316 2017-08-08T12:47:03  *** BashCo_ has joined #bitcoin-core-dev
317 2017-08-08T12:49:42  *** BashCo has quit IRC
318 2017-08-08T12:55:47  *** promag has quit IRC
319 2017-08-08T13:03:37  <jnewbery> ~.
320 2017-08-08T13:03:49  <sipa> agree.
321 2017-08-08T13:04:50  * sdaftuar wonders what sipa's reasoning is
322 2017-08-08T13:07:17  *** Chris_Stewart_5 has joined #bitcoin-core-dev
323 2017-08-08T13:09:51  *** promag has joined #bitcoin-core-dev
324 2017-08-08T13:33:21  *** justanotheruser has quit IRC
325 2017-08-08T13:39:23  *** Austindoggie has joined #bitcoin-core-dev
326 2017-08-08T13:45:51  *** justanotheruser has joined #bitcoin-core-dev
327 2017-08-08T13:45:56  *** justanotheruser has quit IRC
328 2017-08-08T13:46:18  *** justanotheruser has joined #bitcoin-core-dev
329 2017-08-08T13:52:50  *** laurentmt has joined #bitcoin-core-dev
330 2017-08-08T13:56:48  *** laurentmt has quit IRC
331 2017-08-08T14:00:39  *** SopaXorzTaker has quit IRC
332 2017-08-08T14:01:35  *** timothy has quit IRC
333 2017-08-08T14:01:35  *** drizztbsd has joined #bitcoin-core-dev
334 2017-08-08T14:19:26  *** promag has quit IRC
335 2017-08-08T14:23:29  *** btcdrak has quit IRC
336 2017-08-08T14:25:45  *** Guyver2_ has joined #bitcoin-core-dev
337 2017-08-08T14:26:15  *** Murch has joined #bitcoin-core-dev
338 2017-08-08T14:26:50  *** promag has joined #bitcoin-core-dev
339 2017-08-08T14:28:17  *** Guyver2 has quit IRC
340 2017-08-08T14:28:18  *** Guyver2_ is now known as Guyver2
341 2017-08-08T14:32:00  *** miknotauro has joined #bitcoin-core-dev
342 2017-08-08T14:34:49  *** SopaXorzTaker has joined #bitcoin-core-dev
343 2017-08-08T14:40:07  *** promag has quit IRC
344 2017-08-08T14:46:41  *** miknotauro has quit IRC
345 2017-08-08T14:48:30  <wumpus> jnewbery: maybe reverting the default mempool size increase for 0.15.0 will make this easier?
346 2017-08-08T14:50:10  *** miknotauro has joined #bitcoin-core-dev
347 2017-08-08T14:50:34  *** promag has joined #bitcoin-core-dev
348 2017-08-08T15:02:03  *** timothy has joined #bitcoin-core-dev
349 2017-08-08T15:03:01  *** drizztbsd has quit IRC
350 2017-08-08T15:03:10  <Chris_Stewart_5> How long do we support gcc versions? Curious for #8469
351 2017-08-08T15:03:12  <gribble> https://github.com/bitcoin/bitcoin/issues/8469 | [POC] Introducing property based testing to Core by Christewart · Pull Request #8469 · bitcoin/bitcoin · GitHub
352 2017-08-08T15:04:17  <wumpus> Chris_Stewart_5: there's no fixed policy on that - the preference is to support all compilers unless there's a good reason not to, such as lack of c++11 support
353 2017-08-08T15:15:32  *** dcousens has quit IRC
354 2017-08-08T15:19:02  <jnewbery> wumpus: yes, that would resolve the upgrade issue, although I'm sure gmaxwell will argue against that
355 2017-08-08T15:33:22  <wumpus> sure, I just mean maybe we can't have everything in a timeframe soon enough for 0.15.0 and we need to make choices
356 2017-08-08T15:35:11  <luke-jr> Chris_Stewart_5: it would be pretty bad to not build on popular stable OSs
357 2017-08-08T15:36:07  *** Aaronvan_ has joined #bitcoin-core-dev
358 2017-08-08T15:36:12  <luke-jr> so I'd check Fedora, Debian, RedHat/CentOS, Gentoo, Ubuntu at least
359 2017-08-08T15:36:28  <luke-jr> IIRC typically the hold-back is CentOS
360 2017-08-08T15:37:32  <wumpus> luke-jr: it doesn't even work on travis
361 2017-08-08T15:37:54  <luke-jr> heh
362 2017-08-08T15:38:16  <wumpus> gcc 4.8 isn't *that* old and crappy yet, I doubt we should drop support just to be able to do tests in a certain way
363 2017-08-08T15:38:24  *** AaronvanW has quit IRC
364 2017-08-08T15:38:52  <wumpus> and the tests working on travis is kind of important, so disabling them on gcc 4.8 isn't going to fly either
365 2017-08-08T15:39:02  <luke-jr> right
366 2017-08-08T15:39:30  <bitcoin-git> [bitcoin] practicalswift opened pull request #11007: [qt] Fix memory leak when loading a corrupted wallet file (master...wallet-corrupted-leak) https://github.com/bitcoin/bitcoin/pull/11007
367 2017-08-08T15:39:54  <Chris_Stewart_5> Yes, 4.8.4 was released in Dec of 2014
368 2017-08-08T15:40:22  <Chris_Stewart_5> ancient in terms of bitcoin! :P
369 2017-08-08T15:40:22  <sam_c> gentoo is about to mask 4.8
370 2017-08-08T15:43:46  <wumpus> besides that, we already got lots of complaints that we're requiring a c++11 compiler, I"d prefer to keep the compiler requirement the same for the forseeable future
371 2017-08-08T15:46:01  *** BashCo_ has quit IRC
372 2017-08-08T15:46:37  *** BashCo has joined #bitcoin-core-dev
373 2017-08-08T15:46:44  <Chris_Stewart_5> wumpus: Do we have a 'rough' estimate on when 0.15.0 will be done?
374 2017-08-08T15:46:56  *** Austindoggie_ has joined #bitcoin-core-dev
375 2017-08-08T15:50:16  *** Austindoggie_ has quit IRC
376 2017-08-08T15:50:52  *** Austindoggie has quit IRC
377 2017-08-08T15:51:00  *** BashCo has quit IRC
378 2017-08-08T15:52:45  <wumpus> Chris_Stewart_5: yes, rc1 was planned for the 6th, we've slipped two days now
379 2017-08-08T15:54:41  <wumpus> (which is perfectly normal)
380 2017-08-08T15:54:59  *** ekerstein has joined #bitcoin-core-dev
381 2017-08-08T15:56:28  *** jamesob has joined #bitcoin-core-dev
382 2017-08-08T16:04:56  <luke-jr> sam_c: is it? why?
383 2017-08-08T16:05:30  <luke-jr> actually, it looks like it's been masked since May, just because it's old O.o
384 2017-08-08T16:08:02  <Chris_Stewart_5> cool, was just curious
385 2017-08-08T16:09:32  *** BashCo has joined #bitcoin-core-dev
386 2017-08-08T16:30:01  *** promag has quit IRC
387 2017-08-08T16:31:07  *** Aaronvan_ is now known as AaronvanW
388 2017-08-08T16:32:04  *** miknotauro has quit IRC
389 2017-08-08T16:43:36  *** JackH has quit IRC
390 2017-08-08T16:49:23  *** promag has joined #bitcoin-core-dev
391 2017-08-08T16:55:52  *** jimpo has joined #bitcoin-core-dev
392 2017-08-08T16:56:32  *** timothy has quit IRC
393 2017-08-08T16:57:14  *** d_t has joined #bitcoin-core-dev
394 2017-08-08T16:58:10  *** promag has quit IRC
395 2017-08-08T17:03:30  *** JackH has joined #bitcoin-core-dev
396 2017-08-08T17:16:03  *** corinrose_ has joined #bitcoin-core-dev
397 2017-08-08T17:32:49  *** chjj has quit IRC
398 2017-08-08T17:38:54  *** Aaronvan_ has joined #bitcoin-core-dev
399 2017-08-08T17:39:13  *** AaronvanW has quit IRC
400 2017-08-08T17:41:16  *** Chris_Stewart_5 has quit IRC
401 2017-08-08T17:54:17  *** Chris_Stewart_5 has joined #bitcoin-core-dev
402 2017-08-08T18:12:52  *** Chris_Stewart_5 has quit IRC
403 2017-08-08T18:18:15  *** chjj has joined #bitcoin-core-dev
404 2017-08-08T18:21:28  *** clarkmoody has joined #bitcoin-core-dev
405 2017-08-08T18:23:32  *** wvr has joined #bitcoin-core-dev
406 2017-08-08T18:25:55  *** Chris_Stewart_5 has joined #bitcoin-core-dev
407 2017-08-08T18:26:37  *** jannes has quit IRC
408 2017-08-08T18:27:06  *** Aaronvan_ is now known as AaronvanW
409 2017-08-08T18:32:59  *** kexkey has joined #bitcoin-core-dev
410 2017-08-08T18:33:07  *** jannes has joined #bitcoin-core-dev
411 2017-08-08T18:33:23  *** clarkmoody_ has joined #bitcoin-core-dev
412 2017-08-08T18:37:07  *** clarkmoody has quit IRC
413 2017-08-08T18:37:41  *** kexkey has quit IRC
414 2017-08-08T18:38:36  *** chjj has quit IRC
415 2017-08-08T18:39:24  *** kexkey has joined #bitcoin-core-dev
416 2017-08-08T18:43:40  *** abpa has joined #bitcoin-core-dev
417 2017-08-08T18:49:07  *** kexkey has quit IRC
418 2017-08-08T18:51:57  *** chjj has joined #bitcoin-core-dev
419 2017-08-08T18:53:34  *** Dizzle has joined #bitcoin-core-dev
420 2017-08-08T18:56:21  *** echonaut has quit IRC
421 2017-08-08T18:56:36  *** echonaut has joined #bitcoin-core-dev
422 2017-08-08T18:58:16  <jcorgan> congrats
423 2017-08-08T18:59:10  *** ekerstein has quit IRC
424 2017-08-08T18:59:43  <AaronvanW> +1
425 2017-08-08T19:00:40  <arubi> when's "status" going to change to "locked-in" ?
426 2017-08-08T19:00:40  <arubi> I mean, congrats!  :)  but also that
427 2017-08-08T19:00:46  <gmaxwell> oh I guess, barring no reorgs segwit lockin is now guarenteed.
428 2017-08-08T19:01:21  <gmaxwell> arubi: the thing happening now is that there aren't enough blocks left to prevent a lock in, but it won't lock in for another day or so IIRC.
429 2017-08-08T19:01:40  <arubi> so I was just informed it happens at the end of the period
430 2017-08-08T19:01:48  *** timothy has joined #bitcoin-core-dev
431 2017-08-08T19:01:55  <gmaxwell> so even if all blocks for the next of the window don't signal segwit it will still lock in.
432 2017-08-08T19:02:31  <jcorgan> something like 16-17 hours from now
433 2017-08-08T19:02:41  <arubi> right, my confusion was re. the 'getblockchaininfo' "status" filed to changing after 95% rather than at the end of the period
434 2017-08-08T19:02:43  <jcorgan> for end of period
435 2017-08-08T19:03:04  <Chris_Stewart_5> Mmmm ok
436 2017-08-08T19:04:02  *** chjj has quit IRC
437 2017-08-08T19:16:18  *** chjj has joined #bitcoin-core-dev
438 2017-08-08T19:19:58  <gmaxwell> 12:19:19 <@betawaffle> gmaxwell: can you congratulate everyone involved in segwit for us?
439 2017-08-08T19:20:10  <sdaftuar> i guess we can cross "activate segwit" off the list of action items
440 2017-08-08T19:21:47  <gmaxwell> \O/
441 2017-08-08T19:22:16  <jnewbery> gmaxwell morcos wumpus (and anyone else who's interested in wallet topup). I've pushed a branch which I think covers gmaxwell's requested logic:
442 2017-08-08T19:22:19  <jnewbery> - don't shutdown the node on startup if keypool is < keypool_critical
443 2017-08-08T19:22:21  <jnewbery>     - don't shutdown the node if wallet is current
444 2017-08-08T19:22:24  <jnewbery>     - DO shutdown the node if rescanning and keypool drops below
445 2017-08-08T19:22:26  <jnewbery>     keypool_critical
446 2017-08-08T19:22:27  <jnewbery> #10882
447 2017-08-08T19:22:30  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
448 2017-08-08T19:22:32  <gmaxwell> When does the period end? exactly, it would kinda be nice if rc1 lands just after that, so that miners running BIP-91 code can safely try rc1.
449 2017-08-08T19:23:00  <jnewbery> getnewaddress should also be safe
450 2017-08-08T19:23:28  <gmaxwell> sdaftuar: now we just need to get people to modulate their hashrate so that the activation coincides with the solar eclipse. It's pretty close.
451 2017-08-08T19:23:29  <jnewbery> also includes ryanofsky's m_update_best_block
452 2017-08-08T19:23:39  <gmaxwell> (quick, someone make more spinoffs)
453 2017-08-08T19:23:54  <luke-jr> lol
454 2017-08-08T19:24:57  <gmaxwell> jnewbery: where is the branch?
455 2017-08-08T19:25:36  <jnewbery> #10882
456 2017-08-08T19:25:39  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
457 2017-08-08T19:26:07  <gmaxwell> jnewbery: oh great
458 2017-08-08T19:27:08  <jnewbery> This should fix the upgrade issue. bitcoind v0.15 won't shutdown first time you run it with an old encrypted HD wallet
459 2017-08-08T19:31:17  <jnewbery> but if you don't unlock and topup your keypool before you shutdown the node, then I expect it'll refuse to start the next time (since your node tip will advance but your wallet best block won't)
460 2017-08-08T19:31:42  <gmaxwell> Right. We may need to do a bit of warning related to this pattern still:
461 2017-08-08T19:31:55  <gmaxwell> wallet is below 500 because it's old and encrypted.
462 2017-08-08T19:32:09  <gmaxwell> you start, life is good, but you are not updating the best block.
463 2017-08-08T19:32:29  <gmaxwell> then you restart, and now the wallet is behind.
464 2017-08-08T19:32:39  <gmaxwell> it rescans, and life becomes sad.
465 2017-08-08T19:33:59  <jnewbery> yes, sad, but better than any alternative as far as I can see
466 2017-08-08T19:34:35  <jnewbery> there is a warning that your keypool is below keypool_min and your best block isn't advancing
467 2017-08-08T19:35:30  <jnewbery> 2017-08-08 19:19:20 Keypool has fallen below keypool_min (500). Wallet will no longer watch for new transactions and best block height will not be advanced.
468 2017-08-08T19:35:33  <jnewbery> 2017-08-08 19:19:20 Unlock wallet, top up keypool and rescan to resume watching for new transactions.
469 2017-08-08T19:35:55  <jnewbery> Perhaps that could be made more explicit "TOP UP KEYPOOL BEFORE NEXT RESTART!"
470 2017-08-08T19:39:33  <luke-jr> Can we add a new checkpoint? (Minimally invasive way to lock Segwit in hard)
471 2017-08-08T19:40:04  <gmaxwell> luke-jr: No.
472 2017-08-08T19:40:18  <gmaxwell> Thats not a proper or interesting thing to do.
473 2017-08-08T19:40:37  <luke-jr> just to eliminate the FUD around miners potentially reorging it out
474 2017-08-08T19:41:11  <gmaxwell> luke-jr: you should do what we would eventually do, make a patch that removes the activation and just replaces it with a height comparison.
475 2017-08-08T19:41:24  <luke-jr> Isn't it too late for that in 0.15?
476 2017-08-08T19:42:37  <gmaxwell> probably, but we should still write the patch.
477 2017-08-08T19:43:03  <luke-jr> to be clear: I meant the new checkpoint specifically for 0.15.0 only
478 2017-08-08T19:43:26  <gmaxwell> luke-jr: but just pinning the consensus isn't a right thing to do, and especially violating the rules for setting the things.
479 2017-08-08T19:43:39  <luke-jr> "violating the rules for setting the things."?
480 2017-08-08T19:44:10  <gmaxwell> luke-jr: there is a spec for setting that requires them to be set at least 2016 blocks in the past.
481 2017-08-08T19:44:23  <luke-jr> it will be by the time 0.15.0 is released
482 2017-08-08T19:44:23  <gmaxwell> to avoid legislating the ledger state.
483 2017-08-08T19:44:59  <gmaxwell> also neighter of these things will prevent the reorg around the activation and steal outputs along the way.
484 2017-08-08T19:45:21  <gmaxwell> luke-jr: it's important to enforce these things by the time we'd issue SW addresses, less so before.
485 2017-08-08T19:45:37  <luke-jr> hmm, true, I guess we'd need to be checkpointing the actual activation block for that. and there isn't 2016 blocks there.
486 2017-08-08T19:46:02  <luke-jr> gmaxwell: Core is more than just a wallet, and wallets besides Core exist..
487 2017-08-08T19:46:07  <sdaftuar> while on the subject of cleaning up technical debt associated with prior soft fork deployments: #10695 could use review
488 2017-08-08T19:46:08  <gribble> https://github.com/bitcoin/bitcoin/issues/10695 | [qa] Rewrite BIP65/BIP66 functional tests by sdaftuar · Pull Request #10695 · bitcoin/bitcoin · GitHub
489 2017-08-08T19:46:13  <sdaftuar> or merge
490 2017-08-08T19:46:49  <sdaftuar> we should write the csv-height-activation patch first imo
491 2017-08-08T19:46:59  <jnewbery> 10695 looks good to me
492 2017-08-08T19:48:07  <sdaftuar> jnewbery: did we ever fix the ComparisonTestFramework for the bug i mentioned in the OP of that PR?
493 2017-08-08T19:48:11  *** Giszmo has joined #bitcoin-core-dev
494 2017-08-08T19:49:37  *** chjj has quit IRC
495 2017-08-08T19:50:03  <jnewbery> I think we had buy-in for rewriting the ComparisonTestFramework scripts using the standard BitcoinTestFramework
496 2017-08-08T19:50:34  <jnewbery> In fact I've already done that for some of them, but it depends on TestNode and will several months of rebasing
497 2017-08-08T19:50:45  <jnewbery> *will need
498 2017-08-08T19:50:49  <sdaftuar> re-reading my message there though i think there was an immediate bug that makes our comparisonTestFramework tests (perhaps including p2p-fullblocktest?) not test anything.
499 2017-08-08T19:50:52  <gmaxwell> jnewbery: I'm still feeling this solution is incomplete. For example, one of my wallets (though not hd) is a encrypted wallet I use for watching cold funds. I'm never going to unlock the darn thing, its keypool is empty right now.   Maybe what we should do for this release is merge the scan and mark as use and topup if possible but instead of shutting down, spam errors that the keypool ran out a
500 2017-08-08T19:50:58  <gmaxwell> nd your wallet balance may be incorrect.
501 2017-08-08T19:51:50  <jnewbery> we don't shutdown for non-HD wallets
502 2017-08-08T19:52:18  <gmaxwell> jnewbery: I know we don't, but my useage pattern wouldn't be any different if this wallet were HD.
503 2017-08-08T19:53:52  <gmaxwell> I still don't think we have this handling cooked, in that we'll make 0.15 unusable for at least some people or very awkward.  I think we can do better even if I don't know how yet. But it's critical we do something, because without the mark use thing there are several ways to result in serious funds loss. (including giving customers addresses you already gave to other customers, then crediting t
504 2017-08-08T19:53:58  <gmaxwell> he wrong accounts when people make payments..)
505 2017-08-08T19:58:08  *** spudowiar has joined #bitcoin-core-dev
506 2017-08-08T19:58:12  *** spudowiar has left #bitcoin-core-dev
507 2017-08-08T19:59:32  *** paveljanik has joined #bitcoin-core-dev
508 2017-08-08T19:59:32  *** paveljanik has joined #bitcoin-core-dev
509 2017-08-08T20:00:00  <sdaftuar> gmaxwell: in your watching cold funds example, would restarting with a -keypoolmin=0 be sufficient for the use case?
510 2017-08-08T20:01:17  <sdaftuar> (perhaps i don't quite understand the PR though, so ignore me if my question doesn't make sense)
511 2017-08-08T20:01:49  *** chjj has joined #bitcoin-core-dev
512 2017-08-08T20:03:38  <gmaxwell> Yes. It is. I suppose we can just release note that. "If you want to run like this, set that." good point.
513 2017-08-08T20:04:52  <jnewbery> thanks sdaftuar. I think that covers greg's use case
514 2017-08-08T20:05:48  <jnewbery> gmaxwell can we try to evealuate whether this fixes the immediate problem (potential funds loss). We can fix up awkward behaviour post-branch
515 2017-08-08T20:06:42  <gmaxwell> my concern there is that we also need to not introduce something that leaves lots (?) of people stuck. But sdaftuar answers how it doesn't.
516 2017-08-08T20:13:12  <sdaftuar> has anyone given thought to how these keypool parameters work with multiwallet?
517 2017-08-08T20:13:25  *** somenick_29345 has joined #bitcoin-core-dev
518 2017-08-08T20:14:04  *** chjj has quit IRC
519 2017-08-08T20:14:44  *** chjj has joined #bitcoin-core-dev
520 2017-08-08T20:15:33  <sdaftuar> eg if we advise people to use -keypoolmin=0 to solve one situation, and they load up two wallets, are they jeopardizing themselves?
521 2017-08-08T20:17:25  *** SopaXorzTaker has quit IRC
522 2017-08-08T20:17:54  <jnewbery> yes, if you use -keypoolmin=0 you can get into a state where your best block has advanced while your keypool is depleted
523 2017-08-08T20:17:55  *** clarkmoody_ has quit IRC
524 2017-08-08T20:18:08  <jnewbery> and so potentially miss transactions
525 2017-08-08T20:18:19  <bitcoin-git> [bitcoin] gmaxwell opened pull request #11008: Enable disablesafemode by default. (master...no-safe-mode) https://github.com/bitcoin/bitcoin/pull/11008
526 2017-08-08T20:19:12  <jnewbery> but keypoolmin is a hidden debug option. I don't think we're going to be advertising it widely
527 2017-08-08T20:20:14  <bitcoin-git> [bitcoin] rawodb opened pull request #11009: Add information about the next state in getblockchaininfo rpc request (master...master) https://github.com/bitcoin/bitcoin/pull/11009
528 2017-08-08T20:23:44  <jonasschnelli> sdaftuar, luke-jr: about NODE_NETWORK_LIMITED_*. If we would treat them independently, would this mean, a peer signalling NODE_NETWORK_LIMITED_HIGH will always singal NODE_NETWORK_LIMITED_LOW?
529 2017-08-08T20:23:55  <jonasschnelli> And there would be no special case for a peer signaling both bits?
530 2017-08-08T20:24:10  <jonasschnelli> I agree that would make most sense.
531 2017-08-08T20:24:16  <sdaftuar> jonasschnelli: in my opinion, a peer should signal both if it offers both
532 2017-08-08T20:24:24  <bitcoin-git> [bitcoin] rawodb closed pull request #11009: Add information about the next state in getblockchaininfo rpc request (master...master) https://github.com/bitcoin/bitcoin/pull/11009
533 2017-08-08T20:24:57  <sdaftuar> that way if someone wants to write a bitcoin network client that only worries about one of those properties, they don't need to do any complicated reasoning
534 2017-08-08T20:25:01  <jonasschnelli> We would just loose the third state for the two new bits for pure logical consistence reasons. But I guess this makes sense.
535 2017-08-08T20:25:42  <sdaftuar> i would actually suggest that we rename the service bits too, if it's not too much bikeshedding...  NODE_NETWORK is a non-descriptive term for full node to begin with
536 2017-08-08T20:25:57  *** jimpo has quit IRC
537 2017-08-08T20:25:57  <sdaftuar> and putting LIMITED in the name implies denial of "full" services, which i think isn't want we want
538 2017-08-08T20:25:58  <jonasschnelli> We only have 24 useful service bits (~10 are gone)
539 2017-08-08T20:26:19  <bitcoin-git> [bitcoin] rawodb opened pull request #11010: Add information about the next state in getblockchaininfo rpc request (master...pr/getblockchaininfo) https://github.com/bitcoin/bitcoin/pull/11010
540 2017-08-08T20:26:46  <jonasschnelli> sdaftuar: Agree with renaming
541 2017-08-08T20:26:56  <jonasschnelli> sdaftuar: any proposals?
542 2017-08-08T20:26:57  <sdaftuar> i guess i haven't come up with good alternatives yet, but something which means "NODE_RECENT_BLOCKS" or "NODE_REALLY_RECENT_BLOCKS" is along the lines of what i'm thinking
543 2017-08-08T20:27:07  <sdaftuar> (those are not actual proposals, just intuition)
544 2017-08-08T20:28:02  <jonasschnelli> The question is also if NODE_NETWORK implies more then just historical block responses...
545 2017-08-08T20:29:50  <sdaftuar> yeah that's a good question, it's not obvious to me
546 2017-08-08T20:30:09  <sdaftuar> ie we have non-NODE_NETWORK nodes (pruning nodes) which do all the rest, already
547 2017-08-08T20:31:00  <jonasschnelli> I guess there is no concrete definition what a peer must confirm to to allow signalling NODE_NETWORK (expect the reference implementation)
548 2017-08-08T20:31:31  <sipa> yeah, node_network just means "all the things" :)
549 2017-08-08T20:31:37  <jonasschnelli> also, SPV is also ~NODE_NETWORK (!= pruned)
550 2017-08-08T20:31:53  <jonasschnelli> ~(== bit unset)
551 2017-08-08T20:32:07  <sdaftuar> right, but SPV is certainly allowed to eg relay addresses... not sure if any do.
552 2017-08-08T20:32:21  <jonasschnelli> Which results in that NODE_NETWORK_LIMITED somehow includes tx/block relay
553 2017-08-08T20:35:30  <sipa> i guess it's reasonable to suggest that every node can participate in address murmuring, regardless of service flags
554 2017-08-08T20:35:36  <sipa> it's an ootional thing regardless
555 2017-08-08T20:36:04  <sipa> since compact blocks, it doesn't really make sense to have separate services for tx relay and block relay
556 2017-08-08T20:36:47  <sipa> so the only question is whether we want to have a separate bit for relay vs historical fetch (even if it's just 144 blocks deep)
557 2017-08-08T20:37:38  <sipa> any client right now will want both
558 2017-08-08T20:38:16  <jonasschnelli> sipa: for a possible use case where a peer serves historical blocks but won't relay (pure block storage)?
559 2017-08-08T20:38:36  <sdaftuar> if no clients currently offer one but not the other, i don't think it makes sense to split that now.  we can wait til someone wants to build that.
560 2017-08-08T20:39:05  <jonasschnelli> Agree
561 2017-08-08T20:40:21  *** ula|2 has joined #bitcoin-core-dev
562 2017-08-08T20:41:00  <sipa> sdaftuar: right... the only downside is that it'd cost another bit later
563 2017-08-08T20:41:07  *** AaronvanW has quit IRC
564 2017-08-08T20:42:07  <sdaftuar> sipa: we either take that 1 bit cost now or later, right?  unless there's a proposal for distinguishing now that saves a bit, which i missed?
565 2017-08-08T20:43:37  *** Aaronvan_ has joined #bitcoin-core-dev
566 2017-08-08T20:46:46  <sipa> sdaftuar: if we now have 3 bits (relay_tx_and_blocks, blocks_144, blocks_1008), we're good. if we only have (relay_tx_and_blocks_up_to_144, blocks_1008), we may need 2 extra bits later (relay_tx_blocks, blocks_144) totalling 4 bits, where one just implies two other ones
567 2017-08-08T20:47:18  *** timothy has quit IRC
568 2017-08-08T20:48:58  <jonasschnelli> I think the current definition of NODE_NETWORK_LIMITED_LOW is:
569 2017-08-08T20:49:05  <jonasschnelli> NODE_NETWORK_LIMITED_LOW means the same as NODE_NETWORK with the limitation of only serving the last 288 (2 day) blocks
570 2017-08-08T20:49:23  <jonasschnelli> (coupled to the definition of NODE_NETWORK)
571 2017-08-08T20:49:42  <jonasschnelli> Changing the block value is possible though the client/protocol version?
572 2017-08-08T20:50:46  <jonasschnelli> If we want relay as an extra option (assume full block SPV may want to service historical blocks but not relay) another bit would be required
573 2017-08-08T20:51:47  *** ula|2 has quit IRC
574 2017-08-08T20:54:13  <sdaftuar> sipa: i think blocks_144 and blocks_1008 without relay_tx_and_blocks are sort of nonsensical; by definition you're always willing to provide the current tip, and you never have guarantees that transactions will be relayed to you eg due to policy differences, so i don't think of that as much of a promise.
575 2017-08-08T20:54:18  <sdaftuar> but putting that aside:
576 2017-08-08T20:54:35  <sdaftuar> i think we could downgrade those service bits in the future to "may or may not relay blocks and transactions"
577 2017-08-08T20:54:43  <sdaftuar> if we wanted to save a bit
578 2017-08-08T20:54:48  <sdaftuar> oh
579 2017-08-08T20:55:02  <sdaftuar> i guess i'm not sure what i'm comparing...
580 2017-08-08T20:55:58  <sdaftuar> i'm taking jonas' proposal as relay_tx_and_blocks_up_to_144, relay_tx_and_blocks_up_to_1008
581 2017-08-08T20:56:04  <sipa> sdaftuar: but there is a technical difference
582 2017-08-08T20:56:21  <sdaftuar> and suggesting we could in the future add a single bit saying relay_tx_and_blocks
583 2017-08-08T20:56:51  <sdaftuar> and at the point we add that, we say that there are no promises on the quality of service you get from older nodes that aren't setting the new bit
584 2017-08-08T20:57:03  <sipa> sdaftuar: for example relay_blocks_tx could imply support for tx/inv(tx)/cmpctblock/blocktxn/getdata(tx), while the other implies support for getblock/getdata(block)/inv(block)
585 2017-08-08T20:57:21  <sdaftuar> cmpctblock and blocktxn are already separately negotiated
586 2017-08-08T20:57:55  <sipa> but not advertized
587 2017-08-08T20:58:01  <sipa> (right?)
588 2017-08-08T20:58:04  <sdaftuar> right
589 2017-08-08T20:58:15  <sipa> they're implied by NODE_NETWORK + sufficient protocol version
590 2017-08-08T20:58:32  <sdaftuar> well they're implied by NODE_SEGWIT, for all practical purposes
591 2017-08-08T20:58:39  <sdaftuar> but interesting point
592 2017-08-08T20:58:50  *** Giszmo has quit IRC
593 2017-08-08T20:59:04  <sdaftuar> i'm not sure it's reasonable for service bits to keep up with p2p protocol improvements
594 2017-08-08T20:59:09  <sipa> agree
595 2017-08-08T21:00:46  <gmaxwell> I'm not enthused by all these flags because in general we should not be segmenting up the topology if we can avoid it.
596 2017-08-08T21:01:43  <sdaftuar> i agree with that.  i also generally want to punt on features we are not sure anyone will implement.
597 2017-08-08T21:02:26  <sipa> agree
598 2017-08-08T21:02:57  <sipa> just pointing out that it may cost us an extra bit in the future - perhaps that's ok
599 2017-08-08T21:03:05  *** Giszmo has joined #bitcoin-core-dev
600 2017-08-08T21:06:16  *** ula has joined #bitcoin-core-dev
601 2017-08-08T21:07:21  <bitcoin-git> [bitcoin] MarcoFalke pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/2507fd55568b...929fd7276c0f
602 2017-08-08T21:07:22  <bitcoin-git> bitcoin/master d4f0d87 Suhas Daftuar: [qa] Rewrite BIP65 functional tests...
603 2017-08-08T21:07:22  <bitcoin-git> bitcoin/master 4ccc12a Suhas Daftuar: [qa] Rewrite BIP66 functional tests...
604 2017-08-08T21:07:23  <bitcoin-git> bitcoin/master 929fd72 MarcoFalke: Merge #10695: [qa] Rewrite BIP65/BIP66 functional tests...
605 2017-08-08T21:07:46  <bitcoin-git> [bitcoin] MarcoFalke closed pull request #10695: [qa] Rewrite BIP65/BIP66 functional tests (master...2017-06-fix-bip65-test) https://github.com/bitcoin/bitcoin/pull/10695
606 2017-08-08T21:11:09  *** Guyver2 has quit IRC
607 2017-08-08T21:27:37  <jonasschnelli> Should peers avoid serving blocks above the NODE_NETWORK_LIMITED_LOW|HIGH threshold to reduce finger printing opportunities?
608 2017-08-08T21:28:49  <jonasschnelli> Assume you have pruned down to 10k blocks and signalling NODE_NETWORK_LIMITED_HIGH, one could fingerprint by checking how deep down you can serve blocks
609 2017-08-08T21:30:22  <sipa> yes, i think so
610 2017-08-08T21:30:36  *** somenick_29345 has quit IRC
611 2017-08-08T21:30:37  <jonasschnelli> Okay. Will update the BIP
612 2017-08-08T21:31:06  <jonasschnelli> Or maybe just the implementation... need to check
613 2017-08-08T21:31:34  *** clarkmoody has joined #bitcoin-core-dev
614 2017-08-08T21:36:02  <bitcoin-git> [bitcoin] rawodb closed pull request #11010: Add information about the next state in getblockchaininfo rpc request (master...pr/getblockchaininfo) https://github.com/bitcoin/bitcoin/pull/11010
615 2017-08-08T22:02:37  *** instagibbs has quit IRC
616 2017-08-08T22:07:16  *** instagibbs has joined #bitcoin-core-dev
617 2017-08-08T22:23:25  *** Yogaqueef has quit IRC
618 2017-08-08T22:25:17  *** chjj has quit IRC
619 2017-08-08T22:26:38  *** Cheeseo has joined #bitcoin-core-dev
620 2017-08-08T22:27:03  <molz> Dear Core Devs, Thank you for all your hard work!  https://i.imgflip.com/1trxh9.jpg
621 2017-08-08T22:29:55  <sipa> molz: yw!
622 2017-08-08T22:32:16  <mryandao> could anyone from here answer this: https://bitcoin.stackexchange.com/questions/57416/since-bitcoin-core-0-14-0-how-does-a-node-with-default-settings-compute-the-dus
623 2017-08-08T22:35:52  <Murch> sipa: Transaction overhead is 10 bytes for P2PKH transactions, since segwit adds flag and marker, which however go into the witness part am I correct to calculate the weight of the tx overhead as 42?
624 2017-08-08T22:36:24  <Murch> For a transaction that spends at least one P2SHP2PWSH input
625 2017-08-08T22:37:03  <sipa> Murch: looks right to me
626 2017-08-08T22:37:10  <sipa> mryandao: yes, will do if i find the time
627 2017-08-08T22:37:13  <Murch> thank you :)
628 2017-08-08T22:37:34  *** chjj has joined #bitcoin-core-dev
629 2017-08-08T22:40:16  *** Chris_Stewart_5 has quit IRC
630 2017-08-08T22:53:03  <sipa> mryandao: achow101 beat me to it :)
631 2017-08-08T22:53:12  <achow101> :)
632 2017-08-08T22:54:39  <bitcoin-git> [bitcoin] gmaxwell opened pull request #11011: [Trivial] Add a comment on the use of prevector in script. (master...201708-prevector-comment) https://github.com/bitcoin/bitcoin/pull/11011
633 2017-08-08T23:11:06  *** jouke has quit IRC
634 2017-08-08T23:13:13  *** jouke has joined #bitcoin-core-dev
635 2017-08-08T23:13:13  *** jouke has quit IRC
636 2017-08-08T23:13:13  *** jouke has joined #bitcoin-core-dev