1 2017-08-03T00:00:41  <jnewbery> #10882 is squashed and ready for reACK/merge
   2 2017-08-03T00:00:44  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
   3 2017-08-03T00:01:32  <sipa> jnewbery: thanks!
   4 2017-08-03T00:05:09  *** http_GK1wmSU has joined #bitcoin-core-dev
   5 2017-08-03T00:05:12  *** http_GK1wmSU has left #bitcoin-core-dev
   6 2017-08-03T00:10:13  *** AaronvanW has quit IRC
   7 2017-08-03T00:13:46  *** derbumi has quit IRC
   8 2017-08-03T00:15:05  *** derbumi has joined #bitcoin-core-dev
   9 2017-08-03T00:21:20  *** LampTreadStone07 has joined #bitcoin-core-dev
  10 2017-08-03T00:26:19  *** blznblzn2 has quit IRC
  11 2017-08-03T00:31:42  *** jtimon has quit IRC
  12 2017-08-03T00:50:44  *** LampTreadStone07 has quit IRC
  13 2017-08-03T00:53:21  *** Cheeseo has joined #bitcoin-core-dev
  14 2017-08-03T00:54:43  *** lifeofguenter has quit IRC
  15 2017-08-03T00:56:40  *** cheese_ has quit IRC
  16 2017-08-03T01:00:56  *** lifeofguenter has joined #bitcoin-core-dev
  17 2017-08-03T01:07:25  *** dabura667 has joined #bitcoin-core-dev
  18 2017-08-03T01:16:13  *** Orion3k has quit IRC
  19 2017-08-03T01:29:28  *** Orion3k has joined #bitcoin-core-dev
  20 2017-08-03T01:29:38  *** Ylbam has quit IRC
  21 2017-08-03T01:30:20  *** sam_c has quit IRC
  22 2017-08-03T01:32:30  *** sam_c has joined #bitcoin-core-dev
  23 2017-08-03T01:35:01  *** Orion3k has quit IRC
  24 2017-08-03T01:45:46  *** Orion3k has joined #bitcoin-core-dev
  25 2017-08-03T01:49:15  *** james0909 has joined #bitcoin-core-dev
  26 2017-08-03T01:49:28  <james0909> hi, i'm having an issue with core, could someone help?
  27 2017-08-03T01:51:33  <instagibbs> james0909, #bitcoin please, someone can help there
  28 2017-08-03T02:20:48  *** veleiro has joined #bitcoin-core-dev
  29 2017-08-03T02:24:16  *** sam_c has quit IRC
  30 2017-08-03T02:41:30  *** jr2014 has joined #bitcoin-core-dev
  31 2017-08-03T02:47:19  <praxeology> Would it be worth adding some ECC to chainstate and the block index?  Maybe running a memtest before the first synch?
  32 2017-08-03T02:47:58  <praxeology> Memtest could auto start on the first synch from genesis... but be skippable
  33 2017-08-03T02:48:33  <praxeology> Although... sure memtest is better to be done w/ something like Memtest86+
  34 2017-08-03T02:50:59  *** Deacydal has joined #bitcoin-core-dev
  35 2017-08-03T02:54:16  *** Deacyde has quit IRC
  36 2017-08-03T02:56:33  *** veleiro has quit IRC
  37 2017-08-03T03:20:13  *** jeep-ss has quit IRC
  38 2017-08-03T03:28:30  *** blznblzn2 has joined #bitcoin-core-dev
  39 2017-08-03T03:42:04  *** Giszmo1 has joined #bitcoin-core-dev
  40 2017-08-03T03:43:02  *** Giszmo has quit IRC
  41 2017-08-03T03:48:00  *** ekerstein has quit IRC
  42 2017-08-03T03:50:52  *** chjj has quit IRC
  43 2017-08-03T03:55:36  *** miknotauro has joined #bitcoin-core-dev
  44 2017-08-03T03:55:59  <gmaxwell> praxeology: error correction wouldn't really help unless it was very very high overhead.
  45 2017-08-03T04:03:15  <praxeology> I guess I wasn't really calling for error correction, more error detection.  But yea I guess it already does detect an error eventually
  46 2017-08-03T04:04:51  <gmaxwell> There is error detection.
  47 2017-08-03T04:06:04  <praxeology> Maybe if the client gets stuck on an old block, it should suggest to the user that maybe its database is corrupted... and give some suggestions on what to do about it
  48 2017-08-03T04:12:10  <sipa> it does
  49 2017-08-03T04:12:15  <sipa> and asks if you want to reindex
  50 2017-08-03T04:14:41  *** james0909 has quit IRC
  51 2017-08-03T04:18:24  <praxeology> ok.  james0909 in #bitcoin said his error log said...  "ERROR: invalid header received 2017-08-03 01:48:18 ProcessMessages(headers, 11422 bytes) FAILED peer=38"?  and his tip was stuck at 478645.
  52 2017-08-03T04:18:56  <praxeology> But he didn't say anything about it asking him to reindex... although he did ask if he should reindex.
  53 2017-08-03T04:19:48  <gmaxwell> praxeology: why do you think there is corruption...
  54 2017-08-03T04:20:26  <gmaxwell> praxeology: the invalid headers messages are bcash nodes getting banned. And he may not catch up if he's been offline until the next block if the peer he picked for the initial header sync was broken or malicious.
  55 2017-08-03T04:21:17  <praxeology> he said he tried connecting to other peers.
  56 2017-08-03T04:21:47  <gmaxwell> what does that even mean
  57 2017-08-03T04:21:48  <praxeology> I guess I didn't confirm that he actually connected to a known good peer
  58 2017-08-03T04:22:11  <gmaxwell> doesn't matter f he is 'connected' to a good peer, he won't continue syncing until there is a new block.
  59 2017-08-03T04:22:31  <praxeology> oh.  I didn't know that
  60 2017-08-03T04:22:42  *** miknotauro has quit IRC
  61 2017-08-03T04:22:45  <gmaxwell> (if he was behind and the peer he elected for initial headers was bogus)
  62 2017-08-03T04:22:54  <praxeology> So he needs to be connected to a good peer while a new block comes in
  63 2017-08-03T04:23:29  <gmaxwell> praxeology: sure, just doing nothing will do that with very high likelyhood since it'll be connected to at least 8 peers.
  64 2017-08-03T04:24:01  <praxeology> when bitcoin starts up, it only chooses one peer to ask for latest headers?
  65 2017-08-03T04:24:22  <sdaftuar> praxeology: yes
  66 2017-08-03T04:24:30  <praxeology> and if that fails... it doesn't ask another?
  67 2017-08-03T04:24:30  <gmaxwell> praxeology: right, and if doesn't give all of them it won't learn that it's behind until someone else advertises a block.
  68 2017-08-03T04:24:37  <sdaftuar> praxeology: in 0.15, there will be a timeout on that request
  69 2017-08-03T04:24:38  <gmaxwell> it can't tell when it 'fails'
  70 2017-08-03T04:24:55  <gmaxwell> except for a total timeout, which sdaftuar notes is detected in 0.15
  71 2017-08-03T04:25:13  <praxeology> hm, well if the time of his last block is really old... or if the peer he is asking has a different tip...
  72 2017-08-03T04:25:44  <gmaxwell> praxeology: different tip than what
  73 2017-08-03T04:26:24  <praxeology> if your own node has a different tip than the one you are requesting the latest headers from
  74 2017-08-03T04:26:46  <gmaxwell> praxeology: of course its different or fetching headers from it was pointless...
  75 2017-08-03T04:26:48  <praxeology> like, their tip is in a different chain or behind or something
  76 2017-08-03T04:27:08  <praxeology> bah ok you right
  77 2017-08-03T04:27:17  <sdaftuar> you don't request headers from a peer who is on a less-work chain than your tip
  78 2017-08-03T04:27:44  <gmaxwell> And any countermeasure has to not do things like terminate a long header sync from a valid peer just because it's a lot of data... otherwise a newly syncing node could get stuck if it's link was too low in bandwidth.
  79 2017-08-03T04:28:07  <sdaftuar> actually, i said that wrong.  you don't request blocks from such a peer
  80 2017-08-03T04:28:36  *** Deacydal has quit IRC
  81 2017-08-03T04:30:11  <gmaxwell> praxeology: there are a bunch of places we could improve responsiveness by punting less helpful peers, but such rules require just a lot of care because false positives are potententially severe attack vectors.
  82 2017-08-03T04:31:35  *** chjj has joined #bitcoin-core-dev
  83 2017-08-03T04:35:09  <praxeology> alright... well, its unfortunate I misdiagnosed the guy's issue.  Thanks for the help.  Hopefully he will come back and ask again on #bitcoin and one of sees him... for his sake... before he -reindex
  84 2017-08-03T04:36:56  <gmaxwell> it's also important to not let malicious peers cause users to get any kind of error to the greatest extent that we can avoid it.
  85 2017-08-03T04:37:24  <gmaxwell> because unfortunately any warning you show users will cause a small percentage of them to jump off bridges.
  86 2017-08-03T04:38:16  <gmaxwell> "I saw a warning that wouldn't go away so I deleted my wallet, and now I still have the warning and where are my bitcoins?"
  87 2017-08-03T04:42:36  *** ivan has quit IRC
  88 2017-08-03T04:44:08  *** JackH has quit IRC
  89 2017-08-03T04:46:59  <mryandao> LOL
  90 2017-08-03T04:48:23  <gmaxwell> not really funny though. :(
  91 2017-08-03T04:48:53  <gmaxwell> it's not like people are stupid, first they try all the things they think are reasonable, then they try other things...
  92 2017-08-03T04:49:17  <gmaxwell> and with enough users, someone is going to think they should try sawing off their hands or whatever.
  93 2017-08-03T04:49:39  <Emcy_> throwing your wallet in a lake isnt reasonable
  94 2017-08-03T04:49:43  <mryandao> i can't connect the dots between deleting wallet.dat and addressing the problem.
  95 2017-08-03T04:50:08  <gmaxwell> mryandao: because it's state that could be causing the error.
  96 2017-08-03T04:50:19  <gmaxwell> Emcy_: but they saved the addresses first!
  97 2017-08-03T04:50:32  <gmaxwell> you and I know that doesn't help, but it's not exactly obvious
  98 2017-08-03T04:50:42  <Emcy_> it was even called "wallet" as a kind of linguistic skeumorph to get people to understand what it is
  99 2017-08-03T04:50:45  <mryandao> hmm, this sounds like motivation to decouple key from wallet.dat?
 100 2017-08-03T04:51:04  <mryandao> so at least even if wallet.dat was removed out of a panic attack, at least keys is still somewhere safe?
 101 2017-08-03T04:52:06  <Emcy_> youre right about explaining pubkey crypto to normies though oh boy
 102 2017-08-03T04:52:15  <gmaxwell> well we've talked before about writing wallet backup files periodically but there is a counter argument that doing so will make it more likely for users to get their wallets stolen. e.g. clear wallet off a disk before giving it to someone else...
 103 2017-08-03T04:55:37  <Emcy_> i meant to ask why the default datadir on windows in is the domain roaming profile folder, actually, instead of the local profile
 104 2017-08-03T04:55:41  *** e4xit has quit IRC
 105 2017-08-03T04:55:45  <Emcy_> since the subject came up
 106 2017-08-03T04:57:51  <praxeology> does bitcoin still use windows registry?
 107 2017-08-03T04:58:21  <praxeology> windows configuration/file system usage needs a rework :p
 108 2017-08-03T04:59:55  <praxeology> HKEY_CURRENT_USER\Software\Bitcoin\Bitcoin-Qt
 109 2017-08-03T05:00:54  <praxeology> Not that I am actually making demands for your time, sorry.  Have a good night!
 110 2017-08-03T05:03:44  *** wfef has joined #bitcoin-core-dev
 111 2017-08-03T05:04:33  *** clarkmoody_ has quit IRC
 112 2017-08-03T05:12:34  *** miknotauro has joined #bitcoin-core-dev
 113 2017-08-03T05:17:29  <sipa> praxeology: why?
 114 2017-08-03T05:50:57  *** chjj has quit IRC
 115 2017-08-03T05:54:11  *** chjj has joined #bitcoin-core-dev
 116 2017-08-03T05:59:45  <goatpig> isn't that just the URI registration?
 117 2017-08-03T06:00:47  <praxeology> windows registry is not very compatible with multiple installations in use on the same OS
 118 2017-08-03T06:02:32  <praxeology> magic hidden settings that one wouldn't know to move to a new system
 119 2017-08-03T06:27:57  *** Drabiv has joined #bitcoin-core-dev
 120 2017-08-03T06:29:42  *** Ylbam has joined #bitcoin-core-dev
 121 2017-08-03T06:47:50  *** felco has quit IRC
 122 2017-08-03T06:51:18  *** BashCo has quit IRC
 123 2017-08-03T06:58:48  *** marcoagner has quit IRC
 124 2017-08-03T07:06:59  *** JackH has joined #bitcoin-core-dev
 125 2017-08-03T07:10:25  *** marcoagner has joined #bitcoin-core-dev
 126 2017-08-03T07:11:10  *** BashCo has joined #bitcoin-core-dev
 127 2017-08-03T07:12:57  *** BashCo_ has joined #bitcoin-core-dev
 128 2017-08-03T07:16:32  *** BashCo has quit IRC
 129 2017-08-03T07:20:22  *** Orie has quit IRC
 130 2017-08-03T07:41:03  *** clarkmoody has joined #bitcoin-core-dev
 131 2017-08-03T07:41:22  *** timothy has joined #bitcoin-core-dev
 132 2017-08-03T07:46:36  *** timothy has quit IRC
 133 2017-08-03T07:47:13  *** timothy has joined #bitcoin-core-dev
 134 2017-08-03T07:47:40  *** Cheeseo has quit IRC
 135 2017-08-03T07:47:54  *** timothy has quit IRC
 136 2017-08-03T07:50:56  *** timothy has joined #bitcoin-core-dev
 137 2017-08-03T07:52:31  *** jtimon has joined #bitcoin-core-dev
 138 2017-08-03T08:01:55  *** AaronvanW has joined #bitcoin-core-dev
 139 2017-08-03T08:02:21  *** yRDIUTgn has joined #bitcoin-core-dev
 140 2017-08-03T08:03:58  <gmaxwell> I wonder if we should adopt a patch to instantly disconnect bcash nodes based on their service flags.  Sucks to burn a service bit forever to their recklessness though. :(
 141 2017-08-03T08:03:58  *** Aaronvan_ has joined #bitcoin-core-dev
 142 2017-08-03T08:04:16  <gmaxwell> and presumably the constant inadvertant dos attack of connecting to nodes on another network will move them onto another port eventually.
 143 2017-08-03T08:06:05  *** AaronvanW has quit IRC
 144 2017-08-03T08:07:41  *** Deacyde has joined #bitcoin-core-dev
 145 2017-08-03T08:09:05  <praxeology> potentially in the future the burned service code could be recovered in the future after the problem goes away
 146 2017-08-03T08:09:52  <praxeology> are they refusing to move to a different port?
 147 2017-08-03T08:10:43  <gmaxwell> they refused people people asked them previously.
 148 2017-08-03T08:10:59  <gmaxwell> it's not all that easy to recover a service bit where its use results in instantly being disconnected!
 149 2017-08-03T08:11:13  <praxeology> Maybe in a month bcash won't have anyone mining it anymore
 150 2017-08-03T08:11:52  <gmaxwell> perhaps we shouldn't worry much about burning one because we know we're due for some other pretty substantial p2p revisions, and other updates can add new capabilities flags.
 151 2017-08-03T08:12:54  <gmaxwell> I think we only use service flags now for things we absolutely need to have in addr messages, and if we create a addr message replacement (to support NG hidden services and I2P, we'd probably give it a different capabilities signaling tool)
 152 2017-08-03T08:14:05  <praxeology> service flags... "flags" implies a 32 bit or 64 bit number?
 153 2017-08-03T08:14:50  <praxeology> Maybe switch to a set of service tags instead?
 154 2017-08-03T08:15:08  <gmaxwell> well, they need to be small because they're rumored everwhere in addr messages.
 155 2017-08-03T08:15:25  *** d_t has quit IRC
 156 2017-08-03T08:15:30  <gmaxwell> making them fat sidechannels would likely have clowns using them for file trading. :)
 157 2017-08-03T08:17:17  <praxeology> ok, well you could put a size constraint on the tag set... but whatever I'm just sleep dep and over engineering something I don't know enough about
 158 2017-08-03T08:25:00  *** tucenaber has quit IRC
 159 2017-08-03T08:25:40  *** tucenaber has joined #bitcoin-core-dev
 160 2017-08-03T08:25:40  *** tucenaber has joined #bitcoin-core-dev
 161 2017-08-03T08:55:33  *** goatpig has quit IRC
 162 2017-08-03T09:02:25  *** ivan has joined #bitcoin-core-dev
 163 2017-08-03T09:10:08  *** bincap has quit IRC
 164 2017-08-03T09:10:16  *** rafalcpp has quit IRC
 165 2017-08-03T09:11:19  *** miknotauro has quit IRC
 166 2017-08-03T09:27:08  <Eliel> jonasschnelli: is bitcoind's code that does that too difficulty to understand?
 167 2017-08-03T09:27:19  <Eliel> (never mind, was looking at the past)
 168 2017-08-03T09:27:23  <jonasschnelli> Eliel: depends on you experience
 169 2017-08-03T09:27:39  <Eliel> ah true
 170 2017-08-03T09:34:59  *** jimpo has joined #bitcoin-core-dev
 171 2017-08-03T09:34:59  *** LeMiner has joined #bitcoin-core-dev
 172 2017-08-03T09:59:41  *** Austindoggie has joined #bitcoin-core-dev
 173 2017-08-03T10:01:45  *** dabura667 has quit IRC
 174 2017-08-03T10:03:07  *** karelb has joined #bitcoin-core-dev
 175 2017-08-03T10:03:52  <gmaxwell> Hi all, karelb is working on the trezor wallet, and they've been trying to use these patches to bitcoin that implement the bitcore address indexing stuff, but they're finding it really slow to the point where performance is problematic.
 176 2017-08-03T10:04:10  <karelb> we are using it for a while now :)
 177 2017-08-03T10:04:29  <karelb> but right now we are reindexing bitcoin blockchain and it takes foreved
 178 2017-08-03T10:04:34  <gmaxwell> Maybe someone would be interested in giving them a bit of a hand at looking at it? (I need to get to bed); I've already provided the standard disclaimers about the inherent non-scalability of address-indexes-of-all-history. :)
 179 2017-08-03T10:04:46  *** AaronvanW has joined #bitcoin-core-dev
 180 2017-08-03T10:05:10  <karelb> (It is actually bitcoin-abc, I hope I won't be banned :D, but this issue popped up in bitcoin core too)
 181 2017-08-03T10:05:17  <gmaxwell> karelb: one question would be what kind of system is this running on? e.g. is it on some VPS with remote storage that may have poor IO speed?
 182 2017-08-03T10:05:47  <gmaxwell> karelb: we'll forgive you for your sins. though obviously can't help with any abc specific issues.
 183 2017-08-03T10:05:47  <karelb> it is our local server that has SSD, lots of RAM and processors
 184 2017-08-03T10:05:59  <gmaxwell> darn.
 185 2017-08-03T10:06:56  <karelb> actually one issue was sort-of ABC related, but it was because of a commit ported from master from bitcoin core, so it will be relevant anyway
 186 2017-08-03T10:07:04  <karelb> https://github.com/Bitcoin-ABC/bitcoin-abc/issues/43 , https://github.com/satoshilabs/bitcoin-abc/commit/5337f8f210eaa34d1212103f700698dd4989f479
 187 2017-08-03T10:07:26  <gmaxwell> so since I doubt that addrindex has a useful caching layer, you could look for the leveldb::Options object for the database it creates and try increasing the options.block_cache and options.write_buffer_size to really large levells.
 188 2017-08-03T10:07:38  <karelb> it's because the bitcore patches do address index in ConnectBlock/DisconnectBlock, even on the start during the testing thing
 189 2017-08-03T10:07:49  *** Aaronvan_ has quit IRC
 190 2017-08-03T10:08:04  <karelb> (jpochyla is my colleague)
 191 2017-08-03T10:08:41  <gmaxwell> karelb: probably code based on Bitcoin Core master (including ABC) will not be reliably compatible with that address indexing stuff until it is changed.
 192 2017-08-03T10:08:50  <karelb> And the introduction of ApplyBlockUndo somehow caused that
 193 2017-08-03T10:09:35  <karelb> yeah. We want to write our own indexing thing that you won't need to put inside the C++ code, since that is a little insane and we need to keep on rebasing that
 194 2017-08-03T10:09:46  *** erts has joined #bitcoin-core-dev
 195 2017-08-03T10:10:08  *** erts is now known as Guest16355
 196 2017-08-03T10:10:13  <karelb> gmaxwell: just fyi, ABC is not based on master, but on 0.14.1, but they took that thing from master
 197 2017-08-03T10:10:16  <gmaxwell> karelb: bitcoin core master changed the atomiticity requirements for the backend database, but a side effect of this is that it needs special replay logic to handle crash recovery.  ABC has partially ported some of these changes. I am not sure, but I wouldn't be surprised if the address indexing would get corrupted until updated to have the right synchronization behavior.
 198 2017-08-03T10:10:32  <gmaxwell> I know it's based on 0.14.1, but they copied some of these database changes.
 199 2017-08-03T10:10:36  <karelb> ok
 200 2017-08-03T10:11:21  <gmaxwell> in any case, beyond the cache options I mentioned above, I am out of ideas for making it faster without substantial design changes.
 201 2017-08-03T10:11:55  <gmaxwell> I think you should probably also make an extra effort to always cleanly start and stop that node, because I wouldn't be confident that it is durable across crashes without corruption.
 202 2017-08-03T10:13:36  <Austindoggie> Did it take a long time to reindex because you went back a version of bitcoin core?
 203 2017-08-03T10:14:33  <Austindoggie> Sorry if im not allowed to talk here...
 204 2017-08-03T10:14:47  <karelb> gmaxwell: hah, that is not a good news. We also noticed the node randomly crashes once in a while and we are not sure why
 205 2017-08-03T10:15:24  <gmaxwell> ::sigh::
 206 2017-08-03T10:15:29  <karelb> yeah
 207 2017-08-03T10:16:31  <karelb> block_cache and write_buffer_size can be set via conf, or only in code?
 208 2017-08-03T10:16:39  <karelb> wait I will have a look
 209 2017-08-03T10:16:44  <gmaxwell> karelb: run more nodes, reindex anytime one crashes?  At least until you can test if the address index is accurate across crashes?  I'm not completely confident that it won't be, but it would take some careful review and testing to be sure (especially in the context of ABC that has really scrambled things up a lot)
 210 2017-08-03T10:17:02  <gmaxwell> karelb: I think only in the code, I don't think there is an external way to override.
 211 2017-08-03T10:17:21  <gmaxwell> in any case, I have to go to bed. Hopefully someone else will wake up and have some other suggestions.
 212 2017-08-03T10:17:32  <karelb> (I really hate how ABC reformatted everything for no good reason, but that is another issue from this)
 213 2017-08-03T10:17:44  *** jannes has joined #bitcoin-core-dev
 214 2017-08-03T10:18:05  *** Guest16355 has left #bitcoin-core-dev
 215 2017-08-03T10:18:26  <gmaxwell> yes, it makes it really hard to see what exactly has changed because the reformats were heavily intermixed with real changes. :(
 216 2017-08-03T10:19:58  <karelb> @austindoggie nope we started to download blockchain from 0. And it is always stuck on IO for insanely long times
 217 2017-08-03T10:20:16  <karelb> and normal bitcoin without bitcore patches doesn't do it, on the same HW
 218 2017-08-03T10:20:39  <gmaxwell> karelb: gross final suggestion before I really go:  if you really have a lot of ram, create a tmpfs mount big enough for the entire datadir, and sync in there, when it finishes copy it to disk.
 219 2017-08-03T10:21:44  <gmaxwell> karelb: the challenge there is that we have insanely optimized bitcoin core's sync process.. we avoid writing to leveldb at all costs, basically, and a significant fraction of UTXO never hit the database at all during normal sync because they're spent before the dbcache fills)
 220 2017-08-03T10:24:27  *** dobak has joined #bitcoin-core-dev
 221 2017-08-03T10:25:00  <karelb> thanks a lot for your help
 222 2017-08-03T10:25:18  <karelb> going to IRC was a wild shot but it seems it might help :)
 223 2017-08-03T10:25:33  *** dobak has left #bitcoin-core-dev
 224 2017-08-03T10:26:50  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/659c09613408...2e857bb619f5
 225 2017-08-03T10:26:50  <bitcoin-git> bitcoin/master 49d903e Alex Morcos: Eliminate fee overpaying edge case when subtracting fee from recipients
 226 2017-08-03T10:26:51  <bitcoin-git> bitcoin/master 2e857bb Wladimir J. van der Laan: Merge #10942: Eliminate fee overpaying edge case when subtracting fee from recipients...
 227 2017-08-03T10:27:25  <bitcoin-git> [bitcoin] laanwj closed pull request #10942: Eliminate fee overpaying edge case when subtracting fee from recipients (master...subtractfee) https://github.com/bitcoin/bitcoin/pull/10942
 228 2017-08-03T10:27:25  *** LordCow has joined #bitcoin-core-dev
 229 2017-08-03T10:34:40  <karelb> Hm. options.block_cache and options.write_buffer_size are derived from dbcache
 230 2017-08-03T10:35:16  <karelb> dbcache option
 231 2017-08-03T10:35:35  <jonasschnelli> Updated to Debian 9 (stretch) and suddenly get gitian build errors: init.lxc: failed to mount /dev/shm : No such file or directory
 232 2017-08-03T10:35:46  <jonasschnelli> Anyone else experiences this?
 233 2017-08-03T10:39:18  *** Austindoggie has quit IRC
 234 2017-08-03T10:39:32  *** Austindoggie has joined #bitcoin-core-dev
 235 2017-08-03T10:39:49  *** Deacydal has joined #bitcoin-core-dev
 236 2017-08-03T10:40:22  <karelb> maximum dbcache is 16384 MB hard-capped in code; is there a reason for that?
 237 2017-08-03T10:40:38  <jonasschnelli> karelb: why would you need more?
 238 2017-08-03T10:41:58  *** Deacyde has quit IRC
 239 2017-08-03T10:42:23  *** sam_c has joined #bitcoin-core-dev
 240 2017-08-03T10:47:00  <karelb> see the past discussion... we are running bitcore address index patches and they are very slow and stuck on disk operations. And it happens probably because it gets stuck on adding things to address index and commiting it to hard disk on every block
 241 2017-08-03T10:47:11  <karelb> probably
 242 2017-08-03T10:48:13  <karelb> it must be because of the address index somehow, because on the same PC, bitcoin core without bitcore (sigh) patches runs fine
 243 2017-08-03T10:48:46  *** sam_c has quit IRC
 244 2017-08-03T10:49:00  <karelb> I am actually talking about ABC, but the same issue crops up in bitcoin, plus it might be because ABC added some db logic from master
 245 2017-08-03T10:49:45  *** sam_c has joined #bitcoin-core-dev
 246 2017-08-03T10:50:54  *** paveljanik has joined #bitcoin-core-dev
 247 2017-08-03T10:52:00  *** Austindoggie has quit IRC
 248 2017-08-03T10:53:08  <karelb> hm, we stopped the bitcoind and it got into some crashed state which is inconsistent and nothing happens
 249 2017-08-03T10:53:11  <karelb> :/
 250 2017-08-03T10:53:22  <karelb> this is hell
 251 2017-08-03T10:54:16  <gmaxwell> karelb: the leveldb caching isn't very useful for the leveldb databases in core because we've already cached the heck out of those things at a higher level...  but for the options set for your custom address index it may be very helpful.
 252 2017-08-03T10:55:52  *** jr2014 has quit IRC
 253 2017-08-03T11:03:44  *** sam_c has quit IRC
 254 2017-08-03T11:05:08  <karelb> well it started chugging again, but is still gets stuck. With dbcache set at 16384. Memory has 1GB full and 63GB free
 255 2017-08-03T11:06:15  <gmaxwell> karelb: what does stuck mean exactly
 256 2017-08-03T11:07:37  *** felco has joined #bitcoin-core-dev
 257 2017-08-03T11:08:44  <karelb> Log says nothing, and iotop shows 100% and doing a lot of reading/writing in the leveldb
 258 2017-08-03T11:09:10  <karelb> and after about 20 minutes, log messages start to appear again
 259 2017-08-03T11:09:59  <gmaxwell> dear lord. :(
 260 2017-08-03T11:13:45  <karelb> and the binary doesn't reply even to kill signals
 261 2017-08-03T11:19:03  *** paveljanik has quit IRC
 262 2017-08-03T11:19:31  *** paveljanik has joined #bitcoin-core-dev
 263 2017-08-03T11:19:39  *** paveljanik has joined #bitcoin-core-dev
 264 2017-08-03T11:24:43  *** brg444 has quit IRC
 265 2017-08-03T11:26:20  *** jr2014 has joined #bitcoin-core-dev
 266 2017-08-03T11:27:11  *** brg444 has joined #bitcoin-core-dev
 267 2017-08-03T11:29:56  <karelb> data/blocks/index is compacting like crazy when it is stuck
 268 2017-08-03T11:30:33  <karelb> https://pastebin.com/SvsyQyiL
 269 2017-08-03T11:33:56  *** SopaXorzTaker has quit IRC
 270 2017-08-03T11:34:05  <karelb> The bitcoind is stuck and at the same time when this is hapenning
 271 2017-08-03T11:34:23  <karelb> and it keeps writing compacting
 272 2017-08-03T11:34:50  *** SopaXorzTaker has joined #bitcoin-core-dev
 273 2017-08-03T11:34:55  *** laurentmt has joined #bitcoin-core-dev
 274 2017-08-03T11:40:08  *** laurentmt has quit IRC
 275 2017-08-03T11:55:21  *** snq has joined #bitcoin-core-dev
 276 2017-08-03T12:02:21  *** drizztbsd has joined #bitcoin-core-dev
 277 2017-08-03T12:03:04  *** timothy has quit IRC
 278 2017-08-03T12:11:19  *** blznblzn2 has quit IRC
 279 2017-08-03T12:23:29  *** jimpo has quit IRC
 280 2017-08-03T12:31:00  *** dobak has joined #bitcoin-core-dev
 281 2017-08-03T13:03:45  *** Cheeseo has joined #bitcoin-core-dev
 282 2017-08-03T13:04:24  *** jeep-ss has joined #bitcoin-core-dev
 283 2017-08-03T13:07:13  *** rafalcpp has joined #bitcoin-core-dev
 284 2017-08-03T13:07:48  <bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/2e857bb619f5...e222618a32a1
 285 2017-08-03T13:07:48  <bitcoin-git> bitcoin/master 3498a8d Cory Fields: depends: fix fontconfig with newer glibc...
 286 2017-08-03T13:07:49  <bitcoin-git> bitcoin/master e222618 Wladimir J. van der Laan: Merge #10851: depends: fix fontconfig with newer glibc...
 287 2017-08-03T13:08:13  <bitcoin-git> [bitcoin] laanwj closed pull request #10851: depends: fix fontconfig with newer glibc (master...fontconfig-bump) https://github.com/bitcoin/bitcoin/pull/10851
 288 2017-08-03T13:14:56  *** dobak has quit IRC
 289 2017-08-03T13:16:54  *** dobak has joined #bitcoin-core-dev
 290 2017-08-03T13:17:48  *** cheese_ has joined #bitcoin-core-dev
 291 2017-08-03T13:18:42  *** dobak has left #bitcoin-core-dev
 292 2017-08-03T13:20:23  *** spudowiar has joined #bitcoin-core-dev
 293 2017-08-03T13:20:42  *** Cheeseo has quit IRC
 294 2017-08-03T13:21:50  *** Guyver2 has joined #bitcoin-core-dev
 295 2017-08-03T13:24:35  *** dobak has joined #bitcoin-core-dev
 296 2017-08-03T13:27:12  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 297 2017-08-03T13:56:16  *** dobak has quit IRC
 298 2017-08-03T14:30:04  *** jtimon has quit IRC
 299 2017-08-03T14:30:53  *** PaulCapestany has quit IRC
 300 2017-08-03T14:34:33  *** PaulCapestany has joined #bitcoin-core-dev
 301 2017-08-03T14:36:01  *** timothy has joined #bitcoin-core-dev
 302 2017-08-03T14:37:04  *** drizztbsd has quit IRC
 303 2017-08-03T14:45:21  *** yRDIUTgn has quit IRC
 304 2017-08-03T14:45:34  *** yRDIUTgn has joined #bitcoin-core-dev
 305 2017-08-03T14:46:02  <bitcoin-git> [bitcoin] NicolasDorier opened pull request #10980: [WIP] Decouple CKeyStore from CWatchOnlyStore (master...decouplewatchonly) https://github.com/bitcoin/bitcoin/pull/10980
 306 2017-08-03T14:49:43  *** clarkmoody has quit IRC
 307 2017-08-03T14:50:04  *** clarkmoody has joined #bitcoin-core-dev
 308 2017-08-03T14:54:40  *** BashCo_ has quit IRC
 309 2017-08-03T14:55:18  *** BashCo has joined #bitcoin-core-dev
 310 2017-08-03T14:56:26  *** Murch has joined #bitcoin-core-dev
 311 2017-08-03T14:58:11  *** Mirobit has joined #bitcoin-core-dev
 312 2017-08-03T14:59:27  *** BashCo has quit IRC
 313 2017-08-03T15:00:32  <Mirobit> Doesn't anyone know BCash nodes aren't sending BCash transactions to Core peers? My node has only recieved 2 BCash tx and banned both peers. But the other ~5-10 peers don't seem to send any txs. Weird?
 314 2017-08-03T15:16:25  *** jr2014 has quit IRC
 315 2017-08-03T15:20:52  *** BashCo has joined #bitcoin-core-dev
 316 2017-08-03T15:30:20  *** snkey has joined #bitcoin-core-dev
 317 2017-08-03T15:33:31  *** snq has quit IRC
 318 2017-08-03T15:41:54  *** jimpo has joined #bitcoin-core-dev
 319 2017-08-03T15:42:00  *** Dizzle has joined #bitcoin-core-dev
 320 2017-08-03T15:52:04  *** Drabiv has quit IRC
 321 2017-08-03T15:52:12  *** Deacydal has quit IRC
 322 2017-08-03T15:52:37  *** Drabiv has joined #bitcoin-core-dev
 323 2017-08-03T15:59:04  <BlueMatt> achow101: re: #10952: Do you have any idea *how* these folks' wallets got corrupted like that?
 324 2017-08-03T15:59:05  <gribble> https://github.com/bitcoin/bitcoin/issues/10952 | [wallet] Remove vchDefaultKey and have better first run detection by achow101 · Pull Request #10952 · bitcoin/bitcoin · GitHub
 325 2017-08-03T15:59:15  <BlueMatt> I'm highly skeptical that "just write a new default key" is the right solution here
 326 2017-08-03T15:59:40  <BlueMatt> their wallet got confused somehow, auto-fixing without telling them something may be wrong is probably not a good idea
 327 2017-08-03T16:05:09  *** abpa has joined #bitcoin-core-dev
 328 2017-08-03T16:12:53  *** sam_c has joined #bitcoin-core-dev
 329 2017-08-03T16:16:49  *** snq has joined #bitcoin-core-dev
 330 2017-08-03T16:19:05  *** snkey has quit IRC
 331 2017-08-03T16:19:50  *** JackH has quit IRC
 332 2017-08-03T16:24:04  *** laurentmt has joined #bitcoin-core-dev
 333 2017-08-03T16:27:22  *** gribble has quit IRC
 334 2017-08-03T16:31:57  *** laurentmt has quit IRC
 335 2017-08-03T16:38:41  *** gribble has joined #bitcoin-core-dev
 336 2017-08-03T16:40:15  *** sam_c has quit IRC
 337 2017-08-03T16:42:17  *** sam_c has joined #bitcoin-core-dev
 338 2017-08-03T16:44:30  *** fizzwont has quit IRC
 339 2017-08-03T16:45:09  *** fizzwont has joined #bitcoin-core-dev
 340 2017-08-03T16:45:32  *** fizzwont is now known as Guest95816
 341 2017-08-03T16:46:57  *** Yogaqueef has joined #bitcoin-core-dev
 342 2017-08-03T16:48:53  *** timothy has quit IRC
 343 2017-08-03T16:55:27  <achow101> BlueMatt: absolutely no idea how they got corrupted
 344 2017-08-03T16:55:40  <achow101> I was not able to get a copy of their wallet files either so I couldn't examine them
 345 2017-08-03T16:56:23  <achow101> BlueMatt: the solution isn't to "just write a new default key". The solution is to change the first run checker from "there's a valid default key" to "there are keys in the wallet"
 346 2017-08-03T16:57:09  <achow101> so with 10952, those wallets would not be considered to be new first run wallets as they have keys but not a valid default key
 347 2017-08-03T17:00:48  *** jtimon has joined #bitcoin-core-dev
 348 2017-08-03T17:06:48  <wumpus> default key should completely go (for post-0.15 though)
 349 2017-08-03T17:08:41  <BlueMatt> achow101: well my point is more broadly that their wallets clearly got corrupted
 350 2017-08-03T17:08:44  <wumpus> the check for a new allet should be replaced by a proper "is this an empty database" check
 351 2017-08-03T17:08:55  <BlueMatt> achow101: so silently continuing isnt really the right solution
 352 2017-08-03T17:09:07  <wumpus> a wallet without any keys yet could in principle be valid
 353 2017-08-03T17:09:26  <wumpus> though a bit strange as we initially generate a mempool
 354 2017-08-03T17:09:34  <karelb> ok we got abc node working, but it was some crazy solution of switching between various binaries and invalidating nodes; it did not crash (yet). But it seems that bitcore patches inside connectblock/disconnectblock are really not the way for the future; it will probably break again (both in abc and later in bitcoin core once you release the db changes)
 355 2017-08-03T17:09:34  <wumpus> (and we don't allow mempoolsize=0)
 356 2017-08-03T17:10:16  <achow101> BlueMatt: that would require a specific check for the case that a default key is invalid
 357 2017-08-03T17:10:38  *** Chris_Stewart_5 has quit IRC
 358 2017-08-03T17:10:57  <achow101> wumpus: a database is initially created when the wallet file is created. I'm not sure how to check it is empty except that when the database is then read in, there are no keys
 359 2017-08-03T17:11:17  <achow101> that's that 10952 does; it checks if there are any keys in the database
 360 2017-08-03T17:11:35  <wumpus> yes, that sounds sane
 361 2017-08-03T17:12:15  <wumpus> for a new format I'd suggest adding a "database-type" "wallet" k/v pair to distinguish it from other bdb databases, and check for that, but for backwards compatibility that works better
 362 2017-08-03T17:12:37  <wumpus> (and no one would be so stupid to put a random other berkeleydb database as wallet.dat? right? :-)
 363 2017-08-03T17:15:16  <Lightsword> anyone working on switching bdb out for something else like sqlite?
 364 2017-08-03T17:15:30  <achow101> Lightsword: there's a pr somewhere for that I think
 365 2017-08-03T17:15:37  <sipa> meh, overkill
 366 2017-08-03T17:15:54  <sipa> all we need is a key-value store that's effectively read into memory entirely anyway
 367 2017-08-03T17:16:22  <sipa> i had a patch years ago to switch it to an append-only flat file
 368 2017-08-03T17:16:27  <Lightsword> the main advantage to sqlite though is that it has good data integrity protection
 369 2017-08-03T17:16:43  <achow101> we could upgrade to bdb whatever latest and change how records are stored there
 370 2017-08-03T17:16:54  <wumpus> Lightsword: it's quite easy to swap the database for any k/v store, I have a local branch that uses leveldb
 371 2017-08-03T17:17:12  <BlueMatt> achow101: if you have a wallet with keys, and a default key that is invalid or missing, your wallet was clearly either corrupted or generated by something other than bitcoin core
 372 2017-08-03T17:17:22  <wumpus> achow101: bdb latest (6.x) has serious license issues
 373 2017-08-03T17:17:22  <BlueMatt> achow101: continuing silently in that case is not what we want
 374 2017-08-03T17:18:35  <Lightsword> sipa, would there not be some potential future use cases where having sql support be helpful?
 375 2017-08-03T17:18:37  <achow101> BlueMatt: if there is corruption in the keys themselves or other data, that will be caught elsewhere
 376 2017-08-03T17:18:48  <BlueMatt> achow101: we can also change the "first run" check to ignore vchDefaultKey, which we clearly should, but you're pointing to wallets in the wild that have been corrupted and suggesting we should silently continue
 377 2017-08-03T17:18:50  <achow101> BlueMatt: I don't think we should care that much about corruption in the default key as it has no use
 378 2017-08-03T17:19:01  <sipa> Lightsword: i don't wee how
 379 2017-08-03T17:19:04  <sipa> *see
 380 2017-08-03T17:19:13  <BlueMatt> achow101: yes we should!
 381 2017-08-03T17:19:18  <achow101> BlueMatt: only one of those wallets would silently continue, and the guy had no problems with older versions of core
 382 2017-08-03T17:19:19  <BlueMatt> the user should stop using that computer for a wallet!
 383 2017-08-03T17:19:27  <wumpus> sql could be useful for metadata kind of things, but meh, I don't think there's really an advantage to it for us we don't index anything
 384 2017-08-03T17:19:31  <achow101> BlueMatt: the other wallet ran into other corruption problems
 385 2017-08-03T17:19:38  <BlueMatt> that user should be told to throw out their hard drive and get a better one
 386 2017-08-03T17:19:39  <BlueMatt> or a new computer
 387 2017-08-03T17:19:41  <wumpus> although the current 'keep everything in memory' is kind of dumb
 388 2017-08-03T17:19:43  <BlueMatt> not continue
 389 2017-08-03T17:20:30  <Lightsword> yeah, I was thinking maybe metadata/multiple accounts or something along those lines could possibly make use of sql
 390 2017-08-03T17:20:33  <wumpus> if the wallet would keep things in the database instead and query them when needed, indexes could be useful
 391 2017-08-03T17:20:48  <wumpus> accounts are deprecated, if anything we're simplifying the wallet
 392 2017-08-03T17:21:44  <achow101> BlueMatt: sure they should be warned, but a corrupted default key should not be a reason to halt the software entirely as default key is useless
 393 2017-08-03T17:21:53  <Lightsword> wumpus, what about multiwallet?
 394 2017-08-03T17:22:03  <sipa> Lightsword: that uses multiple wallet files
 395 2017-08-03T17:22:10  <achow101> with older versions of core, IIRC the default key would have either been overwritten or ignored and allowed the user to continue
 396 2017-08-03T17:22:14  <BlueMatt> achow101: if someone's wallet is corrupted, we should exit with an error....if they wish to then restart with -salvagewallet or equivalent, that is also ok
 397 2017-08-03T17:22:22  <BlueMatt> achow101: and that is a bug!
 398 2017-08-03T17:22:53  <BlueMatt> if we read something and see that someone's hardware is silently corrupting their wallet, we should exit the same way we do with any other wallet corruption errors
 399 2017-08-03T17:23:00  <BlueMatt> silently fixing wallets is not ok
 400 2017-08-03T17:23:30  <sipa> wumpus: well if we expect to ever need indexes on the future because we won't keep everything in memory, i'd say that sqlite is a good choice
 401 2017-08-03T17:23:32  <wumpus> with older versions of core the lack of the default key record would make it assume it's a new wallet, which could do all kinds of bad things? I don't think that's abetter
 402 2017-08-03T17:23:51  <BlueMatt> there are clearly bugs here that achow101 identified (that i think need fixing for 15)
 403 2017-08-03T17:24:00  <sipa> wumpus: not sure if it's worthwhile though
 404 2017-08-03T17:24:03  <BlueMatt> I'm just not sure that silently correcting /anything/ is every good in wallets
 405 2017-08-03T17:24:05  <wumpus> for 0.15 it's too lte imo
 406 2017-08-03T17:24:14  <wumpus> we should do rc1 asap
 407 2017-08-03T17:24:15  <BlueMatt> wumpus: we've seen it in the wild, and it can be a simple fix
 408 2017-08-03T17:24:16  <BlueMatt> :(
 409 2017-08-03T17:24:25  <wumpus> not add new stuff
 410 2017-08-03T17:24:27  <sipa> have we even identified the bug?
 411 2017-08-03T17:24:38  <wumpus> but that's just my opinion (and being harried at all sides to do 0.15 asap)
 412 2017-08-03T17:24:41  <BlueMatt> sipa: as far as anyone knows its hardware/bdb silently corrupting things
 413 2017-08-03T17:24:42  <sipa> as opposed to "we have ween a wallet with no default key, somehow"
 414 2017-08-03T17:24:43  <achow101> sipa: I can replicate the problem, not necessarily the cause
 415 2017-08-03T17:25:20  <wumpus> but sure if it's a clearly defined problem, with a clearly defined fix, and it can be reviewed in the next days, we can include it
 416 2017-08-03T17:25:20  <sipa> achow101: elaborate
 417 2017-08-03T17:25:45  <achow101> sipa: encrypt a wallet, use db_dump to dump it, remove the default key, load with db_load, start core, runtime exception
 418 2017-08-03T17:26:03  <sipa> well, ok
 419 2017-08-03T17:26:04  <wumpus> that's what I expect if you just remove a record
 420 2017-08-03T17:26:10  <wumpus> don't do that.
 421 2017-08-03T17:26:21  <sipa> but if you permit random file changes, you can make anything happen
 422 2017-08-03T17:26:26  <BlueMatt> wumpus: yes, sorry, my understanding is its that kind of "simple fix" that would be easy to get in in a day or two
 423 2017-08-03T17:26:28  <achow101> if you do it unencrypted, it works fine
 424 2017-08-03T17:26:30  <wumpus> if you delete a random record from the utxo database you'll also be in for a world of pain
 425 2017-08-03T17:26:41  *** jimpo has quit IRC
 426 2017-08-03T17:26:59  <BlueMatt> the issue is that this encourages people to downgrade wallets
 427 2017-08-03T17:27:02  <wumpus> we're not resistant to that kind of corruption, and one special case is not going to change that
 428 2017-08-03T17:27:14  <achow101> I suppose then a check for a valid default key can be added and that would just be a corruption warning?
 429 2017-08-03T17:27:15  <BlueMatt> just making it a nice error message and making sure -salvagewallet works correctly is the Correct (tm) fix, imo
 430 2017-08-03T17:27:17  <wumpus> could just as well have been a key record that disappears
 431 2017-08-03T17:27:26  <BlueMatt> achow101: yes
 432 2017-08-03T17:27:32  <achow101> fine
 433 2017-08-03T17:27:34  <wumpus> salvagewallet should work as long as there is any private key left
 434 2017-08-03T17:27:45  <achow101> btw the current corruption warnings are kinda bad
 435 2017-08-03T17:27:55  <wumpus> that's a known issue...
 436 2017-08-03T17:28:18  <wumpus> there's even a chance of running into asserts on db corruption
 437 2017-08-03T17:29:30  <wumpus> would be nice if it could detect corruption at run time and offer the user a chance to repair
 438 2017-08-03T17:29:56  <wumpus> though if a wallet gets corrupted that's a good indication you should stop using that computer for bitcoin, now
 439 2017-08-03T17:30:21  <wumpus> so I agree with BlueMatt in that regard
 440 2017-08-03T17:31:32  <achow101> the main problem I have with that is it encourages people who don't know what they are doing (i.e. don't ask, can't use -salvagewallet, etc.) to downgrade
 441 2017-08-03T17:31:44  <sipa> ?
 442 2017-08-03T17:31:55  <wumpus> why would they think downgrading would work?
 443 2017-08-03T17:32:01  <achow101> wumpus: because it does.
 444 2017-08-03T17:32:04  <wumpus> say, your wallet is suddenly corrupted one day
 445 2017-08-03T17:32:16  <wumpus> why would you consider downgrading?
 446 2017-08-03T17:32:26  <wumpus> only if it happens on upgrade I suppose
 447 2017-08-03T17:32:28  <achow101> with this specific corruption problem, downgrading to non-hd wallet software (0.12.0- or -usehd=0) will ignore this problem
 448 2017-08-03T17:32:31  <sipa> if the corruption ransomly happens right after uograding, i can see why someone would think that
 449 2017-08-03T17:32:42  <sipa> *randomly
 450 2017-08-03T17:32:45  <sipa> not ransomly
 451 2017-08-03T17:32:57  <wumpus> yes, if it happens right after upgrading, but then it's probably a version issue and not a random corruption in any case, would be extremely unlikely otherwise
 452 2017-08-03T17:33:13  <sipa> achow101: update wallet version number?
 453 2017-08-03T17:33:18  <BlueMatt> anyway, this is why i think we should give an error message
 454 2017-08-03T17:33:19  <achow101> sipa: bleh
 455 2017-08-03T17:33:29  <sipa> oh, we can't because the optional stuff uses it
 456 2017-08-03T17:33:40  <sipa> we really need wallet features grr
 457 2017-08-03T17:33:40  <BlueMatt> the crash is better than continuing silently, but if it makes people downgrade, we should instead tell them whats up
 458 2017-08-03T17:34:18  <achow101> IIRC salvagewallet didn't fix their problems either, so that will need to be updated
 459 2017-08-03T17:34:34  <BlueMatt> sipa: arent you the one who added wallet versioning?
 460 2017-08-03T17:34:44  <BlueMatt> achow101: yes, lets do that :)
 461 2017-08-03T17:35:04  <wumpus> wallet versioning is used for *optional* features?
 462 2017-08-03T17:35:13  <BlueMatt> oh, no
 463 2017-08-03T17:35:17  <achow101> I think it would be fine to write some random default key right? it isn't used for anything
 464 2017-08-03T17:35:30  <achow101> wumpus: yes, hd and hd chain split or optional, but they have version numbers
 465 2017-08-03T17:35:38  <BlueMatt> achow101: its used for pay-to-IP :p
 466 2017-08-03T17:36:02  <BlueMatt> (I think)
 467 2017-08-03T17:36:07  <sipa> BlueMatt: yes, and now we can't use it
 468 2017-08-03T17:36:18  <sipa> because the version number is used for optional features
 469 2017-08-03T17:36:59  <sipa> like HD and split HD
 470 2017-08-03T17:37:25  <sipa> those should have been a separate record, rather the version number
 471 2017-08-03T17:37:31  <wumpus> would have been better to use a new key to indicate those (as well as bump the version number, but not in itself)
 472 2017-08-03T17:37:46  <wumpus> well teh version should be bumped too to make it incompatible, but yeah
 473 2017-08-03T17:38:10  <sipa> right
 474 2017-08-03T17:39:05  *** tyngdekraften has joined #bitcoin-core-dev
 475 2017-08-03T17:39:35  <BlueMatt> Make Bitcoin Great Again: Bring back Pay-2-IP
 476 2017-08-03T17:40:31  <sipa> BlueMatt: i tried :(
 477 2017-08-03T17:40:53  <wumpus> with .onion addresses or some other way to authenticate an IP at the transport layer (such as ipsec) it even could work securely!
 478 2017-08-03T17:41:07  <BlueMatt> wumpus: I'm currently working on tcpcrypt for linux patchset :p
 479 2017-08-03T17:41:28  <wumpus> BlueMatt: I have no idea what that is
 480 2017-08-03T17:41:38  <BlueMatt> wumpus: tcp-crypt :p
 481 2017-08-03T17:42:19  <sipa> en..crypt...TCP?
 482 2017-08-03T17:42:24  <Lightsword> http://www.tcpcrypt.org/
 483 2017-08-03T17:42:26  <BlueMatt> yes, that
 484 2017-08-03T17:43:35  <sipa> ah, nice
 485 2017-08-03T17:44:17  *** tyngdekraften has quit IRC
 486 2017-08-03T17:45:18  <wumpus> "and your network connections will continue to work even if the remote end does not support Tcpcrypt, in which case connections will gracefully fall back to standard clear-text TCP" - that screams downgrade attack :)
 487 2017-08-03T17:46:11  <achow101> oh damn, now default key can't be removed :(
 488 2017-08-03T17:46:12  <sipa> wumpus: read on
 489 2017-08-03T17:46:21  <sipa> achow101: ?
 490 2017-08-03T17:46:41  <sipa> wumpus: it only protects against passive attacks
 491 2017-08-03T17:46:44  <achow101> sipa: to check if default key is valid, it needs to be in CWallet
 492 2017-08-03T17:46:52  <sipa> achow101: ?
 493 2017-08-03T17:46:53  *** Chris_Stewart_5 has joined #bitcoin-core-dev
 494 2017-08-03T17:46:56  <BlueMatt> wumpus: yes, if the application layer does nto support it it only protects against passive mitm attacks, if the application layer supports it you can disconnect if tcpcrypt failed (but dont do that, cause middleware sucks)
 495 2017-08-03T17:47:17  <achow101> I need some way to get default key out of database into a checker, which also checks if default key exists
 496 2017-08-03T17:47:28  <sipa> achow101: you can just check when loading the wallet
 497 2017-08-03T17:47:31  <achow101> the only vehicle for that right now is cwallet I think
 498 2017-08-03T17:48:05  <sipa> i'm sure there is or can be a LoadDefaultKey function on CWallet, called from walletdb
 499 2017-08-03T17:48:18  <sipa> switch that to do a check, but not store
 500 2017-08-03T17:48:57  <achow101> I guess I could also just add more parameters too
 501 2017-08-03T17:49:49  *** Mirobit has quit IRC
 502 2017-08-03T18:10:53  *** miknotauro has joined #bitcoin-core-dev
 503 2017-08-03T18:14:29  <earlz> I'm trying to get gitian working with 0.14.0. Has anyone seen an error like this? lxcbr0: ERROR while getting interface flags: No such device
 504 2017-08-03T18:14:32  <earlz> SIOCSIFADDR: No such device
 505 2017-08-03T18:14:57  <earlz> just following the gitian-building.md document exactly and getting that error
 506 2017-08-03T18:18:18  <earlz> ifconfig
 507 2017-08-03T18:18:22  <earlz> er, woops
 508 2017-08-03T18:19:48  <earlz> do I need a kernel flag or something? It's failing at the sudo ifconfig lxcbr0 up 10.0.2.2 bit
 509 2017-08-03T18:20:43  <sipa> do you have the lxc kernel module loaded?
 510 2017-08-03T18:21:20  <earlz> I pretty much just followed the gitian build instructions and I don't see anywhere in them that the kernel module was explicitly installed unless the lxc package does that
 511 2017-08-03T18:24:59  <wumpus> looks like you don't have a lxcbr0 interface for bridging between the lxc and host
 512 2017-08-03T18:25:09  <wumpus> not sure how and where that gets created though
 513 2017-08-03T18:25:37  <earlz> yea, I'm not understanding where that gets created in the guide. Last time I setup gitian it was for Bitcoin 0.9 so not sure what all has changed since then
 514 2017-08-03T18:25:39  *** Mattie161 has joined #bitcoin-core-dev
 515 2017-08-03T18:27:54  <earlz> maybe related: https://github.com/lxc/lxc/issues/1218
 516 2017-08-03T18:28:16  <earlz> LXC seems to be broken in some subtle way every time I try to do a new setup of gitian
 517 2017-08-03T18:28:36  <sipa> wumpus, BlueMatt: i think we really need to fix the versioning issues
 518 2017-08-03T18:28:47  <BlueMatt> wallet versioning?
 519 2017-08-03T18:28:55  <sipa> that would make it so much easier (you could say, past wallet version X, no default key is needed anymore)
 520 2017-08-03T18:29:02  <sipa> i want to move hd to a separate record
 521 2017-08-03T18:29:12  <sipa> rather than store it in the version number
 522 2017-08-03T18:29:29  * BlueMatt is actually confused why
 523 2017-08-03T18:29:41  <sipa> because we can't upgrade the wallet format, ever, now
 524 2017-08-03T18:29:51  <sipa> in a backward incompatible way
 525 2017-08-03T18:29:55  <BlueMatt> version number = backwards-incompatible change (ie hd), new-key is backwards-compatible extra features
 526 2017-08-03T18:29:57  <sipa> without turning the wallet into an hd wallet
 527 2017-08-03T18:30:06  <sipa> yes, that's what it's supposed to be
 528 2017-08-03T18:30:11  <sipa> but HD is a version number
 529 2017-08-03T18:30:11  <BlueMatt> so?
 530 2017-08-03T18:30:23  <sipa> we can't increment the version number to indicate something incompatible now
 531 2017-08-03T18:30:25  <BlueMatt> what do you want to add that needs to upgrade without being hd?
 532 2017-08-03T18:30:34  <sipa> anything
 533 2017-08-03T18:30:38  <BlueMatt> we should implement hd upgrade
 534 2017-08-03T18:30:45  <BlueMatt> thats something we very much need to do anyway
 535 2017-08-03T18:30:52  <sipa> what does that mean?
 536 2017-08-03T18:31:13  <BlueMatt> taking a non-hd wallet and adding an hd key
 537 2017-08-03T18:31:34  <sipa> yes, ok, that too
 538 2017-08-03T18:31:43  <sipa> but do you want to force everyone to have an hd wallet?
 539 2017-08-03T18:31:50  <BlueMatt> (probably also need an hd-key-rotate option, but thats separate and I think not related to hd)
 540 2017-08-03T18:31:52  <BlueMatt> ^
 541 2017-08-03T18:32:07  <BlueMatt> I'm happy to force everyone into an hd wallet if we have an hd-master-key-rotate option
 542 2017-08-03T18:32:16  <sipa> yeah, ok...
 543 2017-08-03T18:32:24  <sipa> but those are all bigger features
 544 2017-08-03T18:32:31  <BlueMatt> well i think all of those are relatively limited code changes
 545 2017-08-03T18:32:44  <BlueMatt> and at least hd-master-key-rotate can happen with no format change, I think
 546 2017-08-03T18:32:52  <sipa> i am talking about 0.15
 547 2017-08-03T18:32:55  <BlueMatt> even if "big" features
 548 2017-08-03T18:33:11  <BlueMatt> do we need to add anything else? why rush the no vchdefaultkey thing?
 549 2017-08-03T18:33:15  <sipa> otherwise we'll complicate things further for ourselves
 550 2017-08-03T18:33:24  <sipa> to add a compatibility layer for split hd
 551 2017-08-03T18:33:37  <BlueMatt> ok, I'm missing something...need context
 552 2017-08-03T18:33:39  <sipa> no, ignore the vchdefaultkey thing
 553 2017-08-03T18:34:07  <sipa> hd and hdsplit, i think, are optional features - things that people can choose not to use
 554 2017-08-03T18:34:32  <BlueMatt> now, yes, but i have no problem with them not being optional in the future
 555 2017-08-03T18:34:36  <sipa> as they break existing wallet's expectation
 556 2017-08-03T18:35:43  <sipa> hmm, ok
 557 2017-08-03T18:36:00  <BlueMatt> not being optional in our traditional -walletupgrade sense, that is
 558 2017-08-03T18:36:40  <BlueMatt> i mean I'm happy for someone to disagree, I just dont see much downside to it (as long as we have an hd-master-key-rotate option and good documentation on it)
 559 2017-08-03T18:36:52  <sipa> i think i agree
 560 2017-08-03T18:37:03  <sipa> except i'm not sure we'll have that feature implemented by the time we need it
 561 2017-08-03T18:37:18  <BlueMatt> well hd-master-key-rotate is ~trivial with today's format
 562 2017-08-03T18:37:24  <sipa> okay
 563 2017-08-03T18:37:29  <BlueMatt> hd-upgrade may be slightly less so, I havent thought about it
 564 2017-08-03T18:37:35  <BlueMatt> but I expect it to be
 565 2017-08-03T18:37:49  <sipa> in that case, i guess it makes sense that those things are in fact in the version number
 566 2017-08-03T18:39:17  <sipa> i just wasn't seeing hd as a 'next version', and more as an optional but recommended feature
 567 2017-08-03T18:40:55  <BlueMatt> i mean its possible i did /because/ of our versioning scheme, but it is simpler to see it as such and there seem to be relatively limited downsides for it, given some code that doesnt exist yet :p
 568 2017-08-03T18:53:31  <wumpus> I'd say using hdsplit isn't really optional, given that you're using hd
 569 2017-08-03T18:54:06  <wumpus> hdsplit is a pure improvement on hd
 570 2017-08-03T18:57:13  <sipa> agree
 571 2017-08-03T18:57:35  <sipa> but do we support upgrading from hd to hdsplit?
 572 2017-08-03T18:57:37  <sipa> (right now)
 573 2017-08-03T18:58:30  <gmaxwell> ugh. how could we except via invalidating backups which people wouldn't expect...
 574 2017-08-03T18:58:49  <wumpus> no, we don't support that right now
 575 2017-08-03T18:59:02  <wumpus> hdsplit was sort-of rushed to make 0.15, so that at least new wallets would use it
 576 2017-08-03T18:59:28  <sipa> so, until we find a way to (forcefully) upgrade non-hd and hd to hdsplit, we should consider hdsplit an optional feature
 577 2017-08-03T18:59:42  <wumpus> but it's strictly superior to hd without hdsplit, so there should be no way to choose that for new wallets
 578 2017-08-03T18:59:54  <gmaxwell> I don't think forcefully upgrading is at all possible, because it will invalidate backups.
 579 2017-08-03T19:00:11  <wumpus> it wouldn't need to be 'forceful' upgrading, just a *way*
 580 2017-08-03T19:00:34  <wumpus> and in any case we don't ever automatically upgrade wallets
 581 2017-08-03T19:00:41  <wumpus> (because of backwards compatibility)
 582 2017-08-03T19:00:46  <gmaxwell> switch to using segwit, and that will upgrade you. :)
 583 2017-08-03T19:00:54  <wumpus> #startmeeting
 584 2017-08-03T19:00:54  <lightningbot> Meeting started Thu Aug  3 19:00:54 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
 585 2017-08-03T19:00:54  <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
 586 2017-08-03T19:01:06  <achow101> hi
 587 2017-08-03T19:01:18  <jonasschnelli> hi
 588 2017-08-03T19:01:20  <sipa> hi
 589 2017-08-03T19:01:24  <wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101
 590 2017-08-03T19:01:36  <kanzure> hi.
 591 2017-08-03T19:01:48  <jtimon> hi
 592 2017-08-03T19:01:49  <cfields> hi
 593 2017-08-03T19:01:50  <wumpus> 0.15.0rc1 is planned for the 6th (next sunday), so let's go over the open issues again
 594 2017-08-03T19:02:50  <wumpus> there's not much anymore
 595 2017-08-03T19:02:57  <wumpus> https://github.com/bitcoin/bitcoin/milestones/0.15.0
 596 2017-08-03T19:03:16  <paveljanik> Hi
 597 2017-08-03T19:03:17  <wumpus> (and the scripted-diffs are option, and should be done at the last minute to not conflict with anything else)
 598 2017-08-03T19:03:46  <wumpus>  Keypool topup #10882  is the most complicated PR open still, but should be almost ready
 599 2017-08-03T19:03:50  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
 600 2017-08-03T19:04:01  <BlueMatt> https://github.com/bitcoin/bitcoin/issues/10977
 601 2017-08-03T19:04:02  <BlueMatt> could go
 602 2017-08-03T19:04:09  <BlueMatt> (makes test_bitcoin valgrind-better)
 603 2017-08-03T19:04:11  <BlueMatt> and is trivial
 604 2017-08-03T19:04:32  <jnewbery> yeah, I've been working on 10882 today. Should be able to push my commits in the next hour or two
 605 2017-08-03T19:04:34  <wumpus> I'm not really fishing for new things to add to 0.15
 606 2017-08-03T19:04:55  <gmaxwell> jnewbery made a suggestion to fix my outstanding concern in review.
 607 2017-08-03T19:05:00  <wumpus> but if there are things that could be merged without affecting anything else that's ok
 608 2017-08-03T19:05:07  <jtimon> after #10758, #10919 seems simple to review, it's only +14-13
 609 2017-08-03T19:05:09  <gribble> https://github.com/bitcoin/bitcoin/issues/10758 | Fix some chainstate-init-order bugs. by TheBlueMatt · Pull Request #10758 · bitcoin/bitcoin · GitHub
 610 2017-08-03T19:05:10  <gribble> https://github.com/bitcoin/bitcoin/issues/10919 | Fix more init bugs. by TheBlueMatt · Pull Request #10919 · bitcoin/bitcoin · GitHub
 611 2017-08-03T19:05:35  <sipa> it's also already marked 0.15
 612 2017-08-03T19:05:35  <cfields> i think there's a one-liner that could be used to fix the issue in 10977, if it's deemed too much of a change
 613 2017-08-03T19:05:41  <gmaxwell> BlueMatt: I'd like to see 10977 fixed! but darn I wish that patch was smaller and easier to review.
 614 2017-08-03T19:05:49  <wumpus> yes, just needs some ACKs
 615 2017-08-03T19:05:55  <BlueMatt> could be smaller, but is easy to review imo
 616 2017-08-03T19:05:57  <sdaftuar> there's one commit in #10919 that i'm hoping others can review/ack
 617 2017-08-03T19:05:59  <gribble> https://github.com/bitcoin/bitcoin/issues/10919 | Fix more init bugs. by TheBlueMatt · Pull Request #10919 · bitcoin/bitcoin · GitHub
 618 2017-08-03T19:07:14  <wumpus> which one?
 619 2017-08-03T19:07:21  <BlueMatt> the first, i believe
 620 2017-08-03T19:07:39  <sdaftuar> yep
 621 2017-08-03T19:08:13  <wumpus> the threadgroup one? seems obviously sane to me, though it does mean things need to be interrupted too
 622 2017-08-03T19:08:31  *** goatpig has joined #bitcoin-core-dev
 623 2017-08-03T19:08:45  <BlueMatt> well i think the point is that there is a comment there that notes we dont do it "because dragons"
 624 2017-08-03T19:09:00  <BlueMatt> i believe strongly that it is safe, and qt does it, but "dragons"
 625 2017-08-03T19:09:04  <wumpus> it is very bad practice not to wait for threads before exiting
 626 2017-08-03T19:09:37  <wumpus> yes, qt does it already, it's somewhat less scared of dragons it seems :)
 627 2017-08-03T19:09:51  <BlueMatt> isnt qt's logo a dragon or something?
 628 2017-08-03T19:10:08  <cfields> heh, think you're thinking of kde
 629 2017-08-03T19:10:14  <BlueMatt> oh, yea
 630 2017-08-03T19:10:22  <wumpus> (e.g. due to qt's handling of shutdown we also needed #10832)
 631 2017-08-03T19:10:24  <gribble> https://github.com/bitcoin/bitcoin/issues/10832 | init: Factor out AppInitLockDataDirectory and fix startup core dump issue by laanwj · Pull Request #10832 · bitcoin/bitcoin · GitHub
 632 2017-08-03T19:10:35  <wumpus> anyhow that commit looks good to me, I don't think there's any dragons left
 633 2017-08-03T19:11:24  <sdaftuar> sounds-good-to-me-ack
 634 2017-08-03T19:12:01  <wumpus> ok, wow, that seems to be all that is left between here and 0.15.0rc1
 635 2017-08-03T19:12:13  <BlueMatt> !
 636 2017-08-03T19:12:19  <cfields> :)
 637 2017-08-03T19:13:10  <morcos> wumpus: i had an assert crash this morning, i imagine it'll be a simple bug.. hopefully i'll have a PR this afternoon, just haven't had time to look at it yet
 638 2017-08-03T19:13:11  <wumpus> (there's another PR #10971 by cfields for fixing depends builds, but I don't think that needs disussion, it's a one-liner in the build system)
 639 2017-08-03T19:13:12  <gribble> https://github.com/bitcoin/bitcoin/issues/10971 | build: fix missing warnings and sse42 in depends builds by theuni · Pull Request #10971 · bitcoin/bitcoin · GitHub
 640 2017-08-03T19:13:45  <wumpus> morcos: ouch, can you open an issue to track it?
 641 2017-08-03T19:13:48  <cfields> yea, nothing major
 642 2017-08-03T19:14:06  <morcos> yes will open one or the other shortly
 643 2017-08-03T19:14:29  <wumpus> ok, thanks
 644 2017-08-03T19:15:06  <wumpus> do we need any updates to bips.md for 0.15?
 645 2017-08-03T19:15:18  <sipa> hmm, good question
 646 2017-08-03T19:15:19  <wumpus> (that's the item in the release process that still has a ? here)
 647 2017-08-03T19:15:46  <BlueMatt> is there a bip that recommends hd split?
 648 2017-08-03T19:15:49  *** chjj has quit IRC
 649 2017-08-03T19:15:53  <sipa> BlueMatt: bip32? :p
 650 2017-08-03T19:16:15  <wumpus> there's also "Update `src/chainparams.cpp` chainTxData with statistics about the transaction count and rate." left
 651 2017-08-03T19:16:29  <sipa> want me to do that?
 652 2017-08-03T19:16:31  <wumpus> and the BLOCK_CHAIN_SIZE, but that's straightforward
 653 2017-08-03T19:16:49  <wumpus> yes, if you know what's exactly to be done there that'd help :)
 654 2017-08-03T19:17:01  <sipa> sure
 655 2017-08-03T19:18:38  <wumpus> thanks
 656 2017-08-03T19:18:55  <sipa> short topic: bip173 unit tests issue
 657 2017-08-03T19:19:09  <wumpus> #topic bip173 unit tests issue
 658 2017-08-03T19:19:13  <jnewbery> There are a few more things for release notes
 659 2017-08-03T19:19:40  *** clarkmoody_ has joined #bitcoin-core-dev
 660 2017-08-03T19:19:43  <sipa> so, bip173 specifies how to translate address strings to witness version/program, and defers to bip141 for encoding that to scriptPubKeys
 661 2017-08-03T19:19:59  <sipa> however, the unit tests actually test the whole step from address to scriptPubKey
 662 2017-08-03T19:20:07  <sipa> turns out, incorrectly
 663 2017-08-03T19:20:26  <sipa> the tests and reference implementation (of the tests) was wrong, and every reimplementation copied it
 664 2017-08-03T19:21:05  <gmaxwell> The the error was that it confused hex and dec values.
 665 2017-08-03T19:21:07  <sipa> i've made a PR to update the BIP, and all reference implementations i could find, but this is kind of scary
 666 2017-08-03T19:21:17  <cfields> corner-cases wrong? or in all cases?
 667 2017-08-03T19:21:24  <wumpus> jnewbery: agreed, but release notes need to be finished before -final, not -rc1, so it's not a blocker
 668 2017-08-03T19:21:36  <sipa> cfields: it assumed OP_n was encoded as 0x80 + n, rather than 80 + n
 669 2017-08-03T19:21:57  <BlueMatt> sipa: so they generate garbage scripts?
 670 2017-08-03T19:22:01  <jnewbery> got it. Thanks wumpus
 671 2017-08-03T19:22:06  <sipa> BlueMatt: the tests, yes
 672 2017-08-03T19:22:26  <sipa> the code itself doesn't contain a conversion to scriptPubKey at all, only a conversion to witness version/program
 673 2017-08-03T19:22:27  <gmaxwell> cfields: It was wrong for witness program versions other than 0
 674 2017-08-03T19:22:34  <cfields> yikes
 675 2017-08-03T19:22:39  <wumpus> oops
 676 2017-08-03T19:22:44  <gmaxwell> so this could happily have been deployed and started causing problems when v1 was used.
 677 2017-08-03T19:22:47  <sipa> but the tests contain a version/program -> scriptPubKey converter in order to be able the test
 678 2017-08-03T19:23:01  *** marcoagner has quit IRC
 679 2017-08-03T19:23:05  <BlueMatt> well if it generated garbage scripts, not much that can be done but fix it
 680 2017-08-03T19:23:13  <BlueMatt> are you asking if we should like change the prefix now?
 681 2017-08-03T19:23:23  <sipa> no
 682 2017-08-03T19:23:28  <sipa> just raising awareness
 683 2017-08-03T19:23:32  <BlueMatt> ok, cool
 684 2017-08-03T19:23:38  <sdaftuar> perhaps an email to the -dev list would also be good?
 685 2017-08-03T19:23:38  <gmaxwell> Also, it highlighes an implementation footgun, I suggested some warning text in the BIP itself.   One protection here is that the particular error in sipas' code would result in non-standard outputs.
 686 2017-08-03T19:23:42  <sipa> sdaftuar: yes
 687 2017-08-03T19:24:06  <gmaxwell> BlueMatt: I did make a suggestion that we consider changing it to break the checksum, but there doesn't appear to be reason to.
 688 2017-08-03T19:24:16  *** clarkmoody has quit IRC
 689 2017-08-03T19:24:18  <BlueMatt> ok
 690 2017-08-03T19:24:22  *** jonasschnelli has quit IRC
 691 2017-08-03T19:24:24  <morcos> just to be clear what we are talking about, we're not talking about anything merged into Core, but code referenced from the BIP
 692 2017-08-03T19:24:25  <BlueMatt> awareness raised!
 693 2017-08-03T19:24:27  <gmaxwell> Especially since if someone had slavishly reimplemented the error in the reference, they'd produce non-standard outputs.
 694 2017-08-03T19:24:31  <sipa> morcos: indeed.
 695 2017-08-03T19:24:52  <morcos> still, a bit scary
 696 2017-08-03T19:25:06  <sipa> (though i'd like to PR it to core soon - apparently last week it was suggested to do that in 0.15.1?)
 697 2017-08-03T19:25:21  <gmaxwell> Don't start a debate about the name of the version. :P
 698 2017-08-03T19:25:25  <sipa> haha
 699 2017-08-03T19:25:42  <sipa> (though i'd like to PR it to core soon - apparently last week it was suggested to do that in some soon next version)
 700 2017-08-03T19:25:48  *** jonasschnelli has joined #bitcoin-core-dev
 701 2017-08-03T19:25:51  <gmaxwell> It also suggests that BIP173 support's test plan should include testing it with other witness version numbers. :)
 702 2017-08-03T19:25:51  <sdaftuar> prs welcome :)
 703 2017-08-03T19:26:11  <gmaxwell> sipa: well fix the testing shortfalls I found in the C++ version please. :)
 704 2017-08-03T19:26:17  <wumpus> PRs to improve the tests are always welcome anyhow
 705 2017-08-03T19:26:17  <sipa> gmaxwell: of course
 706 2017-08-03T19:26:23  <sipa> anyway, end topic
 707 2017-08-03T19:26:28  <wumpus> ok, other topics?
 708 2017-08-03T19:26:32  *** jonasschnelli has quit IRC
 709 2017-08-03T19:26:32  *** jonasschnelli has joined #bitcoin-core-dev
 710 2017-08-03T19:28:04  <gmaxwell> uh
 711 2017-08-03T19:28:05  <gmaxwell> yes.
 712 2017-08-03T19:28:14  <gmaxwell> So service bits and altcoins.
 713 2017-08-03T19:28:21  <wumpus> #topic service bits and altcoins
 714 2017-08-03T19:28:29  <BlueMatt> wait are altcoins using serice bits now?
 715 2017-08-03T19:28:41  <BlueMatt> oh, right 2x did
 716 2017-08-03T19:28:43  <gmaxwell> Bcash is using our port, p2pmagic, etc. They distinguish themselve with a blinking service bit.
 717 2017-08-03T19:28:44  <BlueMatt> what is wrong with people
 718 2017-08-03T19:28:54  <gmaxwell> (also 2x will do this too it seems)
 719 2017-08-03T19:29:03  <BlueMatt> gmaxwell: can someone open a pr to change this? or do they refuse to work properly?
 720 2017-08-03T19:29:04  <cfields> gmaxwell: i was under the impression that they were planning to change the magic soon
 721 2017-08-03T19:29:09  <gmaxwell> We mostly ban them when they send us transactions or headers.
 722 2017-08-03T19:29:19  <gmaxwell> cfields: not when I looked three days ago, maybe now.
 723 2017-08-03T19:29:36  <karelb> OK, maybe I will ask here. What format are the bitcoin .dat files in data/blocks/*.dat? is that leveldb? what is it exactly?
 724 2017-08-03T19:29:47  <jonasschnelli> karelb: meeting, not now
 725 2017-08-03T19:29:47  <gmaxwell> If so then the issue becomes moot, otherwise I was going to suggest we ban these bits on connect. The downside is we lose the bits basically forever.
 726 2017-08-03T19:29:48  <sipa> they are p2p network format
 727 2017-08-03T19:29:51  <karelb> ok sorry
 728 2017-08-03T19:29:52  <sipa> oops, yes, layer
 729 2017-08-03T19:30:02  <karelb> sorry going out
 730 2017-08-03T19:30:14  * karelb apologizes
 731 2017-08-03T19:30:17  <BlueMatt> gmaxwell: yes, first should be someone bludgening them to work properly
 732 2017-08-03T19:30:23  <BlueMatt> gmaxwell: before we start burning service bits
 733 2017-08-03T19:30:44  <gmaxwell> BlueMatt: people have been since before they started. Obviously I'll go monitor but I'm not super confident.
 734 2017-08-03T19:30:52  <sdaftuar> gmaxwell: why not just ban for eg the next 3 months or something?
 735 2017-08-03T19:30:55  <achow101> BlueMatt: gmaxwell IIRC they will be changing their magic and port
 736 2017-08-03T19:30:56  <sdaftuar> if we need to do anything at all
 737 2017-08-03T19:31:03  <achow101> dunno about 2x
 738 2017-08-03T19:31:04  <gmaxwell> One reason burning service bits may not be so bad is because we are due to replace the addr message for i2p and NG-HS support.
 739 2017-08-03T19:31:04  <BlueMatt> achow101: what about 2x?
 740 2017-08-03T19:31:22  <gmaxwell> So we could at that point just define a new service flagging mechenism.
 741 2017-08-03T19:31:26  <BlueMatt> gmaxwell: yea, does anyone have a spec for that?
 742 2017-08-03T19:31:33  <gmaxwell> Not yet.
 743 2017-08-03T19:31:36  <BlueMatt> k
 744 2017-08-03T19:31:51  <gmaxwell> Well if they're finally going to change it, it becomes moot.. but the same issue arises with 2x.
 745 2017-08-03T19:32:16  <wumpus> how would a service bit help here?
 746 2017-08-03T19:32:19  <BlueMatt> well someone needs to bludgen the 2x folks to change it, otherwise we start banning for a few months
 747 2017-08-03T19:32:20  <wumpus> just ban everyone without NODE_SEGWIT? :p
 748 2017-08-03T19:32:37  <gmaxwell> wumpus: we still want to support non-upgraded nodes.
 749 2017-08-03T19:32:48  <wumpus> but they won't have any new version bit either
 750 2017-08-03T19:32:58  <wumpus> that was my point, not to suggest that seriously :)
 751 2017-08-03T19:33:11  <gmaxwell> wumpus: oh no, you misunderstand: ABC and 2x both set randomly generated service bits.
 752 2017-08-03T19:33:23  <cfields> gmaxwell: eh?
 753 2017-08-03T19:33:26  <BlueMatt> I think sdaftuar's suggestion is reasonable, assuming they refuse to do something sane
 754 2017-08-03T19:33:27  <gmaxwell> (which they've helpfully ignored the gigantic comment in the code that tells you to at least inform the list.)
 755 2017-08-03T19:33:44  <cfields> oh
 756 2017-08-03T19:33:45  <gmaxwell> sdaftuar: I hadn't considered a time limited ban. Good point.
 757 2017-08-03T19:33:46  <wumpus> oh you mean banning everything that sets their version bit?
 758 2017-08-03T19:33:53  <wumpus> yes, that'd be doable
 759 2017-08-03T19:33:56  <BlueMatt> wumpus: yes, with a time limit
 760 2017-08-03T19:33:59  <gmaxwell> wumpus: well disconnecting, not banning.
 761 2017-08-03T19:34:06  <BlueMatt> nah, ban for 3 months
 762 2017-08-03T19:34:08  <gmaxwell> Okay thanks, Time limit makes sense. duh.
 763 2017-08-03T19:34:16  <wumpus> temporarily, yes
 764 2017-08-03T19:34:24  <morcos> wumpus: opened issue #10981, easy fix, but i'll let someone else do the PR as i'm not in the office for next week.  please tag with 0.15.
 765 2017-08-03T19:34:25  <gribble> https://github.com/bitcoin/bitcoin/issues/10981 | resendwallettransactions asserts if walletbroadcast=0 · Issue #10981 · bitcoin/bitcoin · GitHub
 766 2017-08-03T19:34:32  <gmaxwell> BlueMatt: banning creates problems when you run multiple things on one machine.
 767 2017-08-03T19:34:40  <BlueMatt> gmaxwell: meh
 768 2017-08-03T19:34:40  <wumpus> morcos: thanks, will tag later (not logged in now)
 769 2017-08-03T19:35:03  <BlueMatt> gmaxwell: they refused to do something that wasnt astoundingly broken, if it means their users get fucked, its not really my problem
 770 2017-08-03T19:35:13  <wumpus> (or if someone else can do it)
 771 2017-08-03T19:35:21  <morcos> BlueMatt: so chaincode ip will be banned.  nice.
 772 2017-08-03T19:35:37  <BlueMatt> morcos: -connect=altcoin.dns.seed
 773 2017-08-03T19:35:38  <BlueMatt> :)
 774 2017-08-03T19:36:07  <wumpus> agree that banning goes too far, just not allow connections
 775 2017-08-03T19:36:13  <sipa> maye just disconnect?
 776 2017-08-03T19:36:15  <wumpus> there's no point in banning everything after that
 777 2017-08-03T19:36:16  <gmaxwell> what? no. matt, doing that will ban Bitcoin Core users when someone on the same IP ran crapware.
 778 2017-08-03T19:36:19  <jtimon> perhaps better for after the meeting, but I'm still not sure why #8498 wasn't suitable for 0.15 ...
 779 2017-08-03T19:36:21  <gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
 780 2017-08-03T19:36:55  <gmaxwell> To be clear this is important because these useless altcoin nodes waste connection slots, and are potentially at risk of gobbling up our initial headers fetch.
 781 2017-08-03T19:36:57  <BlueMatt> gmaxwell: ugh
 782 2017-08-03T19:37:04  <BlueMatt> fine, disconnect
 783 2017-08-03T19:37:15  <BlueMatt> at some point someone is gonna create some altcoin crapware that reconnects agressively, though
 784 2017-08-03T19:37:16  <wumpus> disconnect up until a certain date
 785 2017-08-03T19:37:20  <BlueMatt> some spv forks probably will
 786 2017-08-03T19:37:23  <wumpus> what would be that point?
 787 2017-08-03T19:37:33  <BlueMatt> because crapware
 788 2017-08-03T19:37:34  <wumpus> it would disconnect immediately after the version message
 789 2017-08-03T19:37:37  <gmaxwell> Also, looks like ABC has some kind of deadlocking bug, because I see a few of them just going unresponsive to anything but pings, which delays them getting banned for being on the wrong network.
 790 2017-08-03T19:37:38  <morcos> +1 disconnect up to certain date, but i think it should be 12 mos and not 3
 791 2017-08-03T19:37:41  <BlueMatt> do not make assumptions about crapware working in a sane way
 792 2017-08-03T19:37:42  <wumpus> banning would ban 1 message sooner
 793 2017-08-03T19:37:47  <morcos> no reason we'll need that next service bit right away
 794 2017-08-03T19:37:59  <sdaftuar> morcos: sure, that sounds fine
 795 2017-08-03T19:38:04  <wumpus> s/ban/disconnect
 796 2017-08-03T19:38:05  <BlueMatt> morcos: seems reasonable
 797 2017-08-03T19:38:19  <gmaxwell> ack on disconnect based on service bits for 12months or similar.
 798 2017-08-03T19:38:38  <wumpus> unless we start adding banned nodes to the local firewall, there's no serious difference between disconnecting on connect or after the version message
 799 2017-08-03T19:38:51  <gmaxwell> though in general, one of these clowns is going to squat service buts we're in the process of trying to use. :( I have no suggestion on dealing with that.
 800 2017-08-03T19:39:12  *** CubicEarth has joined #bitcoin-core-dev
 801 2017-08-03T19:39:12  <morcos> but i think it'd be worth a quick email/github message to jgarzik to check that they aren't imminently changing their plan
 802 2017-08-03T19:39:14  <BlueMatt> lets deal with that when we get there
 803 2017-08-03T19:39:16  <gmaxwell> s/buts/bits/
 804 2017-08-03T19:39:23  <wumpus> well if they change the magic and port we can stop worrying about the service bits they claim
 805 2017-08-03T19:39:31  <BlueMatt> morcos: yes, as I stated previously we should tell these guys to change network magic *first*
 806 2017-08-03T19:39:33  <wumpus> also we could at that point check for NODE_SEGWIT + our service bit
 807 2017-08-03T19:39:34  <morcos> yes but what if they change to a different service bit
 808 2017-08-03T19:39:41  <gmaxwell> morcos: there is some PR where people have been arguing for ages, about this, so I'm doubtful but sure.
 809 2017-08-03T19:39:54  <morcos> might as well ask first and tell him what we're planning on doing
 810 2017-08-03T19:39:57  <wumpus> change to a different version bit? what would that accomplish?
 811 2017-08-03T19:40:07  <morcos> whooo knows?!!
 812 2017-08-03T19:40:11  <gmaxwell> At the end we're doing them a favor, there are a lot more bitcoin nodes than random altcoin nodes, so these incorrect connections tend to cause them a lot more problems than us.
 813 2017-08-03T19:40:22  <BlueMatt> yup
 814 2017-08-03T19:40:27  <BlueMatt> ok, probalem solved
 815 2017-08-03T19:40:34  <BlueMatt> who wants to go tell them that we're gonna disconnect them?
 816 2017-08-03T19:40:39  <wumpus> if avoiding detection is the point, they could better stop setting their version bti at that point is better than randomly cycling it according to moon phases
 817 2017-08-03T19:41:00  <gmaxwell> BlueMatt: perhaps we should just open the disconnect PR and ping them to comment on it?
 818 2017-08-03T19:41:06  <wumpus> bleh
 819 2017-08-03T19:41:15  <BlueMatt> gmaxwell: wfm, but seems like someone should open an issue
 820 2017-08-03T19:41:18  <BlueMatt> I vote morcos does it
 821 2017-08-03T19:41:39  <gmaxwell> throw him to the wolves... enh? what did he do so wrong?
 822 2017-08-03T19:41:42  <BlueMatt> actually, it was sdaftuar's idea, he can do it
 823 2017-08-03T19:41:49  <gmaxwell> throw him to the wolves... enh? what did he do so wrong?
 824 2017-08-03T19:42:04  <wumpus> seems we'd rather not invite certain discussions to our github but eh
 825 2017-08-03T19:42:25  <BlueMatt> gmaxwell: fine, I'll deal with it
 826 2017-08-03T19:42:35  <gmaxwell> BlueMatt: good, I know what you did so wrong. :P
 827 2017-08-03T19:43:04  <jtimon> but if the bits are selected randomly, how does burning them help?
 828 2017-08-03T19:43:34  <sipa> jtimon: s/randomly/arbitrarily/
 829 2017-08-03T19:43:38  <wumpus> they aren't selected randomly, they're not doing service bit hopping or something like that
 830 2017-08-03T19:43:42  <sipa> they're not different every time
 831 2017-08-03T19:43:50  <sipa> they just arbitrarily picked one
 832 2017-08-03T19:43:50  <jtimon> sipa: I see, thanks
 833 2017-08-03T19:43:54  <gmaxwell> jtimon: I don't follow your question. The altcoin efforts have selected randomly but hardcoded the result or their fair dice roll. :)
 834 2017-08-03T19:44:13  <morcos> +1 sdaftuar doing it...  i'm trying to pack
 835 2017-08-03T19:44:13  <jtimon> gmaxwell: yeah, got it
 836 2017-08-03T19:44:16  <gmaxwell> and just failed to follow the giant comment in the code to make a public announcement about it even.
 837 2017-08-03T19:44:22  <wumpus> e.g. they have monkeys throw darts to select one when they need it, not every connection
 838 2017-08-03T19:44:51  <morcos> oh nm, or bluematt
 839 2017-08-03T19:45:22  <BlueMatt> I think morcos is clearly just trying to throw anyone else under the bus, sounds like he should do it, then :p
 840 2017-08-03T19:45:25  <BlueMatt> anyway, next topic?
 841 2017-08-03T19:45:49  <gmaxwell> ;;action bluematt goes under the bus
 842 2017-08-03T19:45:50  * gribble bluematt goes under the bus
 843 2017-08-03T19:46:02  <gmaxwell> see, the robot overlords agree
 844 2017-08-03T19:46:04  <BlueMatt> ;;action goes under the bus
 845 2017-08-03T19:46:04  * gribble goes under the bus
 846 2017-08-03T19:46:17  <achow101> we can't/shouldn't ban 2x peers until they fork
 847 2017-08-03T19:46:23  <BlueMatt> achow101: yes we should
 848 2017-08-03T19:46:37  *** jonasschnelli has quit IRC
 849 2017-08-03T19:46:40  <wumpus> I think this was mainly about BCC which already forked
 850 2017-08-03T19:46:51  <achow101> BlueMatt: why? we won't be giving them invalid stuff until they fork, and vice versa
 851 2017-08-03T19:47:13  <gmaxwell> hash that out outside of the meeting plz.
 852 2017-08-03T19:47:24  <wumpus> achow101: on the other hand, adding logic to the code to check whether they've forked would complicate things more than just disconnecting on a service bit
 853 2017-08-03T19:47:29  <wumpus> yeah
 854 2017-08-03T19:47:40  <BlueMatt> next topic?
 855 2017-08-03T19:47:46  <gmaxwell> But in general if someone is going to make broken software, we can only go so far to accomidate it.
 856 2017-08-03T19:47:49  <wumpus> we've run out of topics
 857 2017-08-03T19:48:00  <achow101> wumpus: we know when they activate. at block X start banning them
 858 2017-08-03T19:48:18  <gmaxwell> I doubt we do.
 859 2017-08-03T19:48:31  <jtimon> perhaps better for after the meeting, but I'm still not sure why #8498 wasn't suitable for 0.15 ...
 860 2017-08-03T19:48:33  <BlueMatt> achow101: I'm not jumping through hoops to make sure altcoins stay in consensus until *right before* they fork...
 861 2017-08-03T19:48:33  <gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
 862 2017-08-03T19:48:33  <wumpus> #endmeeting
 863 2017-08-03T19:48:33  <lightningbot> Meeting ended Thu Aug  3 19:48:33 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
 864 2017-08-03T19:48:33  <lightningbot> Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-08-03-19.00.html
 865 2017-08-03T19:48:33  <lightningbot> Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-08-03-19.00.txt
 866 2017-08-03T19:48:33  <lightningbot> Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-08-03-19.00.log.html
 867 2017-08-03T19:48:36  <gmaxwell> since they still don't have a correct published specification and keep changing things.
 868 2017-08-03T19:48:53  <achow101> gmaxwell: they have stayed on using 12960 blocks after segwit activation
 869 2017-08-03T19:50:29  <gmaxwell> achow101: and what happens if they pull it up to 0 blocks, with a weeks notice? They're clearly being adversarial and trying to harm things, otherwise they'd change port/magic.
 870 2017-08-03T19:50:37  <BlueMatt> achow101: and what if they change it next week? No point, if they want to make sure they stay in consensus, they can run bridge nodes, its not hard
 871 2017-08-03T19:51:41  <achow101> we risk partitioning the network since there are miners that use btc1
 872 2017-08-03T19:52:05  *** jonasschnelli has joined #bitcoin-core-dev
 873 2017-08-03T19:52:07  <BlueMatt> achow101: that can be remedied otherwise
 874 2017-08-03T19:52:09  <BlueMatt> (and will be)
 875 2017-08-03T19:52:22  <BlueMatt> and, mostly, hopefully, they just change network magic
 876 2017-08-03T19:52:27  <BlueMatt> and then they can do whatever they want
 877 2017-08-03T19:53:07  *** SopaXorzTaker has quit IRC
 878 2017-08-03T19:54:05  <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #10982: Disconnect network service bits 6 and 8 until Aug 1, 2018 (master...2017-08-bad-service-bits) https://github.com/bitcoin/bitcoin/pull/10982
 879 2017-08-03T19:57:13  *** chjj has joined #bitcoin-core-dev
 880 2017-08-03T19:59:55  <jnewbery> gmaxwell BlueMatt new commits in #10882 ready for review
 881 2017-08-03T19:59:57  <gribble> https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
 882 2017-08-03T20:02:58  <BlueMatt> thanks
 883 2017-08-03T20:08:55  *** Cobra-Bitcoin has joined #bitcoin-core-dev
 884 2017-08-03T20:09:24  <Cobra-Bitcoin> Any core dev around?
 885 2017-08-03T20:09:25  *** Dizzle has quit IRC
 886 2017-08-03T20:09:34  <sipa> many
 887 2017-08-03T20:12:29  <Cobra-Bitcoin> Anyone know what btcdrak means when he says "future releases will be linked" here https://github.com/bitcoin-core/bitcoincore.org/issues/33?
 888 2017-08-03T20:12:49  <Cobra-Bitcoin> So the release notes will point users to bitcoincore.org binaries?
 889 2017-08-03T20:15:10  <sipa> BlueMatt: ^
 890 2017-08-03T20:17:52  <BlueMatt> Cobra-Bitcoin: that is the intention, yes, I mailed you about this a while back :). I think the goal is to start pointing people to bitcoincore.org, but also sicne y'all wanted to keep mirroring on bitcoin.org (and cause no point telling people to go somewhere else if they're already used to bitcoin.org), they'd be in both places
 891 2017-08-03T20:18:16  <BlueMatt> ie cause security isnt helped by making people make hops
 892 2017-08-03T20:18:51  <BlueMatt> Cobra-Bitcoin: see-also #10651
 893 2017-08-03T20:18:53  <gribble> https://github.com/bitcoin/bitcoin/issues/10651 | Verify binaries from bitcoincore.org and bitcoin.org by TheBlueMatt · Pull Request #10651 · bitcoin/bitcoin · GitHub
 894 2017-08-03T20:23:02  <Cobra-Bitcoin> But is bitcoin.org not doing enough to already distribute the binaries effectively?
 895 2017-08-03T20:23:49  <Cobra-Bitcoin> We have no choice but to mirror, since we have a lot of pages structured around the binaries and Core
 896 2017-08-03T20:25:21  <achow101> wumpus: BlueMatt so what do we do about salvagewallet and this default key thing?
 897 2017-08-03T20:25:51  <achow101> write a new one?
 898 2017-08-03T20:27:11  *** snq has quit IRC
 899 2017-08-03T20:27:17  *** snq has joined #bitcoin-core-dev
 900 2017-08-03T20:27:37  <BlueMatt> Cobra-Bitcoin: see pm
 901 2017-08-03T20:27:59  <BlueMatt> achow101: I havent looked at salvage, I mean certainly making it return an error instead of throwing like it does is not too hard
 902 2017-08-03T20:28:10  <wumpus> Cobra-Bitcoin: hosting on both bitcoin.org and bitconcore.org is perfectly acceptable, preferable even IMO
 903 2017-08-03T20:28:25  <achow101> BlueMatt: I have it return a db corrupt error now (local change, not yet pushed)
 904 2017-08-03T20:28:33  <wumpus> Cobra-Bitcoin: more (trusted) places to get the binaries from gives some redundancy
 905 2017-08-03T20:28:41  <BlueMatt> plus what wumpus said
 906 2017-08-03T20:29:23  <achow101> but my concern is that people will downgrade because downgrading works. there's no way to prevent downgrade is to use a new wallet version, but that's a mess that I don't want to deal with right now
 907 2017-08-03T20:30:02  <BlueMatt> achow101: well a reasonable error message should help, then
 908 2017-08-03T20:30:31  <BlueMatt> I dont care about preventing users from doing X or Y, but giving them an error message informing them that their hardware appears to have silently corrupted their wallet is appropriate
 909 2017-08-03T20:30:50  <BlueMatt> ideally salvagewallet would handle this case (does it not, my understanding was it should?)
 910 2017-08-03T20:31:16  <achow101> currently it says "Error loading wallet.dat: Wallet corrupted" with a more specific "invalid default key" message in the debug.log
 911 2017-08-03T20:31:27  <BlueMatt> thats fine?
 912 2017-08-03T20:31:40  <achow101> salvagewallet doesn't handle uncorrupting keys, it only pulls data that it can find being valid and passes it through
 913 2017-08-03T20:31:52  <achow101> so any corrupted keys will get passed through to a salvaged wallet
 914 2017-08-03T20:32:02  <achow101> (not like you could uncorrupt a key anyways)
 915 2017-08-03T20:32:06  <wumpus> is there even such a thing as 'uncorrupting keys'?
 916 2017-08-03T20:32:19  *** Cobra-Bitcoin has quit IRC
 917 2017-08-03T20:32:22  <wumpus> it's not like we encode in some redundant way that allows recovery
 918 2017-08-03T20:32:29  <wumpus> we could do that, but we don't
 919 2017-08-03T20:32:34  <achow101> wumpus: well you could easily uncorrupt a default key by just writing a generic valid key like the generator
 920 2017-08-03T20:32:54  <wumpus> you could erase/ignore corrupted keys, though that's somewhat scary...
 921 2017-08-03T20:33:05  <BlueMatt> wumpus: isnt salvagewallet always scary?
 922 2017-08-03T20:33:09  <wumpus> (but not much different from what bdb salvage already does)
 923 2017-08-03T20:33:18  <wumpus> yes, sure, but this isn't black and white
 924 2017-08-03T20:33:31  <BlueMatt> true
 925 2017-08-03T20:33:34  <wumpus> there are certainly ways to make it even scarier :)
 926 2017-08-03T20:33:35  <sipa> so there is an added point here: any key that got corrupted ever probably resulted in the wallet just failing, and the user starting over or finding other solution
 927 2017-08-03T20:33:54  <sipa> but if it was the default key that was corrupted, it likely just meant that the wallet replenished the keypool at every startup, ever
 928 2017-08-03T20:34:02  <sipa> and everything kept working
 929 2017-08-03T20:34:05  <wumpus> yes
 930 2017-08-03T20:34:26  <sipa> so there may be a survivalship bias, resulting in currently a higher than expected number of people with a corrupted default key
 931 2017-08-03T20:34:46  <wumpus> there probably should be a way to salvage and ignore corrupt keys...
 932 2017-08-03T20:34:53  <sipa> yes.
 933 2017-08-03T20:35:03  <wumpus> (another salvage level!)
 934 2017-08-03T20:35:17  <sipa> unfortunately, not possible for encrypted keys
 935 2017-08-03T20:35:24  <sipa> at least not without passphrase
 936 2017-08-03T20:35:35  <wumpus> yeah...
 937 2017-08-03T20:35:48  <wumpus> repair for an encrypted wallet should ideally ask for the passphrase
 938 2017-08-03T20:36:00  <wumpus> it's the only way to recover most effectively
 939 2017-08-03T20:38:52  <achow101> it also seems like the two reported cases of this involve wallets that have corrupted keys, which salvagewallet doesn't help with
 940 2017-08-03T20:39:14  <achow101> it's just that the user doesn't notice those keys were corrupted until they try to spend
 941 2017-08-03T20:39:18  *** davec has quit IRC
 942 2017-08-03T20:39:26  <achow101> then the wallet fails to decrypt
 943 2017-08-03T20:40:05  *** davec has joined #bitcoin-core-dev
 944 2017-08-03T20:42:55  <wumpus> yes that's not good
 945 2017-08-03T20:44:56  <achow101> so i guess we could just leave this as a warning about wallet corruption and not do any salvaging which would exacerbate the problem?
 946 2017-08-03T20:45:41  *** Guest95816 is now known as fizzwont
 947 2017-08-03T20:45:48  *** fizzwont has joined #bitcoin-core-dev
 948 2017-08-03T20:47:42  *** dgenr8 has quit IRC
 949 2017-08-03T20:57:00  *** Dizzle has joined #bitcoin-core-dev
 950 2017-08-03T21:00:24  <wumpus> agreed, any 'salvaging' should be at user initiative
 951 2017-08-03T21:17:22  *** jeep-ss has quit IRC
 952 2017-08-03T21:26:02  *** Giszmo1 has quit IRC
 953 2017-08-03T21:26:30  *** Giszmo has joined #bitcoin-core-dev
 954 2017-08-03T21:32:49  *** elias19r has joined #bitcoin-core-dev
 955 2017-08-03T21:33:11  *** elias19r has left #bitcoin-core-dev
 956 2017-08-03T21:38:49  *** chjj has quit IRC
 957 2017-08-03T21:42:32  *** Guyver2 has quit IRC
 958 2017-08-03T21:50:19  *** chjj has joined #bitcoin-core-dev
 959 2017-08-03T22:00:57  *** spudowiar has quit IRC
 960 2017-08-03T22:10:12  *** Chris_Stewart_5 has quit IRC
 961 2017-08-03T22:14:52  <earlz> Is there any easy way to make gitian skip building dependencies? I'm having some trouble with a modification I made breaking the gitian build for bitcoind itself, but the 2 hours spent compiling dependencies is killing me
 962 2017-08-03T22:16:20  *** ekerstein has joined #bitcoin-core-dev
 963 2017-08-03T22:19:55  *** fizzwont has quit IRC
 964 2017-08-03T22:20:42  *** fizzwont has joined #bitcoin-core-dev
 965 2017-08-03T22:21:05  *** fizzwont is now known as Guest17799
 966 2017-08-03T22:21:32  <cfields> earlz: the dependencies are cached, they'll only be built once
 967 2017-08-03T22:21:36  *** Guest17799 is now known as fizzwont
 968 2017-08-03T22:21:45  *** fizzwont has joined #bitcoin-core-dev
 969 2017-08-03T22:25:06  *** LampTreadStone07 has joined #bitcoin-core-dev
 970 2017-08-03T22:26:33  *** Drabiv has quit IRC
 971 2017-08-03T22:27:37  <earlz> cfields: that does not seem to be the case if gbuild fails
 972 2017-08-03T22:29:50  <earlz> is there some command line argument or something I'm missing to prevent it from saving the cache, cause right now I basiclaly do gbuild, it compiles all deps, then encounters failure, I'll apply a fix and commit/push it, then update the gbuild command for the new commit hash and I observe it compiles all deps again
 973 2017-08-03T22:30:00  <cfields> earlz: do a vanilla build, make sure it finishes, get the deps cached
 974 2017-08-03T22:30:03  <cfields> then rebuild with your changes
 975 2017-08-03T22:30:18  <earlz> oh I see, so it only caches if the build succeeds?
 976 2017-08-03T22:30:40  <cfields> yes
 977 2017-08-03T22:31:37  *** Mattie161 has quit IRC
 978 2017-08-03T22:35:05  *** Dizzle has quit IRC
 979 2017-08-03T22:48:43  <bitcoin-git> [bitcoin] TheBlueMatt opened pull request #10984: Allow 2 simultaneous (compact-)block downloads (master...2017-08-paralell-block-downloads) https://github.com/bitcoin/bitcoin/pull/10984
 980 2017-08-03T23:05:30  <BlueMatt> cfields: when you do the next round of cleanups on #10756, can you not move InitializeNode/FinalizeNode? I'm starting work on building on top and have patches that conflict :/
 981 2017-08-03T23:05:32  <gribble> https://github.com/bitcoin/bitcoin/issues/10756 | net processing: swap out signals for an interface class by theuni · Pull Request #10756 · bitcoin/bitcoin · GitHub
 982 2017-08-03T23:06:37  <cfields> BlueMatt: heh, i thought we agreed that was the next step?
 983 2017-08-03T23:06:52  <BlueMatt> cfields: nono, i mean dont move the code down in the file
 984 2017-08-03T23:06:54  <BlueMatt> not dont move
 985 2017-08-03T23:07:28  *** chjj has quit IRC
 986 2017-08-03T23:08:45  <cfields> BlueMatt: oh, I see what you mean
 987 2017-08-03T23:08:54  <cfields> BlueMatt: they have to move out of the anon namespace though :(
 988 2017-08-03T23:09:18  *** d_t has joined #bitcoin-core-dev
 989 2017-08-03T23:09:30  *** ekerstein has quit IRC
 990 2017-08-03T23:09:46  <BlueMatt> ohh, didnt realize they were in a namespace
 991 2017-08-03T23:09:48  <BlueMatt> damn, ok, fine
 992 2017-08-03T23:09:54  <BlueMatt> I'll just have to do some dirty rebase
 993 2017-08-03T23:10:08  <cfields> yea, that's the only reason i moved them
 994 2017-08-03T23:10:39  <cfields> could do something ugly like closing/reopening the namespace, but i'd rather not :)
 995 2017-08-03T23:19:27  *** chjj has joined #bitcoin-core-dev
 996 2017-08-03T23:23:09  *** laurentmt has joined #bitcoin-core-dev
 997 2017-08-03T23:23:55  *** laurentmt has quit IRC
 998 2017-08-03T23:26:15  <bitcoin-git> [bitcoin] sipa opened pull request #10985: Add undocumented -forcecompactdb to force LevelDB compactions (master...20170803_forcecompactdb) https://github.com/bitcoin/bitcoin/pull/10985
 999 2017-08-03T23:39:03  *** Yogaqueef has quit IRC
1000 2017-08-03T23:40:54  <bitcoin-git> [bitcoin] sipa opened pull request #10986: Update chain transaction statistics (master...20170803_txstats) https://github.com/bitcoin/bitcoin/pull/10986
1001 2017-08-03T23:47:29  <gmaxwell> #10985: \0/
1002 2017-08-03T23:47:31  <gribble> https://github.com/bitcoin/bitcoin/issues/10985 | Add undocumented -forcecompactdb to force LevelDB compactions by sipa · Pull Request #10985 · bitcoin/bitcoin · GitHub
1003 2017-08-03T23:47:50  <sipa> gmaxwell: ^ untested
1004 2017-08-03T23:56:19  *** marcoagner has joined #bitcoin-core-dev
1005 2017-08-03T23:59:34  *** praxeology has quit IRC
1006 2017-08-03T23:59:36  *** abpa has quit IRC