1 2017-01-03T00:27:26  *** justan0theruser has joined #bitcoin-core-dev
  2 2017-01-03T00:30:31  *** justanotheruser has quit IRC
  3 2017-01-03T00:35:20  *** dviola has joined #bitcoin-core-dev
  4 2017-01-03T00:37:28  *** lejitz has joined #bitcoin-core-dev
  5 2017-01-03T00:50:27  *** justan0theruser is now known as justanotheruser
  6 2017-01-03T00:55:15  <dviola> hello, the archlinux bitcoin-qt package maintainer builds bitcoin-qt using "--with-incompatible-bdb" and bdb 5.3.28, I have heard there could be potential problems by taking a wallet.dat that was created with newer bdb and then using it on a bitcoin-qt that was compiled with an older version of bdb, are there any tests I could run to know I won't run into issues?
  7 2017-01-03T00:57:26  *** Ylbam has quit IRC
  8 2017-01-03T01:00:23  <phantomcircuit> dviola, 5.x has a different format than 4.8 which is not backwards compatible
  9 2017-01-03T01:00:33  <phantomcircuit> if you open a wallet in 5.x it wont open in 4.x
 10 2017-01-03T01:00:43  <phantomcircuit> so that's a one way trip
 11 2017-01-03T01:08:09  <dviola> phantomcircuit: I see, so would it be better if I create the wallet using the bitcoin-qt builds from bitcoin.org?
 12 2017-01-03T01:09:21  <dviola> I'm a little confused because I created a wallet.dat using Arch's build, then used the wallet.dat with bitcoin-qt from bitcoin.org, and it seems to have worked just fine, but I don't have funds in this wallet of course
 13 2017-01-03T01:09:34  <dviola> I could still see the address and such
 14 2017-01-03T01:09:41  <dviola> which was the same
 15 2017-01-03T01:13:28  *** Giszmo has quit IRC
 16 2017-01-03T01:14:49  *** MarcoFalke has quit IRC
 17 2017-01-03T01:15:10  <dviola> just tested again
 18 2017-01-03T01:15:23  <luke-jr> dviola: *using* the wallet.dat with bdb5 should break using it with bdb4.. no matter how it was created.
 19 2017-01-03T01:17:57  <dviola> hrm
 20 2017-01-03T01:19:13  <dviola> here's what I did: I installed bitcoin-qt with pacman, started a fresh ~/.bitcoin and wallet.dat, I created 5 addresses with name, etc. I copied the wallet.dat to my home dir, I downloaded bitcoin-qt from bitcoin.org, started that with the old wallet.dat and all the address were there
 21 2017-01-03T01:22:30  <dviola> luke-jr: I understand, but isn't what I did the equivalent of what you said? I'm a little confused over this
 22 2017-01-03T01:27:24  <dviola> let me show you something...
 23 2017-01-03T01:28:43  <dviola> this is bitcoin-qt from archlinux (bdb 5.3.28): https://i.imgur.com/iL6MVB2.png
 24 2017-01-03T01:28:54  <dviola> this is bitcoin-qt from bitcoin.org (bdb 4.8.x): https://i.imgur.com/G50bTJi.png
 25 2017-01-03T01:28:57  <dviola> same wallet.dat
 26 2017-01-03T01:29:08  <dviola> I can still read the data from wallet
 27 2017-01-03T01:29:40  <dviola> I'm confused and I apologize if I should have asked in #bitcoin instead, please let me know if I should go there
 28 2017-01-03T01:31:02  <luke-jr> dviola: are you sure Arch isn't using bdb4 for the wallet?
 29 2017-01-03T01:31:19  <luke-jr> if both are installed, I think we prefer it even if incompatible-bdb is allowed via option
 30 2017-01-03T01:35:23  <dviola> I'm sure, this is how they build it: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/bitcoin#n166
 31 2017-01-03T01:35:30  <dviola> ldd shows this also: libdb_cxx-5.3.so => /usr/lib/libdb_cxx-5.3.so (0x00007f3aaa8bc000)
 32 2017-01-03T01:36:38  <dviola> I asked Timothy Redaelli (archlinux bitcoin maintainer) and he said this: https://i.imgur.com/bfvsEcZ.png
 33 2017-01-03T01:42:53  <dviola> not saying I don't believe what you just said, I guess I'm puzzled as to why it works
 34 2017-01-03T01:55:28  <dviola> luke-jr: hrm, I only have bdb5 system-wide
 35 2017-01-03T02:01:17  *** abpa has quit IRC
 36 2017-01-03T02:04:42  <dviola> even with an intact ~/.bitcoin it still works fine with both binaries
 37 2017-01-03T02:04:44  <dviola> bdb4 and 5
 38 2017-01-03T02:05:10  <dviola> I can see the info from getwalletinfo and do dumpwallet, etc
 39 2017-01-03T02:06:17  <dviola> I wonder if there is a way to exhaust this and see where it starts to fail
 40 2017-01-03T02:14:54  *** lejitz has quit IRC
 41 2017-01-03T02:14:56  *** lejitz1 has joined #bitcoin-core-dev
 42 2017-01-03T02:21:06  *** AaronvanW has quit IRC
 43 2017-01-03T03:07:32  *** dviola has quit IRC
 44 2017-01-03T03:08:11  *** abpa has joined #bitcoin-core-dev
 45 2017-01-03T03:09:21  *** abpa has quit IRC
 46 2017-01-03T03:17:59  *** Victorsueca has joined #bitcoin-core-dev
 47 2017-01-03T03:20:20  *** Victor_sueca has quit IRC
 48 2017-01-03T03:27:07  <gmaxwell> anyone interested in picking up #8660 ? it's a nice feature, but has gone fallow after main was killed.
 49 2017-01-03T03:27:09  <gribble> https://github.com/bitcoin/bitcoin/issues/8660 | txoutsbyaddress index (take 2) by djpnewton · Pull Request #8660 · bitcoin/bitcoin · GitHub
 50 2017-01-03T03:28:51  <gmaxwell> droark: it should just fail to even open, always did in the past.
 51 2017-01-03T03:29:42  <gmaxwell> maybe some time in BDB 5 they changed the format to be backwards compatible if shut down cleanly?
 52 2017-01-03T03:34:45  *** justanotheruser has quit IRC
 53 2017-01-03T03:35:03  *** justanotheruser has joined #bitcoin-core-dev
 54 2017-01-03T03:35:10  *** justanotheruser has quit IRC
 55 2017-01-03T03:39:06  *** justanotheruser has joined #bitcoin-core-dev
 56 2017-01-03T03:54:22  *** robert__ has quit IRC
 57 2017-01-03T03:59:21  <phantomcircuit> gmaxwell, does it have to handle reorgs correctly?
 58 2017-01-03T04:00:45  <phantomcircuit> i dont think it would actually if you were saving address -> [block hash/tx index/out index]
 59 2017-01-03T04:01:07  <phantomcircuit> but that's maybe slightly bigger than just address > txout
 60 2017-01-03T04:06:59  <phantomcircuit> oh it's just for the utxo?
 61 2017-01-03T04:07:01  <phantomcircuit> yeah i guess
 62 2017-01-03T04:10:27  *** robert__ has joined #bitcoin-core-dev
 63 2017-01-03T04:19:39  *** Squidicc has joined #bitcoin-core-dev
 64 2017-01-03T04:22:49  *** squidicuz has quit IRC
 65 2017-01-03T05:31:14  <gmaxwell> phantomcircuit: it's just for the utxo set, the PR is somewhat poorly named!
 66 2017-01-03T05:48:10  <CodeShark> utxosbyaddress? :)
 67 2017-01-03T05:48:54  <CodeShark> utxosbyscriptpubkey might even be more useful
 68 2017-01-03T05:53:59  <CodeShark> I might take a look at it, it is a pretty nice feature
 69 2017-01-03T05:55:26  <gmaxwell> assuming the database is constructed correctly (lets hope) the difference between by address vs by scriptpubkey is purely a RPC parameter one.
 70 2017-01-03T05:55:53  <CodeShark> yeah, indeed :)
 71 2017-01-03T05:56:06  <CodeShark> but we don't have address formats for all scriptpubkey types
 72 2017-01-03T05:58:13  <CodeShark> how big is the extra index here?
 73 2017-01-03T06:00:14  <gmaxwell> CodeShark: should be roughly similar in size to the utxo set.
 74 2017-01-03T06:00:40  <CodeShark> it's something of a reverse lookup, no?
 75 2017-01-03T06:00:56  <CodeShark> when validating we need to grab scriptpubkey from blockhash:txindex
 76 2017-01-03T06:01:02  <gmaxwell> right.
 77 2017-01-03T06:01:13  *** windsok has quit IRC
 78 2017-01-03T06:01:20  <gmaxwell> I don't recall what it implements, but
 79 2017-01-03T06:01:32  <CodeShark> or txhash:outindex, rather ;)
 80 2017-01-03T06:01:35  <gmaxwell> e.g. it could map(h(scrippubkey)[64 bits]) -> [txid:vout,...]  which would potentially equal in size or smaller even though it's going to be more sparse.
 81 2017-01-03T06:01:52  <gmaxwell> Well my right was responding to what you meant, of course.
 82 2017-01-03T06:02:57  <gmaxwell> at least if I'd written this from scratch I would have done that kind of hash, probably with a per database salt to keep clowns from intentionally making your queries slow by causing collisions.
 83 2017-01-03T06:05:34  <CodeShark> I'm a little ambivalent about using an in-process storage engine for things not required for full validation. I'd love to see an architecture with an external storage engine that can be configured for different kinds of queries. But I guess this index isn't too huge
 84 2017-01-03T06:07:19  <CodeShark> pulling the consensus-critical stuff out to an external storage engine is perhaps a bit risky, though
 85 2017-01-03T06:07:36  <CodeShark> and perhaps there are optimization issues as well here with the caching
 86 2017-01-03T06:07:45  <CodeShark> I'm not too familiar with the cache stuff
 87 2017-01-03T06:09:21  *** veleiro has joined #bitcoin-core-dev
 88 2017-01-03T06:09:23  *** robert__ is now known as spinNspiral
 89 2017-01-03T06:09:29  <CodeShark> the main advantage here is that different people could extend the functionality of the storage engine and queries supported without having to touch any consensus code at all
 90 2017-01-03T06:09:42  <CodeShark> nor having to rebuild the validation engine
 91 2017-01-03T06:11:36  <CodeShark> but we already have the existing RPC framework...which encourages just adding more in-process RPC calls
 92 2017-01-03T06:23:12  <CodeShark> people always talk about keeping these indices external to bitcoind...but without a framework for it that people can easily extend everyone has to reinvent the wheel
 93 2017-01-03T06:24:18  <CodeShark> ...poorly :p
 94 2017-01-03T06:30:07  <gmaxwell> You can always index whatever you want talking to things over the rpc. but there are plenty of uses for this just from the commandline.
 95 2017-01-03T06:32:38  <CodeShark> no question it's useful. but there could be a bunch other useful queries for app developers (or even for interactive use from commandline) - and having to merge another in-process RPC call everytime is a bottleneck
 96 2017-01-03T06:33:49  <CodeShark> or it encourages people to have to maintain their own forks of the entire project so they can customize it
 97 2017-01-03T06:35:15  <CodeShark> as for an external index, we could get far better performance foregoing the RPC and using a different IPC
 98 2017-01-03T06:35:58  <CodeShark> also, the indices could be built up during IBD
 99 2017-01-03T06:36:33  <CodeShark> but these are longer term objectives - I still like this PR :)
100 2017-01-03T06:48:55  *** windsok has joined #bitcoin-core-dev
101 2017-01-03T06:52:37  *** spinNspiral has quit IRC
102 2017-01-03T06:52:38  *** Ylbam has joined #bitcoin-core-dev
103 2017-01-03T07:06:27  *** Squidicc is now known as squidicuz
104 2017-01-03T07:13:39  *** udiWertheimer has joined #bitcoin-core-dev
105 2017-01-03T07:27:31  *** tunafizz has quit IRC
106 2017-01-03T07:48:32  *** udiWertheimer has quit IRC
107 2017-01-03T07:58:15  *** veleiro has quit IRC
108 2017-01-03T07:59:51  <jonasschnelli> Is it okay to use GPG's (v2.1+) Ed25516 or even secp256k1 keys for gitian signing?
109 2017-01-03T08:00:11  <jonasschnelli> *25519
110 2017-01-03T08:01:31  <wumpus> well, it's not forbidden, but I can't count them right now (my gpg is at 1.4.20, ubuntu 16.04 lts)
111 2017-01-03T08:02:27  <jonasschnelli> Yes. It's probably to early.
112 2017-01-03T08:02:29  <wumpus> I can look at compiling my own for 0.14, but not for 0.13.2 today
113 2017-01-03T08:02:44  <phantomcircuit> wumpus, you can install gnupg 2.x
114 2017-01-03T08:02:49  <phantomcircuit> it's like gnupg-2 or something
115 2017-01-03T08:02:53  <jonasschnelli> gpg2
116 2017-01-03T08:02:57  <wumpus> phantomcircuit: oh!
117 2017-01-03T08:03:24  <jonasschnelli> I may start with signing with Ed25519, just to have some variety.
118 2017-01-03T08:03:44  <gmaxwell> jonasschnelli: distributions ship the table version of gpg, mostly (exclusively?) so AFAICT relatively few people have it.
119 2017-01-03T08:03:55  <wumpus> phantomcircuit: jonasschnelli: that gets me 2.1.11
120 2017-01-03T08:05:35  <jonasschnelli> I guess for all ecdsa curves (including secp256k1) you need 2.1.16 (or 2.1.17?)
121 2017-01-03T08:05:35  <wumpus> gah, it doesn't use the debian alternatives system though, so I have to manually patch up scripts to use gpg2 instead of gpg
122 2017-01-03T08:05:54  <jonasschnelli> But watch out when using gpg2
123 2017-01-03T08:06:00  <jonasschnelli> It will upgrade you .gpg folder
124 2017-01-03T08:06:08  <jonasschnelli> And will no longer be backward comp.
125 2017-01-03T08:06:14  <jonasschnelli> So,... make a backup first
126 2017-01-03T08:06:21  <wumpus> oops
127 2017-01-03T08:06:24  <jcorgan> eww
128 2017-01-03T08:06:35  <jonasschnelli> maybe this was too late. :)
129 2017-01-03T08:06:45  <CodeShark> lol
130 2017-01-03T08:06:51  <wumpus> let's see :) I only looked at the help message, didn't encrypt anything yet
131 2017-01-03T08:07:57  <wumpus> good, v1 still works for verifying signatures
132 2017-01-03T08:08:05  <wumpus> I'll look at this for 0.14 okay
133 2017-01-03T08:08:15  <jonasschnelli> Yes. No hurry.
134 2017-01-03T08:08:43  <jonasschnelli> If secp256k1 is supported through GPG, this would allow signing over a hardware wallet without new curves added to them.
135 2017-01-03T08:11:01  <wumpus> yea that could be interesting, though it would have to assign a single purpose to keys, don't use keys for signing that are used for bitcoin
136 2017-01-03T08:11:14  <jonasschnelli> Yes. Indeed.
137 2017-01-03T08:11:56  *** BashCo has quit IRC
138 2017-01-03T08:12:25  <jonasschnelli> I'd also prefer Ed25519,... not supported by (all?) available hardware wallets I guess.
139 2017-01-03T08:12:33  *** BashCo has joined #bitcoin-core-dev
140 2017-01-03T08:14:59  <wumpus> ed25519 would have the advantage it can be used for ssh too
141 2017-01-03T08:15:14  <jonasschnelli> Oh. Yes.
142 2017-01-03T08:15:33  <wumpus> (even if you compile it without openssl :p)
143 2017-01-03T08:16:03  <jonasschnelli> Right. ed25519 & chacha20Poly1305@openssh. Nice combo.
144 2017-01-03T08:17:25  *** BashCo has quit IRC
145 2017-01-03T08:17:54  <wumpus> yes
146 2017-01-03T08:18:37  *** udiWertheimer has joined #bitcoin-core-dev
147 2017-01-03T08:32:26  *** BashCo has joined #bitcoin-core-dev
148 2017-01-03T08:32:32  <bitcoin-git> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/53442af0aac3...510c0d9c79a5
149 2017-01-03T08:32:32  <bitcoin-git> bitcoin/master 9e351c9 Jonas Schnelli: SetMerkleBranch: remove unused code, remove cs_main lock requirement
150 2017-01-03T08:32:33  <bitcoin-git> bitcoin/master 510c0d9 Jonas Schnelli: Merge #9446: SetMerkleBranch: remove unused code, remove cs_main lock requirement...
151 2017-01-03T08:32:48  <bitcoin-git> [bitcoin] jonasschnelli closed pull request #9446: SetMerkleBranch: remove unused code, remove cs_main lock requirement (master...2016/12/merklebranch) https://github.com/bitcoin/bitcoin/pull/9446
152 2017-01-03T08:59:30  <wumpus> uploaded binaries and pushed 0.13.2 release announcement to the mailing lists, waiting for travis on bitcoin.org (https://github.com/bitcoin-dot-org/bitcoin.org/pull/1470), feel free to get the rest of the announcement cycle started
153 2017-01-03T09:02:09  <gmaxwell> jonasschnelli: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/013397.html  A spv wallet should _never_ be showing transactions from untrusted peers.
154 2017-01-03T09:02:28  <gmaxwell> jonasschnelli: because the peers can feed them any garbage they want and the spv client has no idea about it.
155 2017-01-03T09:02:51  <jonasschnelli> (phonecall, will response soon)
156 2017-01-03T09:04:01  <gmaxwell> and you get lovely results like https://people.xiph.org/~greg/21mbtc.png
157 2017-01-03T09:08:10  *** paveljanik has quit IRC
158 2017-01-03T09:08:16  <sipa> gmaxwell: but what if you trust a peer enough to not relay invalid txn, but not enough to not spy on you?
159 2017-01-03T09:08:42  <jonasschnelli> gmaxwell: I see.
160 2017-01-03T09:09:07  <jonasschnelli> I mentioned this in the bitcoin-dev mail, because users relay on it,... even if its fundamentally broken.
161 2017-01-03T09:09:21  <sipa> rely
162 2017-01-03T09:09:35  <jonasschnelli> rely. Thanks sipa. :)
163 2017-01-03T09:09:50  <jonasschnelli> Also,... it looked like, that my feeler PR: https://github.com/bitcoin/bitcoin/pull/9238 did get some rejects.
164 2017-01-03T09:11:11  <jonasschnelli> In a long shot, I could imagine, BFD (filter digest / filter commitments) for filtering from untrusted peers, and BIP39 for filtering from trusted peers (once BIP150 has ben established).
165 2017-01-03T09:11:23  <jonasschnelli> Even if BIP39 is not the most efficient way.
166 2017-01-03T09:11:30  <sipa> BIP39?
167 2017-01-03T09:11:33  <sipa> are you sure?
168 2017-01-03T09:11:35  <jonasschnelli> 37! argh
169 2017-01-03T09:11:56  <jonasschnelli> Bloom Filter
170 2017-01-03T09:13:06  *** jannes has joined #bitcoin-core-dev
171 2017-01-03T09:13:17  <jonasschnelli> But I think there is no solution for a non-BIP37 0-conf "filtering" without running into these privacy issues related to bip37
172 2017-01-03T09:13:33  <gmaxwell> jonasschnelli: for your wokrin in 9238 just transfer all transactions, any filtering scheme will trash privacy, and transfering transactions is an aggregate of 14 kilobit/s.
173 2017-01-03T09:13:53  <gmaxwell> But displaying unconfirmed transactions in any non-validating wallet is just an invite to abuse.
174 2017-01-03T09:14:35  <gmaxwell> Many lite wallets that do this today are utterly crippled by it, because they also spend unconfirmed coins, so a single troublemaker giving them some fake payments makes them unusable. (or at least they did a half year ago.)
175 2017-01-03T09:15:11  <jonasschnelli> gmaxwell: Yes. Agree. But the user-experience without displaying incoming funds will probably be not acceptable "in the field".
176 2017-01-03T09:15:25  <gmaxwell> please.
177 2017-01-03T09:16:39  <jonasschnelli> I don't know. I also don't like the 0-conf displaying. While working on the Core wallets SPV mode, I just had the feeling that people will not use it because of the missing 0-conf display.
178 2017-01-03T09:16:47  <gmaxwell> I think it's a bad idea to release features that would require an emergency software update to disable them as soon as one board person spends an hour making a few like hack that sends out garbage transactions and spins up a bunch of sybil nodes.
179 2017-01-03T09:17:56  <jonasschnelli> 0-conf / spv should probably only be possible in conjunction with BIP150 and a trusted peer (at home or somewhere)
180 2017-01-03T09:18:35  <gmaxwell> In any case, just send all the data. As mentioned it's 14kbit/sec.  The history is really the only annoying part about that.
181 2017-01-03T09:19:26  <jonasschnelli> gmaxwell: do you mean send all the data if someone request a filtered mempool?
182 2017-01-03T09:19:34  <jonasschnelli> or just for tv invs?
183 2017-01-03T09:19:38  <jonasschnelli> *tx
184 2017-01-03T09:20:10  <gmaxwell> I mean don't do any tx filtering.
185 2017-01-03T09:21:25  <gmaxwell> Same as we do today. Please. implementing spv mode in core doesn't mean implementing all the flaws and vulnerabilities in other software slavishly. Thats pointless. If someone wants something that is going to blast out there addresses, validate nothing, etc.. they can already use bitcoinj based software.
186 2017-01-03T09:21:41  <jonasschnelli> Heh. True.
187 2017-01-03T09:22:01  <gmaxwell> :) at least the spv behavior in core can be private.
188 2017-01-03T09:22:10  <jonasschnelli> I have implemented the BFD relevant hooks. IsBlockRelevantToWallet(pindex).
189 2017-01-03T09:23:39  <jonasschnelli> gmaxwell: The only usability issues are: catch-up a couple of weeks (144*<day> MBs, take a while) and the missing 0-conf. Maybe this is acceptable to prevent privacy.
190 2017-01-03T09:24:50  <gmaxwell> I think it's not bad, especially if we had better facilities for keeping it caught up in the background.
191 2017-01-03T09:24:55  <jonasschnelli> Though, I haven't fully understood the semi.-trusted oracle idea for the BDF. I would expect to have the BDF in the coinbase...
192 2017-01-03T09:26:00  <jonasschnelli> But somehow the BFD is required for older blocks as well
193 2017-01-03T09:26:09  <gmaxwell> It's not really great to go from nothing to consensus rule if it can be avoided: No design survives first contact with users.  Better to prove out the idea and refine it before its required, which is possible here.
194 2017-01-03T09:27:29  <jonasschnelli> From where would you get the BFD (maybe its not bloom filters in the end)? A new p2p command?
195 2017-01-03T09:28:13  <gmaxwell> just another type of element that you can getdata, perhaps more than one.
196 2017-01-03T09:29:53  <jonasschnelli> this would result in trusting the peer, right? The whole signing thing of BFDs would still be a thing? Or just compare BFDs from the data retrived from other peers?
197 2017-01-03T09:30:20  <jonasschnelli> Probably easy to sybil you with fake data
198 2017-01-03T09:30:57  <gmaxwell> I thought you were referring to the filter itselt and not the hash root... Unclear, multiple models are possible and _all_ are better than BIP37.
199 2017-01-03T09:31:38  <jonasschnelli> Right, tx withhold is also possible with BIP37.
200 2017-01-03T09:32:18  <gmaxwell> and asking multiple peers is very inefficient with BIP37.. needs n-times the bandwidth.
201 2017-01-03T09:33:19  <jonasschnelli> good point.
202 2017-01-03T09:36:22  *** slsdhl has joined #bitcoin-core-dev
203 2017-01-03T09:36:34  *** slsdhl has quit IRC
204 2017-01-03T09:37:14  <btcdrak> Requesting review for release blog post https://github.com/bitcoin-core/bitcoincore.org/pull/305
205 2017-01-03T09:40:23  <gmaxwell> jonasschnelli: but pulling data from peers is just one option, you could-- for example-- go get a hash root (root of a tree of digest hashes) from a website and pop that into a configuration.
206 2017-01-03T09:41:09  <gmaxwell> (and then use that for filtering the past-- then the future, past that root, you just fetch whole blocks)
207 2017-01-03T09:49:31  *** jeremias has quit IRC
208 2017-01-03T09:50:22  *** jeremias has joined #bitcoin-core-dev
209 2017-01-03T09:52:28  *** timothy has quit IRC
210 2017-01-03T09:54:18  *** Guyver2 has joined #bitcoin-core-dev
211 2017-01-03T10:01:23  *** timothy has joined #bitcoin-core-dev
212 2017-01-03T10:12:32  *** timothy has quit IRC
213 2017-01-03T10:13:13  *** timothy has joined #bitcoin-core-dev
214 2017-01-03T10:19:03  *** timothy has quit IRC
215 2017-01-03T10:19:32  *** MarcoFalke has joined #bitcoin-core-dev
216 2017-01-03T10:23:41  <BlueMatt> cfields: ohhh, static CNodeState::cs....missed the static part
217 2017-01-03T10:24:28  <BlueMatt> cfields: ok, I'll make it static in the pr and add a TODO: do something smarter
218 2017-01-03T10:24:38  <BlueMatt> cfields: oh, wait, no, really dont want to do that
219 2017-01-03T10:25:40  *** Giszmo has joined #bitcoin-core-dev
220 2017-01-03T10:26:35  <BlueMatt> cfields: that would mean that #9375 + multithreaded processmessages wouldnt accomplish anything :/
221 2017-01-03T10:26:37  <gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
222 2017-01-03T10:28:06  <sipa> BlueMatt: but the same is true for the current code, no?
223 2017-01-03T10:28:13  <sipa> which uses cs_main
224 2017-01-03T10:31:40  <BlueMatt> sipa: no, I dont believe so, #9375 + #9419 + a fix for https://github.com/bitcoin/bitcoin/pull/9375#discussion_r94180477 (which just means making fWantsCmpctWitness atomic) + multithreaded ProcessMessages will work
225 2017-01-03T10:31:42  <gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
226 2017-01-03T10:31:44  <gribble> https://github.com/bitcoin/bitcoin/issues/9419 | Stop Using cs_main for CNodeState/State() by TheBlueMatt · Pull Request #9419 · bitcoin/bitcoin · GitHub
227 2017-01-03T10:33:09  <sipa> BlueMatt: ah, i see
228 2017-01-03T10:33:14  <sipa> right
229 2017-01-03T10:33:14  <BlueMatt> it may be the case that we dont need State() at all for that, though
230 2017-01-03T10:35:38  <BlueMatt> yea, no, so to make it all work we need to have a State()->fWantsCmpctWitness and pfrom->GetSendVersion() call without cs_main
231 2017-01-03T10:40:05  *** timothy has joined #bitcoin-core-dev
232 2017-01-03T10:51:53  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to 0.13: https://github.com/bitcoin/bitcoin/commit/77eaadb6c9619370b09513fecba13cfe656d63d6
233 2017-01-03T10:51:54  <bitcoin-git> bitcoin/0.13 77eaadb Wladimir J. van der Laan: doc: Clean out release notes on 0.13.x branch...
234 2017-01-03T10:52:56  <bitcoin-git> [bitcoin] laanwj pushed 1 new commit to master: https://github.com/bitcoin/bitcoin/commit/03e1d6ce349cc83b92140fec7d0c5f88893c0a9c
235 2017-01-03T10:52:56  <bitcoin-git> bitcoin/master 03e1d6c Wladimir J. van der Laan: doc: Add historical release notes for 0.13.2
236 2017-01-03T10:54:45  <jonasschnelli> gmaxwell: thanks.
237 2017-01-03T10:55:22  <jonasschnelli> gmaxwell: you mentioned here (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-May/012637.html) a rough specs for a better filter structure then bloom
238 2017-01-03T10:55:30  <jonasschnelli> Do you have any specification for that?
239 2017-01-03T10:59:30  <BlueMatt> cfields: hmmm, regarding #9441, do we really want to run the entire ProcessMessages loop (calling SendMessages potentially umpteen times) just to process multiple messages from the same node?
240 2017-01-03T10:59:32  <gribble> https://github.com/bitcoin/bitcoin/issues/9441 | Net: Massive speedup. Net locks overhaul by theuni · Pull Request #9441 · bitcoin/bitcoin · GitHub
241 2017-01-03T10:59:49  <BlueMatt> (ie remove the loop inside ProcessMessages and move it to ThreadProcessMessages)
242 2017-01-03T11:01:02  <timothy> buiding bitcoin-core 0.13.2 on archlinux...
243 2017-01-03T11:03:18  <sipa> BlueMatt: i was wondering about that too
244 2017-01-03T11:03:52  <sipa> BlueMatt: but there is hardly any (contentious) locking going on anymore in between
245 2017-01-03T11:04:04  <BlueMatt> yea, but SendMessages.....
246 2017-01-03T11:04:09  <sipa> Ah.
247 2017-01-03T11:04:31  <sipa> i hadn't considered that
248 2017-01-03T11:04:37  <sipa> that defeats the purpose
249 2017-01-03T11:05:10  <sipa> hmm, i was about to say that it negatively impacts batching of invs and addrs
250 2017-01-03T11:05:27  <sipa> but since we have explicit random delays for those, i don't think that's really an issue anymore
251 2017-01-03T11:05:40  <BlueMatt> yea, I dont think it breaks anything
252 2017-01-03T11:05:47  <BlueMatt> just repeatedly calls SendMessages to do nothing
253 2017-01-03T11:06:12  <sipa> oh, it won't break anything
254 2017-01-03T11:06:53  <BlueMatt> I assume you didnt touch cs_vSend due to https://github.com/bitcoin/bitcoin/pull/9419/commits/c214d120a363a05ba9afdccff6b4bda6e29ae7c4, cfields?
255 2017-01-03T11:10:06  <BlueMatt> cfields: other than that, I got through #9441 and it looks good to me
256 2017-01-03T11:10:08  <gribble> https://github.com/bitcoin/bitcoin/issues/9441 | Net: Massive speedup. Net locks overhaul by theuni · Pull Request #9441 · bitcoin/bitcoin · GitHub
257 2017-01-03T11:10:38  * BlueMatt will copy discussion into pr thread
258 2017-01-03T11:10:45  *** AaronvanW has joined #bitcoin-core-dev
259 2017-01-03T11:10:46  *** AaronvanW has quit IRC
260 2017-01-03T11:10:46  *** AaronvanW has joined #bitcoin-core-dev
261 2017-01-03T11:17:14  <sipa> thanks
262 2017-01-03T11:17:22  * sipa -> NYC
263 2017-01-03T11:17:45  <BlueMatt> sipa: cool, see you tomorrow afternoon
264 2017-01-03T11:17:55  <BlueMatt> anyone else at RWC? sipa, cfields and I likely will be
265 2017-01-03T11:24:38  *** bitcfan has joined #bitcoin-core-dev
266 2017-01-03T11:25:34  *** Saucery has quit IRC
267 2017-01-03T11:45:22  *** tiago_ has joined #bitcoin-core-dev
268 2017-01-03T11:51:21  <timothy> is MALLOC_ARENA_MAX=1 still "recommended" on linux?
269 2017-01-03T12:01:18  <sipa> timothy: i expect that setting it so low may come with lower performancr
270 2017-01-03T12:01:27  <sipa> but i don't think anyone ever benchmarked it
271 2017-01-03T12:03:16  <sipa> it's certainly recommended if you are very tight on memory
272 2017-01-03T12:03:59  <timothy> is -reindex-chainstate is faster than -reindex?
273 2017-01-03T12:04:06  <timothy> without the second is :P
274 2017-01-03T12:05:26  *** windsok has quit IRC
275 2017-01-03T12:09:32  <sipa> yes
276 2017-01-03T12:09:53  <sipa> but it only works when your blocks and blocks/index are not corrupted
277 2017-01-03T12:19:23  *** kadoban has quit IRC
278 2017-01-03T12:19:24  *** jl2012 has quit IRC
279 2017-01-03T12:19:24  *** kinlo has quit IRC
280 2017-01-03T12:19:51  *** kinlo has joined #bitcoin-core-dev
281 2017-01-03T12:19:51  *** kadoban has joined #bitcoin-core-dev
282 2017-01-03T12:20:02  *** tiago_ has quit IRC
283 2017-01-03T12:20:53  *** jl2012 has joined #bitcoin-core-dev
284 2017-01-03T12:32:52  *** BashCo_ has joined #bitcoin-core-dev
285 2017-01-03T12:36:01  *** BashCo has quit IRC
286 2017-01-03T12:36:16  *** windsok has joined #bitcoin-core-dev
287 2017-01-03T13:21:37  *** JackH has quit IRC
288 2017-01-03T13:25:50  *** ryanofsky_ has quit IRC
289 2017-01-03T13:27:28  *** ryanofsky has joined #bitcoin-core-dev
290 2017-01-03T13:30:04  *** goregrind has quit IRC
291 2017-01-03T13:30:09  *** goregrin1 has joined #bitcoin-core-dev
292 2017-01-03T13:30:56  *** belcher has quit IRC
293 2017-01-03T13:33:14  *** belcher has joined #bitcoin-core-dev
294 2017-01-03T13:46:55  *** juscamarena has quit IRC
295 2017-01-03T13:47:16  *** juscamarena has joined #bitcoin-core-dev
296 2017-01-03T13:47:23  *** Aleph0 has quit IRC
297 2017-01-03T13:47:23  *** ybit has quit IRC
298 2017-01-03T13:47:23  *** jrayhawk has quit IRC
299 2017-01-03T13:47:23  *** aj has quit IRC
300 2017-01-03T13:47:30  *** aj has joined #bitcoin-core-dev
301 2017-01-03T13:47:31  *** Aleph0 has joined #bitcoin-core-dev
302 2017-01-03T13:47:32  *** jrayhawk has joined #bitcoin-core-dev
303 2017-01-03T13:47:37  *** ybit has joined #bitcoin-core-dev
304 2017-01-03T13:47:41  <jonasschnelli> bitcoin-qt tells me today when running with a fresh datadir: "Last received block was generated 7 year(s) and 52 week(s) ago."
305 2017-01-03T13:47:44  *** ibrightly has quit IRC
306 2017-01-03T13:48:24  *** blkdb has quit IRC
307 2017-01-03T13:49:31  <wumpus> jonasschnelli: so it regards it as an edge case, 52 weeks but not a year yet. curious :)
308 2017-01-03T13:50:02  <jonasschnelli> qint64 remainder = secs % YEAR_IN_SECONDS;
309 2017-01-03T13:50:09  <jonasschnelli> Na. I don't fix that.
310 2017-01-03T13:50:30  <wumpus> yeah it's silly but not a serious issue
311 2017-01-03T13:51:01  *** ibrightly has joined #bitcoin-core-dev
312 2017-01-03T13:55:32  *** roasbeef has quit IRC
313 2017-01-03T13:56:59  *** roasbeef has joined #bitcoin-core-dev
314 2017-01-03T13:57:51  *** laurentmt has joined #bitcoin-core-dev
315 2017-01-03T13:58:30  *** laurentmt has quit IRC
316 2017-01-03T13:58:55  *** laurentmt has joined #bitcoin-core-dev
317 2017-01-03T14:07:37  <bitcoin-git> [bitcoin] laanwj opened pull request #9460: Fix a few typos in translated strings (master...2017_01_messages_update) https://github.com/bitcoin/bitcoin/pull/9460
318 2017-01-03T14:13:27  *** dermoth has joined #bitcoin-core-dev
319 2017-01-03T14:15:40  *** laurentmt has quit IRC
320 2017-01-03T14:18:11  *** bsm117532 has quit IRC
321 2017-01-03T14:18:12  *** fengling has quit IRC
322 2017-01-03T14:41:13  <luke-jr> wumpus: btcdrak: bad signature on sendy release ann
323 2017-01-03T14:42:03  <wumpus> luke-jr: are you checking the html or text attachment?
324 2017-01-03T14:42:57  *** helo has quit IRC
325 2017-01-03T14:43:09  <wumpus> in case of the html you should run gpg on the raw html file
326 2017-01-03T14:43:19  <wumpus> (both attachments verify OK here)
327 2017-01-03T14:44:05  <luke-jr> both
328 2017-01-03T14:44:20  <luke-jr> how do you run GPG on the raw HTML file, seeing as the HTML is outside the signature? :/
329 2017-01-03T14:44:37  <bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/03e1d6ce349c...e0dc3d70c6ec
330 2017-01-03T14:44:38  <bitcoin-git> bitcoin/master a9d6151 Wladimir J. van der Laan: qt,wallet: Fix a few typos in messages...
331 2017-01-03T14:44:39  <bitcoin-git> bitcoin/master d45b21e Wladimir J. van der Laan: qt: Fill in English numerusforms...
332 2017-01-03T14:44:39  <bitcoin-git> bitcoin/master e0dc3d7 Wladimir J. van der Laan: Merge #9460: Fix a few typos in translated strings...
333 2017-01-03T14:44:53  <btcdrak> luke-jr: matches for me
334 2017-01-03T14:44:53  <bitcoin-git> [bitcoin] laanwj closed pull request #9460: Fix a few typos in translated strings (master...2017_01_messages_update) https://github.com/bitcoin/bitcoin/pull/9460
335 2017-01-03T14:45:07  <wumpus> that mail has a text/plain as well as a text/html mime attachment - just save them to a file and run gpg on them?
336 2017-01-03T14:46:30  <luke-jr> hmm, that works. I wonder what copy/paste is changing
337 2017-01-03T14:46:55  <btcdrak> Thunderbird also automatically validated it for me.
338 2017-01-03T14:47:03  <jonasschnelli> Same here
339 2017-01-03T14:47:11  <luke-jr> my mail client normally auto-validates, but it didn't like the formatting or something
340 2017-01-03T14:53:42  <bitcoin-git> [bitcoin] jonasschnelli opened pull request #9461: [Qt] Improve progress display during headers-sync and peer-finding (master...2017/01/qt_sync) https://github.com/bitcoin/bitcoin/pull/9461
341 2017-01-03T15:14:12  *** Cheeseo has joined #bitcoin-core-dev
342 2017-01-03T15:14:12  *** Cheeseo has joined #bitcoin-core-dev
343 2017-01-03T15:14:23  *** Cheeseo has left #bitcoin-core-dev
344 2017-01-03T15:14:49  <jonasschnelli> luke-jr: I think squashing makes sense when one commit overwrites many lines from a former commit in the same PR.
345 2017-01-03T15:14:56  <jonasschnelli> But Okay for #8877
346 2017-01-03T15:14:58  <gribble> https://github.com/bitcoin/bitcoin/issues/8877 | Qt RPC console: history sensitive-data filter, and saving input line when browsing history by luke-jr · Pull Request #8877 · bitcoin/bitcoin · GitHub
347 2017-01-03T15:15:36  <luke-jr> jonasschnelli: there are some cases where I think it may make sense, yes
348 2017-01-03T15:15:56  *** andytoshi has left #bitcoin-core-dev
349 2017-01-03T15:16:05  <jonasschnelli> But your right, squashing and loosing "logical history" is not ideal
350 2017-01-03T15:17:13  *** bsm117532 has joined #bitcoin-core-dev
351 2017-01-03T15:20:09  <jonasschnelli> can anyone give https://github.com/bitcoin/bitcoin/pull/8877 a final review?
352 2017-01-03T15:21:22  *** chris2000 has joined #bitcoin-core-dev
353 2017-01-03T15:34:52  *** helo has joined #bitcoin-core-dev
354 2017-01-03T15:37:29  *** blkdb has joined #bitcoin-core-dev
355 2017-01-03T15:40:41  *** BashCo_ has quit IRC
356 2017-01-03T15:41:18  *** BashCo has joined #bitcoin-core-dev
357 2017-01-03T15:45:35  *** BashCo has quit IRC
358 2017-01-03T15:50:29  *** fengling has joined #bitcoin-core-dev
359 2017-01-03T15:54:46  <cfields> BlueMatt: yes, cs_vSend wasn't touched because of your PR. I've just been operating under the assumption that the lock around SendMessages will be removed by one PR or another.
360 2017-01-03T15:55:44  <btcdrak> jonasschnelli: done.
361 2017-01-03T15:56:30  *** jtimon has joined #bitcoin-core-dev
362 2017-01-03T15:57:40  <jonasschnelli> thanks btcdrak
363 2017-01-03T15:57:52  <jonasschnelli> now I own you a review. :)
364 2017-01-03T15:57:53  <bitcoin-git> [bitcoin] jonasschnelli pushed 12 new commits to master: https://github.com/bitcoin/bitcoin/compare/e0dc3d70c6ec...6dc4c43d326e
365 2017-01-03T15:57:54  <bitcoin-git> bitcoin/master fc95daa Jonas Schnelli: Qt/RPCConsole: Save current command entry when browsing history...
366 2017-01-03T15:57:54  <bitcoin-git> bitcoin/master 9044908 Jonas Schnelli: Qt/RPCConsole: Don't store commands with potentially sensitive information in the history...
367 2017-01-03T15:57:55  <bitcoin-git> bitcoin/master de8980d Luke Dashjr: Bugfix: Do not add sensitive information to history for real...
368 2017-01-03T15:58:05  <bitcoin-git> [bitcoin] jonasschnelli closed pull request #8877: Qt RPC console: history sensitive-data filter, and saving input line when browsing history (master...qt_console_history_filter) https://github.com/bitcoin/bitcoin/pull/8877
369 2017-01-03T16:02:08  <jtimon> #9271 needs rebase but could still get some general feedback
370 2017-01-03T16:02:11  <gribble> https://github.com/bitcoin/bitcoin/issues/9271 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
371 2017-01-03T16:02:14  <instagibbs> can someone explain ProcessBlockAvailability? The comment for it is unclear: "/** Check whether the last unknown block a peer advertised is not yet known. */"
372 2017-01-03T16:02:28  <instagibbs> check whether an unknown block is unknown... ok!
373 2017-01-03T16:05:04  <jtimon> also, friendly reminder https://github.com/bitcoin/bitcoin/pull/8855 https://github.com/bitcoin/bitcoin/pull/9279 and https://github.com/bitcoin/bitcoin/pull/8498 still *don't* need rebase
374 2017-01-03T16:07:18  *** udiWertheimer has quit IRC
375 2017-01-03T16:07:21  <instagibbs> nevermind, think I got it
376 2017-01-03T16:07:44  *** xiangfu has quit IRC
377 2017-01-03T16:07:49  *** jannes has quit IRC
378 2017-01-03T16:07:58  *** fengling has quit IRC
379 2017-01-03T16:08:56  *** xiangfu has joined #bitcoin-core-dev
380 2017-01-03T16:09:11  *** BashCo has joined #bitcoin-core-dev
381 2017-01-03T16:13:07  * luke-jr rebases #9152 to add history filter before he forgets ☺
382 2017-01-03T16:13:09  <gribble> https://github.com/bitcoin/bitcoin/issues/9152 | Wallet/RPC: sweepprivkeys method to scan UTXO set and send to local wallet by luke-jr · Pull Request #9152 · bitcoin/bitcoin · GitHub
383 2017-01-03T16:19:47  *** jannes has joined #bitcoin-core-dev
384 2017-01-03T16:25:56  <BlueMatt> cfields: hmmm...yes, re: moving the loop into net_processing I figured that was just due to not complicating that pr too much, thanks
385 2017-01-03T16:26:30  <cfields> BlueMatt: np, thanks for review
386 2017-01-03T16:26:55  <BlueMatt> cfields: as for not having a per-node loop....hum...can we re-add it in net.cpp with two lines so that its not a (proabbly barely noticeable) regression?
387 2017-01-03T16:27:58  <BlueMatt> bool fMoreNodeWork = ... while (fMoreNodeWork) ProcessMessages
388 2017-01-03T16:28:07  <cfields> BlueMatt: I'm not sure there's any actual change from the current behavior
389 2017-01-03T16:28:11  <sdaftuar> instagibbs: ProcessBlockAvailability is for when you get block announcements via inv (before you have the header)
390 2017-01-03T16:28:38  <BlueMatt> cfields: you mean like from current master or from current pr?
391 2017-01-03T16:29:02  <BlueMatt> sdaftuar: we still have crap for that? can we just remove it and replace with "getheaders"
392 2017-01-03T16:29:10  <cfields> BlueMatt: oh.. it doesn't swallow all messages at once for fairness. I'm pretty sure emptying the node's full queue before moving on would be a DDoS vector
393 2017-01-03T16:29:15  <cfields> BlueMatt: from master
394 2017-01-03T16:29:42  <sdaftuar> BlueMatt: i think we could!
395 2017-01-03T16:29:54  <cfields> BlueMatt: see the comment here: https://github.com/bitcoin/bitcoin/pull/9441/commits/99599884147ea4db3f960b0e706b039ba1553c80#r94270593
396 2017-01-03T16:32:36  <BlueMatt> cfields: I believe the improvements against master come largely from removing lock contention between the message processing loop and the read-from-wire loop?
397 2017-01-03T16:32:42  <BlueMatt> or do I have that backwards?
398 2017-01-03T16:33:20  <BlueMatt> cfields: re: swallowing from one node...that isnt how I read the old code?
399 2017-01-03T16:34:01  <BlueMatt> I mean certainly if we got a packet which only had one message we'd not process more than one message at a time, but if the loop is slow to go around (it appears to be sometimes due to various messages being slow) then we might have multiple to process per-peer
400 2017-01-03T16:34:14  <cfields> BlueMatt: sorry, by no change in current behavior, i meant the swallowing-one-message part. Obviously fixing the lock contention is a behavioural change :)
401 2017-01-03T16:34:26  <BlueMatt> and in that case I think we'd prefer to only call SendMessages less often (though, to be fair, doing them back-to-back is less likely to be slow than otherwise)
402 2017-01-03T16:35:06  *** fengling has joined #bitcoin-core-dev
403 2017-01-03T16:35:09  <cfields> BlueMatt: take a look at the current code. I really don't think the new behavior is different, other than the case of corrupted messages
404 2017-01-03T16:35:15  <BlueMatt> cfields: to answer in a different way: why do we need to change from the current behavior
405 2017-01-03T16:35:25  <BlueMatt> cfields: whats the break you're referring to in that comment?
406 2017-01-03T16:35:29  <BlueMatt> (line numbers are confusing here)
407 2017-01-03T16:35:40  <BlueMatt> the !message.complete() one?
408 2017-01-03T16:36:00  <cfields> and yes, sendmessages needs to be broken up and fixed in lots of different ways, but imo not here
409 2017-01-03T16:36:01  <cfields> sec
410 2017-01-03T16:36:12  <BlueMatt> definitely agreed, not here
411 2017-01-03T16:36:31  <cfields> BlueMatt: whoops, that should be 2538
412 2017-01-03T16:36:31  <BlueMatt> just trying to figure out if re-adding a while somewhere in that pr is better or worse, since this is all we can hope for in 0.14 :)
413 2017-01-03T16:36:58  <BlueMatt> ohhhhhh, I hadnt seen /that/ break
414 2017-01-03T16:37:23  <cfields> yes. So the loop only continues in 2 cases, iirc.
415 2017-01-03T16:37:37  <cfields> bad hash, or uhmm.. bad start chars maybe?
416 2017-01-03T16:37:42  <BlueMatt> wait, so on current master we will continue the loop if the message is somehow corrupted, but otherwise wont? wtf.....
417 2017-01-03T16:37:57  <BlueMatt> oh, no, just bad hash
418 2017-01-03T16:38:00  <BlueMatt> yea, this makes no sense
419 2017-01-03T16:38:38  <cfields> BlueMatt: indeed, those are the cases that the node should be punished for, but instead they get a free pass.
420 2017-01-03T16:38:39  <cfields> go figure.
421 2017-01-03T16:38:47  *** xiangfu has quit IRC
422 2017-01-03T16:39:15  <BlueMatt> well, i guess if your checksum is bad its not like we did any work for you anyway...
423 2017-01-03T16:39:33  <cfields> sure, but that should at least toss you to the back of the line
424 2017-01-03T16:39:57  *** xiangfu has joined #bitcoin-core-dev
425 2017-01-03T16:40:29  <cfields> BlueMatt: so you agree now that the loop behavior isn't changed here significantly? (or, at least, only for the better?)
426 2017-01-03T16:41:14  <BlueMatt> yes, agreed
427 2017-01-03T16:41:25  <cfields> ok
428 2017-01-03T16:41:27  <BlueMatt> I suppose I wouldnt want to go audit for whether it was a dos vuln to change it in this pr
429 2017-01-03T16:41:29  <BlueMatt> so, good :)
430 2017-01-03T16:42:03  <cfields> heh
431 2017-01-03T16:42:32  <cfields> BlueMatt: as a follow-up for 0.14, if you're concerned about it, we could add a quick hack to quickly exit SendMessages if it hasn't been at least x msec since the last call
432 2017-01-03T16:42:43  <cfields> but i don't think there's any real need, since this has been the behavior all along
433 2017-01-03T16:44:05  <BlueMatt> yea, I'm less concerned about that...still hoping for parallel processmessages but....that seems stretchy now :/
434 2017-01-03T16:44:30  <cfields> :(
435 2017-01-03T16:45:11  <BlueMatt> depends on how many reviewers pop their heads up this week, I suppose
436 2017-01-03T16:46:25  <cfields> BlueMatt: other than the current PRs, how much remains there? I think that 9441 should cut down on the scary races a good bit?
437 2017-01-03T16:46:55  <BlueMatt> yea, I think after current PRs its just one or five std::atomics in CNode and then the one commit which fuckes around with the ProcessMessages loop
438 2017-01-03T16:47:06  <BlueMatt> which would need rebased on your stuff, but probably wouldnt be too hard
439 2017-01-03T16:47:53  <cfields> these are atomics that fix more than just CNodeStats which has always been racy anyway?
440 2017-01-03T16:48:14  <BlueMatt> I havent looked as of the latest stuff so you'd know better than I
441 2017-01-03T16:48:22  <BlueMatt> I mean we should probably atomic-ify CNodeStats...
442 2017-01-03T16:48:26  <BlueMatt> easy enough
443 2017-01-03T16:48:28  <cfields> I'll check out your branch again
444 2017-01-03T16:48:53  <BlueMatt> cool
445 2017-01-03T16:49:16  <BlueMatt> I'll probably not get much done until thurs....early flight tomorrow so will likely be super tired during the day at rwc
446 2017-01-03T16:49:30  <cfields> yea, I was thinking about that yesterday. I think it may make sense to just actually keep the stats in CNodeStats with one lock. See nRecvBytes as an example
447 2017-01-03T16:49:41  <cfields> then when we need the stats, just do a quick lock and copy out
448 2017-01-03T16:49:54  <BlueMatt> iirc that like automagically ended up with lock inversions
449 2017-01-03T16:50:04  <BlueMatt> i think i looked into that, but dont remember now
450 2017-01-03T16:50:12  <BlueMatt> seems weird that it would, but....something something...
451 2017-01-03T16:50:40  <cfields> hmm, not sure how. I'll look again
452 2017-01-03T16:50:57  <cfields> BlueMatt: btw, you're good with the boost changes that 9441 include?
453 2017-01-03T16:51:28  <BlueMatt> which boost changes? you mean #9289?
454 2017-01-03T16:51:30  <cfields> if so, mind re-acking #9289 ?
455 2017-01-03T16:51:30  <gribble> https://github.com/bitcoin/bitcoin/issues/9289 | net: drop boost::thread_group by theuni · Pull Request #9289 · bitcoin/bitcoin · GitHub
456 2017-01-03T16:51:33  <gribble> https://github.com/bitcoin/bitcoin/issues/9289 | net: drop boost::thread_group by theuni · Pull Request #9289 · bitcoin/bitcoin · GitHub
457 2017-01-03T16:51:34  <cfields> yes
458 2017-01-03T16:53:02  <BlueMatt> aww fuck, got rebased, yea, I'm pretty sure nothing changed that I didnt think was reasonable or good...I'll try to take another look tomorrow, but I'm pretty sure that ack is still valid
459 2017-01-03T16:53:05  <bitcoin-git> [bitcoin] MarcoFalke opened pull request #9462: [qt] Do not translate tilde character (master...Mf1701-qtTransTilde) https://github.com/bitcoin/bitcoin/pull/9462
460 2017-01-03T16:54:59  <cfields> BlueMatt: np, thanks
461 2017-01-03T16:56:08  <cfields> BlueMatt: if i could fixup the racy stuff that would affect parallel processing, want me to PR? Or does that just complicate the current PR maze?
462 2017-01-03T16:57:01  <BlueMatt> cfields: up to you...I had assumed you were waiting on the current prs to clean up a bit, but if you can do it without stepping on your own toes feel free
463 2017-01-03T16:58:22  <cfields> ok. I'll at least try to get something ready on top of the queued-up changes. If it happens to not depend on them, I'll go ahead and submit it.
464 2017-01-03T16:58:46  <BlueMatt> sounds good, thanks
465 2017-01-03T16:59:16  <BlueMatt> I'll look into the same tomorrow on the flight if I'm not asleep the whole time
466 2017-01-03T16:59:40  <cfields> heh, ok
467 2017-01-03T17:00:08  <BlueMatt> ehh, by "the same" I meant redoing the parallel-processmessages pr
468 2017-01-03T17:00:12  <BlueMatt> that was super unclear....
469 2017-01-03T17:08:26  *** abpa has joined #bitcoin-core-dev
470 2017-01-03T17:08:27  *** davec has quit IRC
471 2017-01-03T17:08:58  *** udiWertheimer has joined #bitcoin-core-dev
472 2017-01-03T17:08:59  *** davec has joined #bitcoin-core-dev
473 2017-01-03T17:09:08  *** udiWertheimer has quit IRC
474 2017-01-03T17:17:23  *** lejitz1 has quit IRC
475 2017-01-03T17:17:25  *** lejitz has joined #bitcoin-core-dev
476 2017-01-03T17:27:37  *** xiangfu has quit IRC
477 2017-01-03T17:30:50  <instagibbs> is there a plan for multiwallet for 0.14, or is it too late
478 2017-01-03T17:31:13  *** fengling has quit IRC
479 2017-01-03T17:35:00  <gmaxwell> we're not at the feature freeze yet, and it seems done +/- things that need to be fixed due to review.
480 2017-01-03T17:35:14  <gmaxwell> Several people have said they would review it.
481 2017-01-03T17:35:43  <gmaxwell> It would be really great it multiwallet and the utxo scriptpubkey index could make it in.
482 2017-01-03T17:39:53  *** fengling has joined #bitcoin-core-dev
483 2017-01-03T17:40:30  *** lejitz has quit IRC
484 2017-01-03T17:42:18  <instagibbs> #8694 needs rebase though
485 2017-01-03T17:42:20  <gribble> https://github.com/bitcoin/bitcoin/issues/8694 | Basic multiwallet support by luke-jr · Pull Request #8694 · bitcoin/bitcoin · GitHub
486 2017-01-03T17:42:27  *** xiangfu has joined #bitcoin-core-dev
487 2017-01-03T17:44:10  <luke-jr> instagibbs: #8776 goes first
488 2017-01-03T17:44:12  <gribble> https://github.com/bitcoin/bitcoin/issues/8776 | Wallet refactoring leading up to multiwallet by luke-jr · Pull Request #8776 · bitcoin/bitcoin · GitHub
489 2017-01-03T17:44:26  <instagibbs> ok, ill review
490 2017-01-03T17:45:28  <luke-jr> actually #8776 looks ready to merge. 3 reviews already
491 2017-01-03T17:45:30  <gribble> https://github.com/bitcoin/bitcoin/issues/8776 | Wallet refactoring leading up to multiwallet by luke-jr · Pull Request #8776 · bitcoin/bitcoin · GitHub
492 2017-01-03T17:45:36  <luke-jr> yes we know gribble --.
493 2017-01-03T17:46:55  *** jannes has quit IRC
494 2017-01-03T17:49:24  <BlueMatt> luke-jr: good time to go rebase 8694 then :p
495 2017-01-03T17:49:34  <luke-jr> sure
496 2017-01-03T17:51:23  <luke-jr> oh, need #8775 still as well
497 2017-01-03T17:51:25  <gribble> https://github.com/bitcoin/bitcoin/issues/8775 | RPC refactoring: Access wallet using new GetWalletForJSONRPCRequest by luke-jr · Pull Request #8775 · bitcoin/bitcoin · GitHub
498 2017-01-03T17:52:20  <luke-jr> instagibbs: ^ could use review
499 2017-01-03T18:03:36  <luke-jr> (once those two get in, looks reasonable to split up 8694 further into base/Qt/RPC PRs)
500 2017-01-03T18:17:57  *** paveljanik has joined #bitcoin-core-dev
501 2017-01-03T18:17:57  *** paveljanik has joined #bitcoin-core-dev
502 2017-01-03T18:20:05  *** bsm117532 has quit IRC
503 2017-01-03T18:36:36  *** dermoth has quit IRC
504 2017-01-03T18:46:14  *** arubi has quit IRC
505 2017-01-03T18:46:33  *** arubi has joined #bitcoin-core-dev
506 2017-01-03T18:50:28  *** lainknight has joined #bitcoin-core-dev
507 2017-01-03T18:55:22  *** Sosumi has joined #bitcoin-core-dev
508 2017-01-03T19:05:58  *** lejitz has joined #bitcoin-core-dev
509 2017-01-03T19:09:55  *** jtimon has quit IRC
510 2017-01-03T19:11:49  *** arubi has quit IRC
511 2017-01-03T19:12:04  *** arubi has joined #bitcoin-core-dev
512 2017-01-03T19:14:40  *** LeMiner has quit IRC
513 2017-01-03T19:31:58  *** Sosumi has quit IRC
514 2017-01-03T19:35:44  *** jl2012 has quit IRC
515 2017-01-03T19:35:44  *** kadoban has quit IRC
516 2017-01-03T19:35:44  *** kinlo has quit IRC
517 2017-01-03T19:35:44  *** Ylbam has quit IRC
518 2017-01-03T19:35:44  *** Victorsueca has quit IRC
519 2017-01-03T19:35:44  *** nOgAnOo has quit IRC
520 2017-01-03T19:35:44  *** limpkin has quit IRC
521 2017-01-03T19:35:44  *** PaulCape_ has quit IRC
522 2017-01-03T19:35:44  *** luke-jr has quit IRC
523 2017-01-03T19:35:44  *** murr4y has quit IRC
524 2017-01-03T19:35:45  *** _mn3monic has quit IRC
525 2017-01-03T19:35:45  *** Taek has quit IRC
526 2017-01-03T19:35:53  *** _mn3monic has joined #bitcoin-core-dev
527 2017-01-03T19:35:58  *** luke-jr has joined #bitcoin-core-dev
528 2017-01-03T19:36:01  *** luke-jr has quit IRC
529 2017-01-03T19:36:01  *** luke-jr has joined #bitcoin-core-dev
530 2017-01-03T19:36:23  *** murr4y has joined #bitcoin-core-dev
531 2017-01-03T19:36:54  *** Taek has joined #bitcoin-core-dev
532 2017-01-03T19:36:55  *** kadoban has joined #bitcoin-core-dev
533 2017-01-03T19:37:32  *** Victorsueca has joined #bitcoin-core-dev
534 2017-01-03T19:37:40  *** PaulCapestany has joined #bitcoin-core-dev
535 2017-01-03T19:38:22  *** profall has quit IRC
536 2017-01-03T19:39:40  *** jl2012 has joined #bitcoin-core-dev
537 2017-01-03T19:40:29  *** limpkin has joined #bitcoin-core-dev
538 2017-01-03T19:42:05  *** Ylbam has joined #bitcoin-core-dev
539 2017-01-03T19:46:20  *** profall has joined #bitcoin-core-dev
540 2017-01-03T19:48:58  *** nOgAnOo has joined #bitcoin-core-dev
541 2017-01-03T19:49:15  *** lejitz_ has joined #bitcoin-core-dev
542 2017-01-03T20:00:37  *** Chris_Stewart_5 has quit IRC
543 2017-01-03T20:09:48  <sdaftuar> BlueMatt: if I understand 9375 correctly, I think that when there is a fork, and some nodes have tip A and others tip B, #9375 would cause all nodes to end up downloading both A and B.
544 2017-01-03T20:09:51  <gribble> https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
545 2017-01-03T20:10:12  <sdaftuar> i guess that could result in a testnet DoS vector?
546 2017-01-03T20:10:30  <sdaftuar> maybe that's the least of our testnet problems though
547 2017-01-03T20:11:14  *** lejitz1 has joined #bitcoin-core-dev
548 2017-01-03T20:12:31  *** lejitz has quit IRC
549 2017-01-03T20:12:31  *** lejitz_ is now known as lejitz
550 2017-01-03T20:12:44  *** wvr has joined #bitcoin-core-dev
551 2017-01-03T20:16:53  *** Chris_Stewart_5 has joined #bitcoin-core-dev
552 2017-01-03T20:17:01  *** CubicEarth has joined #bitcoin-core-dev
553 2017-01-03T20:19:56  *** lainknight has quit IRC
554 2017-01-03T20:20:40  *** lejitz1 has quit IRC
555 2017-01-03T20:40:51  *** CubicEarth has quit IRC
556 2017-01-03T20:41:06  *** bsm117532 has joined #bitcoin-core-dev
557 2017-01-03T20:43:00  *** CubicEarth has joined #bitcoin-core-dev
558 2017-01-03T20:53:04  *** udiWertheimer has joined #bitcoin-core-dev
559 2017-01-03T20:53:53  *** dermoth has joined #bitcoin-core-dev
560 2017-01-03T20:58:33  *** laurentmt has joined #bitcoin-core-dev
561 2017-01-03T21:01:13  *** squidicuz has quit IRC
562 2017-01-03T21:01:42  *** squidicuz has joined #bitcoin-core-dev
563 2017-01-03T21:05:04  *** Squidicc has joined #bitcoin-core-dev
564 2017-01-03T21:05:55  *** laurentmt has quit IRC
565 2017-01-03T21:08:25  *** squidicuz has quit IRC
566 2017-01-03T21:52:40  <bitcoin-git> [bitcoin] sipa pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/6dc4c43d326e...ce5c1f4acae4
567 2017-01-03T21:52:41  <bitcoin-git> bitcoin/master 680b0c0 Suhas Daftuar: Release cs_main before calling ProcessNewBlock (cmpctblock handling)
568 2017-01-03T21:52:41  <bitcoin-git> bitcoin/master bd02bdd Suhas Daftuar: Release cs_main before processing cmpctblock as header
569 2017-01-03T21:52:42  <bitcoin-git> bitcoin/master ce5c1f4 Pieter Wuille: Merge #9252: Release cs_main before calling ProcessNewBlock, or processing headers (cmpctblock handling)...
570 2017-01-03T21:52:55  <bitcoin-git> [bitcoin] sipa closed pull request #9252: Release cs_main before calling ProcessNewBlock, or processing headers (cmpctblock handling) (master...cb-lock) https://github.com/bitcoin/bitcoin/pull/9252
571 2017-01-03T22:11:48  <bitcoin-git> [bitcoin] sipa pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/ce5c1f4acae4...2a524b8e8fe6
572 2017-01-03T22:11:48  <bitcoin-git> bitcoin/master fb0c934 Luke Dashjr: Wallet: Let the interval-flushing thread figure out the filename
573 2017-01-03T22:11:49  <bitcoin-git> bitcoin/master 5394b39 Luke Dashjr: Wallet: Split main logic from InitLoadWallet into CreateWalletFromFile
574 2017-01-03T22:11:50  <bitcoin-git> bitcoin/master 2a524b8 Pieter Wuille: Merge #8776: Wallet refactoring leading up to multiwallet...
575 2017-01-03T22:15:53  <gmaxwell> \O/
576 2017-01-03T22:16:29  *** Squidicuz has joined #bitcoin-core-dev
577 2017-01-03T22:18:37  *** Squidicc has quit IRC
578 2017-01-03T22:19:04  *** Squidicc has joined #bitcoin-core-dev
579 2017-01-03T22:21:37  *** Squidicuz has quit IRC
580 2017-01-03T22:30:10  *** Guyver2 has quit IRC
581 2017-01-03T22:31:12  <Chris_Stewart_5> Can we have OP_1NEGATE used as a push op code for witness program versioning?
582 2017-01-03T22:32:57  <sipa> no, only OP_0, and OP_1..OP_16
583 2017-01-03T22:34:16  <Chris_Stewart_5> Interesting, thanks sipa
584 2017-01-03T22:35:08  <Chris_Stewart_5> Actually, is there any reason for this?
585 2017-01-03T22:35:16  <sipa> not really
586 2017-01-03T22:35:24  <sipa> it was considered sufficient :)
587 2017-01-03T22:38:57  <luke-jr> by some* maybe; IMO it was overlooked.
588 2017-01-03T22:40:02  <Chris_Stewart_5> Defintely a protocol 'quirk' since the BIP does say '1 byte push opcodes'
589 2017-01-03T22:40:27  <luke-jr> not worth the effort to fix at this point though
590 2017-01-03T22:41:02  <sipa> fix the bip :)
591 2017-01-03T22:41:11  <sipa> (to match the code)
592 2017-01-03T22:41:30  <Chris_Stewart_5> but it is so much easier to criticize! :P. I'll submit a pull req later..
593 2017-01-03T22:44:05  *** fanquake has joined #bitcoin-core-dev
594 2017-01-03T22:45:00  <sipa> thanks!
595 2017-01-03T23:12:12  <sipa> Chris_Stewart_5: the BIP clearly says "for 0 to 16", no?
596 2017-01-03T23:12:25  <sipa> (i missed that when suggesting you send a PR, sorry)
597 2017-01-03T23:12:56  <Chris_Stewart_5> sipa: What is '0 to 16'? Bytes? Technically OP_1 is 0x51
598 2017-01-03T23:13:12  <Chris_Stewart_5> I don't like the wording in that regard either.
599 2017-01-03T23:13:22  <sipa> the value being pushed
600 2017-01-03T23:13:29  <Chris_Stewart_5> Why not explicitly state what ops they are?
601 2017-01-03T23:13:33  <sipa> sure
602 2017-01-03T23:13:45  <sipa> i trying to find the misunderstanding
603 2017-01-03T23:13:47  <Chris_Stewart_5> instead having to do the conversion in your head?
604 2017-01-03T23:13:53  <sipa> if it isn't clear, reformulating helps
605 2017-01-03T23:14:21  <Chris_Stewart_5> I don't think I have a misunderstanding per se, but I think can be clearer. Instead of having to infer the op codes from the text I think they should just be explicitly stated
606 2017-01-03T23:14:49  <Chris_Stewart_5> Plus it falls more inline with the actual implementation IMO
607 2017-01-03T23:14:54  <sipa> fair enough
608 2017-01-03T23:17:06  <fanquake> sipa You shouldn't see significant speedup with #8610 if you have a large -dbcache set (like 2048), should you?
609 2017-01-03T23:17:08  <gribble> https://github.com/bitcoin/bitcoin/issues/8610 | Share unused mempool memory with coincache by sipa · Pull Request #8610 · bitcoin/bitcoin · GitHub
610 2017-01-03T23:17:47  <sipa> fanquake: depends how large, but the effect should certainly be less dramativ
611 2017-01-03T23:17:50  <sipa> *dramatic
612 2017-01-03T23:19:09  <fanquake> Ok. Just ran through with -dbcache=2048 on master and that PR. Both basically the same (~800s) to reindex to 280k blocks.
613 2017-01-03T23:19:37  <fanquake> I think I'll run through again without uping the dbcache at all. That seemed to show the most significant speedup.
614 2017-01-03T23:20:20  *** droark has quit IRC
615 2017-01-03T23:21:54  <sipa> if you set dbcache to 10 i'm sure the difference will be spectacular :D
616 2017-01-03T23:22:23  <luke-jr> Chris_Stewart_5: adding "valid" seems to make it less clear IMO
617 2017-01-03T23:23:00  <Chris_Stewart_5> luke-jr: How so? A invalid 1 byte op code is OP_1NEGATE
618 2017-01-03T23:23:44  <sipa> how about "a select subset of 1-byte push opcodes" ?
619 2017-01-03T23:23:46  <Chris_Stewart_5> before it was a blanket statement, a 'one byte push op code' or something like that
620 2017-01-03T23:23:54  <fanquake> heh yes I'm sure it would. When I first tested with 300mb dbcache there was something like a 30% speedup.
621 2017-01-03T23:23:54  <sipa> oh, and OP_0 is not a 1-byte push
622 2017-01-03T23:24:03  <Chris_Stewart_5> oo interesting
623 2017-01-03T23:24:09  <Chris_Stewart_5> good point :/
624 2017-01-03T23:24:42  <sipa> fanquake: if you feel like benchmarking, i have a few more utxo cache changes to test coming up :)
625 2017-01-03T23:26:07  <fanquake> sipa sure
626 2017-01-03T23:26:41  <Chris_Stewart_5> sipa: Then maybe just a 'select set of op codes' and then enumerate them like I have?
627 2017-01-03T23:27:20  <sipa> Chris_Stewart_5: sgtm
628 2017-01-03T23:28:27  <luke-jr> Chris_Stewart_5: OP_1NEGATE is valid though
629 2017-01-03T23:28:50  <luke-jr> it's just not matched for witness purposes
630 2017-01-03T23:29:03  <Chris_Stewart_5> (*this)[0] < OP_1 does it fail this predicate?
631 2017-01-03T23:31:07  <Chris_Stewart_5> luke-jr: Here: https://github.com/bitcoin/bitcoin/blob/14d01309bed59afb08651f2b701ff90371b15b20/src/script/script.cpp#L228
632 2017-01-03T23:32:34  <sipa> Chris_Stewart_5: depends what *this is...
633 2017-01-03T23:34:05  *** justanotheruser has quit IRC
634 2017-01-03T23:34:36  <BlueMatt> sdaftuar: yes, your comment about only calling NewPoWValidBlock on things that are better than our current tip is a good one
635 2017-01-03T23:34:45  <BlueMatt> sdaftuar: and build on our tip
636 2017-01-03T23:36:00  <BlueMatt> sdaftuar: re: https://github.com/bitcoin/bitcoin/pull/9375#discussion_r94483881 I think I'm missing your point here? did you mean to comment a bit further down where the check is to set bool send?
637 2017-01-03T23:37:09  *** justanotheruser has joined #bitcoin-core-dev
638 2017-01-03T23:43:49  *** droark has joined #bitcoin-core-dev
639 2017-01-03T23:46:44  *** kinlo has joined #bitcoin-core-dev