19:00:44 #startmeeting 19:00:44 Meeting started Thu Jul 6 19:00:44 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:44 Useful Commands: #action #agreed #help #info #idea #link #topic. 19:00:51 #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier 19:00:56 proposed topic: multiwallet endpoint vs json item 19:00:56 LO 19:00:56 topics? 19:01:01 hi 19:01:04 hi 19:01:10 hi. 19:01:15 jonasschnelli: yeah, apparently we have to discuss that again, with all the competing PRs 19:01:22 heh. Yes 19:01:29 jonasschnelli: though in principle we settled on endpoint a few weeks ago 19:01:31 begging for review... lots of fee/wallet/estimate stuff that needs to make 0.15 19:01:41 i already have 3 on high priority... sheepish grin 19:01:53 yes, high priority for review will as usual be first topic 19:01:54 morcos: We should do the things. 19:01:55 We set endpoints, but some where also in favor of the JSON item solution 19:02:01 #topic high priority for review 19:02:04 PSA: if you're running master, be very careful not to swap -txindex on your db: the check to prevent you from doing so is broken and you could corrupt your chainstate 19:02:17 https://github.com/bitcoin/bitcoin/projects/8 19:02:29 by swap txindex he means turn it on/off on an already running node. 19:02:44 I'll remove my #10240 from the list for now 19:02:46 good to know... 19:02:47 without a reindex-chainstate I guess 19:02:49 gmaxwell: you mean already created db? 19:02:50 https://github.com/bitcoin/bitcoin/issues/10240 | Add HD wallet auto-restore functionality by jonasschnelli · Pull Request #10240 · bitcoin/bitcoin · GitHub 19:03:01 jonasschnelli: ok 19:03:22 It's to big and will re-focus during early 0.16 19:03:27 maybe put #8498 in project 8 ? 19:03:30 https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub 19:03:33 wumpus: the arguments for endpoint seem strong IMO 19:03:38 morcos, doesn;t help that they're an unconfirmed chain of PRs :) 19:04:06 instagibbs: i know! :) high priority review ones arent though 19:04:07 we need a chain length limit on PRs 19:04:08 guess we're not on that topic yet 19:04:11 jtimon: is that high priority to get into 0.15? 19:04:17 luke-jr: next topic 19:04:25 wumpus: I think 10179 is ready(ish) for a merge, which makes my high-prio of 10652 cleaner 19:04:49 wumpus: I haven't benchmarked, but it's an optimization and now also a "near bugfix" 19:04:56 BlueMatt: i was just rereviewing that, but don't wait on me, i'm out of sync these days and doing all posthumous review 19:05:51 BlueMatt: agree 19:06:16 #10179 19:06:19 https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub 19:07:13 yes, second PSA: never shy away from postumous review! the feeling that its not contributing to moving things forward is wrong, if you think something got merged without enough acks, just review it! 19:07:27 or if you want to be sure to understand the new code! 19:07:32 well, that too 19:07:44 Or if you want to understand the code in the first place! :) 19:08:21 :) 19:08:51 ok, so anything that needs to be added to project 8? 19:09:15 i have other things needed for 0.15, but they are dependent on the ones i already have in 8 19:09:28 also i already have 3 19:09:28 ok, just tag them for 0.15 then, don't need to be in that project 19:09:36 #topic RPC interface for multiwallet (again) 19:09:36 wumpus: doesn't that qualify for priority? 19:09:55 can someone give an overview of what people are thinking on interface for multiwallet... i missed this 19:09:56 jtimon: if the gain is unclear, I don't think so 19:09:57 Again we should decide wether we use Endpoints of JSON objects for multiwallet switch... helps to continue on PRs 19:10:05 instagibbs: please read the current PRs: 19:10:08 wumpus: can we have #10571 #10579 in 0.15? 19:10:09 https://github.com/bitcoin/bitcoin/issues/10571 | [RPC]Move transaction combining from signrawtransaction to new RPC by achow101 · Pull Request #10571 · bitcoin/bitcoin · GitHub 19:10:10 https://github.com/bitcoin/bitcoin/issues/10579 | [RPC] Split signrawtransaction into wallet and non-wallet RPC command by achow101 · Pull Request #10579 · bitcoin/bitcoin · GitHub 19:10:10 The JSON object is simpler... less impect 19:10:10 wumpus: I think it's clearly a gain 19:10:36 the endpoint approach may allow more in future... 19:11:02 I don't understand the criterion then 19:11:16 #10650 #10653 19:11:17 In the JSON object approach (where you choose the wallet bases on a JSON array item), I don't like that the actual switch in in the JSON layer. 19:11:17 https://github.com/bitcoin/bitcoin/issues/10650 | Multiwallet: add RPC endpoint support by jonasschnelli · Pull Request #10650 · bitcoin/bitcoin · GitHub 19:11:18 https://github.com/bitcoin/bitcoin/issues/10653 | Simple, backwards compatible RPC multiwallet support by ryanofsky · Pull Request #10653 · bitcoin/bitcoin · GitHub 19:11:26 I like the JSON interface, but I worry that when we split out the wallet it will break 19:11:29 wumpus, add those to multiwallet project? 19:11:35 #10650 19:11:36 It also only works with the new named based argumenst 19:11:36 https://github.com/bitcoin/bitcoin/issues/10650 | Multiwallet: add RPC endpoint support by jonasschnelli · Pull Request #10650 · bitcoin/bitcoin · GitHub 19:11:40 eh what's the third one? 19:12:00 endpoints seemed okay, until the API bump got tacked on.. 19:12:02 I guess the third one (based on Auth) has already been "rejected"? right? 19:12:08 I don't like the JSON based interface, having to add an optional wallet argument on every wallet call is easy to forget 19:12:14 #10661 works with positional arguments, not just named arguments 19:12:15 https://github.com/bitcoin/bitcoin/issues/10661 | Add optional wallet=filename arguments to wallet RPCs by ryanofsky · Pull Request #10661 · bitcoin/bitcoin · GitHub 19:12:21 and if you forget it it defaults to the 'default wallet' 19:12:26 that's just too easy to mess up 19:12:38 ryanofsky: thanks for clearing this up... wasn't aware, sry 19:12:40 the endpoint makes sure you can be connected to only one wallet with one RPC connection 19:12:49 jonasschnelli: right! 19:12:54 i think we should just get rid of the concept of default wallet 19:13:04 ryanofsky: on the long run, yes, but that's no option for 0.15 19:13:05 ryanofsky: definitely not in 0.15 19:13:13 what about, if more than one wallet, then default wallet must be explicitly specified 19:13:17 if there's more than one wallet, it should just be an error not to specify a wallet 19:13:20 let's focus on what we want to do now 19:13:30 then you break all existing sw 19:13:31 I think for 0.15 we should simply do the endpoint-based interface 19:13:34 we can do that right now 19:13:36 wumpus: what do you think about the concern that the endpoint stuff establishes a new API that we'll be stuck supporting but haven't given much thought to? 19:13:37 no need to wait 19:13:52 i think as an end goal, endpoint-based selection is awesome, because it prepares for process separation 19:13:53 gmaxwell: the same is true for any RPC change 19:13:56 gmaxwell: we can mark it unstable? 19:14:00 v1 == unstable? 19:14:09 use / (v0) if you want stability 19:14:15 but if endpoints can't for example remove the non-wallet RPCs, that's sort of not really achieving that goal anyway 19:14:18 gmaxwell: adding an argument to every wallet RPC call is also such a massive change 19:14:29 wumpus: with named args it's trivial, no? 19:14:41 I can work on splitting the RPC calls in wallet / nonwallet 19:14:47 (I don't have strong opinions, just raising it) 19:14:48 if we agree on endpoints 19:14:51 wumpus, are you saying 10661 is a massive change? 19:14:51 and it shouldn't be adding it to every RPC; just catch it in the rpc handler 19:14:55 but the point is that it'd be something that has to be supported virtually forever 19:14:58 (only supporting the default wallet, per rpcauth user, seems the best for backward/forward compatibility still) 19:14:59 and imo it's poorly thought out 19:15:16 but I don't care deeply 19:15:21 at this point we should simply make a choice 19:15:25 if we don't make a choice today and stick with it 19:15:26 I don't really think named arguments is a great thing. It would make support easier in some software in the short term. 19:15:30 we can forget multiwallet for 0.15 19:15:36 ack! 19:15:36 I think every criticism wumpus has on that one is spot on. 19:15:56 * luke-jr suggests rpcauth-based default wallet, and we can figure out endpoints for 0.16 19:16:18 gmaxwell: indeed - most RPC client libraries don't even support named arguments yet 19:16:22 that gives more time to think out API change 19:16:25 gmaxwell: while changing the endpoint is easy, just change the URI 19:16:48 luke-jr: please don't bring back a third option 19:16:52 * BlueMatt always kinda assumed named args would allow us to add things like multiwallet/different number precision/etc in the future, as a simple add-on to every RPC without any massive code change everywhere 19:16:53 luke-jr: that's not going to make it easier 19:16:54 but, ok 19:16:59 luke-jr: are wallets assigned per rpcauth user already? 19:17:07 no 19:17:10 no 19:17:12 uh.. 19:17:26 guess I had a different impression than everyone else, then 19:17:26 kanzure: there is no way to use multiwallet right now 19:17:47 BlueMatt: it's possible, and not hard to implement, but just not the right choice for this IMO 19:17:56 What about using v1/wallet/y and mark it unstable (experimental?) for 0.15? 19:17:59 BlueMatt, that was my impression too, it's the basis for 10661 & 10653 19:18:00 -y 19:18:05 i officially no longer have an opibion on approach 19:18:07 jonasschnelli: sounds good to me 19:18:10 ok, I mean I dont have a very strong impression, i just always thought that seemed natural 19:18:18 wumpus: we should have a per-user default wallet *regardless* of the other options; merging it first is a clean way to defer choosing between the others 19:18:20 but, really, can we flip a coin? 19:18:32  19:18:38 let's just go with endpoints for now 19:18:55 Somone disagree? 19:18:57 how the heck did you send a blank line 19:18:59 Anyone 19:19:01 I think we could say that the endpoint version totally unstable and will change to answer the concern that we're setting an api there premately. 19:19:04 if no one cares deeply, let's just stick with the decision of a few weeks ago 19:19:25 gmaxwell, ack 19:19:29 gmaxwell: we could mark the whole multiwallet (incl. endpoint) as experimental in 0.15 19:19:33 And stable in 0.16 19:19:34 midnightmagic: that wasn't a blank line, it as \x7f characters 19:19:42 gmaxwell: yes, multiwallet is unstable in 0.15, +1 19:19:51 gmaxwell: there's probably quite some things that need to change, still 19:19:58 no opinon on the issue, but ACK on making a decision. 19:20:17 ryanofsky: could you live with the endpoint solution? 19:20:43 I think in general we should get into a practice of new API's being explicitly unstable in their first release. We've mulliganed quite a few times. 19:20:53 gmaxwell: yes 19:20:53 of course, yeah 19:21:13 okay. Let me finish the endpoint PR and hope it will make it into 0.15 19:21:18 jonasschnelli: great! 19:21:33 /topic 19:21:40 ryanofsky: thanks 19:21:59 jonasschnelli: can you do it on top of 7b73f24311639fdc79c22608c21e4bfcbc6d4243 ? 19:22:06 pr #? 19:22:08 any other topics? 19:22:53 remember morcos was saying something about fee PRs, but not sure it was aimed as a topic 19:22:56 wifi just fied in the BS office 19:22:58 jonasschnelli: it's part of #10615 19:22:59 I just want to say that master continues to be mostly awesome and performing great. I'm really excited about this next release. (esp if we get our act togeather on multiwallet) :) 19:23:00 https://github.com/bitcoin/bitcoin/issues/10615 | RPC: Allow rpcauth configs to specify a 4th parameter naming a specific wallet (multiwallet RPC support) by luke-jr · Pull Request #10615 · bitcoin/bitcoin · GitHub 19:23:11 https://github.com/bitcoin/bitcoin/pull/10615/commits/7b73f24311639fdc79c22608c21e4bfcbc6d4243 19:23:13 gmaxwell: yeah! 19:23:15 luke-jr: 7b73f24311639fdc79c22608c21e4bfcbc6d4243 pollutes server.h with CWallet... :/ 19:23:18 (later) 19:23:29 there are a bunch of fee PRs which I think are very useful, and we should try desperately to make progress on them for 15 19:23:34 So yes, there are a number of fee/change handling PRs which are urgent for 0.15. 19:23:38 yeah i don't really have a topic, but i need some review 19:23:42 some are bug fixes 19:23:47 But I don't know what to say beyond that since they're already on the high prio list. 19:23:48 #topic fee PRs 19:23:53 some are RPC api finalization which would be good to get right 19:24:03 another topic proposal could be: txoutsbyaddress (it's marked with the 0.15 milestone) 19:24:38 I'm not sure if I broke it up in the easiest way possible for review, but am hesitant to try to reorganize this late in the game... 19:24:42 jonasschnelli: bleh, server.h should definitely not get a CWallet reference, it's meant to be not specific to bitcoin, let alone wallet 19:25:02 wumpus: yes. I think the same 19:25:16 wumpus: jonasschnelli: I don't see a better alternative. 19:25:43 sounds like jonasschnelli also has his hands full with multiwallet and i think it would have been nice to get access to longer fee estimates in the GUI 19:25:44 (keep in mind not all calls will come from the RPC server) 19:25:48 but seems like that is not going to happen 19:25:52 luke-jr: https://github.com/bitcoin/bitcoin/pull/10650/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR36 19:26:00 luke-jr: there are certainly alternatives, more general ways to attach custom data to a structure, but let's leave this for another time 19:26:06 morcos: oh the gui doesn't have access to the new estimates? thats unfortunate. 19:26:14 I guess I need to do some gui testing, haven't used it in a while. 19:26:16 gmaxwell: no 19:26:16 [13bitcoin] 15ryanofsky closed pull request #10661: Add optional wallet=filename arguments to wallet RPCs (06master...06pr/multiopt) 02https://github.com/bitcoin/bitcoin/pull/10661 19:26:23 gmaxwell: just not the conf target > 26+ 19:26:27 jonasschnelli: that doesn't work for GUI or tests 19:26:35 no way to ask for non-conservative. but at least after one of my PR's it'll default to that if tx is replaceable 19:26:39 anyhow, later.. 19:26:57 non-conservative would be simple (a checkbox?) 19:27:15 a slider with fix positions make little sense... sliders are ment to be linear 19:27:22 A dropdown could make more sense 19:27:26 * luke-jr likes dropdown 19:27:30 jonasschnelli: yes sort of. the way it is implemented elsewhere is it defaults to the opposite of opt in rbf, but you coudl force it either way 19:27:43 "ASAP", "today", "this week", "eventually" 19:28:05 luke-jr: names tend to bikeshed... but at least "conf-target in block | time" 19:28:13 * luke-jr shrugs 19:28:31 Or maybe "time | blocks | feerate" 19:28:52 Ideally we would run coinselection when opening the dropdown to tell the (possible) absolute fee 19:29:03 jonasschnelli: please no 19:29:08 feerate selection first 19:29:12 then coin selection 19:29:14 heh.. I though somebody will complain. :) 19:29:18 coin selection needs fee rate.. 19:29:20 we can't realistically do that. We need the feerate to perform selection. 19:29:32 in the future coin selection may be different depending on feerate anyway 19:29:38 (and will need it more in the future) 19:29:39 gmaxwell: do it for all options ... *duck* 19:29:48 >_< 19:29:59 coin selection can be slow, unless that's been optimised 19:30:00 Thoughs about the dropbox? 19:30:10 "The next Bitcoin-Qt version requires a 4k screen for coin selection" 19:30:14 in any case, i think _something_ simple would be ideal so users have access to longer than 25 confirms 19:30:15 lol 19:30:17 jonasschnelli: well we'd like to be able to do good selections which won't be instant. thats something that could be expiremented with later. 19:30:18 I'm happy to do it if it's general accaptable (the dropbown) 19:30:21 sipa: ooo, I have those, sounds gogod! 19:30:23 good 19:31:08 well we do want multiple near term options because of market effects. e.g. 2,3,4,5,6,72,today,two days, three days, five days, 1 week... or something. 19:31:17 what about a box with number of blocks instead of a dropbox ? 19:31:26 jonasschnelli: sure. something, anything. but recommend you build off my other PR #10706 19:31:27 https://github.com/bitcoin/bitcoin/issues/10706 | Improve wallet fee logic and fix GUI bugs by morcos · Pull Request #10706 · bitcoin/bitcoin · GitHub 19:31:29 I'll try the dropdown and see how it feels.. should not be that hard 19:31:31 er dropdown 19:31:34 morcos: will do 19:31:35 do we have to do GUI design in this meeting? 19:31:43 XD 19:31:50 gmawell, it could estimate with a blocktarget of ~6 and do that before the window opens? ;) 19:31:52 no i was just hoping for someone else to volunteer since seems jonas has a lot to do 19:32:11 or whatever the default is nowaays 19:32:14 and 0.15 is fast approaching 19:32:16 sipa: that wasn't GUI design, quite the opposite, there are economic reasons that gui clumping people wouldn't be great. 19:32:19 I could give it a shot, I guess. 19:32:44 luke-jr: I just started... :) 19:32:50 heh 19:32:57 either or, i'll let you guys work it out, but i'm bad at gUI, but i did make several changes already in the prior PR. thanks!! 19:33:50 #topic txoutsbyaddress (it's marked with the 0.15 milestone) 19:33:55 I think we should remove that milestone 19:34:06 Sadly. Has anyone been working on it? 19:34:12 #9806 has been quite inactive 19:34:14 https://github.com/bitcoin/bitcoin/issues/9806 | txoutsbyaddress index (take 3) by droark · Pull Request #9806 · bitcoin/bitcoin · GitHub 19:34:15 I'm more interested about should we or or not add an index for that 19:34:24 not publicly at least 19:34:37 The best index implementations is currently the one from Bitpay,.. not? 19:34:43 jonasschnelli: I think we should; unlike many other things it's actually sustainable. 19:34:59 I tend to also think we should 19:35:06 (after installed a BWS index) 19:35:08 the bitpay index stuff is utterly unmaintable and borderline abandonware; fwiw. 19:35:21 why not by scriptpubkey ? that seems more generic 19:35:25 there is some indexd project 19:35:30 The most stables index I could find so far is that one: https://github.com/bitcoin/bitcoin/pull/10370 19:35:36 jtimon: yes, I think that was suggested on the PR. 19:35:42 dcousens has an external index project which I think sipa is referring to 19:35:50 (it's the one BWS [Bitpay wallet service] is unsing in the fileld since a couple of years) 19:36:27 jonasschnelli: the UTXO indexes are special because they actually have viable scalablity... 19:36:43 https://github.com/bitcoinjs/indexd 19:36:48 I think anyone depending on complete blockchain indexes will eventually be forced onto centeralized servers, unfortunately. 19:37:02 so I have much less interest in internal support (hooks for external things sound fine though) 19:37:44 Agree. Maybe internal txoutsbyaddress and for the rest, use something like indexd that sipa mentioned 19:38:08 https://github.com/bitcoinjs/indexd 19:38:09 UTXO indexes would be nice, I'd love to have more query functionality for the UTXO set, we have to track that anyway with a full node 19:38:14 i've taken to using zmq to notify of new txes/blocks and the REST API to retrieve parsed info about them, for indexing externally 19:38:25 I've got to run, but someone please tag #10557 #10589 #10707 #10712 for 0.15 19:38:26 https://github.com/bitcoin/bitcoin/issues/10557 | Make check to distinguish between orphan txs and old txs more efficient. by morcos · Pull Request #10557 · bitcoin/bitcoin · GitHub 19:38:27 I guess one reason why some of the centralized services (like the BWS) still is based on 0.12.1 is because the indexes where never added to Core master branch 19:38:28 https://github.com/bitcoin/bitcoin/issues/10589 | More economical fee estimates for RBF and RPC options to control by morcos · Pull Request #10589 · bitcoin/bitcoin · GitHub 19:38:29 https://github.com/bitcoin/bitcoin/issues/10707 | Better API for estimatesmartfee RPC by morcos · Pull Request #10707 · bitcoin/bitcoin · GitHub 19:38:30 https://github.com/bitcoin/bitcoin/issues/10712 | Add change output if necessary to reduce excess fee by morcos · Pull Request #10712 · bitcoin/bitcoin · GitHub 19:39:06 huh what is gribble doing 19:39:12 wumpus: right. also if ever we support having pruned wallets (wallets that don't know their full history, but do have their full coins), the txout index is something we need for it... but not other indexes. 19:39:12 oh sure morcos 19:39:12 (sorry) 19:39:21 gmaxwell: yes! 19:39:30 tagged 19:39:56 gmaxwell: yes. For HD restores in pruned env. the utxo index is handy 19:39:57 gmaxwell: would be very niec to instantly query the balance, if history isn't important 19:40:33 rescan has become so slow for me at least that I'm kinda desperate for something like that. 19:40:41 I've lost days of time waiting on rescans. 19:41:42 next topic? 19:42:09 nope. 19:42:25 wumpus: can you remind me of the 0.15 schedule? 19:42:30 eh, UTXO isn't a substitute for rescanning.. you'd miss historical txs 19:42:48 #9961 19:42:48 https://github.com/bitcoin/bitcoin/issues/9961 | Release schedule for 0.15.0 · Issue #9961 · bitcoin/bitcoin · GitHub 19:42:58 luke-jr: thus pruned wallet, ... it's fine to not have historical txs if you know you don't. 19:43:13 BlueMatt: thanks! 19:43:15 luke-jr: what if you don't care about history, and just want balance + possibly spending? 19:43:21 T-10 days to branch 19:43:22 gmaxwell: but you'd end up with *some* historical tx in this case, with no deterministic reason why some are missing 19:43:27 luke-jr: you can also say that txes found on the blockchain aren't a replacement for having their metadata... :) 19:43:35 I suppose we could import them as all change-only or something? :/ 19:43:38 luke-jr: you can scan in the background for the history in a very slow manner once you have done it via the UTXO set index 19:43:40 BlueMatt: no, not to branch, to feature freeze 19:43:50 wumpus: it'd be incompatible with hardware wallets, before segwit 19:43:51 jonasschnelli: hmm, good idea 19:43:58 as you need the full crediting txn 19:44:00 luke-jr: no you wouldn't: my suggestion is that a pruned wallet basically have a line shown in the UI where nothing is there before it except a move like ledger entry that shows the earlier balance. 19:44:13 oh, sorry, yes, freeze 19:44:16 BlueMatt: branch is 2017-08-06, so a month away 19:44:20 luke-jr: I made an issue describing some ideas for that a while back. 19:44:21 gmaxwell: I see, makes sense 19:45:05 wumpus: can #10571 and #10579 be tagged for 0.15? 19:45:07 https://github.com/bitcoin/bitcoin/issues/10571 | [RPC]Move transaction combining from signrawtransaction to new RPC by achow101 · Pull Request #10571 · bitcoin/bitcoin · GitHub 19:45:08 https://github.com/bitcoin/bitcoin/issues/10579 | [RPC] Split signrawtransaction into wallet and non-wallet RPC command by achow101 · Pull Request #10579 · bitcoin/bitcoin · GitHub 19:45:21 There are other reasons why building such things are attractive... (e.g. UTXO based sync can't provide the data to give history...) 19:45:32 achow101: sure 19:45:44 thanks 19:45:48 ack 19:48:55 early lunch? 19:49:52 ye fine with me 19:49:54 #endmeeting