19:00:07 #startmeeting 19:00:07 Meeting started Thu Jun 15 19:00:07 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:07 Useful Commands: #action #agreed #help #info #idea #link #topic. 19:00:24 PSA: v0.14.2 has been tagged, please start your gitian builders 19:00:30 topics? 19:00:34 #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier 19:00:34 #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 instagibbs achow101 19:00:36 hi. 19:00:40 Hi 19:00:41 hi 19:00:42 jonasschnelli: i just wrote a quick mutex locktime reporter, ping me after meeting if you'd like to discuss 19:00:44 here 19:00:45 yow 19:00:49 (I guess I should update my list) 19:00:57 cfields: awesome! 19:01:03 wumpus: you're requesting the presence of 2 instagibbses? 19:01:07 cfields: finally someone did that, awesome. 19:01:20 well it's very dumb, but it's something :) 19:01:21 sipa: yes! 19:01:34 cfields: -DDEBUG_LOCKCONTENTION? 19:01:38 lol 19:01:38 cfields: dumb is good 19:01:51 #topic high priority for review 19:02:04 https://github.com/bitcoin/bitcoin/projects/8 19:02:08 #10148 plz *puppyeyes* 19:02:10 https://github.com/bitcoin/bitcoin/issues/10148 | Use non-atomic flushing with block replay by sipa · Pull Request #10148 · bitcoin/bitcoin · GitHub 19:02:26 10148 is already on the list, I'm testing it 19:02:30 cool 19:02:33 i'm working on an rpc test as well 19:02:39 sdaftuar: awesome! 19:02:42 can i help? 19:03:00 i'll let you know! 19:03:05 i think i almos thave it 19:04:08 having thought more about it, i don't think #10339 will have any significant performance impact 19:04:11 https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub 19:04:22 basic multiwallet was merged. I hope to have the next step (RPC support) later today. ACK to add to priority then? 19:04:48 RPC support would be great 19:04:53 agree 19:04:55 You mean addressing wallet via RPC endpoint? 19:04:55 luke-jr: yes please 19:05:08 jonasschnelli: I mean each username has a different single wallet 19:05:22 luke-jr: hmm... 19:05:24 sipa: ok, removing it from high priority then 19:05:27 I'd prefere endpoints 19:05:39 AUTH for wallet switching seems hackish 19:05:41 jonasschnelli: endpoints can be done later; I'm just trying to get the simplest stuff done first 19:05:50 endpoint is 10lines of code 19:05:54 I'd also prefer endpoints, makes it easier to move wallets to external processes etc just by changing the url 19:05:57 I can post it to you later 19:06:00 jonasschnelli: not securely ;p 19:06:10 jonasschnelli: I don't want JoinMarket to have access to my main wallet 19:06:21 there is no security in our RPC implementation :/ 19:06:24 yay, wallet ACLs 19:06:25 * sipa hides 19:06:30 please don't use auth, it's not supposed to be a multi-user authentication mult-wallet, that just adds another nightmare difficult to support like accounts 19:06:31 you could do some combination of both? 19:06:31 wumpus: different username is also a simple change to the URI 19:06:43 we already support different usernames/passwords 19:06:49 use a passphase as wallet name 19:06:55 it's an authentication feature, should not affect the wallet 19:06:56 same security as basic auth? 19:07:23 i think the first step should be either endpoint or a generic optional named parameter to select the wallet 19:07:24 wumpus: I see no distinction 19:07:28 sipa: yes 19:07:28 endpoints won't be ten lines of code. After all, we'll need to add support for them to bitcoin-cli, the test framework, etc. 19:07:34 sipa: agreed 19:07:43 the choice of which user can access which wallets is orthogonal, i think 19:07:50 gmaxwell: which is easy, the underlying stuff (rpcproxy, libevent) obviously supports it 19:07:55 but i would prefer not to tie users to wallets at the auth level 19:07:57 gmaxwell: yes Indeed 19:08:48 first stage, each user should be able to access each wallet.... 19:08:48 obviously we do want to have username/wallet binding, right? This lets you be more confident e.g. that your joinmarket install isn't going to screw up your ordinary wallet, for example. 19:08:48 sipa: me neither, it just seems a level violation, and causes wrong expectations that giving access to RPC to one wallet is secure in any way 19:08:54 (eventually) 19:08:57 jonasschnelli: no -.- 19:09:11 gmaxwell: I really think that's going too far 19:09:23 then why are we bothering? 19:09:27 securing RPC for multiple users is absolutely a nightmare 19:09:32 luke-jr: the first logical extendable step would be that, no? Adding wallet selecting via AUTH is something you need to throw away later 19:09:49 well, if I can't isolate JoinMarket this way, I have no interest in doing it.. so I can just move on to GUI and leave RPC support in Knots only 19:09:51 wumpus: i think it's inevitable that we'll need that 19:09:53 anyhow a security layer could always be added could be later if endpoint-based multiwallet is in place 19:09:58 sipa: I think it's a mistake 19:10:05 sipa: just like accounts was 19:10:13 it's something that bitcoind shouldn't handle 19:10:27 I think what luke would like to accomplish is making multiwallet immediately useful for the application of combining multiple applicatoins onto one bitcoind; rather than having to run seperate bitcoinds for each thing that needs a wallet that you're running. 19:10:30 jonasschnelli: no? 19:10:59 gmaxwell: i think that's an interesting use case; i don't think it should be the first step 19:11:04 that's just inviting bugs, there's no way we can make that secure, the RPC is not a secure endpoint and is regarded as compeltely trusted 19:11:34 it would escalate a bug in e.g. a single RPC command to a security issue, right now RPC access = fully trusted 19:11:36 luke-jr: I don't know the JoinMarket use case very well.. but if you give it access to your node, it could shutdown, add peers, etc. (in case you don't trust that software) 19:11:45 This is also important to us at blockstream and we will end up maintaining a fork of Bitcoin with it. (though luke wasn't doing this work at our request). 19:11:58 jonasschnelli: even if we add endpoint multiwallet and ACLs later, we still want a way to select a default wallet for each user 19:12:20 wumpus: then why do we have auth at all? 19:12:32 I really think we should keep hands away from multi-user/multi-wallet setup 19:12:34 luke-jr: to gain access 19:12:47 jonasschnelli: me too... seems something that needs to be a level on top, not handled by bitcoind itself 19:12:55 For now we should focus on single-user/multi-wallet (1:n) 19:13:15 n:n smells like a account-like-problem-re-incarnation 19:13:20 anyhow if we have endpoint multi-wallet access, it'spossible to slap on a wallet/user auth mapping later 19:13:34 or vice-versa.. 19:13:38 that's "just" a matter of access control 19:13:44 yes. n:n may make sense.. but endpoint first seems much more logical 19:13:48 yes 19:13:51 jonasschnelli: I do not follow your comment with account like problems. The problem with accounts is that they weren't wallets but users expected them to be and treated them like ones. 19:14:06 gmaxwell: Yes. Not directly related. 19:14:10 i don't think access control is necessarily that complicated; have a global permission and wallet specific permission; configure which users have which 19:14:13 I just think that making bitcoind multi-user is a grave mistake 19:14:18 but I"ll shut up about it... 19:14:24 I think the complexity is huge,.. leads to permission groups, etc. 19:14:35 yes, exactly, some people wnat everything in bitcoind 19:14:44 jonasschnelli: what? no it doesn't. 19:14:47 well it seems that multiple people want multiwallet for multiple reasons 19:14:53 i don't think that's a problem 19:15:04 and should not be a blocker for the basic functionality 19:15:07 this is one the reason why the wallet should have been split off to a separate process / library I guess... now it all needs to be compounded 19:15:11 making bitcoind some kind of systemd 19:15:11 if we start to use n:n, enterprises will probably use it for multi-user wallet backends... 19:15:32 wumpus: user:wallet makes a split off later simpler 19:15:33 and removing – if it gets to complicated – is hard or even impossible (like the accounting) 19:15:34 jonasschnelli: yes exactly... and what if there's a bug in that 19:15:37 endpoints makes splitting off later complex 19:15:54 it moves all the (perceived) responsiblity for managing multi-user setups secure to us 19:16:00 luke-jr: endpoint would even work if each wallet runs in its own process 19:16:04 How do we split wallets if we are using endpoints? 19:16:08 (though auth probably also) 19:16:17 jonasschnelli: huh? not really..? 19:16:23 but multi-wallet doesn't imply multi-user, does it? 19:16:25 gmaxwell: what do you mean with "split wallets"? 19:16:36 split wallets int oseperate processes 19:16:51 gmaxwell: different wallets have different URLs then 19:17:04 gmaxwell: so it's just another change: change the port... 19:17:17 wumpus: ALL of these options are simple URI changes.. 19:17:27 how would different endpoints work with bitcoin-cli or the debug console? 19:17:28 although some tools don't allow changing the URI right now 19:17:31 http://127.0.0.1:8333/wallet1 versus http://127.0.0.1:8334/wallet2 19:17:39 achow101: thats why it isn't ten lines of code. 19:17:42 achow101: you can add: -wallet=filename 19:17:55 achow101: just add an option 19:18:03 s/filename/name/ 19:18:09 endpoints in bitcoin-cli is not really complex... 19:18:10 achow101: debug console isn't via RPC anyway 19:18:11 sipa: yes 19:18:14 but it certainly wouldn't work with debug console 19:18:14 sipa +1 19:18:39 as for debug console: you could ask the same question about authentication 19:18:40 luke-jr: I think we'll need endpoints in any case regardless of auth to set a default. 19:18:52 gmaxwell: maybe 19:18:52 achow101: the whole GUI has no multiwallet interface 19:18:58 luke-jr: because that will be what you need to make it usable to work with multiple wallets as a single user. 19:18:59 debug console is not authenticated at all - so adding endpoint/auth support is likely the similar amount of work 19:19:24 if access control is implemented, a single user could want to have access to multiple wallets anyhow 19:19:25 debug console should perhaps get a dropdown, and yes, it will be the same work either way.. probably easier with endpoints. 19:19:31 so user=wallet is a bad abstraction 19:19:36 I already have the GUI done BTW 19:19:43 yes you'll want to have access to multiple wallets from a single user regardless. 19:19:44 it's just based on the RPC branch 19:19:51 luke-jr: how does it let you select the wallet? 19:19:52 luke-jr: how does gui handle the debug console? 19:19:56 A very hackish (and very old) endpoint impl 19:19:57 sipa: comboboxes 19:20:03 A very hackish (and very old) endpoint impl for bitcoin-cli: https://github.com/jonasschnelli/bitcoin/blob/2015/05/corewallet/src/bitcoin-cli.cpp#L134 19:20:05 one in the main window, and one in the debug window 19:20:20 sounds more or less okay. 19:20:56 luke-jr: so why not implement endpoints first? surely even if your own use needs account you can carry a 5 line patch to allow accounts to select the default wallet. 19:20:57 wumpus: even if a single user can access multiple wallets, we still want a way to choose the default 19:21:05 see above 19:21:21 gmaxwell: it's more code, and not done yet 19:21:23 well the default wallet could depend on the user, I don't really care 19:21:31 I can implement it, but IMO it will delay things to make it the next step 19:21:32 though I'd prefer to get rid of 'default wallet', in time 19:21:35 maybe the GUI should have a node window (network, peers) and a wallet-window per wallet... 19:21:52 jonasschnelli: ugh 19:21:58 the user=>wallet stuff is literally done and well-tested (in Knots), just needs to be rebased 19:21:59 jonasschnelli: that doesn't sound like a good UI. :P 19:22:07 jonasschnelli: /me remembers browsers before tabs 19:22:08 at least not mandatory. 19:22:13 what sipa says. :P 19:22:20 yeah... I like windows.. but I'm pretty alone nowadays with that 19:22:31 Yeah. Tabs make more sense I guess. 19:22:37 anyway, separate discussion 19:22:52 * jonasschnelli think sipa certainly browses with lynx 19:23:04 i would really prefer endpoints or optional named argument to select a wallet, and deal with the authentication question later 19:23:15 sipa: +1 19:23:16 luke-jr: in any case, seems to me the path forward is to do the endpoints thing, and having auth pick default is a simple change which is either sufficiently non-objectionable or at least a trivial patch to carry. 19:23:21 sipa: ack 19:23:22 sipa: same for me 19:23:27 would anyone NACK if I go forward with user->wallet mappings since they're basically ready, and then do endpoints based on that? 19:24:07 probably 19:24:09 luke-jr: does it also support one user with many wallets? 19:24:12 well as we determined above, a user may want to have access to multiple wallets, so a single user->wallet mapping just doesn't cut it, even if you want to add access control 19:24:20 what wumpus says. 19:24:28 gmaxwell: the current code does not, but there's no reason the endpoints couldn't add that 19:24:29 I really think we should just start with endpoints as sipa says 19:24:52 Isn't rpcuser deprecated anyway? 19:24:54 Yes. Let's start with endpoint.. I'll can write it next week because I already did once... 19:24:57 I'm not going to NACK anything that makes progress though 19:24:59 jnewbery: it's rpcauth 19:25:00 jnewbery: rpcauth isn't 19:25:01 *I can 19:25:29 jnewbery: rpcuser is, but this is rpcauth (rpcuser doesn't even have multiple users) 19:25:29 jnewbery: is rpcuser deprecated? since when? 19:25:38 jtimon: a year? 19:25:39 jtimon: it's deprecated since a long time ago 19:25:42 it prints out a notice! 19:25:50 rpcuser is deprecated, people are encouraged to use either rpcauth or cookie auth 19:25:59 we won't remove it just yet ofcourse 19:26:09 oh, deprecated as in "we want to remove this", but it actually still works, no? 19:26:14 26 minutes... 19:26:14 right 19:26:15 yes 19:26:17 that is what deprecated means, yes 19:26:28 yeah, sorry 19:26:29 paveljanik, 34 to go :P 19:26:43 paveljanik: what's so special about 26? 19:26:49 jonasschnelli: any progress on GUI for database upgrade? 19:26:58 #topic GUI for database upgrade? 19:27:00 And we've wasted a perfectly good half hour. :P luke should put up patches and we can yell at him on github, but I think I would really prefer if the first cut does multiple wallets for a user... (if nothing else, that is the easiest thing to test) 19:27:00 sipa: I sadly had only little time last and this week 19:27:07 wumpus: 21 is half of 42 19:27:18 sipa: I looked into it and wanted to ask you how I get the max size of a db cursor (to calc progress) 19:27:42 jonasschnelli: it's not hard to estimate as txids are randomly distributed 19:27:51 so you just look at the txid... 19:27:54 i can add code for that 19:27:55 they're done in order. 19:28:07 if it's at 0x01... then it's done 1/256 of it. 19:28:12 txid -> arith_uint256 -> * 100/2^256 19:28:29 Okay. The rest is simple (debug.log non newline [10%] progress / GUI splash screen progress with abort) 19:28:39 BTW: jnewbery rebased the label API pull (#7729), a lot of thanks for that 19:28:41 https://github.com/bitcoin/bitcoin/issues/7729 | rpc: introduce label API for wallet by laanwj · Pull Request #7729 · bitcoin/bitcoin · GitHub 19:28:50 jonasschnelli: i'll write code for the progress estimation 19:29:08 sipa: Okay. Pass me over a commit and I'll finish the rest 19:29:14 wumpus: no problem. I wanted to test drive it :) 19:30:38 wumpus: \O/ on the label rebase. 19:30:53 must have been a nightmare 19:31:27 any other topics? 19:31:51 what about the rpc splitting,... has that been discussed so far? 19:32:04 There had been quite a few refactors. I think the rebase was good, but reviewers should look out for anything that looks off 19:32:06 signrawwithkey, etc.? 19:32:09 rpc splitting? 19:32:26 the PRs I made to unfuck signrawtx and validateaddress 19:32:42 ah https://github.com/bitcoin/bitcoin/pull/10583 19:32:47 #10583 19:32:48 achow101: yes. Those.. 19:32:48 https://github.com/bitcoin/bitcoin/issues/10583 | [RPC] Split part of validateaddress into getaddressinfo by achow101 · Pull Request #10583 · bitcoin/bitcoin · GitHub 19:33:04 #topic split off wallet functionality from mixed wallet/non-wallet RPC calls 19:33:08 or something 19:33:22 #10570 19:33:23 https://github.com/bitcoin/bitcoin/issues/10570 | [RPC] Split signrawtransaction into multiple distinct RPCs · Issue #10570 · bitcoin/bitcoin · GitHub 19:33:45 concept ack (haven't gotten around to reviewing anything) 19:33:48 I think those changes all make sense. Someone commented about breaking compatibility, but its for a new major version and it will be easy for callers to update their behavior. 19:33:59 i'm here now.. haven't caught up on backlog 19:34:06 it breaks compatibility with a never documented or advertized feature :) 19:34:07 well we could allow both, for one major version 19:34:10 Though we might want to rename the old calls at the same time. (a suggestion for discussion) 19:34:20 (concatenating multiple tx hex strings, yuck) 19:34:25 wut? 19:34:32 ok, we're talking about different things 19:34:37 aside from mixing wallet/nonwallet, what's the issue with validateaddress? 19:34:38 sipa: it was certantly advertised and known. 19:34:40 I mean validateaddress/getaddressinfo 19:34:42 (or is that the issue) 19:34:50 instagibbs: that's the issue 19:34:51 wumpus: ah, i'm talking about signrawtransaction 19:34:52 wumpus: he's talking about the signraw split to create the combine call. 19:34:52 instagibbs: that is the issue 19:34:57 ok, are we killing off getinfo then too: 19:34:59 :) 19:35:05 hopefully :D 19:35:11 instagibbs: yes, but that's not the topic now 19:35:16 I would miss getinfo. all the other commands take more typing. :P 19:35:17 we're already confusing two things, let's add more! 19:35:25 wumpus: okay! 19:35:25 eh, we're talking about blowing away validateaddress as is, sorry 19:35:26 gmaxwell: hey I have a pull that implements it client side 19:35:46 wumpus: lol. can you name the rpc call "gi" :P even less typing. 19:35:47 IMO having a non wallet sign rawtx where priv keys are passed aroung in a shell over a possible TCP channel is not ideal.. but we already have it... I though instead of splitting it off, move it to the tool 19:35:55 But I see the point with getting the UTXOs 19:36:01 let's rename all RPCs to get*info... for example s/sendtoaddress/getnewpaymenttxid/ 19:36:08 jonasschnelli: we need the UTXOs or its all awfulsauce. 19:36:11 sipa, lol 19:36:16 #8843 19:36:17 https://github.com/bitcoin/bitcoin/issues/8843 | rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` by laanwj · Pull Request #8843 · bitcoin/bitcoin · GitHub 19:36:18 lol 19:36:28 getsignedtransaction 19:36:31 gmaxwell: I though the node RPC can spitout what you need to pass it into bitcoin-tx or so... 19:36:48 but I know... very inconvinient 19:36:48 getthistransactionbroadcast 19:36:50 :p 19:36:51 jonasschnelli: sure it can; it's just more convenient to not need that 19:36:51 eventually closed it because the only person responding was luke-jr and he kept arguing against it 19:37:08 jonasschnelli: and that adds steps to the process.. which is already long enough that it's prone to error. 19:37:14 jonasschnelli: it's called listunspent 19:37:16 It's just another source how people can shoot themselfs with exposing priv keys 19:37:28 luke-jr: why do you hate freedom? 19:37:32 gmaxwell: !⁈ 19:37:35 haha 19:38:30 jonasschnelli: but the functionality already exists, and i very much like removing it from the wallet (so people at least won't accidentally mix up privkey based operations with wallet stuff) 19:38:54 sipa, ok I see the motivation there 19:38:58 anyways, I can just say that I have tried to stop using getinfo and failed. Mostly because typing getnetworkinfo getblockchainfo getfooooooooooinfo and then wading through a bunch of things when I want to see: How many connections, which block am I at, and what wallet am I running (which I can tell via the balance). :P just personal feedback. 19:39:01 gmaxwell: I didn't even NACK it :o 19:39:04 Yes... I guess that makes sense. I kinda hoped once we touch that we could move it away from the node into a sep. process 19:39:21 gmaxwell: use the GUI for that! :p 19:39:31 jonasschnelli: i have a vague proposal for that too, but it's more complicated 19:39:37 luke-jr: again, why do you hate freedom? :P 19:39:50 jonasschnelli: and involves a new format for partially signed transactions... 19:39:57 gmaxwell: anyhow we can easily reopen and rebase that PR, bitcoin-cli hardly changed since then 19:40:01 wumpus: ack 19:40:03 sipa: that contains everything you need to sign? 19:40:08 gmaxwell: make a shell alias to all the calls ;) 19:40:23 it's so much easier to have a signing blob thing post segwit. :( 19:40:37 [13bitcoin] 15laanwj reopened pull request #8843: rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` (06master...062016_09_getinfo_clientside) 02https://github.com/bitcoin/bitcoin/pull/8843 19:40:39 jonasschnelli: yes, contains amounts, change info, prevouts being spent, ... 19:40:40 yes, let's activate segwit 19:40:46 luke-jr: I don't like customizing my expirence of bitcoin too much because then I'll just patch around everything that stinks. 19:40:47 * sipa revives segnet 19:41:09 sipa: That's also something we could re-use for the detatched signing standard (a.k.a hardware wallet standard) 19:41:11 we could just leave getinfo how it is ;) 19:41:26 in any case, we're offtopic. I think that achow's PRs are all nice incremental improvements and we should take them (after review) 19:41:28 jonasschnelli: pre-segwit however, it also needs to contain the full spent transactions :( 19:41:33 bitcoin-cli -getinfo only handles 1 sortof-use-case, and leaves the other 2 supported use cases unaddressed 19:41:35 gmaxwell: agree 19:41:37 after promising to deprecate it for years... yeah, of course.... 19:41:52 (disclaimer: achow101'w my intern this summer, i asked him to work on those) 19:42:02 sipa: [full spent transactions], I guess that's okay. 19:42:25 jonasschnelli: bitcoind unfortunately can't do that generically (you need the wallet for that) 19:42:25 getinfo is going away, there's no going back now 19:42:44 let's add a getallinfo then 19:42:46 /s 19:42:47 I've exactly documented what information you can find on what get*info command 19:42:54 I am fine with it going away, but I don't believe we replaced it as well as we thought we did. 19:43:01 and the client-side getinfo is ther for user friendlyness, if people want it 19:43:01 i have no problem with removing it, with or without 8843 19:43:15 wumpus: it doesn't work in the debug window 19:43:16 i'm sure i'll curse a bit that getinfo isn't around anymore, and then change my habits 19:43:34 sipa: I tried blocking it, you won't. the replacements right now are not usable. 19:43:43 :/ 19:43:50 But thats okay, wumpus suggestion would be fine, though luke has a point about the debug console. 19:43:53 luke-jr: isn't all the information in the debug window *without* typing anything? 19:43:57 removing getinfo will mess with a ton of things that use getinfo for basic rpc connection checking too... 19:44:11 achow101: and you're starting to bring that up *now*? 19:44:12 but I think it should be removed anyways 19:44:12 let's merge the getuptime rpc thing 19:44:13 wumpus: if it isn't we should make it. problem solved. 19:44:22 [13bitcoin] 15ryanofsky opened pull request #10605: Add AssertLockHeld assertions in CWallet::ListCoins (06master...06pr/listlock) 02https://github.com/bitcoin/bitcoin/pull/10605 19:44:23 achow101, move to dumpprivkey obv 19:44:33 gmaxwell: the first tab of the debug window pretty much shows everything, and indeed, if it isn't it could be added 19:44:49 oh, another topic: non-hardened key derivation 19:44:51 wumpus: well most of those things are old website scripts that were written once and never touched again by the authors 19:44:54 sipa, ACK 19:44:54 luke-jr: I think ^ is how we should handle the gui. (also important to make it copy/pasteable if it isn't.) 19:45:01 I'm really disappointed that years after deciding to deprecate getinfo we're still having this discussion 19:45:12 wumpus: lol maybe :D 19:45:14 anyhow next topic 19:45:20 wumpus: sometimes you have to try things out to know their effects completely! 19:45:30 #topic non-hardened key derivation 19:45:32 so 19:45:53 non-hardened key derivation has many use cases in addition to hardened 19:46:02 I guess NicolasDorier made good work there 19:46:07 achow101: we should carefully note it in the release notes of course 19:46:13 however, they also have a gaping wide security hole when child private keys are exposed 19:46:23 achow101: we could even make getinfo fail with a custom message 19:46:29 that plus dumpwallet will give you sads 19:46:30 thus, suggestion: allow a new wallet to be created with either harderned or unhardened keys 19:46:40 when you choose unhardened, dumpprivkey is disabled 19:46:44 xpub is only accessible through dumpwallet right now AFAIK 19:46:45 wumpus: returning null would probably not mess with anything 19:46:49 (but dumpmasterkey or whatever is still available) 19:47:11 sipa: there is no dumpmasterkey 19:47:11 instagibbs: dumpprivkey must be disabled anyways 19:47:19 sipa: I'd want to be able to mix them.. 19:47:23 achow101: dumpmasterkey must be added 19:47:30 luke-jr, .... why 19:47:31 this is a lot more interesting with multiwallet support in place, since the cases where you want that are mostly secondary wallets (like incoming payments with keys generated by your webserver) 19:47:37 dumpmasterkey, isn't there a PR for that? 19:47:42 no 19:47:47 instagibbs: to generate reusable payment tokens 19:47:49 luke-jr: i guess that's fine; just disable dumpwallet, and dumpprivkey selectively for keys derived in a non-hardened fashion 19:47:51 sipa said or whatever for a reason. :P 19:47:54 jonasschnelli: I'm stil lwaiting #9504 19:47:55 https://github.com/bitcoin/bitcoin/issues/9504 | [RPC] dumpmasterprivkey command by achow101 · Pull Request #9504 · bitcoin/bitcoin · GitHub 19:48:04 oh... 19:48:05 sipa: sgtm 19:48:09 see, I wasn't crazy 19:48:25 wumpus: well, technically it doesn't prove that... 19:48:26 achow101: how can I not be aware of that PR... sorry for the missinfo 19:48:34 gmaxwell: true 19:48:37 getmissinfo 19:48:47 :/ 19:48:56 (I even commited on the PR ^^) 19:48:59 commented 19:49:15 getmisinfo would really fit with the spirit of the times 19:49:19 Anyways, I think that sounds okay. These applications will likely need a couple of extra RPCs too. no need to design here however. (e.g. it will need to export the extended public key.) 19:49:33 not much more to say about the topic - just pointing out that if we disable dumping child private keys, my concern with non-hardered derivation largely goes away 19:49:47 anyhow, no problem with non-hardened key support 19:49:58 I guess with allowing xpub derivation, flexible keypath would be welcome.. 19:50:04 People want to use BIP44 19:50:06 as an option, not as default 19:50:08 w/ disabling and it not being a default thing. sounds great to me. 19:50:25 sgtm 19:50:51 Yes. Would be a great change... 19:50:59 also so long as we do the extra rpcs to make it actually useful (like extract the extended public keys, and whatever else is needed to handle an external address generator.) 19:51:13 I guess keypool handling would be much simpler with xpub derivation 19:51:22 I doubt it? 19:51:30 I think it's the same-ish 19:51:33 Why would you need a keypool with xpub derivation? 19:51:41 for scanning. 19:51:41 you derive when you need 19:51:50 you have master seed already, you can already do that 19:51:50 you always derive the lookup window in mem 19:51:53 you need to scan forward to know when you're paid. 19:52:05 +1000 keys in mem should be ackish 19:52:18 (pubkeys) 19:52:26 yes, the keypool doesn't technically need to be saved on disk, though it may be faster to do so then to generate thousands of addresses at every wallet load. 19:52:39 but otherwise I think its the same. 19:52:56 I'd say loading the keypool at wallet load takes almost the same (or longer) 19:53:28 sipa: why would people want to dump child private keys? 19:53:32 okay, this would make the change much more intrusive then I think. 19:53:39 jtimon, you want to stop them from doing so 19:53:47 jtimon: same reason some people eat paste. (they don't know better) 19:53:52 heh 19:53:57 jonasschnelli: the reason creating new keys is slow, is because we flush them all individually to disk 19:54:11 yes... 19:54:17 gmaxwell: I see 19:54:21 Yes. In-mem non-bdb "keypool" would be much faster 19:54:41 that flushing is not required anymore with hdwallets regardless, I believe. 19:54:45 gmaxwell: exactly 19:55:00 With xpub derivation, would there be really a need to write the pool to disk? I doubt it 19:55:07 (we will still need a flush to know how many we've given out...) 19:55:09 gmaxwell: good point! 19:55:29 All you need is the seed 19:55:44 jonasschnelli: no, not a need, though why result in more distinct codepaths? I would expect it to also make loading faster. 19:55:47 you just need to write the path of the most recent key 19:55:48 gmaxwell: yes. thats a good point 19:56:11 achow101: Yes. 19:56:14 achow101: yes, every new address still needs to do a wallet flush. But on bulk key creation we do not need a flush for each operation. 19:56:38 e.g. starting up for the first time with a keypoool of 10000. 19:58:26 two minutes to go 19:58:33 jonasschnelli: did you ever get around to implementing the change where any key that is noticed used on the blockchain marks all the earlier keys in its chain as used? 19:58:52 gmaxwell: HD restore? 19:59:12 https://github.com/bitcoin/bitcoin/pull/10240 19:59:16 it's part of your restore pr? okay. 19:59:25 That makes all keys as used up to the one found 19:59:30 great. 19:59:34 It can even temp. halt the sync if you prune, etc. 19:59:39 Needs overhaul and rebase... 19:59:46 It's already (too) big 19:59:54 ryanofsky gave me a hard time 20:00:01 #getmeetingendinfo 20:00:12 #endmeeting