19:00:31 <meshcollider> #startmeeting
19:00:31 <lightningbot> Meeting started Fri Jul  3 19:00:31 2020 UTC.  The chair is meshcollider. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:31 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:00:36 <meshcollider> #bitcoin-core-dev Wallet Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball ariard digi_james amiti fjahr
19:00:36 <meshcollider> jeremyrubin emilengler jonatack hebasto jb55
19:00:41 <BlueMatt> sipa: yes, afaict, once you 'find them' you're good, but essentially its super, super, super difficult to find onions from square 1
19:00:45 <achow101> hi
19:01:04 <sipa> BlueMatt: they only really propagate through onion-enabled peers
19:01:07 <BlueMatt> (presumably because they dont relay between non-onion peers nearly at all, and unless you are onion-only you'll be largely blind)
19:01:10 <meshcollider> Now, I know luke-jr still has a proposed topic, were there any more?
19:01:36 <BlueMatt> right, but my observations seem to indicate that you almost have to be onion-only, or at least connect to onion only to learn any, really
19:01:50 <BlueMatt> all anecdotal, of course
19:02:02 <achow101> is it too early to discuss deprecating and removing the legacy wallet?
19:02:10 <meshcollider> Yes :p
19:02:13 <sipa> achow101: yes
19:02:58 <meshcollider> Try again in v0.27
19:03:30 <achow101> i'll try again next year
19:03:44 <meshcollider> Is luke-jr here this time for his topic
19:04:39 <meshcollider> I guess not
19:04:56 <achow101> maybe if we say luke-jr enough times he'll show up
19:05:08 <sipa> tonal tonal tonal
19:05:52 <meshcollider> Alright, short meeting again then :)
19:05:55 <luke-jr> achow101 must be joking
19:06:11 <luke-jr> I need a reminder what my topic is :D
19:06:12 <luke-jr> …
19:06:14 <luke-jr> is my IRC broken?
19:06:19 <achow101> luke-jr: not anymore
19:06:23 <sipa> we can read you
19:06:27 <achow101> 2020-06-17.log:12:15 < luke-jr> #proposedwalletmeetingtopic revert #6550 (conceptually) - merkle branches stored in the wallet would be useful for pruned nodes [w/ watch-only wallets]
19:06:31 <gribble> https://github.com/bitcoin/bitcoin/issues/6550 | Do not store Merkle branches in the wallet. by sipa · Pull Request #6550 · bitcoin/bitcoin · GitHub
19:06:58 <luke-jr> ah right
19:07:14 <luke-jr> apparently Electrum-Personal-Server and similar projects could really use those merkle branches
19:07:41 <meshcollider> #topic Merkle branches stored in wallet (luke-jr)
19:08:17 <luke-jr> sipa: what was the motivation for removing them? is it okay to add them back?
19:08:18 <achow101> I can see the case for wanting these for pruned wallets in general
19:08:36 <achow101> it seems like the original motivation was that they were useless
19:08:36 <sipa> luke-jr: they were just not used by anything at the time
19:08:49 <sipa> no objection to adding them back if that is no longer the case
19:09:21 <luke-jr> should we bother to try to make it compatible with the old thing?
19:09:24 <sipa> vaguely related: we should also store the network fee for wallet transactions
19:09:30 <sipa> i don't think so
19:09:46 <achow101> i would prefer to be wholly new
19:09:59 <achow101> less headaches with possible compatibility states
19:10:10 <luke-jr> k
19:10:27 <luke-jr> what to do with wallets missing the info?
19:10:41 <sipa> a rescan will fix it
19:11:00 <luke-jr> but pruned wallets can't rescan necessarily
19:11:03 <luke-jr> and this is mainly for them..
19:11:12 <achow101> for not pruned wallets, it's easy to compute
19:11:25 <sipa> well if the data isn't there, there is nothing we can do
19:11:57 <achow101> i suppose a related feature would be the fetching of individual blocks over p2p for pruned nodes
19:12:00 <luke-jr> so maybe just omit it in RPC requests for txs missing it?
19:12:11 <sipa> achow101: ah the perennial feature request
19:12:12 <luke-jr> should it be an optional wallet feature?
19:12:38 <luke-jr> achow101: hmm, I suppose if we know the height it's possible, but seems complex
19:12:59 <luke-jr> and even if we get that, there's a period of time before the request is fulfilled
19:13:00 <achow101> luke-jr: I think we know the hash
19:13:26 <sipa> that's not enough to fetch it
19:14:33 <luke-jr> there's no way to get just the merkle tree I suspect
19:14:43 <luke-jr> bloom would get us exactly what we need, but has privacy problems
19:15:00 <luke-jr> in any case, I suspect all such fetching can be a later PR - it doesn't simplify the upfront feature
19:15:41 <achow101> it can just be one of those features that we do from now on and for old wallets that we can get the merkle branch for
19:16:12 <achow101> but for pruned nodes where those blocks have already been disarded, they're SOL
19:16:21 <luke-jr> achow101: I thought we weren't going to try to be compatible? besides, I think 0.12+ deleted that info even in old wallets?
19:16:50 <achow101> luke-jr: if you're adding a new record, it'll still be openable in old wallets, just not useful data
19:17:03 * luke-jr wonders if we should have a pruning setting to never discard blocks your wallet is involved in
19:17:05 <achow101> so it's not a version bump or an incompatible wallet flag
19:17:41 <luke-jr> achow101: oh, I misunderstood "old wallets that we can get the merkle branch for"
19:17:44 <jonatack> hi
19:17:49 <achow101> right
19:18:18 <luke-jr> is there any other info we might want to save?
19:18:33 <luke-jr> perhaps the transactions used as inputs?
19:18:37 <meshcollider> I feel like keeping the entire blocks your wallet is involved in would not be useful other than in weird cases like right now
19:18:51 <luke-jr> meshcollider: definitely don't want to do that ☺
19:18:53 <achow101> didn't we use to save transactions used as inputs
19:19:00 <luke-jr> oh, not in the wallet
19:19:05 <luke-jr> achow101: I think so
19:19:09 <sipa> we should store unconfirmed inputs too, i think
19:19:24 <sipa> but not using the exponential-blowup-fest approach that was used before
19:19:41 <luke-jr> sipa: I forget what we had before for this
19:20:01 <luke-jr> was the problem it saved the txs in the wallettx?
19:20:08 <sipa> yes
19:20:19 <luke-jr> so a new db entry per txid should fix that, right?
19:20:20 <achow101> CWalletTx needs to be trimmed
19:20:51 <luke-jr> refcounted I suppose
19:21:07 * luke-jr feels like he's had this discussion recently already :x
19:21:50 <sipa> i think we can just store the inputs as wallet txn
19:22:04 <achow101> We might want to store just the prev txos?
19:22:12 <sipa> which won't be IsMine(), but just there to fetch
19:22:18 <luke-jr> hmm
19:22:30 <luke-jr> achow101: that won't work if we need to send the entire tx to the hw wallet?
19:22:31 <sipa> achow101: i don't think that's useful
19:22:50 <sipa> ah, in taproot it may be
19:23:04 <sipa> but i was more thinking that this can help rebroadcasting
19:23:26 <achow101> ah.
19:23:50 <luke-jr> it used to be your wallet would keep the transactions it has an interest in alive..
19:23:58 <luke-jr> I guess we lost that
19:24:12 <achow101> I think storing the prevtxs as a wallet txn will confuse many things
19:24:23 <sipa> it may be, i'm not sure
19:24:45 <achow101> I'm pretty sure there's an implicit assumption that things in mapWalletTxs are actually involved in your wallet
19:24:49 <achow101> as in an input or output is yours
19:25:09 <sipa> maybe - but i think most things still filter using IsMine
19:25:21 <luke-jr> harmless to just change the db key
19:25:27 <achow101> I'd like to not do that (as much) in the future :)
19:25:28 <luke-jr> can read it as a wallettx type if that's useful
19:25:56 <achow101> right, just have it as another record would probably be better
19:26:03 <sipa> perhaps yes
19:26:23 <sipa> that also has less risk causing backward compatibility issues
19:28:14 <achow101> topic suggestion: legacy to descriptor wallet migration
19:29:23 <meshcollider> #topic legacy to descriptor wallet migration (achow101)
19:30:08 <achow101> did we determine whether it was possible to make all of the legacy things fit into descriptors?
19:30:44 <meshcollider> What do you mean by possible
19:30:55 <achow101> I would actually like to figure out how/if we can migrate people's wallets to descriptors so they don't lose anything
19:31:21 <sipa> sure, but it may need a huge amount of descriptors
19:31:39 <achow101> as in can we represent everything that could be in a legacy wallet as descriptors, and preferably not as thousands of descriptors
19:32:12 <achow101> I think we previously had concern that IsMine would match to a nearly infinite number of scripts, but I don't think that's possible anymore?
19:32:31 <sipa> i believe that's not the case anymore since bare multisig matching is gone
19:32:34 <achow101> s/possible/concern
19:33:38 <meshcollider> You can probably turn an HD wallet into a descriptor wallet okay
19:33:48 * luke-jr still prefers JBOK wallets
19:34:08 <sipa> infer a descriptor from HD information, expand it, see which scriptpubkeys it does not cover
19:34:26 <sipa> store the remaining ones as additional descriptors
19:34:29 <meshcollider> Obviously a non-HD wallet it's gonna be thousands of individual descriptors
19:34:30 <achow101> for non-HD, that's going to be a lot of descriptors
19:34:37 <sipa> yes
19:34:50 <achow101> what about imports?
19:35:13 <sipa> perhaps it makes sense to have a list() descriptor or something, which contains a list of non-ranged descriptors
19:35:28 <sipa> so that you can have one thing that represents an entire wallet, but still expose it as a "list"
19:35:54 <sipa> s/list/range/
19:35:56 <meshcollider> And have a similar range index thing as a ranged descriptor?
19:36:00 <sipa> right
19:36:21 <achow101> also, with descriptors we don't have the address type mutation thing, so I think migrating would require generating new active descriptors and all the old stuff is no longer used
19:37:34 <luke-jr> achow101: that was never *supposed* to work though
19:37:36 <luke-jr> do we need to support it?
19:38:24 <achow101> luke-jr: i mean retaining possible mutations on the old stuff that has already been used
19:38:38 <sipa> luke-jr: i think it's dangerous to remove things that were treated as IsMine
19:39:14 <luke-jr> I think once an address is used, we don't need to look for it in new blocks anymore <.<
19:39:30 <achow101> unfortunately that's not a reasonable assumption
19:39:48 <sipa> yeah
19:41:49 <achow101> so I think it should be possible to migrate, but it might be a bit ugly
19:42:01 <achow101> that's good to know
19:42:25 <meshcollider> The list descriptor is worth thinking about imo
19:42:54 <sipa> it's easy to add, but may become huge :)
19:43:18 <meshcollider> Certainly would become huge in a lot of cases :p
19:43:27 <luke-jr> I don't really get what there is to gain from this
19:43:45 <meshcollider> Achow just wants eventual legacy deprecation
19:44:16 <achow101> it's so we can ditch the legacy wallet in the future without forcing people to make wholly new wallets
19:44:22 <luke-jr> why?
19:44:43 <meshcollider> Because descriptor wallets are better
19:44:48 <luke-jr> …
19:44:56 <achow101> less stuff to maintaini
19:45:13 <luke-jr> couldn't we just load the old keys as descriptors in memory?
19:45:31 <achow101> yes but that's (probably) expensive
19:45:33 <meshcollider> That loses most of the benefits
19:45:43 <meshcollider> Anyway any other topics?
19:47:02 <meshcollider> #endmeeting