19:04:56 <sipa> #startmeeting
19:04:56 <lightningbot> Meeting started Fri Oct 19 19:04:56 2018 UTC.  The chair is sipa. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:04:56 <lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
19:05:14 <sipa> welcome to the first bitcoin core wallet meeting!
19:05:30 <meshcollider> \o/
19:05:57 <sipa> i think it would be good to start with listing what people are working on
19:07:39 <gmaxwell> I am working on nothing wallet related. (there, now everyone doesn't have to worry, since you can't be doing worse than nothing)
19:08:16 <meshcollider> I've been working on getting importmulti working with all the segwit address types and everything recently, and trying to get through  most of sipa's descriptor PRs for review :)
19:08:29 <gwillen> oh hm, I am glad I happened to see this but I am interested in how one finds out about these meetings and their topics :-)
19:08:46 <sipa> gwillen: discussed yesterday in the bitcoin core meeting :)
19:08:58 <sipa> i'm working on a number of smaller descriptor related improvements, before digging into fully importing descriptors into the wallet
19:09:52 <meshcollider> Should we ping the list of Devs from the main meeting in case anyone who wanted to be here forgot
19:10:04 <sipa> meshcollider: sgtm
19:11:00 <meshcollider> #bitcoin-core-dev Wallet meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
19:11:47 <jonasschnelli> hi
19:12:21 <kanzure> hi.
19:12:49 <gwillen> Wallet-wise: I am still working on the offline signing usecase, modeled after the flow armory uses with gui interfaces for create-sign-broadcast.
19:12:54 <kanzure> gwillen: also it was discussed in tokyo. you were there.
19:13:03 <gwillen> kanzure: like I remember things people say.
19:13:14 <luke-jr> lol
19:13:14 <jamesob> anyone know of an easy way to clear the sig/script caches of a running bitcoind process?
19:13:22 <achow101> hi
19:13:30 <sipa> jamesob: i'm not sure it's possible
19:13:36 <jamesob> oops! sorry to interrupt the meeting :)
19:13:36 <sipa> jamesob: but meeting now :)
19:13:43 <luke-jr> btw, not wallet related, but: my node stats are b0rked for a bit :x
19:13:44 <sipa> yay, we have some people
19:14:19 <kanzure> are descriptors going to be renamed (or descript?)
19:14:23 <luke-jr> sorry, gtg
19:14:39 <meshcollider> kanzure: why would they be renamed?
19:14:45 <sipa> kanzure: we've sort of settled on the name miniscript instead of descript; descriptors are descriptors :)
19:15:01 <achow101> meshcollider: because it's confusing
19:15:02 <sipa> (they're much less related to eachother than the name seemed to imply anyway)
19:15:04 <kanzure> miniscript because it's yer subset of script. alright.
19:15:20 <kanzure> i would have also accepted sipascript
19:15:32 <meshcollider> achow101: descriptors is a good name? Only "descript" might have been a little ;)
19:16:18 <kanzure> http://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-10-08-script-descriptors/
19:16:30 <sipa> so i think my most important blocker is #14150
19:16:33 <gribble> https://github.com/bitcoin/bitcoin/issues/14150 | Add key origin support to descriptors by sipa · Pull Request #14150 · bitcoin/bitcoin · GitHub
19:16:49 <achow101> so what steps do we need to do to get the wallet to be descriptor based?
19:17:25 <gwillen> kanzure: I have also been pushing for "output descriptors" rather than "script descriptors" given the choice of both, the latter being a bit of a tongue twister :-)
19:17:46 <sipa> after that a number of things become possible, including #14477, the ability to add origin info to scantxoutset (which would solve #14503), and it would also allow writing a descriptor/utxoset based PSBT updater
19:17:49 <gribble> https://github.com/bitcoin/bitcoin/issues/14477 | Add ability to convert solvability info to descriptor by sipa · Pull Request #14477 · bitcoin/bitcoin · GitHub
19:17:50 <gribble> https://github.com/bitcoin/bitcoin/issues/14503 | scantxoutset does not identify UTXO origins · Issue #14503 · bitcoin/bitcoin · GitHub
19:17:55 <achow101> gwillen: output script descriptors ;p
19:17:58 <sipa> achow101: yeah, that's the big question :)
19:18:24 <sipa> i think the first step is abstracting out IsMine
19:18:43 <gmaxwell> scanutxoset based psbt updater would be a major win.
19:18:44 <kanzure> need to keep old ismine things?
19:19:15 <sipa> kanzure: i would prefer that, and i also don't think it's that much extra work
19:19:30 <sipa> IsMine is really simple to add things too - just OR the result
19:19:30 <meshcollider> Long term, is making the wallet descriptor-based enough to solve all the existing concerns with same keys being used for different address types and which keys we treat as IsMine, etc?
19:19:38 <sipa> meshcollider: yup
19:21:26 <achow101> sipa: isn't IsMine already separated from the wallet?
19:21:37 <sipa> so one way of seeing it i think is that a wallet will consist of a number of records, each of which has one descriptor plus some metadata (birthdate, change or not, explored how far, gap limit, ...), and one record is designated "here is where you draw payment addresses from" and "here is where you draw change from"
19:22:27 <sipa> and then there can be - for now - a 'legacy' record that corresponds to the behaviour of the existing keypool/mapkeys/mapwatchonly/mapscripts
19:23:08 <sipa> achow101: so the goal would be that there can be multiple implementations of the ismine logic, and one (the legacy one) needs to encapsulate the keypool logic... which is a nontrivial change
19:23:45 <sipa> right now IsMine is just something that operates based on a KeyStore, and that's not enough
19:23:57 <meshcollider> For the payment and change addresses sources, they would just be ranged descriptors too right?
19:24:10 <sipa> yup
19:24:45 <sipa> another missing piece is an "evaluation cache" for descriptors, which would e.g. store pubkeys for hd keys which have hardened steps in them
19:25:36 <meshcollider> Store pubkeys for solvabilty while the wallet is locked, or?
19:25:57 <sipa> meshcollider: well, and to compute the scriptPubKeys to watch for
19:26:25 <sipa> for efficiency maybe it shoud store all pubkeys involved in descriptors, even the ones from unhardened paths
19:26:33 <sipa> or we'd need to rederive them at startup
19:27:50 <sipa> probably the first step is creating an interface for IsMine; just like SigningProvider is an interface for solving/signing
19:28:13 <sipa> initially the wallet itself can implement that, by calling the old IsMine code, but that code can then be moved into one instance of that logic
19:28:37 <sipa> after that, it should be easy to create another descriptor based implementation
19:29:18 <sipa> </monologue>
19:29:29 <sipa> other topics?
19:31:07 <meshcollider> Upgrade wise, descriptor based wallets aren't backwards compatible with old versions, or we are going to allow some more magic not-touching-the-wallet derivation of descriptors at startup every time?
19:31:29 <sipa> yeah, that's a different question; i think there are a number of ways
19:31:51 <sipa> one is to keep them completely separate, and old wallet remains compatible with old versions as long as you don't import anything descriptor based
19:32:49 <sipa> another is to convert the existing stuff to descriptors at startup every time - which would allow removing the existing IsMine logic from the runtime, but not gain us much otherwise (and the conversion is pretty complicated)
19:33:19 <achow101> it also wouldn't work for encrypted wallets
19:33:22 <sipa> and i guess another is to just have 2 types of wallets, and you need an explicit conversion between them
19:33:42 <sipa> achow101: i think it would
19:33:53 <sipa> we know the xpub we derived keys from, no?
19:33:58 <achow101> no
19:34:24 <achow101> xpub and xprv are derived from the seed on the for whenever they are needed
19:34:24 <sipa> no?
19:34:30 <sipa> ah
19:34:59 <achow101> alao hardened derivation, so no xpub
19:35:17 <sipa> that's ok
19:35:24 <sipa> but yeah
19:35:41 <sipa> i think it's actually best to at least initially have the two live side by side
19:36:13 <meshcollider> Can we deprecate the old version and in a future release only accept wallets if they've been upgraded?
19:36:52 <sipa> perhaps
19:37:09 <achow101> I'm afraid that if we have to keep the old one around that it will never go away because people don't upgrade
19:37:29 <meshcollider> That's what I mean
19:37:47 <sipa> i think a bigger question is what to do with things like addmultisigaddress etc
19:38:20 <achow101> can't it construct a descriptor based on what was given?
19:38:47 <meshcollider> Isn't that ok to just make a new multisig descriptor record with the public keys of whatever was specified?
19:38:49 <gmaxwell> that was my thought
19:39:05 <sipa> achow101: yeah, with slightly different (but far more reasonable, actuallly) semantics
19:39:43 <sipa> meshcollider: same with importmulti, actually
19:39:54 <sipa> anything specified there can be converted to a descriptor
19:40:07 <andytoshi> meshcollider: people today show up with 2012 wallets on #bitcoin, and it's a pretty impressive show of backward-compatibility that i can always say "just import it into the latest core"
19:40:20 <andytoshi> even if it's got e.g. p2pk outputs
19:41:50 <booyah> andytoshi: yeap, I think it's safe to assume people will expect that basically forever. Wallets in some deep storate boxes, inherited, burried in backyard
19:41:59 <meshcollider> andytoshi: you could still import it, but it would upgrade it for you I think
19:42:19 <meshcollider> So you just can't import it and then go back and use it on the 2012 software
19:43:21 <sipa> i think that the burden of maintaining compatibility with the old wallet format actually won't be too hard, as due to the necessary refactoring for descriptors in the first place, it will become pretty standalone and not entangled with everything else
19:43:46 <sipa> and i shouldn't say "old format", it's just adding some new fields
19:44:31 <jonasschnelli> I still sometimes have the feeling we should clone the wallet code (make it run with the existing wallet code) and remove everything that is legacy,.. don't promise backward comp. for 1-2 yrs.
19:44:50 <jonasschnelli> All the fancy stuff could go there...
19:44:59 <jonasschnelli> API can break during that 1-2yr period
19:45:04 <sipa> jonasschnelli: maybe, but i think that's not the right granularity
19:45:17 <sipa> you'd be duplicating a lot of things that are perfectly compatible
19:45:21 <sipa> like coin selection
19:45:23 <jonasschnelli> We still can "backport" to the stable wallet
19:45:33 <jonasschnelli> You can factor out the coin selection
19:45:38 <sipa> exactly.
19:45:40 <jonasschnelli> But I agree, some parts would. be duplicated
19:45:44 <sipa> just like you can factor out the ismine logic.
19:45:50 <sipa> :)
19:46:00 <gmaxwell> I don't think we get enough testing/review for one wallet, having two doesn't sound better. :) if it were narrowed down to a pretty small thing, then sure.
19:46:38 <gmaxwell> Being able to _import_ old wallets should always be relatively straight forward though, so it may make sense to not support old wallets except being able to import them at some point.
19:46:47 <sipa> yeah, agree
19:46:56 <jonasschnelli> I think we test less if we go with the two-wallets approach since backward compatibility and the edge-cases that come up with it consumes a major part of the review and fix time
19:47:32 <sipa> i feel that "meh just create a new wallet" is a knee jerk response when facing the complexity of the existing system
19:47:34 <jonasschnelli> At some point in time, you have to abandon old wallets (maybe thats not within the next 10 yrs)
19:47:43 <sipa> it's always appealing to rewrite things you don't understand
19:47:50 <sipa> but that doesn't make it the right choice
19:48:03 <jonasschnelli> Yes. Maybe.
19:48:06 <gmaxwell> And usually a bad idea, since the complexity was almost always there for a reason. :)
19:48:29 <sipa> it is certainly true that _some_ of the complexity can be dropped if we don't need backward compatibility
19:48:42 <sipa> but identifying that is perhaps less work than making a new implementation actually production ready
19:48:46 <jonasschnelli> I don't think its about complexity... more about progress we can't make otherwise. And dragging around legacy stuff like the account system.
19:49:18 <sipa> i think we can make progress fine
19:49:38 <jonasschnelli> Yes. That indeed true.
19:50:04 <sipa> especially with a number of people pulling in the same direction - which is something that hopefully these meetings contribute to
19:50:13 <jonasschnelli> As said, somethimes I think it would be worth do the 2nd wallet... but you guys always convince me the single-wallet approach is more future proof
19:50:54 <meshcollider> Is #8369 in any way helpful to this discussion
19:50:56 <gribble> https://github.com/bitcoin/bitcoin/issues/8369 | [FOR LATER USE][WIP][Wallet] add support for a flexible "set of features" by jonasschnelli · Pull Request #8369 · bitcoin/bitcoin · GitHub
19:51:14 <jonasschnelli> meshcollider: oh. I think we implemented that in a way,.. not?
19:51:27 <jonasschnelli> With the disableprivatekey function
19:51:55 <jonasschnelli> The disableprivatekey function introduced a 64bit bitmap
19:52:18 <sipa> jonasschnelli: very old but relevant blog post: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/
19:52:50 <jonasschnelli> Oh. I smell that this is a good read. Thanks sipa
19:53:45 <jonasschnelli> meshcollider: with that 64bit features flag bitmap, the upper 32 are mandatory (wallet needs those features) where the lower32 bits are optional.
19:54:26 <sipa> jonasschnelli: i'm in favor of something like that, though we shouldn't overuse it as it may lead to an explosion of combinations to test
19:54:26 <jonasschnelli> https://github.com/bitcoin/bitcoin/pull/9662/commits/9995a602a639b64a749545b7c3bafbf67f97324f
19:54:56 <sipa> (which is less of a concern when when the features don't interact, and another argument for abstracting out things, so that it becomes clear they can't interact)
19:56:44 <sipa> oh, i didn't realize the disableprivatekey introduced that already
19:56:50 <meshcollider> A descriptor wallet could just be a mandatory flag then right
19:56:56 <jonasschnelli> Yes. I smuggled it in
19:57:12 <sipa> meshcollider: right
19:57:12 <jonasschnelli> Things like disableprivatekeys is optional and needs a such facility
19:59:00 <sipa> i guess that's it?
19:59:05 <meshcollider> Alright this has been a good meeting IMO, last couple minutes anything else?
19:59:23 <meshcollider> Maybe it's just been good because I've been asking lots of questions lol
19:59:25 <jonasschnelli> We should use the meetingbot I guess?
19:59:25 <jonasschnelli> (next time)
19:59:40 <meshcollider> Well, it's running
19:59:50 <jonasschnelli> Oh. We did.
19:59:50 <meshcollider> Just no explicit actions or topics
19:59:58 <jonasschnelli> I see. Sure.
20:00:03 <sipa> #endmeeting