BIP32 "wallet structure" in use? Remove it?
12 messages, 25 to 26 Apr 2014. Started by Andreas Schildbach.
Tamas Blummer (2), Andreas Schildbach, Jim, Greg Maxwell, Mike Hearn, Aaron Voisine, Thomas Voegtlin, Tier Nolan, +3 more
Does anyone use or plan to use the "wallet structure" part of the BIP32 document?
I suggest removing it from the document and maybe instead point at BIP43. That new specification proposal just defines a "purpose" and leaves everything else to the inventor of that purpose. The idea it that over time a standard will evolve naturally rather than top-down.
https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki
I'd volunteer to submit a pull request if I can see some level of consent here.
Oh dear.
For reasons that are perfectly reasonable we are close to losing any chance of intra-client HD compatibility for BIP32 wallets.
In the next 12 months there will probably be collectively millions of users of our new wallets. I don't want them to suffer from vendor lockin.
Can we not agree on a lowest common denominator that we agree to support ? An "HD Basic" if you like. For entry level users we can keep things simple and any "HD Basic" bitcoin will be fully interoperable.
Sure, if you use anything fancy you'll be locked in to a particular wallet but a lot of users just want somewhere safe to put their bitcoin, spend it and receive it.
I appreciate standising everything is very difficult (if not impossible) but if we don't have a minimum of interoperability I think we'll do our users a disservice.
On Fri, Apr 25, 2014, at 11:23 AM, Andreas Schildbach wrote:
Quoted text (24 lines)
> Does anyone use or plan to use the "wallet structure" part of the BIP32 > document? > > I suggest removing it from the document and maybe instead point at > BIP43. That new specification proposal just defines a "purpose" and > leaves everything else to the inventor of that purpose. The idea it that > over time a standard will evolve naturally rather than top-down. > > https://github.com/bitcoin/bips/blob/master/bip-0043.mediawiki > > I'd volunteer to submit a pull request if I can see some level of > consent here. > > > ------------------------------------------------------------------------------ > Start Your Social Network Today - Download eXo Platform > Build your Enterprise Intranet with eXo Platform Software > Java Based Open Source Intranet - Social, Extensible, Cloud Ready > Get Started Now And Turn Your Intranet Into A Collaboration Platform > http://p.sf.net/sfu/ExoPlatform > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
On Fri, Apr 25, 2014 at 7:53 AM, Jim <jim618@fastmail•co•uk> wrote:
Quoted text (13 lines)
> Oh dear. > > For reasons that are perfectly reasonable we are close to losing any chance of intra-client HD compatibility for BIP32 wallets. > > In the next 12 months there will probably be collectively millions of users of our new wallets. I don't want them to suffer from vendor lockin. > > Can we not agree on a lowest common denominator that we agree to support ? > An "HD Basic" if you like. > For entry level users we can keep things simple and any "HD Basic" bitcoin will be fully interoperable. > > Sure, if you use anything fancy you'll be locked in to a particular wallet but a lot of users just want somewhere safe to put their bitcoin, spend it and receive it. > > I appreciate standising everything is very difficult (if not impossible) but if we don't have a minimum of interoperability I think we'll do our users a disservice.
I don't believe that wallet interoperability at this level is possible in general except as an explicit compatibility feature. I also don't believe that it is a huge loss that it is so.
The structure of the derivation defines and constrains functionality. You cannot be structure compatible unless you have the same features and behavior with respect to key management. To that extent that wallets have the same features, I agree its better if they are compatible— but unless they are dead software they likely won't keep the same features for long.
Even if their key management were compatible there are many other things that go into making a wallet portable between systems; the handling of private keys is just one part: a complete wallet will have other (again, functionality specific) metadata.
I agree that it would be it would be possible to support a compatibility mode where a wallet has just a subset of features which works when loaded into different systems, but I'm somewhat doubtful that it would be widely used. The decision to use that mode comes at the wrong time— when you start, not when you need the features you chose to disable or when you want to switch programs. But the obvious thing to do there is to just specify that a linear chain with no further branching is that mode: then that will be the same mode you use when someone gives you a master public key and asks you to use it for reoccurring changes— so at least the software will get used.
Compatibility for something like a recovery tool is another matter, and BIP32 probably defines enough there that with a bit of extra data about how the real wallet worked that recovery can be successful.
Calling it "vendor lock in" sounds overblown to me. If someone wants to change wallets they can transfer the funds— manual handling of private keys is seldom advisable, and as is they're going to lose their metadata in any case. No one expects to switch banks and to keep their account records at the new bank. And while less than perfect, the price of heavily constraining functionality in order to get another result is just too high.
I generally agree, but I wonder how popular cloning wallets between devices will be in future. Right now if someone wants to have a wallet shared between Hive, blockchain.info and Bitcoin Wallet for Android, we just tell them they're out of luck and they need to pick one, or split their funds up manually.
But probably a lot of people would like to use different UI's to access the same wallets. Sharing key trees is a part of that, though full blown wallet metadata sync would also be needed.
So I guess we're going to end up with some kind of fairly complex compatibility matrix. But I agree it may be unavoidable.
On github I commented on the BIP43 pull request about adding a "purpose" of 0' which would correspond to the BIP32 recommended tree structure for a single account wallet. (m/0'/chain) This will allow for backwards compatibility and interoperability at least for existing single account BIP32 implementations like my own. The only issue is that it would involve a new BIP, and it wouldn't follow the BIP43 recommendation of using the BIP number as the "purpose number", unless I can get BIP0. :)
Aaron
There's no trick to being a humorist when you have the whole government working for you -- Will Rodgers
On Fri, Apr 25, 2014 at 8:49 AM, Mike Hearn <mike@plan99•net> wrote:
Quoted text (24 lines)
> I generally agree, but I wonder how popular cloning wallets between devices > will be in future. Right now if someone wants to have a wallet shared > between Hive, blockchain.info and Bitcoin Wallet for Android, we just tell > them they're out of luck and they need to pick one, or split their funds up > manually. > > But probably a lot of people would like to use different UI's to access the > same wallets. Sharing key trees is a part of that, though full blown wallet > metadata sync would also be needed. > > So I guess we're going to end up with some kind of fairly complex > compatibility matrix. But I agree it may be unavoidable. > > ------------------------------------------------------------------------------ > Start Your Social Network Today - Download eXo Platform > Build your Enterprise Intranet with eXo Platform Software > Java Based Open Source Intranet - Social, Extensible, Cloud Ready > Get Started Now And Turn Your Intranet Into A Collaboration Platform > http://p.sf.net/sfu/ExoPlatform > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
I totally agree with gmaxwell here. The cost of interoperability is too high. It would force us to freeze all features, and to require a broad consensus everytime we want to add something new.
In addition, some partial level of compatibility would probably lead to users not able to recover all their funds when they enter their seed in another wallet. That is not acceptable, and should be avoided.
Le 25/04/2014 17:46, Gregory Maxwell a écrit :
Quoted text (51 lines)
> > I don't believe that wallet interoperability at this level is possible > in general except as an explicit compatibility feature. I also don't > believe that it is a huge loss that it is so. > > The structure of the derivation defines and constrains functionality. > You cannot be structure compatible unless you have the same features > and behavior with respect to key management. To that extent that > wallets have the same features, I agree its better if they are > compatible— but unless they are dead software they likely won't keep > the same features for long. > > Even if their key management were compatible there are many other > things that go into making a wallet portable between systems; the > handling of private keys is just one part: a complete wallet will > have other (again, functionality specific) metadata. > > I agree that it would be it would be possible to support a > compatibility mode where a wallet has just a subset of features which > works when loaded into different systems, but I'm somewhat doubtful > that it would be widely used. The decision to use that mode comes at > the wrong time— when you start, not when you need the features you > chose to disable or when you want to switch programs. But the obvious > thing to do there is to just specify that a linear chain with no > further branching is that mode: then that will be the same mode you > use when someone gives you a master public key and asks you to use it > for reoccurring changes— so at least the software will get used. > > Compatibility for something like a recovery tool is another matter, > and BIP32 probably defines enough there that with a bit of extra data > about how the real wallet worked that recovery can be successful. > > Calling it "vendor lock in" sounds overblown to me. If someone wants > to change wallets they can transfer the funds— manual handling of > private keys is seldom advisable, and as is they're going to lose > their metadata in any case. No one expects to switch banks and to > keep their account records at the new bank. And while less than > perfect, the price of heavily constraining functionality in order to > get another result is just too high. > > ------------------------------------------------------------------------------ > Start Your Social Network Today - Download eXo Platform > Build your Enterprise Intranet with eXo Platform Software > Java Based Open Source Intranet - Social, Extensible, Cloud Ready > Get Started Now And Turn Your Intranet Into A Collaboration Platform > http://p.sf.net/sfu/ExoPlatform > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
Maybe the solution is to have a defined way to import an unknown wallet?
This means that the gap space and a search ordering needs to be defined.
Given a blockchain and a root seed, it should be possible to find all the addresses for that root seed.
The hierarchy that the wallet actually uses could be anything.
On Sat, Apr 26, 2014 at 11:36 AM, Thomas Voegtlin <thomasv1@gmx•de> wrote:
Quoted text (77 lines)
> I totally agree with gmaxwell here. The cost of interoperability is too > high. It would force us to freeze all features, and to require a broad > consensus everytime we want to add something new. > > In addition, some partial level of compatibility would probably lead to > users not able to recover all their funds when they enter their seed in > another wallet. That is not acceptable, and should be avoided. > > > > > Le 25/04/2014 17:46, Gregory Maxwell a écrit : > > > > I don't believe that wallet interoperability at this level is possible > > in general except as an explicit compatibility feature. I also don't > > believe that it is a huge loss that it is so. > > > > The structure of the derivation defines and constrains functionality. > > You cannot be structure compatible unless you have the same features > > and behavior with respect to key management. To that extent that > > wallets have the same features, I agree its better if they are > > compatible— but unless they are dead software they likely won't keep > > the same features for long. > > > > Even if their key management were compatible there are many other > > things that go into making a wallet portable between systems; the > > handling of private keys is just one part: a complete wallet will > > have other (again, functionality specific) metadata. > > > > I agree that it would be it would be possible to support a > > compatibility mode where a wallet has just a subset of features which > > works when loaded into different systems, but I'm somewhat doubtful > > that it would be widely used. The decision to use that mode comes at > > the wrong time— when you start, not when you need the features you > > chose to disable or when you want to switch programs. But the obvious > > thing to do there is to just specify that a linear chain with no > > further branching is that mode: then that will be the same mode you > > use when someone gives you a master public key and asks you to use it > > for reoccurring changes— so at least the software will get used. > > > > Compatibility for something like a recovery tool is another matter, > > and BIP32 probably defines enough there that with a bit of extra data > > about how the real wallet worked that recovery can be successful. > > > > Calling it "vendor lock in" sounds overblown to me. If someone wants > > to change wallets they can transfer the funds— manual handling of > > private keys is seldom advisable, and as is they're going to lose > > their metadata in any case. No one expects to switch banks and to > > keep their account records at the new bank. And while less than > > perfect, the price of heavily constraining functionality in order to > > get another result is just too high. > > > > > ------------------------------------------------------------------------------ > > Start Your Social Network Today - Download eXo Platform > > Build your Enterprise Intranet with eXo Platform Software > > Java Based Open Source Intranet - Social, Extensible, Cloud Ready > > Get Started Now And Turn Your Intranet Into A Collaboration Platform > > http://p.sf.net/sfu/ExoPlatform > > _______________________________________________ > > Bitcoin-development mailing list > > Bitcoin-development@lists•sourceforge•net > > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > Start Your Social Network Today - Download eXo Platform > Build your Enterprise Intranet with eXo Platform Software > Java Based Open Source Intranet - Social, Extensible, Cloud Ready > Get Started Now And Turn Your Intranet Into A Collaboration Platform > http://p.sf.net/sfu/ExoPlatform > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
Yes, it is expensive but possible to discover any funds associated with a seed, provided there are set limits to:
1. gap of address use (e.g. 20) 2. depth of hierarchy (e.g. 6) 3. gap in use of parallel branches (e.g. 0)
I would pick the limits in brackets above.
Regards,
Tamas Blummer http://bitsofproof.com
On 26.04.2014, at 12:48, Tier Nolan <tier.nolan@gmail•com> wrote:
Quoted text (7 lines)
> Maybe the solution is to have a defined way to import an unknown wallet? > > This means that the gap space and a search ordering needs to be defined. > > Given a blockchain and a root seed, it should be possible to find all the addresses for that root seed. > > The hierarchy that the wallet actually uses could be anything.
Actually gap in parallel branches already fails with BIP64 as it starts with m/64'/…. without having m/63'
Regards,
Tamas Blummer http://bitsofproof.com
On 26.04.2014, at 12:59, Tamas Blummer <tamas@bitsofproof•com> wrote:
Quoted text (23 lines)
> Yes, it is expensive but possible to discover any funds associated with a seed, provided there are set limits to: > > 1. gap of address use (e.g. 20) > 2. depth of hierarchy (e.g. 6) > 3. gap in use of parallel branches (e.g. 0) > > I would pick the limits in brackets above. > > Regards, > > Tamas Blummer > http://bitsofproof.com > > On 26.04.2014, at 12:48, Tier Nolan <tier.nolan@gmail•com> wrote: > >> Maybe the solution is to have a defined way to import an unknown wallet? >> >> This means that the gap space and a search ordering needs to be defined. >> >> Given a blockchain and a root seed, it should be possible to find all the addresses for that root seed. >> >> The hierarchy that the wallet actually uses could be anything. >
On 04/26/2014 12:48 PM, Tier Nolan wrote:
Quoted text (1 line)
> Maybe the solution is to have a defined way to import an unknown wallet?
That is nonsense. There is no way how to import the wallet if you don't know its structure.
Quoted text (2 lines)
> Given a blockchain and a root seed, it should be possible to find all the > addresses for that root seed.
Unless the keyspace is almost infinite because:
Quoted text (1 line)
> The hierarchy that the wallet actually uses could be anything.
On Sat, Apr 26, 2014 at 2:24 PM, Pavol Rusnak <stick@gk2•sk> wrote:
Quoted text (5 lines)
> On 04/26/2014 12:48 PM, Tier Nolan wrote: >> Maybe the solution is to have a defined way to import an unknown wallet? > > That is nonsense. There is no way how to import the wallet if you don't > know its structure.
I agree. Especially when multiple chains are combined (multisig) for P2SH usage, defining things like a gap limit becomes impossible without knowing some metadata.
However, perhaps it is possible to define something like "BIP44 import-compatible", meaning that the application doesn't actually support all of BIP44 features, but does guarantee not losing any funds when imported? Similar things could be done for other purpose types.
It is very young in bitcoin's life. We don't know what features will work out best, or need to be radically changed after initial deployment in the field.
Loose coordination is good. Good ideas will spread on their own. Users will demand compatibility with certain features, and fail to care incompatibilities in other features.
Tight interoperability at this stage is too confining.
On Fri, Apr 25, 2014 at 11:46 AM, Gregory Maxwell <gmaxwell@gmail•com> wrote:
Quoted text (64 lines)
> On Fri, Apr 25, 2014 at 7:53 AM, Jim <jim618@fastmail•co•uk> wrote: >> Oh dear. >> >> For reasons that are perfectly reasonable we are close to losing any chance of intra-client HD compatibility for BIP32 wallets. >> >> In the next 12 months there will probably be collectively millions of users of our new wallets. I don't want them to suffer from vendor lockin. >> >> Can we not agree on a lowest common denominator that we agree to support ? >> An "HD Basic" if you like. >> For entry level users we can keep things simple and any "HD Basic" bitcoin will be fully interoperable. >> >> Sure, if you use anything fancy you'll be locked in to a particular wallet but a lot of users just want somewhere safe to put their bitcoin, spend it and receive it. >> >> I appreciate standising everything is very difficult (if not impossible) but if we don't have a minimum of interoperability I think we'll do our users a disservice. > > I don't believe that wallet interoperability at this level is possible > in general except as an explicit compatibility feature. I also don't > believe that it is a huge loss that it is so. > > The structure of the derivation defines and constrains functionality. > You cannot be structure compatible unless you have the same features > and behavior with respect to key management. To that extent that > wallets have the same features, I agree its better if they are > compatible-- but unless they are dead software they likely won't keep > the same features for long. > > Even if their key management were compatible there are many other > things that go into making a wallet portable between systems; the > handling of private keys is just one part: a complete wallet will > have other (again, functionality specific) metadata. > > I agree that it would be it would be possible to support a > compatibility mode where a wallet has just a subset of features which > works when loaded into different systems, but I'm somewhat doubtful > that it would be widely used. The decision to use that mode comes at > the wrong time-- when you start, not when you need the features you > chose to disable or when you want to switch programs. But the obvious > thing to do there is to just specify that a linear chain with no > further branching is that mode: then that will be the same mode you > use when someone gives you a master public key and asks you to use it > for reoccurring changes-- so at least the software will get used. > > Compatibility for something like a recovery tool is another matter, > and BIP32 probably defines enough there that with a bit of extra data > about how the real wallet worked that recovery can be successful. > > Calling it "vendor lock in" sounds overblown to me. If someone wants > to change wallets they can transfer the funds-- manual handling of > private keys is seldom advisable, and as is they're going to lose > their metadata in any case. No one expects to switch banks and to > keep their account records at the new bank. And while less than > perfect, the price of heavily constraining functionality in order to > get another result is just too high. > > ------------------------------------------------------------------------------ > Start Your Social Network Today - Download eXo Platform > Build your Enterprise Intranet with eXo Platform Software > Java Based Open Source Intranet - Social, Extensible, Cloud Ready > Get Started Now And Turn Your Intranet Into A Collaboration Platform > http://p.sf.net/sfu/ExoPlatform > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development