Future Of Bitcoin-Cores Wallet

Hi (excuse my english; it’s not my native language)

As you might noticed, bitcoin-cores wallet didn’t got that much focus during the last month (even years?). Wallet development has mostly moved towards SPV (bitcoinj), thin clients (Electrum), centralized web middleware (Copay, Greenaddress, etc.).

Obviously this direction was highly appreciated by users who now can now run a bitcoin-client (SPV / thin client) on a smartphone or on a computer with tiny available resources.

Full validation slowly gets a privilege of people who can manage to run bitcoin-core on a VPS or different server like system. Thought, i think, running a full node wallet could be end user friendly with some changes in the current concept.

Today a standard user can download a 1080p 10GB movie over iTunes (in background) and simultaneous play a CPU/GPU extensive 3D game on a standard computer. Why do people think (and it might is) running a full node is so painful?

Mainly it could be because bitcoin-core has been focused on doing validation as quick as possible (okay for a server, not desirable for a wallet background service).

I could see the following strategy: - - end user focused full node wallet would have enabled pruning by default (~2GB disk usage). - - throttled validation (flexible CPU usage, user selectable, maybe ~20% by default). - - throttled block download (bandwidth). - - SPV during catch up (initial sync as also when catching up multiple days because user/node was offline). - - Disable bloom filtering if there is enough bandwith, keep blocks for later validation. - - when node is in sync, switch from SPV to full validation (maybe maintain to lists/dbs of wtx or re-validate after full catchup and display potential conflicts). - - participate in p2p, but with limits/throttling (service limited amount of blocks, tx [TBD]).

Why? - - This could increase the amount of participating full nodes while giving users more privacy and security. - - Create a counterweight against SPV/thin clients (avoid wallet development centralization, could be helpful once/if attacks agains SPV/thin clients are becoming real). - - Slowly complete full validation (can take ~1-2 weeks) and thereforce increase privacy (avoid bloomfilters) and security.

What about SPV together with a full node? Sounds good in theory. But who can run a full node (see above)? How is the channel secured (against MITM, privacy) between the SPV client and the trusted full node? How hard is it to setup and maintain a secure tunnel between a smartphone SPV and a full node over the p2p (8333) channel? How about examine the mempool for fee calculations and (maybe) upcoming CPFP like approaches, etc.?

How about smartphones? Obviously the above solution won’t probably work on a smartphone (to much bandwidth and CPU usage). But do you carry your whole saving in your physical wallet with you? Maybe a smartphone does hold keys which protects low value / daily spendings like a physical wallet (=SPV okay). My personal long term vision of that use-case is, that groups of people who trust each other (a family, etc.) might run one or multiple full node(s) on a hardened system (something similar to the bitnodes hardware) where the system could serve smartphones over something like a stratum server (electrum) or bitpays wallet-service does (index blockchain, additional wallet services). Every member still holds it’s keys but they trust the connected full node (full nodes does address index, balance calculation, multisig arrangements and maybe even coin selection).

Since about one year i slowly work toward this direction. It took me a while to commit myself to a strategy (and i still shake from time to time). At the moment I am working on a wallet focused bitcoin-core fork with the ability to re-merge it to the bitcoin-core branch (keep the fork rebased). Long term goal is, to decouple the both (wallet / core) by using bitcoin-core as a library (static or shared) for the wallet side (which is possible already now)

To myself: I work in the bitcoin-space full-time and completely independent (not employed or influenced by a bitcoin related company) with no business interest, though, i think the acquired know-how can be valuable within the next serval years. And i won’t have any regrets if my work turns out to be unless and ends up in trash.

I have set up this mail to avoid parallelism on a works stream (if there is any). Of course i would really appreciate if other developers are willing to join the team by reviewing, concept critism or contributing code at https://github.com/jonasschnelli/bitcoin.

Any forms of criticism and any ideas are highly appreciated.

— /jonas

Hey Jonas,

I think your analysis of what (some) users need is a good one.

We've discussed this before so I know you prefer your current approach, but I personally would take a slightly different path to reach the same end:

   1. Support serving of SPV wallets from pruned storage. This means some
   protocol upgrades, BIPs, etc. It helps all SPV wallets, including on phones.
   2. Then make a bitcoinj based desktop wallet app, that contains a
   bundled bitcoind.
   3. Make the app sync TWO wallets simultaneously, one from the P2P
   network as today, and another from the local bitcoind via a local socket
   (or even just passing buffers around internally)
   4. The app should then switch from using the wallet synced to P2P to the
   wallet synced to localhost when the latter is fully caught up, and back
   again when the local node is behind.
   5. If there's a discrepancy, alert the user.

There are big advantages of taking this path! They are:

   - The switching back and forth between local full-security mode (which
   may be behind) and remote SPV security (fully synced) is instant and
   transparent to the user. This is important for laptop users who don't run a
   local node all the time. The different audit levels can be reflected in the
   UI in some way.

   - The bitcoinj wallet code already has support for things like
   multi-sig, BIP32, seed words, micropayment channels, etc. You can disable
   Bloom filtering if you like (download full blocks).

   - You can do a local RPC or JNI/JNA call to get fee estimates, if wanted.

   - The modern JVM tools and languages are much, much more productive than
   working with C++.

If you want a thing that runs a home server, then the best way to do that IMO would be to bundle Tor and make it auto-register a Tor hidden service. Then you can just define a QR code standard for 'pairing' a wallet to a .onion address. Any bitcoinj based wallet can sync to it, and as it's your own node, you can use a Bloom filter sized to give virtually no false positives. No additional indexing is then required.

On Tue, Aug 11, 2015 at 4:33 PM, Mike Hearn via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:

> Hey Jonas,
>
> I think your analysis of what (some) users need is a good one.
>
> We've discussed this before so I know you prefer your current approach,
> but I personally would take a slightly different path to reach the same end:
>
>    1. Support serving of SPV wallets from pruned storage. This means some
>    protocol upgrades, BIPs, etc. It helps all SPV wallets, including on phones.
>    2. Then make a bitcoinj based desktop wallet app, that contains a
>    bundled bitcoind.
>
> It makes a lot of sense to have single set of reference implementations

for the different components. So unless you also support something you have not explicitly stated - get rid of bitcoin-qt in its current form, the above does not meet the spirit of Jonas' original post - which makes a load of sense to me.

+1 Glad to see this work.

The Bitcoin Core wallet is a bit neglected these days - maintained but not advancing.

On Tue, Aug 11, 2015 at 5:02 AM, Jonas Schnelli via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Hi
> (excuse my english; it’s not my native language)
>
> As you might noticed, bitcoin-cores wallet didn’t got that much focus
> during the last month (even years?).
> Wallet development has mostly moved towards SPV (bitcoinj), thin
> clients (Electrum), centralized web middleware (Copay, Greenaddress,
> etc.).
>
> Obviously this direction was highly appreciated by users who now can
> now run a bitcoin-client (SPV / thin client) on a smartphone or on a
> computer with tiny available resources.
>
> Full validation slowly gets a privilege of people who can manage to
> run bitcoin-core on a VPS or different server like system.
> Thought, i think, running a full node wallet could be end user
> friendly with some changes in the current concept.
>
> Today a standard user can download a 1080p 10GB movie over iTunes (in
> background) and simultaneous play a CPU/GPU extensive 3D game on a
> standard computer.
> Why do people think (and it might is) running a full node is so painful?
>
> Mainly it could be because bitcoin-core has been focused on doing
> validation as quick as possible (okay for a server, not desirable for
> a wallet background service).
>
> I could see the following strategy:
> - - end user focused full node wallet would have enabled pruning by
> default (~2GB disk usage).
> - - throttled validation (flexible CPU usage, user selectable, maybe
> ~20% by default).
> - - throttled block download (bandwidth).
> - - SPV during catch up (initial sync as also when catching up multiple
> days because user/node was offline).
> - - Disable bloom filtering if there is enough bandwith, keep blocks for
> later validation.
> - - when node is in sync, switch from SPV to full validation (maybe
> maintain to lists/dbs of wtx or re-validate after full catchup and
> display potential conflicts).
> - - participate in p2p, but with limits/throttling (service limited
> amount of blocks, tx [TBD]).
>
> Why?
> - - This could increase the amount of participating full nodes while
> giving users more privacy and security.
> - - Create a counterweight against SPV/thin clients (avoid wallet
> development centralization, could be helpful once/if attacks agains
> SPV/thin clients are becoming real).
> - - Slowly complete full validation (can take ~1-2 weeks) and thereforce
> increase privacy (avoid bloomfilters) and security.
>
> What about SPV together with a full node?
> Sounds good in theory. But who can run a full node (see above)? How is
> the channel secured (against MITM, privacy) between the SPV client and
> the trusted full node? How hard is it to setup and maintain a secure
> tunnel between a smartphone SPV and a full node over the p2p (8333)
> channel? How about examine the mempool for fee calculations and
> (maybe) upcoming CPFP like approaches, etc.?
>
> How about smartphones?
> Obviously the above solution won’t probably work on a smartphone (to
> much bandwidth and CPU usage). But do you carry your whole saving in
> your physical wallet with you? Maybe a smartphone does hold keys which
> protects low value / daily spendings like a physical wallet (=SPV okay).
> My personal long term vision of that use-case is, that groups of
> people who trust each other (a family, etc.) might run one or multiple
> full node(s) on a hardened system (something similar to the bitnodes
> hardware) where the system could serve smartphones over something like
> a stratum server (electrum) or bitpays wallet-service does (index
> blockchain, additional wallet services). Every member still holds it’s
> keys but they trust the connected full node (full nodes does address
> index, balance calculation, multisig arrangements and maybe even coin
> selection).
>
> Since about one year i slowly work toward this direction. It took me a
> while to commit myself to a strategy (and i still shake from time to
> time).
> At the moment I am working on a wallet focused bitcoin-core fork with
> the ability to re-merge it to the bitcoin-core branch (keep the fork
> rebased).
> Long term goal is, to decouple the both (wallet / core) by using
> bitcoin-core as a library (static or shared) for the wallet side
> (which is possible already now)
>
> To myself: I work in the bitcoin-space full-time and completely
> independent (not employed or influenced by a bitcoin related company)
> with no business interest, though, i think the acquired know-how can
> be valuable within the next serval years. And i won’t have any regrets
> if my work turns out to be unless and ends up in trash.
>
> I have set up this mail to avoid parallelism on a works stream (if
> there is any). Of course i would really appreciate if other developers
> are willing to join the team by reviewing, concept critism or
> contributing code at https://github.com/jonasschnelli/bitcoin.
>
> Any forms of criticism and any ideas are highly appreciated.
>
>
> —
> /jonas
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBCAAGBQJVyboPAAoJECnUvLZBb1PsgkkQAKySrksCclbfwIsf1L4Jwaxp
> nWmUhlXa3lIoxKG3eYZMPVugFX/LFKtmNqypQJ8whUjF2K5ZwIW1Boosl13cBkE9
> 2EIAMcBrpah0pwzomJ9ViLArl+7t6ksLR5qWJsgJsLnWlaW4c/1KwjGkYTZ6++VC
> 8O3M3HrdwT3fnrK35XJXTIhpFfRKRFrWcW98vMFt9xw7MUq+enhloGF6Nw4UiQbi
> nOV85YleBJEBU5cXuIOznG4EwHH77f8336+GY8d6mLCg7rKj7ZZWqqHztqEQf68Q
> mtwJhag3pxyW2jqb/fCxYD27oPqgMcLT1lyUobpzu7SrlKKmAIikf8uNU1+8rH+W
> U1C1IsWzvoPK7g7mqlmpk1/6kvlJOWNshTATfQS2A/hMB1Oec3zZTCz+P5S5g+F7
> FT4tB2sR2nwGkWNVPNSs92o0f8y55/u+fAFcbmHkkNrfEt4IuMwsqJNW9i7GzU6b
> uOznq4ZgYw5OxUJh8uXLaA1OFIdPTEcTA4nNRSA7v6cRbfWeaCSGzafOYdGQKV2Y
> 5R7VCZJZe8ALzSUr03FsZ5bilcjdJ3ZyeQNikqeYnQLP+qoH6oRhDT0B2GI2E4+4
> EvfIZCmaDxYG0FClWOQiiO0WeW2kNBWyD3tWCpM63ri//OnoN65NDKsbIpJ+oD8m
> OwxleWQP0eC/5knZSFwB
> =7CG9
> -----END PGP SIGNATURE-----
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>