Bitcoin Core 0.12.0 release candidate 1 available
11 messages, 17 to 25 Jan 2016. Started by Wladimir J. van der Laan.
Wladimir J. van der Laan (4), xor (4), Marco Falke, Simon Selitsky, Jonas Schnelli
Binaries for bitcoin Core version 0.12.0rc1 are available from:
https://bitcoin.org/bin/bitcoin-core-0.12.0/test/
Source code can be found on github under the signed tag
https://github.com/bitcoin/bitcoin/tree/v0.12.0rc1
This is a release candidate for a new major version release, bringing new features, bug fixes, as well as other improvements.
Preliminary release notes for the release can be found here:
https://github.com/bitcoin/bitcoin/blob/0.12/doc/release-notes.md
Release candidates are test versions for releases. When no critical problems are found, this release candidate will be tagged as 0.12.0.
Please report bugs using the issue tracker at github:
https://github.com/bitcoin/bitcoin/issues
On Sunday, January 17, 2016 11:08:08 AM Wladimir J. van der Laan via bitcoin- dev wrote:
Quoted text (3 lines)
> Preliminary release notes for the release can be found here: > > https://github.com/bitcoin/bitcoin/blob/0.12/doc/release-notes.md
The part which lists raw Git pull requests says:
Quoted text (1 line)
> #6057 ac5476e re-enable wallet in autoprune
But the main, handwritten part does not mention this. Is pruning really finished, i.e. could I safely use it as a wallet "end-user"?
IMHO it would be one of the most interesting feature for users, as it could fix the issue of taking >60 GB of disk space.
So if it is finished, please mention that - it's finished - how to enable it.
Thanks for your hard work! :)
On Sun, Jan 17, 2016 at 11:57:28PM +0100, xor--- via bitcoin-dev wrote:
Quoted text (12 lines)
> On Sunday, January 17, 2016 11:08:08 AM Wladimir J. van der Laan via bitcoin- > dev wrote: > > Preliminary release notes for the release can be found here: > > > > https://github.com/bitcoin/bitcoin/blob/0.12/doc/release-notes.md > > > The part which lists raw Git pull requests says: > > #6057 ac5476e re-enable wallet in autoprune > > But the main, handwritten part does not mention this. > Is pruning really finished, i.e. could I safely use it as a wallet "end-user"?
It has been tested in git for almost half a year. This RC is the first binary release that contains the functionality.
It is extremely unlikely that the wallet will eat your coins (always backup nevertheless), but I can't guarantee there won't be some issue where the wallet and chain get out of sync and you're forced to redownload the blockchain.
Quoted text (6 lines)
> IMHO it would be one of the most interesting feature for users, as it could > fix the issue of taking >60 GB of disk space. > > So if it is finished, please mention that > - it's finished > - how to enable it.
How to enable it is still the same as mentioned in the 0.11 release notes: https://github.com/bitcoin/bitcoin/blob/v0.11.0/doc/release-notes.md#block-file-pruning
Wladimir
On Monday, January 18, 2016 12:14:16 PM Wladimir J. van der Laan wrote:
Quoted text (7 lines)
> It has been tested in git for almost half a year. This RC is the first > binary release that contains the functionality. > > It is extremely unlikely that the wallet will eat your coins (always backup > nevertheless), but I can't guarantee there won't be some issue where the > wallet and chain get out of sync and you're forced to redownload the > blockchain.
I think I asked the wrong way, sorry: My question was not really meant at whether it is bug-free (testing that is the purpose of a release candidate, so we of course don't know yet), but rather whether it is at least feature complete now.
Remember, the previous v0.11.0 release notes said:
Quoted text (5 lines)
> Block pruning is currently incompatible with running a wallet due to the > fact that block data is used for rescanning the wallet and importing keys > or addresses (which require a rescan.) However, running the wallet with > block pruning will be supported in the near future, subject to those > limitations.
So I'm interested whether this limitation has been lifted, and the whole feature is considered as finished.
If yes, I would highly recommend advertising it in the new release notes - as said, the disk space reduction is a big deal.
Thank you!
Quoted text (3 lines)
> > So I'm interested whether this limitation has been lifted, and the whole > feature is considered as finished.
Yes, it's exactly that limitation that has been lifted!
Quoted text (2 lines)
> If yes, I would highly recommend advertising it in the new release notes - as > said, the disk space reduction is a big deal.
Good idea, has been added by Marco Falke in commit fa31133,
Wladimir
On Monday, January 25, 2016 01:03:17 PM Wladimir J. van der Laan wrote:
Quoted text (4 lines)
> > If yes, I would highly recommend advertising it in the new release notes - > > as said, the disk space reduction is a big deal. > > Good idea, has been added by Marco Falke in commit fa31133,
Thanks. The RC2 changelog now says:
Quoted text (3 lines)
> To enable block pruning set prune=<N> on the command line or in > bitcoin.conf, where N is the number of MiB to allot for raw block & undo > data.
From having read the Bitcoin whitepaper quite a few months ago ago, I have the very very basic understanding that pruning is meant to: - delete old transaction data which merely "moves coins around" - instead only store the "origin" (= block where coins were mined) and "current location" of the coins, i.e. the unspent transactions. Notably, I understood it as "this is as secure as storing everything, since we know where the coins were created, and where they are".
So from that point of view, I would assume that there is a "natural" amount of megabytes which a fully pruned blockchain consists of: It would be defined by the final amount of unspent coins. I thereby am confused why it is possible to configure a number of megabytes "to allot for raw block & undo data". I would rather expect pruning just to be a boolean on/off flag, and the number of megabytes to be an automatically computed result from the natural size of the dataset. And especially, I fear that I could set N too low, and as a result, it would delete "too much". I mean could this result in even security relevant transaction data being deleted?
Thus, it would be nice if you could yet once more edit the release notes to: - explain why a N must be given - what a "safe" value of N is. I.e. how large must N be at least to not delete security-relevant stuff? - maybe mention if there is a "auto" setting for N to ensure that it choses a safe value on its own?
Sorry if my descriptions are from a layman's point of view. I intentionally did *not* re-read the Bitcoin whitepaper to have a better understanding: I think having a layman's understanding is a good usability test for such stuff.
All of this is already implemented in the bitcoind and bitcoin gui.
The theoretic minimum for the prune target would be 0 (just the header of the current best block) as Bitcoin Core already stores the chainstate (about 2 GiB) regardless of what you set for -prune=<N>.
In practice, the minimum is 510, so reorgs and small rescans (may not be implemented in 0.12) are still possible.
The clients won't let you set it below that target: "Prune configured below the minimum of 550 MiB. Please use a higher number."
Also, keep in mind Bitcoin Core comes with a help message explaining -prune and other command line options
--Marco
2016-01-25 13:27 GMT+01:00 xor--- via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org>:
Quoted text (47 lines)
> On Monday, January 25, 2016 01:03:17 PM Wladimir J. van der Laan wrote: >> > If yes, I would highly recommend advertising it in the new release notes - >> > as said, the disk space reduction is a big deal. >> >> Good idea, has been added by Marco Falke in commit fa31133, > > Thanks. The RC2 changelog now says: > >> To enable block pruning set prune=<N> on the command line or in >> bitcoin.conf, where N is the number of MiB to allot for raw block & undo >> data. > > From having read the Bitcoin whitepaper quite a few months ago ago, I have the > very very basic understanding that pruning is meant to: > - delete old transaction data which merely "moves coins around" > - instead only store the "origin" (= block where coins were mined) and > "current location" of the coins, i.e. the unspent transactions. Notably, I > understood it as "this is as secure as storing everything, since we know where > the coins were created, and where they are". > > So from that point of view, I would assume that there is a "natural" amount of > megabytes which a fully pruned blockchain consists of: It would be defined by > the final amount of unspent coins. > I thereby am confused why it is possible to configure a number of megabytes > "to allot for raw block & undo data". I would rather expect pruning just to be > a boolean on/off flag, and the number of megabytes to be an automatically > computed result from the natural size of the dataset. > And especially, I fear that I could set N too low, and as a result, it would > delete "too much". I mean could this result in even security relevant > transaction data being deleted? > > Thus, it would be nice if you could yet once more edit the release notes to: > - explain why a N must be given > - what a "safe" value of N is. I.e. how large must N be at least to not delete > security-relevant stuff? > - maybe mention if there is a "auto" setting for N to ensure that it choses a > safe value on its own? > > Sorry if my descriptions are from a layman's point of view. I intentionally > did *not* re-read the Bitcoin whitepaper to have a better understanding: > I think having a layman's understanding is a good usability test for such > stuff. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
Quoted text (22 lines)
> > To enable block pruning set prune=<N> on the command line or in > > bitcoin.conf, where N is the number of MiB to allot for raw block & undo > > data. > > From having read the Bitcoin whitepaper quite a few months ago ago, I have the > very very basic understanding that pruning is meant to: > - delete old transaction data which merely "moves coins around" > - instead only store the "origin" (= block where coins were mined) and > "current location" of the coins, i.e. the unspent transactions. Notably, I > understood it as "this is as secure as storing everything, since we know where > the coins were created, and where they are". > > So from that point of view, I would assume that there is a "natural" amount of > megabytes which a fully pruned blockchain consists of: It would be defined by > the final amount of unspent coins. > I thereby am confused why it is possible to configure a number of megabytes > "to allot for raw block & undo data". I would rather expect pruning just to be > a boolean on/off flag, and the number of megabytes to be an automatically > computed result from the natural size of the dataset. > And especially, I fear that I could set N too low, and as a result, it would > delete "too much". I mean could this result in even security relevant > transaction data being deleted?
The term 'pruning', unfortunately is very much overused and overloaded in the bitcoin ecosystem. Satoshi's paper refers to UTXO pruning. This is Pieter Wuille's "ultraprune", which has been part of Bitcoin Core for more than three years.
Block pruning is not mentioned in that paper because it is just administrative, the only reason that nodes store historical blocks at all is to serve it to other nodes, as well as to catch up the wallet status and for -reindexes.
Quoted text (1 line)
> Thus, it would be nice if you could yet once more edit the release notes to:
I don't have time to work on the release notes right now, but if someone else wants to contribute that'd be awesome.
Quoted text (1 line)
> - explain why a N must be given
To give a quotum. The point is that the user can choose how much harddisk space they want to dedicate to block storage.
Block data that is stored can be used by other software, or potentially be served to other nodes. The latter is not implemented at the moment - it would require a change to the P2P protocol, thus right now pruning nodes don't serve block data at all.
Quoted text (2 lines)
> - what a "safe" value of N is. I.e. how large must N be at least to not delete > security-relevant stuff?
There is no security compromise with pruning. Any value of N is intended to be safe.
Very low values would delete undo data that may be necessary in a reorganization, but this is prohibited by argument checks.
Release notes are not meant as a replacement or supplement for documentation. The documentation for -prune is this:
-prune=<n>
Reduce storage requirements by pruning (deleting) old blocks. This mode
is incompatible with -txindex and -rescan. Warning: Reverting this
setting requires re-downloading the entire blockchain. (default: 0 =
disable pruning blocks, >550 = target size in MiB to use for block
files)
Quoted text (2 lines)
> - maybe mention if there is a "auto" setting for N to ensure that it choses a > safe value on its own?
As said, there is no safe or unsafe value. The lowest acceptable value is just as safe as storing everything.
Wladimir
Quoted text (2 lines)
>> Block data that is stored can be used by other software, or potentially be >> served to other nodes. The latter is not implemented at the moment - it would require a change to the P2P protocol, thus right now pruning nodes don't serve block data at all.
Why is the minimum storage quota of 550 MiB necessary for pruning nodes if the block data is not served to other nodes ? Could the client just do transaction verification and transaction relaying and only keep the block(s) being verified on disk ?
On Jan 25, 2016, at 10:05 AM, Wladimir J. van der Laan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (76 lines)
>>> To enable block pruning set prune=<N> on the command line or in >>> bitcoin.conf, where N is the number of MiB to allot for raw block & undo >>> data. > >> From having read the Bitcoin whitepaper quite a few months ago ago, I have the >> very very basic understanding that pruning is meant to: >> - delete old transaction data which merely "moves coins around" >> - instead only store the "origin" (= block where coins were mined) and >> "current location" of the coins, i.e. the unspent transactions. Notably, I >> understood it as "this is as secure as storing everything, since we know where >> the coins were created, and where they are". >> >> So from that point of view, I would assume that there is a "natural" amount of >> megabytes which a fully pruned blockchain consists of: It would be defined by >> the final amount of unspent coins. >> I thereby am confused why it is possible to configure a number of megabytes >> "to allot for raw block & undo data". I would rather expect pruning just to be >> a boolean on/off flag, and the number of megabytes to be an automatically >> computed result from the natural size of the dataset. >> And especially, I fear that I could set N too low, and as a result, it would >> delete "too much". I mean could this result in even security relevant >> transaction data being deleted? > > The term 'pruning', unfortunately is very much overused and overloaded in the > bitcoin ecosystem. Satoshi's paper refers to UTXO pruning. This is Pieter Wuille's "ultraprune", > which has been part of Bitcoin Core for more than three years. > > Block pruning is not mentioned in that paper because it is just administrative, > the only reason that nodes store historical blocks at all is to serve it to other nodes, > as well as to catch up the wallet status and for -reindexes. > >> Thus, it would be nice if you could yet once more edit the release notes to: > > I don't have time to work on the release notes right now, but if someone else > wants to contribute that'd be awesome. > >> - explain why a N must be given > > To give a quotum. The point is that the user can choose how much harddisk space they want to > dedicate to block storage. > > Block data that is stored can be used by other software, or potentially be > served to other nodes. The latter is not implemented at the moment - it would require > a change to the P2P protocol, thus right now pruning nodes don't serve block > data at all. > >> - what a "safe" value of N is. I.e. how large must N be at least to not delete >> security-relevant stuff? > > There is no security compromise with pruning. Any value of N is intended to be safe. > > Very low values would delete undo data that may be necessary in a reorganization, > but this is prohibited by argument checks. > > Release notes are not meant as a replacement or supplement for documentation. > The documentation for -prune is this: > > -prune=<n> > Reduce storage requirements by pruning (deleting) old blocks. This mode > is incompatible with -txindex and -rescan. Warning: Reverting this > setting requires re-downloading the entire blockchain. (default: 0 = > disable pruning blocks, >550 = target size in MiB to use for block > files) > >> - maybe mention if there is a "auto" setting for N to ensure that it choses a >> safe value on its own? > > As said, there is no safe or unsafe value. The lowest acceptable value is just as safe > as storing everything. > > Wladimir > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Quoted text (5 lines)
> Why is the minimum storage quota of 550 MiB necessary for pruning > nodes if the block data is not served to other nodes ? Could the > client just do transaction verification and transaction relaying > and only keep the block(s) being verified on disk ? >
We try to allow reorgs ~288 blocks deep also in pruning:
- From code comments: Require that user allocate at least 550MB for block & undo files (blk???.dat and rev???.dat) At 1MB per block, 288 blocks = 288MB. Add 15% for Undo data = 331MB Add 20% for Orphan block rate = 397MB We want the low water mark after pruning to be at least 397 MB and since we prune in full block file chunks, we need the high water mark which triggers the prune to be one 128MB block file + added 15% undo data = 147MB greater for a total of 545MB Setting the target to > than 550MB will make it likely we can respect the target.
</jonas>
On Monday, January 25, 2016 04:05:59 PM Wladimir J. van der Laan wrote:
Quoted text (2 lines)
> I don't have time to work on the release notes right now, but if someone > else wants to contribute that'd be awesome.
I cooked my first pull request to resolve this: https://github.com/bitcoin/bitcoin/pull/7416
Thanks for all the explanations you folks provided! :)