Version bits proposal
14 messages, 27 May to 3 Jun 2015. Started by Pieter Wuille.
Pieter Wuille (2), Jorge Timón (2), Tier Nolan (2), Douglas Roark, Luke Dashjr, Peter Todd, Sergio Lerner, Patrick Strateman, +3 more
Hello everyone,
here is a proposal for how to coordinate future soft-forking consensus changes: https://gist.github.com/sipa/bf69659f43e763540550
It supports multiple parallel changes, as well as changes that get permanently rejected without obstructing the rollout of others.
Feel free to comment. As the gist does not support notifying participants of new comments, I would suggest using the mailing list instead.
This is joint work with Peter Todd and Greg Maxwell.
On 2015/5/26 21:48, Pieter Wuille wrote:
Quoted text (10 lines)
> here is a proposal for how to coordinate future soft-forking > consensus changes: > https://gist.github.com/sipa/bf69659f43e763540550 > > It supports multiple parallel changes, as well as changes that get > permanently rejected without obstructing the rollout of others. > > Feel free to comment. As the gist does not support notifying > participants of new comments, I would suggest using the mailing > list instead.
Hi Pieter. Thanks for posting the proposal. I think the concept itself is pretty solid. I know some people have been proposing alternate methods too. I hope they'll share here, assuming they haven't already. As is, my comments concern typos and general copy editing.
- - Just speaking in general, I found the BIP to be a bit hard to read. AFAIK, the basic facts are accurate. I just found myself having to re-read certain passages two or three times. A little polish wouldn't hurt. For example, using the "it" pronoun can be confusing, such as multiple uses in the abstract. Specifying what "it" is (e.g., "The proposed change relies on...") would really help. In addition, the way the "W" value is handled seems like it could be improved a bit. I know the wording is accurate. Seeing 1000 change to 1001 is still a little weird. - - In "Multi-stage soft forks," I presume the second sentence should end as follows: "[...] with additional validation rules that get enabled one by _one_." Depending on semantics, I'd consider changing "one by one" to "incremental steps," but that's your call. - - I found the "High bits" section to be confusing at first. It looks like you chose to show everything as little endian data, matching what's actually in the block. My personal preference would be to represent the data, for readability purposes, as big endian. I doubt I'm the only one who finds big endian to be much easier to process mentally. - - Some sort of legend showing A, I, W, etc. would really help, as opposed to just running into them as one goes along. Otherwise, the alphabet soup can be a bit confusing.
Thanks again.
- -- - --- Douglas Roark Senior Developer Armory Technologies, Inc. doug@bitcoinarmory•com PGP key ID: 92ADC0D7
On Wednesday, May 27, 2015 1:48:05 AM Pieter Wuille wrote:
Quoted text (2 lines)
> Feel free to comment. As the gist does not support notifying participants > of new comments, I would suggest using the mailing list instead.
I suggest adding a section describing how this interacts with and changes GBT.
Currently, the client tells the server what the highest block version it supports is, and the server indicates a block version to use in its template, as well as optional instructions for the client to forcefully use this version despite its own maximum version number. Making the version a bitfield contradicts the increment-only assumption of this design, and since GBT clients are not aware of overall network consensus state, reused bits can easily become confused. I suggest, therefore, that GBT clients should indicate (instead of a maximum supported version number) a list of softforks by identifier keyword, and the GBT server respond with a template indicating: - An object of softfork keywords to bit values, that the server will accept. - The version number, as presently conveyed, indicating the preferred softfork flags.
Does this sound reasonable, and/or am I missing anything else?
Luke
It would also help to see the actual code changes required, which I'm sure will be much shorter than the explanation itself. On May 27, 2015 5:47 AM, "Luke Dashjr" <luke@dashjr•org> wrote:
Quoted text (36 lines)
> On Wednesday, May 27, 2015 1:48:05 AM Pieter Wuille wrote: > > Feel free to comment. As the gist does not support notifying participants > > of new comments, I would suggest using the mailing list instead. > > I suggest adding a section describing how this interacts with and changes > GBT. > > Currently, the client tells the server what the highest block version it > supports is, and the server indicates a block version to use in its > template, > as well as optional instructions for the client to forcefully use this > version > despite its own maximum version number. Making the version a bitfield > contradicts the increment-only assumption of this design, and since GBT > clients are not aware of overall network consensus state, reused bits can > easily become confused. I suggest, therefore, that GBT clients should > indicate > (instead of a maximum supported version number) a list of softforks by > identifier keyword, and the GBT server respond with a template indicating: > - An object of softfork keywords to bit values, that the server will > accept. > - The version number, as presently conveyed, indicating the preferred > softfork > flags. > > Does this sound reasonable, and/or am I missing anything else? > > Luke > > > ------------------------------------------------------------------------------ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
I think it would be better to have the deadlines set as block counts. That eliminates the need to use the median time mechanism.
The deadline could be matched to a "start-line". The definition would then be something like
BIP 105 Start block: 325000 End block: 350000 Activation: 750 of 1000 Implication: 950 of 1000 Bit: 9
This would allow creation of a simple table of known BIPs. It also keeps multiple users of the bit as strictly separate.
The alternative to the start time is that it is set equal to the deadline or implication time of the previous user of the bit.
Was the intention to change the 95% rule. You need 750 of the last 1000 to activate and then must wait at least 1000 for implication?
On Wed, May 27, 2015 at 4:51 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
Quoted text (51 lines)
> It would also help to see the actual code changes required, which I'm sure > will be much shorter than the explanation itself. > On May 27, 2015 5:47 AM, "Luke Dashjr" <luke@dashjr•org> wrote: > >> On Wednesday, May 27, 2015 1:48:05 AM Pieter Wuille wrote: >> > Feel free to comment. As the gist does not support notifying >> participants >> > of new comments, I would suggest using the mailing list instead. >> >> I suggest adding a section describing how this interacts with and changes >> GBT. >> >> Currently, the client tells the server what the highest block version it >> supports is, and the server indicates a block version to use in its >> template, >> as well as optional instructions for the client to forcefully use this >> version >> despite its own maximum version number. Making the version a bitfield >> contradicts the increment-only assumption of this design, and since GBT >> clients are not aware of overall network consensus state, reused bits can >> easily become confused. I suggest, therefore, that GBT clients should >> indicate >> (instead of a maximum supported version number) a list of softforks by >> identifier keyword, and the GBT server respond with a template indicating: >> - An object of softfork keywords to bit values, that the server will >> accept. >> - The version number, as presently conveyed, indicating the preferred >> softfork >> flags. >> >> Does this sound reasonable, and/or am I missing anything else? >> >> Luke >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists•sourceforge•net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > >
On Wed, May 27, 2015 at 10:35:03AM +0100, Tier Nolan wrote:
Quoted text (2 lines)
> I think it would be better to have the deadlines set as block counts. That > eliminates the need to use the median time mechanism.
The median time mechanism is basically a way for hashing power to show what time they think it is. Equally, the nVersion soft-fork mechanism is a way for hashing power to show what features they want to support.
Block counts are inconvenient for planning, as there's no guarantee they'll actually happen in any particular time frame, forward and back. There's no particular incentive problems here - the median time clearly shows support by a majority of hashing power - so I don't see any reason to make planning more difficult.
Quoted text (12 lines)
> The deadline could be matched to a "start-line". The definition would then > be something like > > BIP 105 > Start block: 325000 > End block: 350000 > Activation: 750 of 1000 > Implication: 950 of 1000 > Bit: 9 > > This would allow creation of a simple table of known BIPs. It also keeps > multiple users of the bit as strictly separate.
If you assume no large reorganizations, your table of known BIPs can just as easily be a list of block heights even if the median time mechanism is used.
If you do assume there may be large reorganizations you can't have a "simple table"
On May 27, 2015 11:35 AM, "Tier Nolan" <tier.nolan@gmail•com> wrote:
Quoted text (1 line)
> Was the intention to change the 95% rule. You need 750 of the last 1000
to activate and then must wait at least 1000 for implication?
You need 75% to start applying it, 95% to start rejecting blocks that don't apply it.
On Wed, May 27, 2015 at 11:15 AM, Peter Todd <pete@petertodd•org> wrote:
Quoted text (5 lines)
> The median time mechanism is basically a way for hashing power to show > what time they think it is. Equally, the nVersion soft-fork mechanism is > a way for hashing power to show what features they want to support. > >
Fair enough. It means slightly more processing, but the median time could be cached in the header index, so no big deal.
Block counts are inconvenient for planning, as there's no guarantee
Quoted text (2 lines)
> they'll actually happen in any particular time frame, forward and back. >
I don't think the deadline needs to be set that accurately. A roughly 6 month deadline should be fine, but as you say a majority of miners is needed to abuse the median time and it is already a miner poll.
Perhaps the number of blocks used in the median could be increased to reduce "noise".
The median time could be median of the last 144 blocks plus 12 hours.
Quoted text (4 lines)
> If you assume no large reorganizations, your table of known BIPs can > just as easily be a list of block heights even if the median time > mechanism is used. >
I think it makes it easier to write the code. It reduced the state that needs to be stored per BIP. You don't need to check if the previous bips were all accepted.
Each bit is assigned to a particular BIP for a particular range of times (or blocks).
If block numbers were used for the deadline, you just need to check the block index for the deadline block.
enum {
BIP_INACTIVE = 0,
BIP_ACTIVE,
BIP_LOCKED
BIP_INVALID_BLOCK,
}
int GetBIPState(block, bip)
{
if (block.height == bip.deadline) // Bit must be set to match
locked/unlocked at deadline
{
int bipState = check_supermajority(...);
if (bipState == BIP_LOCKED && (block.nVersion & bip.bit)
return BIP_LOCKED;
if (bipState != BIP_LOCKED && (block.nVersion & (~bip.bit)))
return BIP_INACTIVE;
return BIP_INVALID_BLOCK;
}
if (block.height > deadline) // Look at the deadline block to determine
if the BIP is locked
return (block_index[deadline].nVersion & bip_bit) != 0 ? BIP_LOCKED
: BIP_INACTIVE;
if (block.height < startline + I) // BIP cannot activate/lock until
startline + implicit window size
return INACTIVE;
return check_supermajority(....) // Check supermajority of bit }
The block at height deadline would indicate if the BIP was locked in.
Block time could still be used as long as the block height was set after that. The deadline_time could be in six months. The startline height could be the current block height and the deadline_height could be startline + 35000.
The gives roughly
start time = now deadline time = now + six months deadline height = now + eight months
The deadline height is the block height when the bit is returned to the pool but the deadline time is when the BIP has to be accepted.
It also helps with the warning system. For each block height, there is a set of known BIP bits that are allowed. Once the final deadline is passed, the expected mask is zeros.
On Wed, May 27, 2015 at 11:15 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
Quoted text (8 lines)
> On May 27, 2015 11:35 AM, "Tier Nolan" <tier.nolan@gmail•com> wrote: > > > Was the intention to change the 95% rule. You need 750 of the last 1000 > to activate and then must wait at least 1000 for implication? > > You need 75% to start applying it, 95% to start rejecting blocks that > don't apply it. >
I think the phrasing is ambiguous. I was just asking for clarification.
"Whenever I out of any W *subsequent* blocks (regardless of the block itself) have bit B set,"
That suggests that the I of W blocks for the 95% rule must happen after activation. This makes the rule checking harder. Easier to use the current system, where blocks that were part of the 750 rule also count towards the 95% rule.
I like the idea but I think we should leave at least 16 bits of the version fixed as an extra-nonce. If we don't then miners may use them as a nonce anyway, and mess with the soft-fork voting system. My original proposal was this: https://github.com/bitcoin/bitcoin/pull/5102
Best regards
There is absolutely no reason to do this.
Any reasonable micro-controller can build merkle tree roots significantly faster than is necessary.
1 Th/s walks the nonce range once every 4.3ms.
The largest valid merkle trees are 14 nodes high.
That translates to 28 SHA256 ops per 4.3ms or 6511 SHA256 ops/second.
For reference an RPi 1 model B does 2451050 SHA256 ops/second.
On 05/27/2015 03:52 PM, Sergio Lerner wrote:
Quoted text (14 lines)
> I like the idea but I think we should leave at least 16 bits of the > version fixed as an extra-nonce. > If we don't then miners may use them as a nonce anyway, and mess with > the soft-fork voting system. > My original proposal was this: https://github.com/bitcoin/bitcoin/pull/5102 > > Best regards > > > ------------------------------------------------------------------------------ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Agreed, there is no need to misuse the version field as well. There is more than enough variability you could roll in the merkle tree including and excluding transactions, and the scriptSig of the coinbase transaction, which also influences the merkle root.
I have a fundamental dislike of retroactively changing semantics, and the version field should be used just for that: a version. I don't even particularly like flagging support for a fork in the version field, but since I have no better solution, count me as supporting Sipa's proposal. We definitely need a more comfortable way of rolling out new features.
Regards, Chris
On Thu, May 28, 2015 at 3:08 AM Patrick Strateman < patrick.strateman@gmail•com> wrote:
Quoted text (40 lines)
> There is absolutely no reason to do this. > > Any reasonable micro-controller can build merkle tree roots > significantly faster than is necessary. > > 1 Th/s walks the nonce range once every 4.3ms. > > The largest valid merkle trees are 14 nodes high. > > That translates to 28 SHA256 ops per 4.3ms or 6511 SHA256 ops/second. > > For reference an RPi 1 model B does 2451050 SHA256 ops/second. > > On 05/27/2015 03:52 PM, Sergio Lerner wrote: > > I like the idea but I think we should leave at least 16 bits of the > > version fixed as an extra-nonce. > > If we don't then miners may use them as a nonce anyway, and mess with > > the soft-fork voting system. > > My original proposal was this: > https://github.com/bitcoin/bitcoin/pull/5102 > > > > Best regards > > > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > > Bitcoin-development mailing list > > Bitcoin-development@lists•sourceforge•net > > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
Or as far as that goes, permuting (the non-dependent) transactions in the block by permuting the internal merkle tree nodes at increasing depths. (Dependent because transactions that depend on each other have to come in-order; but one could eg put the n-1 of each n sequence of in-order transactions in the left-half and unordered in the right half.)
That makes the tree manipulations maximum depth independent, and even transaction independent possibly - just need to know enough depth in the tree of hashes that are permutation safe.
Adam
On 28 May 2015 at 08:51, Christian Decker <decker.christian@gmail•com> wrote:
Quoted text (65 lines)
> Agreed, there is no need to misuse the version field as well. There is more > than enough variability you could roll in the merkle tree including and > excluding transactions, and the scriptSig of the coinbase transaction, which > also influences the merkle root. > > I have a fundamental dislike of retroactively changing semantics, and the > version field should be used just for that: a version. I don't even > particularly like flagging support for a fork in the version field, but > since I have no better solution, count me as supporting Sipa's proposal. We > definitely need a more comfortable way of rolling out new features. > > Regards, > Chris > > On Thu, May 28, 2015 at 3:08 AM Patrick Strateman > <patrick.strateman@gmail•com> wrote: >> >> There is absolutely no reason to do this. >> >> Any reasonable micro-controller can build merkle tree roots >> significantly faster than is necessary. >> >> 1 Th/s walks the nonce range once every 4.3ms. >> >> The largest valid merkle trees are 14 nodes high. >> >> That translates to 28 SHA256 ops per 4.3ms or 6511 SHA256 ops/second. >> >> For reference an RPi 1 model B does 2451050 SHA256 ops/second. >> >> On 05/27/2015 03:52 PM, Sergio Lerner wrote: >> > I like the idea but I think we should leave at least 16 bits of the >> > version fixed as an extra-nonce. >> > If we don't then miners may use them as a nonce anyway, and mess with >> > the soft-fork voting system. >> > My original proposal was this: >> > https://github.com/bitcoin/bitcoin/pull/5102 >> > >> > Best regards >> > >> > >> > >> > ------------------------------------------------------------------------------ >> > _______________________________________________ >> > Bitcoin-development mailing list >> > Bitcoin-development@lists•sourceforge•net >> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> >> >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists•sourceforge•net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
oh my God ...
发自我的 iPhone
Quoted text (93 lines)
> 在 2015年5月27日，19:26，Tier Nolan <tier.nolan@gmail•com> 写道：
>
>
>
>> On Wed, May 27, 2015 at 11:15 AM, Peter Todd <pete@petertodd•org> wrote:
>> The median time mechanism is basically a way for hashing power to show
>> what time they think it is. Equally, the nVersion soft-fork mechanism is
>> a way for hashing power to show what features they want to support.
>
> Fair enough. It means slightly more processing, but the median time could be cached in the header index, so no big deal.
>
>> Block counts are inconvenient for planning, as there's no guarantee
>> they'll actually happen in any particular time frame, forward and back.
>
> I don't think the deadline needs to be set that accurately. A roughly 6 month deadline should be fine, but as you say a majority of miners is needed to abuse the median time and it is already a miner poll.
>
> Perhaps the number of blocks used in the median could be increased to reduce "noise".
>
> The median time could be median of the last 144 blocks plus 12 hours.
>
>> If you assume no large reorganizations, your table of known BIPs can
>> just as easily be a list of block heights even if the median time
>> mechanism is used.
>
> I think it makes it easier to write the code. It reduced the state that needs to be stored per BIP. You don't need to check if the previous bips were all accepted.
>
> Each bit is assigned to a particular BIP for a particular range of times (or blocks).
>
> If block numbers were used for the deadline, you just need to check the block index for the deadline block.
>
> enum {
> BIP_INACTIVE = 0,
> BIP_ACTIVE,
> BIP_LOCKED
> BIP_INVALID_BLOCK,
> }
>
> int GetBIPState(block, bip)
> {
> if (block.height == bip.deadline) // Bit must be set to match locked/unlocked at deadline
> {
> int bipState = check_supermajority(...);
> if (bipState == BIP_LOCKED && (block.nVersion & bip.bit)
> return BIP_LOCKED;
>
> if (bipState != BIP_LOCKED && (block.nVersion & (~bip.bit)))
> return BIP_INACTIVE;
>
> return BIP_INVALID_BLOCK;
> }
>
> if (block.height > deadline) // Look at the deadline block to determine if the BIP is locked
> return (block_index[deadline].nVersion & bip_bit) != 0 ? BIP_LOCKED : BIP_INACTIVE;
>
> if (block.height < startline + I) // BIP cannot activate/lock until startline + implicit window size
> return INACTIVE;
>
> return check_supermajority(....) // Check supermajority of bit
> }
>
> The block at height deadline would indicate if the BIP was locked in.
>
> Block time could still be used as long as the block height was set after that. The deadline_time could be in six months. The startline height could be the current block height and the deadline_height could be startline + 35000.
>
> The gives roughly
>
> start time = now
> deadline time = now + six months
> deadline height = now + eight months
>
> The deadline height is the block height when the bit is returned to the pool but the deadline time is when the BIP has to be accepted.
>
> It also helps with the warning system. For each block height, there is a set of known BIP bits that are allowed. Once the final deadline is passed, the expected mask is zeros.
>
>> On Wed, May 27, 2015 at 11:15 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
>> On May 27, 2015 11:35 AM, "Tier Nolan" <tier.nolan@gmail•com> wrote:
>>
>> > Was the intention to change the 95% rule. You need 750 of the last 1000 to activate and then must wait at least 1000 for implication?
>>
>> You need 75% to start applying it, 95% to start rejecting blocks that don't apply it.
>
> I think the phrasing is ambiguous. I was just asking for clarification.
>
> "Whenever I out of any W subsequent blocks (regardless of the block itself) have bit B set,"
>
> That suggests that the I of W blocks for the 95% rule must happen after activation. This makes the rule checking harder. Easier to use the current system, where blocks that were part of the 750 rule also count towards the 95% rule.
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>Thanks for all the comments.
I plan to address the feedback and work on an implementation next week.
On Tue, May 26, 2015 at 6:48 PM, Pieter Wuille <pieter.wuille@gmail•com> wrote:
Quoted text (16 lines)
> Hello everyone, > > here is a proposal for how to coordinate future soft-forking consensus > changes: https://gist.github.com/sipa/bf69659f43e763540550 > > It supports multiple parallel changes, as well as changes that get > permanently rejected without obstructing the rollout of others. > > Feel free to comment. As the gist does not support notifying participants > of new comments, I would suggest using the mailing list instead. > > This is joint work with Peter Todd and Greg Maxwell. > > -- > Pieter >