Version bits with timeout and delay.
26 messages, 13 to 30 Sep 2015. Started by Rusty Russell.
Rusty Russell (8), Jorge Timón (7), Tier Nolan (6), Tom Harding (2), Btc Drak, Eric Lombrozo, Gavin Andresen
Hi all,
Those who've seen the original versionbits bip, this adds: 1) Percentage checking only on retarget period boundaries. 2) 1 retarget period between 95% and activation. 3) A stronger suggestion for timeout value selection.
https://gist.github.com/rustyrussell/47eb08093373f71f87de
And pasted below, de-formatted a little.
Thanks, Rusty.
BIP: ?? Title: Version bits with timeout and delay Author: Pieter Wuille <pieter.wuille@gmail•com>, Peter Todd <pete@petertodd•org>, Greg Maxwell <greg@xiph•org>, Rusty Russell <rusty@rustcorp•com•au> Status: Draft Type: Informational Track Created: 2015-10-04
==Abstract==
This document specifies a proposed change to the semantics of the 'version' field in Bitcoin blocks, allowing multiple backward-compatible changes (further called called "soft forks") being deployed in parallel. It relies on interpreting the version field as a bit vector, where each bit can be used to track an independent change. These are tallied each retarget period. Once the consensus change succeeds or times out, there is a "fallow" pause after which the bit can be reused for later changes.
==Motivation==
BIP 34 introduced a mechanism for doing soft-forking changes without predefined flag timestamp (or flag block height), instead relying on measuring miner support indicated by a higher version number in block headers. As it relies on comparing version numbers as integers however, it only supports one single change being rolled out at once, requiring coordination between proposals, and does not allow for permanent rejection: as long as one soft fork is not fully rolled out, no future one can be scheduled.
In addition, BIP 34 made the integer comparison (nVersion >= 2) a consensus rule after its 95% threshold was reached, removing 2^31 +2 values from the set of valid version numbers (all negative numbers, as nVersion is interpreted as a signed integer, as well as 0 and 1). This indicates another downside this approach: every upgrade permanently restricts the set of allowed nVersion field values. This approach was later reused in BIP 66, which further removed nVersion = 2 as valid option. As will be shown further, this is unnecessary.
==Specification==
===Mechanism===
'''Bit flags''' We are permitting several independent soft forks to be deployed in parallel. For each, a bit B is chosen from the set {0,1,2,...,28}, which is not currently in use for any other ongoing soft fork. Miners signal intent to enforce the new rules associated with the proposed soft fork by setting bit 1<sup>B</sup> in nVersion to 1 in their blocks.
'''High bits''' The highest 3 bits are set to 001, so the range of actually possible nVersion values is [0x20000000...0x3FFFFFFF], inclusive. This leaves two future upgrades for different mechanisms (top bits 010 and 011), while complying to the constraints set by BIP34 and BIP66. Having more than 29 available bits for parallel soft forks does not add anything anyway, as the (nVersion >= 3) requirement already makes that impossible.
'''States''' With every softfork proposal we associate a state BState, which begins at ''defined'', and can be ''locked-in'', ''activated'', or ''failed''. Transitions are considered after each retarget period.
'''Soft Fork Support''' Software which supports the change should begin by setting B in all blocks mined until it is resolved.
if (BState == defined) {
SetBInBlock();
}
'''Success: Lock-in Threshold''' If bit B is set in 1916 (1512 on testnet) or more of the 2016 blocks within a retarget period, it is considered ''locked-in''. Miners should stop setting bit B.
if (NextBlockHeight % 2016 == 0) {
if (BState == defined && Previous2016BlocksCountB() >= 1916) {
BState = locked-in;
BActiveHeight = NextBlockHeight + 2016;
}
}
'''Success: Activation Delay''' The consensus rules related to ''locked-in'' soft fork will be enforced in the second retarget period; ie. there is a one retarget period in which the remaining 5% can upgrade. At the that activation block and after, the bit B may be reused for a different soft fork.
if (BState == locked-in && NextBlockHeight == BActiveHeight) {
BState = activated;
ApplyRulesForBFromNextBlock();
/* B can be reused, immediately */
}
'''Failure: Timeout''' A soft fork proposal should include a ''timeout''. This is measured as the beginning of a calendar year as per this table (suggested three years from drafting the soft fork proposal):
Timeout Year >= Seconds Timeout Year >= Seconds 2018 1514764800 2026 1767225600 2019 1546300800 2027 1798761600 2020 1577836800 2028 1830297600 2021 1609459200 2029 1861920000 2022 1640995200 2030 1893456000 2023 1672531200 2031 1924992000 2024 1704067200 2032 1956528000 2025 1735689600 2033 1988150400
If the soft fork still not ''locked-in'' and the GetMedianTimePast() of a block following a retarget period is at or past this timeout, miners should cease setting this bit.
if (NextBlockHeight % 2016 == 0) {
if (BState == defined && GetMedianTimePast(nextblock) >= BFinalYear) {
BState = failed;
}
}
After another retarget period (to allow detection of buggy miners), the bit may be reused.
'''Warning system''' To support upgrade warnings, an extra "unknown upgrade" is tracked, using the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask will be non-zero whenever an unexpected bit is set in nVersion. Whenever lock-in for the unknown upgrade is detected, the software should warn loudly about the upcoming soft fork. It should warn even more loudly after the next retarget period.
'''Forks''' It should be noted that the states are maintained along block chain branches, but may need recomputation when a reorganization happens.
===Support for future changes===
The mechanism described above is very generic, and variations are possible for future soft forks. Here are some ideas that can be taken into account.
'''Modified thresholds''' The 95% threshold (based on in BIP 34) does not have to be maintained for eternity, but changes should take the effect on the warning system into account. In particular, having a lock-in threshold that is incompatible with the one used for the warning system may have long-term effects, as the warning system cannot rely on a permanently detectable condition anymore.
'''Conflicting soft forks''' At some point, two mutually exclusive soft forks may be proposed. The naive way to deal with this is to never create software that implements both, but that is a making a bet that at least one side is guaranteed to lose. Better would be to encode "soft fork X cannot be locked-in" as consensus rule for the conflicting soft fork - allowing software that supports both, but can never trigger conflicting changes.
'''Multi-stage soft forks''' Soft forks right now are typically treated as booleans: they go from an inactive to an active state in blocks. Perhaps at some point there is demand for a change that has a larger number of stages, with additional validation rules that get enabled one by one. The above mechanism can be adapted to support this, by interpreting a combination of bits as an integer, rather than as isolated bits. The warning system is compatible with this, as (nVersion & ~nExpectedVersion) will always be non-zero for increasing integers.
== Rationale ==
The failure timeout allows eventual reuse of bits even if a soft fork was never activated, so it's clear that the new use of the bit refers to a new BIP. It's deliberately very course grained, to take into account reasonable development and deployment delays. There are unlikely to be enough failed proposals to cause a bit shortage.
The fallow period at the conclusion of a soft fork attempt allows some detection of buggy clients, and allows time for warnings and software upgrades for successful soft forks.
Rusty,
I think you've covered all the issues discussed now. +1 for submitting to BIPs repo to get an official number.
Are you planning to write the implementation?
On Sun, Sep 13, 2015 at 7:56 PM, Rusty Russell via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (196 lines)
> Hi all, > > Those who've seen the original versionbits bip, this adds: > 1) Percentage checking only on retarget period boundaries. > 2) 1 retarget period between 95% and activation. > 3) A stronger suggestion for timeout value selection. > > https://gist.github.com/rustyrussell/47eb08093373f71f87de > > And pasted below, de-formatted a little. > > Thanks, > Rusty. > > BIP: ?? > Title: Version bits with timeout and delay > Author: Pieter Wuille <pieter.wuille@gmail•com>, Peter Todd < > pete@petertodd•org>, Greg Maxwell <greg@xiph•org>, Rusty Russell < > rusty@rustcorp•com•au> > Status: Draft > Type: Informational Track > Created: 2015-10-04 > > ==Abstract== > > This document specifies a proposed change to the semantics of the > 'version' field in Bitcoin blocks, allowing multiple backward-compatible > changes (further called called "soft forks") being deployed in parallel. It > relies on interpreting the version field as a bit vector, where each bit > can be used to track an independent change. These are tallied each retarget > period. Once the consensus change succeeds or times out, there is a > "fallow" pause after which the bit can be reused for later changes. > > ==Motivation== > > BIP 34 introduced a mechanism for doing soft-forking changes without > predefined flag timestamp (or flag block height), instead relying on > measuring miner support indicated by a higher version number in block > headers. As it relies on comparing version numbers as integers however, it > only supports one single change being rolled out at once, requiring > coordination between proposals, and does not allow for permanent rejection: > as long as one soft fork is not fully rolled out, no future one can be > scheduled. > > In addition, BIP 34 made the integer comparison (nVersion >= 2) a > consensus rule after its 95% threshold was reached, removing 2^31 +2 values > from the set of valid version numbers (all negative numbers, as nVersion is > interpreted as a signed integer, as well as 0 and 1). This indicates > another downside this approach: every upgrade permanently restricts the set > of allowed nVersion field values. This approach was later reused in BIP 66, > which further removed nVersion = 2 as valid option. As will be shown > further, this is unnecessary. > > ==Specification== > > ===Mechanism=== > > '''Bit flags''' > We are permitting several independent soft forks to be deployed in > parallel. For each, a bit B is chosen from the set {0,1,2,...,28}, which is > not currently in use for any other ongoing soft fork. Miners signal intent > to enforce the new rules associated with the proposed soft fork by setting > bit 1<sup>B</sup> in nVersion to 1 in their blocks. > > '''High bits''' > The highest 3 bits are set to 001, so the range of actually possible > nVersion values is [0x20000000...0x3FFFFFFF], inclusive. This leaves two > future upgrades for different mechanisms (top bits 010 and 011), while > complying to the constraints set by BIP34 and BIP66. Having more than 29 > available bits for parallel soft forks does not add anything anyway, as the > (nVersion >= 3) requirement already makes that impossible. > > '''States''' > With every softfork proposal we associate a state BState, which begins > at ''defined'', and can be ''locked-in'', ''activated'', > or ''failed''. Transitions are considered after each > retarget period. > > '''Soft Fork Support''' > Software which supports the change should begin by setting B in all blocks > mined until it is resolved. > > if (BState == defined) { > SetBInBlock(); > } > > '''Success: Lock-in Threshold''' > If bit B is set in 1916 (1512 on testnet) or more of the 2016 blocks > within a retarget period, it is considered ''locked-in''. Miners should > stop setting bit B. > > if (NextBlockHeight % 2016 == 0) { > if (BState == defined && Previous2016BlocksCountB() >= 1916) { > BState = locked-in; > BActiveHeight = NextBlockHeight + 2016; > } > } > > '''Success: Activation Delay''' > The consensus rules related to ''locked-in'' soft fork will be enforced in > the second retarget period; ie. there is a one retarget period in > which the remaining 5% can upgrade. At the that activation block and > after, the bit B may be reused for a different soft fork. > > if (BState == locked-in && NextBlockHeight == BActiveHeight) { > BState = activated; > ApplyRulesForBFromNextBlock(); > /* B can be reused, immediately */ > } > > '''Failure: Timeout''' > A soft fork proposal should include a ''timeout''. This is measured > as the beginning of a calendar year as per this table (suggested > three years from drafting the soft fork proposal): > > Timeout Year >= Seconds Timeout Year >= Seconds > 2018 1514764800 2026 1767225600 > 2019 1546300800 2027 1798761600 > 2020 1577836800 2028 1830297600 > 2021 1609459200 2029 1861920000 > 2022 1640995200 2030 1893456000 > 2023 1672531200 2031 1924992000 > 2024 1704067200 2032 1956528000 > 2025 1735689600 2033 1988150400 > > If the soft fork still not ''locked-in'' and the > GetMedianTimePast() of a block following a retarget period is at or > past this timeout, miners should cease setting this bit. > > if (NextBlockHeight % 2016 == 0) { > if (BState == defined && GetMedianTimePast(nextblock) >= BFinalYear) { > BState = failed; > } > } > > After another retarget period (to allow detection of buggy miners), > the bit may be reused. > > '''Warning system''' > To support upgrade warnings, an extra "unknown upgrade" is tracked, using > the "implicit bit" mask = (block.nVersion & ~expectedVersion) != 0. Mask > will be non-zero whenever an unexpected bit is set in nVersion. Whenever > lock-in for the unknown upgrade is detected, the software should warn > loudly about the upcoming soft fork. It should warn even more loudly after > the next retarget period. > > '''Forks''' > It should be noted that the states are maintained along block chain > branches, but may need recomputation when a reorganization happens. > > ===Support for future changes=== > > The mechanism described above is very generic, and variations are possible > for future soft forks. Here are some ideas that can be taken into account. > > '''Modified thresholds''' > The 95% threshold (based on in BIP 34) does not have to be maintained for > eternity, but changes should take the effect on the warning system into > account. In particular, having a lock-in threshold that is incompatible > with the one used for the warning system may have long-term effects, as the > warning system cannot rely on a permanently detectable condition anymore. > > '''Conflicting soft forks''' > At some point, two mutually exclusive soft forks may be proposed. The > naive way to deal with this is to never create software that implements > both, but that is a making a bet that at least one side is guaranteed to > lose. Better would be to encode "soft fork X cannot be locked-in" as > consensus rule for the conflicting soft fork - allowing software that > supports both, but can never trigger conflicting changes. > > '''Multi-stage soft forks''' > Soft forks right now are typically treated as booleans: they go from an > inactive to an active state in blocks. Perhaps at some point there is > demand for a change that has a larger number of stages, with additional > validation rules that get enabled one by one. The above mechanism can be > adapted to support this, by interpreting a combination of bits as an > integer, rather than as isolated bits. The warning system is compatible > with this, as (nVersion & ~nExpectedVersion) will always be non-zero for > increasing integers. > > == Rationale == > > The failure timeout allows eventual reuse of bits even if a soft fork was > never activated, so it's clear that the new use of the bit refers to a > new BIP. It's deliberately very course grained, to take into account > reasonable development and deployment delays. There are unlikely to be > enough failed proposals to cause a bit shortage. > > The fallow period at the conclusion of a soft fork attempt allows some > detection of buggy clients, and allows time for warnings and software > upgrades for successful soft forks. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Sun, Sep 13, 2015 at 7:56 PM, Rusty Russell via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (6 lines)
> '''States''' > With every softfork proposal we associate a state BState, which begins > at ''defined'', and can be ''locked-in'', ''activated'', > or ''failed''. Transitions are considered after each > retarget period. >
I think the 75% rule should be maintained. It confirms that miners who are setting the bit are actually creating blocks that meet the new rule (though it doesn't check if they are enforcing it).
What is the reason for aligning the updated to the difficulty window?
*defined* Miners set bit If 75% of blocks of last 2016 have bit set, goto tentative
*tentative* Miners set bit Reject blocks that have bit set that don't follow new rule If 95% of blocks of last 2016 have bit set, goto locked-in
*locked-in*
Point of no return Miners still set bit Reject blocks that have bit set that don't follow new rule After 2016 blocks goto notice
*activated*
Miners don't set bit for at least 10080 blocks Reject blocks that don't follow new rule
'''Failure: Timeout'''
Quoted text (2 lines)
> A soft fork proposal should include a ''timeout''. >
I think counting in blocks is easier to be exact here.
If two bits were allocated per proposal, then miners could vote against forks to recover the bits. If 25% of the miners vote against, then that kills it.
In the rationale, it would be useful to discuss effects on SPV clients and buggy miners.
SPV clients should be recommended to actually monitor the version field.
Tier Nolan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> writes:
Quoted text (13 lines)
> On Sun, Sep 13, 2015 at 7:56 PM, Rusty Russell via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation•org> wrote: > >> '''States''' >> With every softfork proposal we associate a state BState, which begins >> at ''defined'', and can be ''locked-in'', ''activated'', >> or ''failed''. Transitions are considered after each >> retarget period. >> > > I think the 75% rule should be maintained. It confirms that miners who are > setting the bit are actually creating blocks that meet the new rule (though > it doesn't check if they are enforcing it).
I couldn't see a use for it, since partial enforcement of a soft fork is pretty useless.
Your point about checking that miners are actually doing it is true, though all stuff being forked in in future will be nonstandard AFAICT.
I bias towards simplicity for this.
Quoted text (1 line)
> What is the reason for aligning the updated to the difficulty window?
Miners already have that date in their calendar, so prefer to anchor to that.
Quoted text (17 lines)
> *defined* > Miners set bit > If 75% of blocks of last 2016 have bit set, goto tentative > > > *tentative* > Miners set bit > Reject blocks that have bit set that don't follow new rule > If 95% of blocks of last 2016 have bit set, goto locked-in > > > *locked-in* > > Point of no return > Miners still set bit > Reject blocks that have bit set that don't follow new rule > After 2016 blocks goto notice
OK, *that* variant makes perfect sense, and is no more complex, AFAICT.
So, there's two weeks to detect bad implementations, then you everyone stops setting the bit, for later reuse by another BIP.
Quoted text (1 line)
> I think counting in blocks is easier to be exact here.
Easier for code, but harder for BIP authors.
Quoted text (3 lines)
> If two bits were allocated per proposal, then miners could vote against > forks to recover the bits. If 25% of the miners vote against, then that > kills it.
You need a timeout: an ancient (non-mining, thus undetectable) node should never fork itself off the network because someone reused a failed BIP bit.
Quoted text (4 lines)
> In the rationale, it would be useful to discuss effects on SPV clients and > buggy miners. > > SPV clients should be recommended to actually monitor the version field.
SPV clients don't experience a security change when a soft fork occurs? They're already trusting miners.
Greg pointed out that soft forks tend to get accompanied by block forks across activation, but SPV clients should *definitely* be taking those into account whenever they happen, right?
Thanks! Rusty.
For enforcing new restrictions on your own blocks (thus at the policy level, not consensus) you don't need to wait for 75%. You can do it from the start (this way all miners setting the bit will enforce the new restrictions. On Sep 16, 2015 4:20 PM, "Rusty Russell via bitcoin-dev" < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (86 lines)
> Tier Nolan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> > writes: > > On Sun, Sep 13, 2015 at 7:56 PM, Rusty Russell via bitcoin-dev < > > bitcoin-dev@lists•linuxfoundation•org> wrote: > > > >> '''States''' > >> With every softfork proposal we associate a state BState, which begins > >> at ''defined'', and can be ''locked-in'', ''activated'', > >> or ''failed''. Transitions are considered after each > >> retarget period. > >> > > > > I think the 75% rule should be maintained. It confirms that miners who > are > > setting the bit are actually creating blocks that meet the new rule > (though > > it doesn't check if they are enforcing it). > > I couldn't see a use for it, since partial enforcement of a soft fork is > pretty useless. > > Your point about checking that miners are actually doing it is true, > though all stuff being forked in in future will be nonstandard AFAICT. > > I bias towards simplicity for this. > > > What is the reason for aligning the updated to the difficulty window? > > Miners already have that date in their calendar, so prefer to anchor to > that. > > > *defined* > > Miners set bit > > If 75% of blocks of last 2016 have bit set, goto tentative > > > > > > *tentative* > > Miners set bit > > Reject blocks that have bit set that don't follow new rule > > If 95% of blocks of last 2016 have bit set, goto locked-in > > > > > > *locked-in* > > > > Point of no return > > Miners still set bit > > Reject blocks that have bit set that don't follow new rule > > After 2016 blocks goto notice > > OK, *that* variant makes perfect sense, and is no more complex, AFAICT. > > So, there's two weeks to detect bad implementations, then you everyone > stops setting the bit, for later reuse by another BIP. > > > I think counting in blocks is easier to be exact here. > > Easier for code, but harder for BIP authors. > > > If two bits were allocated per proposal, then miners could vote against > > forks to recover the bits. If 25% of the miners vote against, then that > > kills it. > > You need a timeout: an ancient (non-mining, thus undetectable) node > should never fork itself off the network because someone reused a failed > BIP bit. > > > In the rationale, it would be useful to discuss effects on SPV clients > and > > buggy miners. > > > > SPV clients should be recommended to actually monitor the version field. > > SPV clients don't experience a security change when a soft fork occurs? > They're already trusting miners. > > Greg pointed out that soft forks tend to get accompanied by block forks > across activation, but SPV clients should *definitely* be taking those > into account whenever they happen, right? > > Thanks! > Rusty. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Wed, Sep 16, 2015 at 9:19 PM, Rusty Russell <rusty@rustcorp•com•au> wrote:
Quoted text (3 lines)
> I couldn't see a use for it, since partial enforcement of a soft fork is > pretty useless. >
It isn't useful for actually using the feature, but some miners might set the bit but not actually create blocks that comply with the new rule.
This would cause their blocks to be orphaned until they fixed it.
OK, *that* variant makes perfect sense, and is no more complex, AFAICT.
Quoted text (4 lines)
> > So, there's two weeks to detect bad implementations, then you everyone > stops setting the bit, for later reuse by another BIP. >
It could be more than two weeks if the support stays between 80% and 90% for a while.
75%+ checks that blocks with the bit set follow the rule.
95%+ enters lock-in and has the same rules as 75%+, but is irreversible at that point.
Quoted text (4 lines)
> You need a timeout: an ancient (non-mining, thus undetectable) node > should never fork itself off the network because someone reused a failed > BIP bit. >
I meant if the 2nd bit was part of the BIP. One of the 2 bits is "FOR" and the other is "AGAINST". If against hits 25%, then it is deemed a failure.
The 2nd bit wouldn't be used normally. This means that proposals can be killed quickly if they are obviously going to fail.
On Wed, Sep 16, 2015 at 9:27 PM, Jorge Timón <jtimon@jtimon•cc> wrote:
Quoted text (5 lines)
> For enforcing new restrictions on your own blocks (thus at the policy > level, not consensus) you don't need to wait for 75%. You can do it from > the start (this way all miners setting the bit will enforce the new > restrictions. >
At 75%, you have a pretty solid super-majority.
You can safely reject blocks that have the bit set but are invalid according to the new rule (as long as everyone who sets the bit does it too).
No, 95% is safer and will produce less orphaned blocks. 0%is fine to do it in your own blocks.
I agree on using height vs time. Rusty, what do you mean by being easier for bip writers? How is writing "block x" any harder than writing "date y". On Sep 16, 2015 4:32 PM, "Tier Nolan via bitcoin-dev" < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (21 lines)
> > > On Wed, Sep 16, 2015 at 9:27 PM, Jorge Timón <jtimon@jtimon•cc> wrote: > >> For enforcing new restrictions on your own blocks (thus at the policy >> level, not consensus) you don't need to wait for 75%. You can do it from >> the start (this way all miners setting the bit will enforce the new >> restrictions. >> > At 75%, you have a pretty solid super-majority. > > You can safely reject blocks that have the bit set but are invalid > according to the new rule (as long as everyone who sets the bit does it > too). > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
On Wed, Sep 16, 2015 at 9:38 PM, Jorge Timón <jtimon@jtimon•cc> wrote:
Quoted text (2 lines)
> No, 95% is safer and will produce less orphaned blocks. >
The point of the 75% is just as a test run. Enforcement wouldn't happen until 95%.
At 75%, if someone sets the bit, then they should be creating valid blocks (under the rule).
On Sep 16, 2015 4:49 PM, "Tier Nolan via bitcoin-dev" < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (8 lines)
> > > > On Wed, Sep 16, 2015 at 9:38 PM, Jorge Timón <jtimon@jtimon•cc> wrote: >> >> No, 95% is safer and will produce less orphaned blocks. > > The point of the 75% is just as a test run. Enforcement wouldn't happen
until 95%.
Quoted text (2 lines)
> > At 75%, if someone sets the bit, then they should be creating valid
blocks (under the rule).
You shouldn't rely on that, some may start applying the restrictions in their own blocks at 0% and others only at 90%. Until it becomes a consensus rule it is just part of the standard policy (and we shouldn't rely on nodes following the standard policy).
Quoted text (5 lines)
> _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Wed, Sep 16, 2015 at 9:54 PM, Jorge Timón <jtimon@jtimon•cc> wrote:
Quoted text (11 lines)
> > On Sep 16, 2015 4:49 PM, "Tier Nolan via bitcoin-dev" < > bitcoin-dev@lists•linuxfoundation•org> wrote: > > At 75%, if someone sets the bit, then they should be creating valid > blocks (under the rule). > > You shouldn't rely on that, some may start applying the restrictions in > their own blocks at 0% and others only at 90%. Until it becomes a consensus > rule it is just part of the standard policy (and we shouldn't rely on nodes > following the standard policy). >
It would be a consensus rule. If >75% of the blocks in the last 2016 window have the bit set, then reject all blocks that have the bit set and fail to meet the rule.
I understand your proposal, but I don't see what it accomplishes compared to applying the new rule from the start (in your own blocks) and wait for 95% for consensus activation (which is my preference and it's much simpler to implement). What are the disadvantages of my approach? What are the advantages of yours? On Sep 16, 2015 4:57 PM, "Tier Nolan via bitcoin-dev" < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (27 lines)
> > > On Wed, Sep 16, 2015 at 9:54 PM, Jorge Timón <jtimon@jtimon•cc> wrote: > >> >> On Sep 16, 2015 4:49 PM, "Tier Nolan via bitcoin-dev" < >> bitcoin-dev@lists•linuxfoundation•org> wrote: >> > At 75%, if someone sets the bit, then they should be creating valid >> blocks (under the rule). >> >> You shouldn't rely on that, some may start applying the restrictions in >> their own blocks at 0% and others only at 90%. Until it becomes a consensus >> rule it is just part of the standard policy (and we shouldn't rely on nodes >> following the standard policy). >> > > It would be a consensus rule. If >75% of the blocks in the last 2016 > window have the bit set, then reject all blocks that have the bit set and > fail to meet the rule. > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
The exact numbers (95% vs. 75% etc) don't need to be completely specified to start working on an implementation. What really matters for now is defining the states and trigger mechanisms. I'd rather we not argue over the optimal values for supermajority requirement at this point.
On September 16, 2015 5:03:43 PM EDT, "Jorge Timón via bitcoin-dev" <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (53 lines)
>I understand your proposal, but I don't see what it accomplishes >compared >to applying the new rule from the start (in your own blocks) and wait >for >95% for consensus activation (which is my preference and it's much >simpler >to implement). >What are the disadvantages of my approach? What are the advantages of >yours? >On Sep 16, 2015 4:57 PM, "Tier Nolan via bitcoin-dev" < >bitcoin-dev@lists•linuxfoundation•org> wrote: > >> >> >> On Wed, Sep 16, 2015 at 9:54 PM, Jorge Timón <jtimon@jtimon•cc> >wrote: >> >>> >>> On Sep 16, 2015 4:49 PM, "Tier Nolan via bitcoin-dev" < >>> bitcoin-dev@lists•linuxfoundation•org> wrote: >>> > At 75%, if someone sets the bit, then they should be creating >valid >>> blocks (under the rule). >>> >>> You shouldn't rely on that, some may start applying the restrictions >in >>> their own blocks at 0% and others only at 90%. Until it becomes a >consensus >>> rule it is just part of the standard policy (and we shouldn't rely >on nodes >>> following the standard policy). >>> >> >> It would be a consensus rule. If >75% of the blocks in the last 2016 >> window have the bit set, then reject all blocks that have the bit set >and >> fail to meet the rule. >> >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >> > > >------------------------------------------------------------------------ > >_______________________________________________ >bitcoin-dev mailing list >bitcoin-dev@lists•linuxfoundation•org >https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
On Wed, Sep 16, 2015 at 11:52 PM, Eric Lombrozo <elombrozo@gmail•com> wrote:
Quoted text (5 lines)
> The exact numbers (95% vs. 75% etc) don't need to be completely specified > to start working on an implementation. What really matters for now is > defining the states and trigger mechanisms. I'd rather we not argue over > the optimal values for supermajority requirement at this point. >
The discussion was about what each state means, not the thresholds exactly. I agree that can be set later.
On Wed, Sep 16, 2015 at 10:03 PM, Jorge Timón <jtimon@jtimon•cc> wrote:
Quoted text (1 line)
> I understand your proposal, but I don't see what it accomplishes compared
to applying the new rule from the start (in your own blocks)
Quoted text (1 line)
> and wait for 95% for consensus activation (which is my preference and
it's much simpler to implement).
Quoted text (1 line)
> What are the disadvantages of my approach? What are the advantages of
yours? I agree that miners should apply the rule from the start in their own blocks.
*defined* Miners set bit Miners apply rule to their own blocks If 75% of blocks of last 2016 have bit set, goto tentative
*tentative* Miners set bit Miners apply rule to their own blocks Miners enforce rule in blocks with bit set (reject invalid blocks) If 95% of blocks of last 2016 have bit set, goto locked-in
*locked-in*
Point of no return Miners set bit Miners apply rule to their own blocks Miners enforce rule in blocks with bit set (reject invalid blocks) After 2016 blocks goto activated
*activated*
Miners don't set bit Reject any block that has the bit set for 10080 blocks (10 diff periods) Reject blocks that don't follow new rule
The advantage of enforcing the rule when 75% is reached (but only for blocks with the bit set) is that miners get early notification that they have implemented the rule incorrectly. They might produce blocks that they think are fine, but which aren't.
On Thu, Sep 17, 2015 at 12:38 PM, Tier Nolan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (4 lines)
> The advantage of enforcing the rule when 75% is reached (but only for blocks > with the bit set) is that miners get early notification that they have > implemented the rule incorrectly. They might produce blocks that they > think are fine, but which aren't.
I'm still unconvinced, but thanks, this is what I was asking for.
Tier Nolan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> writes:
Quoted text (4 lines)
> The advantage of enforcing the rule when 75% is reached (but only for > blocks with the bit set) is that miners get early notification that they > have implemented the rule incorrectly. They might produce blocks that > they think are fine, but which aren't.
Indeed. There are three believable failure possibilties:
1) You don't implement the rule at all, and don't set the bit. 2) You implement it and set bit, but think some valid block is invalid. 3) You implement it and set bit, but think some invalid block is valid.
#1 is by far the most common, and the proposal is designed so they *always* get ~2 weeks warning before those drop to SPV security.
Assuming the mining majority isn't buggy (otherwise, it's arguably not a bug but a feature!) #2 is the worst case: some miners fork off and don't rejoin.
So there is a slight advantage in doing this early: those buggy miners no longer contribute to the 95% threshold. But that's outweighed IMHO by:
1) We would need another delay at 75% so #1 nodes can upgrade.
2) The new feature won't be exercised much before impliciation, since it's useless before then, so it might not find bugs anyway.
In conclusion, I'm not convinced by the extra complexity.
Cheers, Rusty.
Jorge Timón via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> writes:
Quoted text (2 lines)
> I agree on using height vs time. Rusty, what do you mean by being easier > for bip writers? How is writing "block x" any harder than writing "date y".
Three years from drafting is reasonable. How many blocks is that? Hmm, better make it 6 years of blocks just in case we have a hash race.
Deployment speed is measured in months, not blocks. It's hard enough to guess without adding another variable.
Cheers, Rusty.
Tier Nolan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> writes:
Quoted text (12 lines)
> On Wed, Sep 16, 2015 at 9:19 PM, Rusty Russell <rusty@rustcorp•com•au> > wrote: >> You need a timeout: an ancient (non-mining, thus undetectable) node >> should never fork itself off the network because someone reused a failed >> BIP bit. >> > > I meant if the 2nd bit was part of the BIP. One of the 2 bits is "FOR" and > the other is "AGAINST". If against hits 25%, then it is deemed a failure. > > The 2nd bit wouldn't be used normally. This means that proposals can be > killed quickly if they are obviously going to fail.
This could be added if we approach one failed soft fork every 5 weeks, I guess (or it could be just for specific soft forks).
Cheers, Rusty.
I disagree with the importance of this concern and old soft/hardforks will replace this activation mechanism with height, so that's an argument in favor of using the height from the start. This is "being discussed" in a thread branched from bip99's discussion. Anyway, is this proposing to use the block time or the median block time? For some hardforks/softforks the block time complicates the implementation (ie in acceptToMemoryPool) as discussed in the mentioned thread. On Sep 19, 2015 1:24 AM, "Rusty Russell" <rusty@rustcorp•com•au> wrote:
Quoted text (15 lines)
> Jorge Timón via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> > writes: > > I agree on using height vs time. Rusty, what do you mean by being easier > > for bip writers? How is writing "block x" any harder than writing "date > y". > > Three years from drafting is reasonable. How many blocks is that? Hmm, > better make it 6 years of blocks just in case we have a hash race. > > Deployment speed is measured in months, not blocks. It's hard enough to > guess without adding another variable. > > Cheers, > Rusty. >
Jorge Timón <jtimon@jtimon•cc> writes:
Quoted text (4 lines)
> I disagree with the importance of this concern and old soft/hardforks will > replace this activation mechanism with height, so that's an argument in > favor of using the height from the start. This is "being discussed" in a > thread branched from bip99's discussion.
Thanks, I'll have to dig through bitcoin-dev and find it.
Quoted text (3 lines)
> Anyway, is this proposing to use the block time or the median block time? > For some hardforks/softforks the block time complicates the implementation > (ie in acceptToMemoryPool) as discussed in the mentioned thread.
BIP text is pretty clear that it's median block time.
This is only for timeout, not for soft fork rule change (which *is* 2016 blocks after 95% is reached).
Cheers, Rusty.
On Sep 20, 2015 10:58 PM, "Rusty Russell" <rusty@rustcorp•com•au> wrote:
Quoted text (3 lines)
> > Jorge Timón <jtimon@jtimon•cc> writes: > > I disagree with the importance of this concern and old soft/hardforks
will
Quoted text (5 lines)
> > replace this activation mechanism with height, so that's an argument in > > favor of using the height from the start. This is "being discussed" in a > > thread branched from bip99's discussion. > > Thanks, I'll have to dig through bitcoin-dev and find it.
The initial thread is linked to from the BIP document (which is in the bitcoin/bips PR).
Jorge Timón <jtimon@jtimon•cc> writes:
Quoted text (13 lines)
> On Sep 20, 2015 10:58 PM, "Rusty Russell" <rusty@rustcorp•com•au> wrote: >> >> Jorge Timón <jtimon@jtimon•cc> writes: >> > I disagree with the importance of this concern and old soft/hardforks > will >> > replace this activation mechanism with height, so that's an argument in >> > favor of using the height from the start. This is "being discussed" in a >> > thread branched from bip99's discussion. >> >> Thanks, I'll have to dig through bitcoin-dev and find it. > > The initial thread is linked to from the BIP document (which is in the > bitcoin/bips PR).
Thanks, read and digested.
The good news is that timeout via GetMedianTimePast() doesn't have any effect on "should I accept this to mempool", and seems pretty uncontroversial. Activation is by block number once vote hits 95%, so that too is fairly simple to implement.
Cheers, Rusty.
On 9/13/2015 11:56 AM, Rusty Russell via bitcoin-dev wrote:
Quoted text (6 lines)
> '''Success: Activation Delay''' > The consensus rules related to ''locked-in'' soft fork will be enforced in > the second retarget period; ie. there is a one retarget period in > which the remaining 5% can upgrade. At the that activation block and > after, the bit B may be reused for a different soft fork. >
Rather than a simple one-period delay, should there be a one-period "burn-in" to show sustained support of the threshold? During this period, support must continuously remain above the threshold. Any lapse resets to inactivated state.
With a simple delay, you can have the embarrassing situation where support falls off during the delay period and there is far below threshold support just moments prior to enforcement, but enforcement happens anyway.
BIP 101 has this problem too.
I say keep it simple.
If the 75% threshold is hit, then support suddenly drops off below 50%, "meh" -- there will be a big ruckus, everybody will freak out, and miners will refuse to build big blocks because they'll worry that they'll get orphaned.
Adding more complexity for a case that ain't gonna happen (and isn't a disaster if it does) is a mistake, in my humble opinion.
On Wed, Sep 23, 2015 at 2:33 PM, Tom Harding via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (27 lines)
> On 9/13/2015 11:56 AM, Rusty Russell via bitcoin-dev wrote: > >> '''Success: Activation Delay''' >> The consensus rules related to ''locked-in'' soft fork will be enforced in >> the second retarget period; ie. there is a one retarget period in >> which the remaining 5% can upgrade. At the that activation block and >> after, the bit B may be reused for a different soft fork. >> >> > Rather than a simple one-period delay, should there be a one-period > "burn-in" to show sustained support of the threshold? During this period, > support must continuously remain above the threshold. Any lapse resets to > inactivated state. > > With a simple delay, you can have the embarrassing situation where support > falls off during the delay period and there is far below threshold support > just moments prior to enforcement, but enforcement happens anyway. > > BIP 101 has this problem too. > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
Tom Harding via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> writes:
Quoted text (17 lines)
> On 9/13/2015 11:56 AM, Rusty Russell via bitcoin-dev wrote: >> '''Success: Activation Delay''' >> The consensus rules related to ''locked-in'' soft fork will be enforced in >> the second retarget period; ie. there is a one retarget period in >> which the remaining 5% can upgrade. At the that activation block and >> after, the bit B may be reused for a different soft fork. >> > > Rather than a simple one-period delay, should there be a one-period > "burn-in" to show sustained support of the threshold? During this > period, support must continuously remain above the threshold. Any lapse > resets to inactivated state. > > With a simple delay, you can have the embarrassing situation where > support falls off during the delay period and there is far below > threshold support just moments prior to enforcement, but enforcement > happens anyway.
Yeah, but Gavin's right. If you can't account for all the corner cases, all you can do is keep it simple and well defined.
Thanks, Rusty.
On 9/29/2015 7:05 PM, Rusty Russell wrote:
Quoted text (9 lines)
> Tom Harding via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> > writes: >> With a simple delay, you can have the embarrassing situation where >> support falls off during the delay period and there is far below >> threshold support just moments prior to enforcement, but enforcement >> happens anyway. > Yeah, but Gavin's right. If you can't account for all the corner cases, > all you can do is keep it simple and well defined. >
At least you changed the BIP to make it possible to see a fall off in support, even though nothing is done about it.