CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners
19 messages, 19 to 28 Aug 2015. Started by Peter Todd.
Btc Drak (4), Peter Todd (3), Mark F (3), Tier Nolan (3), Jorge Timón (2), Johnson Lau (2), Matt Corallo, Jeff Garzik
Deployment of the proposed CLTV, CSV, etc. soft-forks has been recently complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners. Both mine blocks with nVersion=0x20000007, which would falsely trigger the previously suggested implementation using the IsSuperMajority() mechanism and nVersion=4 blocks. Additionally while the XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's nVersion soft-fork mechanism(3) a key component of it - fork deadlines(3) - is not implemented.
XT/Not-Bitcoin-XT behavior --------------------------
Both implementations produce blocks with nVersion=0x20000007, or in binary: 0b001...111
Neither implementation supports a fork deadline; both Not-Bitcoin-XT and XT will produce blocks with those bits set indefinitely under any circumstance, with the proviso that while XT has a hashing power majority, blocks it produces might not be part of the Bitcoin blockchain after Jan 11th 2016. (though this can flap back and forth if reorgs happen)
Curiously the BIP101 draft was changed(4) at the last minute from using the nVersion bits compliant 0x20000004 block nVersion, to using two more bits unnecessarily. The rational for doing this is unknown; the git commit message associated with the change suggested "compatibility concerns", but what the concerns actually were isn't specified. Equally even though implementing the fork deadline would be very each in the XT implementation, this was not done. (the XT codebase has had almost no peer review)
Options for CLTV/CSV/etc. deployment ------------------------------------
1) Plain IsSuperMajority() with nVersion=4
This option can be ruled out immediately due to the high risk of premature triggering, without genuine 95% miner support.
2) nVersion mask, with IsSuperMajority()
In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would be masked away, prior to applying standard IsSuperMajority() logic:
block.nVersion & ~0x20000007
This means that CLTV/CSV/etc. miners running Bitcoin Core would create blocks with nVersion=8, 0b1000. From the perspective of the CLTV/CSV/etc. IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be advertising blocks that do not trigger the soft-fork.
For the perpose of soft-fork warnings, the highest known version can remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks as well as a future nVersion bits implementation. Equally, XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an unknown bit set.
When nVersion bits is implemented by the Bitcoin protocol, the plan of setting the high bits to 0b001 still works. The three lowest bits will be unusable for some time, but will be eventually recoverable as XT/Not-Bitcoin-XT mining ceases.
Equally, further IsSuperMajority() softforks can be accomplished with the same masking technique.
This option does complicate the XT-coin protocol implementation in the future. But that's their problem, and anyway, the maintainers (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks and/or appear to be in favor of a more centralized mandatory update schedule.(6)
3) Full nVersion bits implementation
The most complex option would be to deploy via full nVersion bits implementation using flag bit #4 to trigger the fork. Compliant miners would advertise 0x20000008 initially, followed by 0x20000000 once the fork had triggered. The lowest three bits would be unusable for forks for some time, although they could be eventually recovered as XT/Not-Bitcoin-XT mining ceases.
The main disadvantage of this option is high initial complexity - the reason why IsSuperMajority() was suggested for CLTV/CSV in the first place. That said, much of the code required has been implemented in XT for the BIP101 hard-fork logic, although as mentioned above, the code has had very little peer review.
References ----------
1) https://github.com/bitcoinxt/bitcoinxt
2) https://github.com/xtbit/notbitcoinxt
3) "Version bits proposal",
Pieter Wuille, May 26th 2015, Bitcoin-development mailing list,
http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html,
https://gist.github.com/sipa/bf69659f43e763540550
4) https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe
5) "On consensus and forks - What is the difference between a hard and soft fork?", Mike Hearn, Aug 12th 2015, https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7
6) 2013 San Jose Bitcoin conference developer round-table
We can use nVersion & 0x8 to signal support, while keeping the consensus rule as nVersion >= 4, right? That way we don't waste a bit after this all clears up. On Aug 18, 2015 10:50 PM, "Peter Todd via bitcoin-dev" < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (125 lines)
> Deployment of the proposed CLTV, CSV, etc. soft-forks has been recently > complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners. Both > mine blocks with nVersion=0x20000007, which would falsely trigger the > previously suggested implementation using the IsSuperMajority() > mechanism and nVersion=4 blocks. Additionally while the > XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's > nVersion soft-fork mechanism(3) a key component of it - fork > deadlines(3) - is not implemented. > > > XT/Not-Bitcoin-XT behavior > -------------------------- > > Both implementations produce blocks with nVersion=0x20000007, > or in binary: 0b001...111 > > Neither implementation supports a fork deadline; both Not-Bitcoin-XT and > XT will produce blocks with those bits set indefinitely under any > circumstance, with the proviso that while XT has a hashing power > majority, blocks it produces might not be part of the Bitcoin blockchain > after Jan 11th 2016. (though this can flap back and forth if reorgs > happen) > > Curiously the BIP101 draft was changed(4) at the last minute from using > the nVersion bits compliant 0x20000004 block nVersion, to using two more > bits unnecessarily. The rational for doing this is unknown; the git > commit message associated with the change suggested "compatibility > concerns", but what the concerns actually were isn't specified. Equally > even though implementing the fork deadline would be very each in the XT > implementation, this was not done. (the XT codebase has had almost no > peer review) > > > Options for CLTV/CSV/etc. deployment > ------------------------------------ > > 1) Plain IsSuperMajority() with nVersion=4 > > This option can be ruled out immediately due to the high risk of > premature triggering, without genuine 95% miner support. > > > 2) nVersion mask, with IsSuperMajority() > > In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would > be masked away, prior to applying standard IsSuperMajority() logic: > > block.nVersion & ~0x20000007 > > This means that CLTV/CSV/etc. miners running Bitcoin Core would create > blocks with nVersion=8, 0b1000. From the perspective of the > CLTV/CSV/etc. IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be > advertising blocks that do not trigger the soft-fork. > > For the perpose of soft-fork warnings, the highest known version can > remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks > as well as a future nVersion bits implementation. Equally, > XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an > unknown bit set. > > When nVersion bits is implemented by the Bitcoin protocol, the plan of > setting the high bits to 0b001 still works. The three lowest bits will > be unusable for some time, but will be eventually recoverable as > XT/Not-Bitcoin-XT mining ceases. > > Equally, further IsSuperMajority() softforks can be accomplished with > the same masking technique. > > This option does complicate the XT-coin protocol implementation in the > future. But that's their problem, and anyway, the maintainers > (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks > and/or appear to be in favor of a more centralized mandatory update > schedule.(6) > > > 3) Full nVersion bits implementation > > The most complex option would be to deploy via full nVersion bits > implementation using flag bit #4 to trigger the fork. Compliant miners > would advertise 0x20000008 initially, followed by 0x20000000 once the > fork had triggered. The lowest three bits would be unusable for forks > for some time, although they could be eventually recovered as > XT/Not-Bitcoin-XT mining ceases. > > The main disadvantage of this option is high initial complexity - the > reason why IsSuperMajority() was suggested for CLTV/CSV in the first > place. That said, much of the code required has been implemented in XT > for the BIP101 hard-fork logic, although as mentioned above, the code > has had very little peer review. > > > References > ---------- > > 1) https://github.com/bitcoinxt/bitcoinxt > > 2) https://github.com/xtbit/notbitcoinxt > > 3) "Version bits proposal", > Pieter Wuille, May 26th 2015, Bitcoin-development mailing list, > > http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html > , > https://gist.github.com/sipa/bf69659f43e763540550 > > 4) > https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe > > 5) "On consensus and forks - What is the difference between a hard and > soft fork?", > Mike Hearn, Aug 12th 2015, > https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7 > > 6) 2013 San Jose Bitcoin conference developer round-table > > -- > 'peter'[:-1]@petertodd.org > 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
Seems like 3 is something we want to do no matter what and therefore is the "most future-proof" solution. I wonder if I can help with that (and I know there's more people that would be interested). Where's the current "non-full" nVersion bits implementation? Why implement a "non-full" version instead of going with the full implementation directly?
On Wed, Aug 19, 2015 at 8:10 AM, Mark Friedenbach via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (136 lines)
> We can use nVersion & 0x8 to signal support, while keeping the consensus > rule as nVersion >= 4, right? That way we don't waste a bit after this all > clears up. > > On Aug 18, 2015 10:50 PM, "Peter Todd via bitcoin-dev" > <bitcoin-dev@lists•linuxfoundation•org> wrote: >> >> Deployment of the proposed CLTV, CSV, etc. soft-forks has been recently >> complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners. Both >> mine blocks with nVersion=0x20000007, which would falsely trigger the >> previously suggested implementation using the IsSuperMajority() >> mechanism and nVersion=4 blocks. Additionally while the >> XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's >> nVersion soft-fork mechanism(3) a key component of it - fork >> deadlines(3) - is not implemented. >> >> >> XT/Not-Bitcoin-XT behavior >> -------------------------- >> >> Both implementations produce blocks with nVersion=0x20000007, >> or in binary: 0b001...111 >> >> Neither implementation supports a fork deadline; both Not-Bitcoin-XT and >> XT will produce blocks with those bits set indefinitely under any >> circumstance, with the proviso that while XT has a hashing power >> majority, blocks it produces might not be part of the Bitcoin blockchain >> after Jan 11th 2016. (though this can flap back and forth if reorgs >> happen) >> >> Curiously the BIP101 draft was changed(4) at the last minute from using >> the nVersion bits compliant 0x20000004 block nVersion, to using two more >> bits unnecessarily. The rational for doing this is unknown; the git >> commit message associated with the change suggested "compatibility >> concerns", but what the concerns actually were isn't specified. Equally >> even though implementing the fork deadline would be very each in the XT >> implementation, this was not done. (the XT codebase has had almost no >> peer review) >> >> >> Options for CLTV/CSV/etc. deployment >> ------------------------------------ >> >> 1) Plain IsSuperMajority() with nVersion=4 >> >> This option can be ruled out immediately due to the high risk of >> premature triggering, without genuine 95% miner support. >> >> >> 2) nVersion mask, with IsSuperMajority() >> >> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would >> be masked away, prior to applying standard IsSuperMajority() logic: >> >> block.nVersion & ~0x20000007 >> >> This means that CLTV/CSV/etc. miners running Bitcoin Core would create >> blocks with nVersion=8, 0b1000. From the perspective of the >> CLTV/CSV/etc. IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be >> advertising blocks that do not trigger the soft-fork. >> >> For the perpose of soft-fork warnings, the highest known version can >> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks >> as well as a future nVersion bits implementation. Equally, >> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an >> unknown bit set. >> >> When nVersion bits is implemented by the Bitcoin protocol, the plan of >> setting the high bits to 0b001 still works. The three lowest bits will >> be unusable for some time, but will be eventually recoverable as >> XT/Not-Bitcoin-XT mining ceases. >> >> Equally, further IsSuperMajority() softforks can be accomplished with >> the same masking technique. >> >> This option does complicate the XT-coin protocol implementation in the >> future. But that's their problem, and anyway, the maintainers >> (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks >> and/or appear to be in favor of a more centralized mandatory update >> schedule.(6) >> >> >> 3) Full nVersion bits implementation >> >> The most complex option would be to deploy via full nVersion bits >> implementation using flag bit #4 to trigger the fork. Compliant miners >> would advertise 0x20000008 initially, followed by 0x20000000 once the >> fork had triggered. The lowest three bits would be unusable for forks >> for some time, although they could be eventually recovered as >> XT/Not-Bitcoin-XT mining ceases. >> >> The main disadvantage of this option is high initial complexity - the >> reason why IsSuperMajority() was suggested for CLTV/CSV in the first >> place. That said, much of the code required has been implemented in XT >> for the BIP101 hard-fork logic, although as mentioned above, the code >> has had very little peer review. >> >> >> References >> ---------- >> >> 1) https://github.com/bitcoinxt/bitcoinxt >> >> 2) https://github.com/xtbit/notbitcoinxt >> >> 3) "Version bits proposal", >> Pieter Wuille, May 26th 2015, Bitcoin-development mailing list, >> >> http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html, >> https://gist.github.com/sipa/bf69659f43e763540550 >> >> 4) >> https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe >> >> 5) "On consensus and forks - What is the difference between a hard and >> soft fork?", >> Mike Hearn, Aug 12th 2015, >> https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7 >> >> 6) 2013 San Jose Bitcoin conference developer round-table >> >> -- >> 'peter'[:-1]@petertodd.org >> 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Wed, Aug 19, 2015 at 10:34 AM, Jorge Timón < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (7 lines)
> Seems like 3 is something we want to do no matter what and therefore > is the "most future-proof" solution. > I wonder if I can help with that (and I know there's more people that > would be interested). > Where's the current "non-full" nVersion bits implementation? > Why implement a "non-full" version instead of going with the full > implementation directly?
There is a simple answer to this, convenience: versionbits has not been implemented yet, and I believe the BIP is still in review stage. As it seems likely the remaining locktime pull requests will be ready by or before the next major release, we need a deployment method if versionbits is not ready (which is unlikely because no-one appears to be working on it at the moment). Pieter indicated he is OK with another IsSuperMajority() rollout in the interim. Personally, I dont think we should let perfection be the enemy of progress here because at the end of the day, the deployment method is less important than the actual featureset being proposed.
That said, the features in the next soft fork proposal are all related and best deployed as one featureset softfork, but moving forward, versionbits seems essential to be able to roll out multiple features in parallel without waiting for activation and enforcement each time.
I don't think just using version=4 for cltv and friends would be a problem if it wasn't for the XT/nonXT issue.
On Wed, Aug 19, 2015 at 12:20 PM, Btc Drak <btcdrak@gmail•com> wrote:
Quoted text (31 lines)
> On Wed, Aug 19, 2015 at 10:34 AM, Jorge Timón > <bitcoin-dev@lists•linuxfoundation•org> wrote: >> >> Seems like 3 is something we want to do no matter what and therefore >> is the "most future-proof" solution. >> I wonder if I can help with that (and I know there's more people that >> would be interested). >> Where's the current "non-full" nVersion bits implementation? >> Why implement a "non-full" version instead of going with the full >> implementation directly? > > > There is a simple answer to this, convenience: versionbits has not been > implemented yet, and I believe the BIP is still in review stage. As it seems > likely the remaining locktime pull requests will be ready by or before the > next major release, we need a deployment method if versionbits is not ready > (which is unlikely because no-one appears to be working on it at the > moment). Pieter indicated he is OK with another IsSuperMajority() rollout in > the interim. Personally, I dont think we should let perfection be the enemy > of progress here because at the end of the day, the deployment method is > less important than the actual featureset being proposed. > > That said, the features in the next soft fork proposal are all related and > best deployed as one featureset softfork, but moving forward, versionbits > seems essential to be able to roll out multiple features in parallel without > waiting for activation and enforcement each time. > > > > >
Wait, why did Bitcoin-XT use that nVersion???
Definitely option 3 is much cleaner technically, and it would be nice to have that code implemented, but I'd be rather concerned about the size of the fork ballooning. It's already four separate features in one fork, which seems pretty big, even if the code isn't too huge. I'd probably prefer slightly to waste another two version bits than add a bunch of code to the fork now (maybe we can take just a part of the bit-based approach and allow lower version numbers again after the fork reaches 95%?). Either way, a proper version of the bit-based soft forking mechanism should be implemented sooner rather than later (maybe merged unused, even).
Still, it is entirely possible that there is relatively little uptake on XT and a competing proposal has a lot of support before we want to ship a LTV fork, so it may all be moot (or we could choose option 1 to fix the XT fork for them - easily forking off XT miners when either fork happens so XT isn't vulnerable to the 25% attack without needing to mine a massive transaction on Bitcoin first :p).
Matt
On August 19, 2015 11:34:54 AM GMT+02:00, "Jorge Timón via bitcoin-dev" <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (182 lines)
>Seems like 3 is something we want to do no matter what and therefore >is the "most future-proof" solution. >I wonder if I can help with that (and I know there's more people that >would be interested). >Where's the current "non-full" nVersion bits implementation? >Why implement a "non-full" version instead of going with the full >implementation directly? > > >On Wed, Aug 19, 2015 at 8:10 AM, Mark Friedenbach via bitcoin-dev ><bitcoin-dev@lists•linuxfoundation•org> wrote: >> We can use nVersion & 0x8 to signal support, while keeping the >consensus >> rule as nVersion >= 4, right? That way we don't waste a bit after >this all >> clears up. >> >> On Aug 18, 2015 10:50 PM, "Peter Todd via bitcoin-dev" >> <bitcoin-dev@lists•linuxfoundation•org> wrote: >>> >>> Deployment of the proposed CLTV, CSV, etc. soft-forks has been >recently >>> complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners. >Both >>> mine blocks with nVersion=0x20000007, which would falsely trigger >the >>> previously suggested implementation using the IsSuperMajority() >>> mechanism and nVersion=4 blocks. Additionally while the >>> XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's >>> nVersion soft-fork mechanism(3) a key component of it - fork >>> deadlines(3) - is not implemented. >>> >>> >>> XT/Not-Bitcoin-XT behavior >>> -------------------------- >>> >>> Both implementations produce blocks with nVersion=0x20000007, >>> or in binary: 0b001...111 >>> >>> Neither implementation supports a fork deadline; both Not-Bitcoin-XT >and >>> XT will produce blocks with those bits set indefinitely under any >>> circumstance, with the proviso that while XT has a hashing power >>> majority, blocks it produces might not be part of the Bitcoin >blockchain >>> after Jan 11th 2016. (though this can flap back and forth if reorgs >>> happen) >>> >>> Curiously the BIP101 draft was changed(4) at the last minute from >using >>> the nVersion bits compliant 0x20000004 block nVersion, to using two >more >>> bits unnecessarily. The rational for doing this is unknown; the git >>> commit message associated with the change suggested "compatibility >>> concerns", but what the concerns actually were isn't specified. >Equally >>> even though implementing the fork deadline would be very each in the >XT >>> implementation, this was not done. (the XT codebase has had almost >no >>> peer review) >>> >>> >>> Options for CLTV/CSV/etc. deployment >>> ------------------------------------ >>> >>> 1) Plain IsSuperMajority() with nVersion=4 >>> >>> This option can be ruled out immediately due to the high risk of >>> premature triggering, without genuine 95% miner support. >>> >>> >>> 2) nVersion mask, with IsSuperMajority() >>> >>> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners >would >>> be masked away, prior to applying standard IsSuperMajority() logic: >>> >>> block.nVersion & ~0x20000007 >>> >>> This means that CLTV/CSV/etc. miners running Bitcoin Core would >create >>> blocks with nVersion=8, 0b1000. From the perspective of the >>> CLTV/CSV/etc. IsSuperMajority() test, XT/Not-Bitcoin-XT miners >would be >>> advertising blocks that do not trigger the soft-fork. >>> >>> For the perpose of soft-fork warnings, the highest known version can >>> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT >blocks >>> as well as a future nVersion bits implementation. Equally, >>> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an >>> unknown bit set. >>> >>> When nVersion bits is implemented by the Bitcoin protocol, the plan >of >>> setting the high bits to 0b001 still works. The three lowest bits >will >>> be unusable for some time, but will be eventually recoverable as >>> XT/Not-Bitcoin-XT mining ceases. >>> >>> Equally, further IsSuperMajority() softforks can be accomplished >with >>> the same masking technique. >>> >>> This option does complicate the XT-coin protocol implementation in >the >>> future. But that's their problem, and anyway, the maintainers >>> (Hearn/Andresen) has strenuously argued(5) against the use of >soft-forks >>> and/or appear to be in favor of a more centralized mandatory update >>> schedule.(6) >>> >>> >>> 3) Full nVersion bits implementation >>> >>> The most complex option would be to deploy via full nVersion bits >>> implementation using flag bit #4 to trigger the fork. Compliant >miners >>> would advertise 0x20000008 initially, followed by 0x20000000 once >the >>> fork had triggered. The lowest three bits would be unusable for >forks >>> for some time, although they could be eventually recovered as >>> XT/Not-Bitcoin-XT mining ceases. >>> >>> The main disadvantage of this option is high initial complexity - >the >>> reason why IsSuperMajority() was suggested for CLTV/CSV in the first >>> place. That said, much of the code required has been implemented in >XT >>> for the BIP101 hard-fork logic, although as mentioned above, the >code >>> has had very little peer review. >>> >>> >>> References >>> ---------- >>> >>> 1) https://github.com/bitcoinxt/bitcoinxt >>> >>> 2) https://github.com/xtbit/notbitcoinxt >>> >>> 3) "Version bits proposal", >>> Pieter Wuille, May 26th 2015, Bitcoin-development mailing list, >>> >>> >http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html, >>> https://gist.github.com/sipa/bf69659f43e763540550 >>> >>> 4) >>> >https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe >>> >>> 5) "On consensus and forks - What is the difference between a hard >and >>> soft fork?", >>> Mike Hearn, Aug 12th 2015, >>> >https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7 >>> >>> 6) 2013 San Jose Bitcoin conference developer round-table >>> >>> -- >>> 'peter'[:-1]@petertodd.org >>> 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d >>> >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists•linuxfoundation•org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> >_______________________________________________ >bitcoin-dev mailing list >bitcoin-dev@lists•linuxfoundation•org >https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
On Wed, Aug 19, 2015 at 11:31 AM, Jorge Timón <jtimon@jtimon•cc> wrote:
Quoted text (2 lines)
> I don't think just using version=4 for cltv and friends would be a > problem if it wasn't for the XT/nonXT issue.
What problem am I missing if we just mask of the offending bits. For my own project which uses auxpow (and thus has weird nVersion), I also used the bitmasking method to get rid of auxpow version bits before making the standard integer comparisons to deploy BIP66 using IsSuperMajority():
if ((block.nVersion & 0xff) >= 4 && CBlockIndex::IsSuperMajority(...))
{ //...}On Wed, Aug 19, 2015 at 7:10 AM, Mark Friedenbach via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (4 lines)
> We can use nVersion & 0x8 to signal support, while keeping the consensus > rule as nVersion >= 4, right? That way we don't waste a bit after this all > clears up. >
What happens if XT gets 40% and this BIP gets 55%? That gives 95% that accept the upgrade. Version 3 and lower blocks need to be rejected after that.
By advertising 0x7 for the last 3 bits, XT is effectively claiming to support the check locktime verify BIPs but they don't have the code.
This sequence could be used, without a specific version-bits proposal.
Until height = N + 5000, if 950 of the last 1000 blocks have the 0x8 bit set, then reject blocks with version numbers less than 8.
At height N, if the above rule is active, then the BIP is permanent.
It applies to any block with bit 0x8 set, once the 75% threshold is met.
From height N + 5000 to N + 10000, reject blocks which have bit 0x8 set.
N could be set 1 year from now, or so.
This gives a period of time after lock that bit 8 is kept and then a period where is is guaranteed to be zero.
This gives software that is only watching the bit time to be upgraded and similarly time where the bit is set to zero before it can be reused.
On Wed, Aug 19, 2015 at 2:15 PM, Btc Drak via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (9 lines)
>
> What problem am I missing if we just mask of the offending bits. For my
> own project which uses auxpow (and thus has weird nVersion), I also used
> the bitmasking method to get rid of auxpow version bits before making the
> standard integer comparisons to deploy BIP66 using IsSuperMajority():
>
> if ((block.nVersion & 0xff) >= 4 && CBlockIndex::IsSuperMajority(...))
> { //...}
>What if version number 257 is used in the future? That would appear to be a version 1 block and fail the test.
A lot of this detail and worry is eliminated by simply asking BIP 102 maintainers to change to a different bit that works best for everyone. Existing nVersion garbage will get buried in sufficient time window to prevent anything from triggering.
Just settle on a specific standard, reserve bits for feature upgrade forks, and move on. There is already a rough standard proposed in sipa's gist.
On Wed, Aug 19, 2015 at 9:22 AM, Tier Nolan via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (40 lines)
> On Wed, Aug 19, 2015 at 7:10 AM, Mark Friedenbach via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation•org> wrote: > >> We can use nVersion & 0x8 to signal support, while keeping the consensus >> rule as nVersion >= 4, right? That way we don't waste a bit after this all >> clears up. >> > What happens if XT gets 40% and this BIP gets 55%? That gives 95% that > accept the upgrade. Version 3 and lower blocks need to be rejected after > that. > > By advertising 0x7 for the last 3 bits, XT is effectively claiming to > support the check locktime verify BIPs but they don't have the code. > > This sequence could be used, without a specific version-bits proposal. > > Until height = N + 5000, if 950 of the last 1000 blocks have the 0x8 bit > set, then reject blocks with version numbers less than 8. > > At height N, if the above rule is active, then the BIP is permanent. > > It applies to any block with bit 0x8 set, once the 75% threshold is met. > > From height N + 5000 to N + 10000, reject blocks which have bit 0x8 set. > > N could be set 1 year from now, or so. > > > This gives a period of time after lock that bit 8 is kept and then a > period where is is guaranteed to be zero. > > This gives software that is only watching the bit time to be upgraded and > similarly time where the bit is set to zero before it can be reused. > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
I think you misunderstand my suggestion Tier. I am suggesting the same as BtcDrak, I think:
Modify IsSuperMajority to take an (optional) mask field. Bits set in that mask are zero'd for the purpose of the IsSuperMajority calculation. For this vote we mask bits 0x20000007.
Thus to signal support for CLTV/CSV/etc. (on Core, XT, or not-XT), you set bit 4. On Core this would mean a minimum version of 0x8, on XT/not-XT a minimum version of 0x20000008.
However the vote is still over whether to enforce BIP 65, 68, etc. for blocks with nVersion>=4. So when this all clears up we haven't needlessly wasted an additional bit.
On Wed, Aug 19, 2015 at 6:22 AM, Tier Nolan via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (40 lines)
> On Wed, Aug 19, 2015 at 7:10 AM, Mark Friedenbach via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation•org> wrote: > >> We can use nVersion & 0x8 to signal support, while keeping the consensus >> rule as nVersion >= 4, right? That way we don't waste a bit after this all >> clears up. >> > What happens if XT gets 40% and this BIP gets 55%? That gives 95% that > accept the upgrade. Version 3 and lower blocks need to be rejected after > that. > > By advertising 0x7 for the last 3 bits, XT is effectively claiming to > support the check locktime verify BIPs but they don't have the code. > > This sequence could be used, without a specific version-bits proposal. > > Until height = N + 5000, if 950 of the last 1000 blocks have the 0x8 bit > set, then reject blocks with version numbers less than 8. > > At height N, if the above rule is active, then the BIP is permanent. > > It applies to any block with bit 0x8 set, once the 75% threshold is met. > > From height N + 5000 to N + 10000, reject blocks which have bit 0x8 set. > > N could be set 1 year from now, or so. > > > This gives a period of time after lock that bit 8 is kept and then a > period where is is guaranteed to be zero. > > This gives software that is only watching the bit time to be upgraded and > similarly time where the bit is set to zero before it can be reused. > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
On Wed, Aug 19, 2015 at 2:24 PM, Tier Nolan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (6 lines)
> On Wed, Aug 19, 2015 at 2:15 PM, Btc Drak via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
>> What problem am I missing if we just mask of the offending bits. For my own project which uses auxpow (and thus has weird nVersion), I also used the bitmasking method to get rid of auxpow version bits before making the standard integer comparisons to deploy BIP66 using IsSuperMajority():
>>
>> if ((block.nVersion & 0xff) >= 4 && CBlockIndex::IsSuperMajority(...)) { //...}
>
> What if version number 257 is used in the future? That would appear to be a version 1 block and fail the test.To clarify this is the code example for how the same problem occurs with auxpow bits when running a an IsSuperMajority() softfork and how it's solved in that instance.
In our case for Bitcoin Core, option 2 we use nVersion=8, apply a bitmask of 0xdffffff8 thus:
if ((block.nVersion & ~0x20000007) >= 4 &&
CBlockIndex::IsSuperMajority(...)) { //...}
With nVersion=8, but using comparison >=4 allows us to recover the bit later, assuming we want it (otherwise we use version >=8).
This should, unless I am missing something, be forward compatible with future softforks. My understanding was if "versionbits softfork" code is not ready by the time we're ready for the deployment, IsSuperMajority() would be acceptable; and this is how we could deploy it in the wake of the XT developers' carelessness.
On Wed, Aug 19, 2015 at 6:25 PM, Btc Drak <btcdrak@gmail•com> wrote:
Quoted text (9 lines)
> In our case for Bitcoin Core, option 2 we use nVersion=8, apply a
> bitmask of 0xdffffff8 thus:
>
> if ((block.nVersion & ~0x20000007) >= 4 &&
> CBlockIndex::IsSuperMajority(...)) { //...}
>
> With nVersion=8, but using comparison >=4 allows us to recover the
> bit later, assuming we want it (otherwise we use version >=8).
>That is the 75% "activation" rule portion? The 95% rule has to apply to all blocks.
The supermajority applies to unmasked blocks?
I think you want it so that a sequence of blocks with version 8 can be followed by version 4 blocks?
If 950 of the last 1000 blocks have bit 0x08 set, then reject any block with a version less than 4.
This means transitioning to the version bits BIP just requires dropping the version back to 4 and adding a rule enforcing the BIPs for version 4 and higher blocks.
This would be part of the version bits BIP enforcement.
On Wed, Aug 19, 2015 at 09:32:49AM -0700, Mark Friedenbach via bitcoin-dev wrote:
Quoted text (14 lines)
> I think you misunderstand my suggestion Tier. I am suggesting the same as > BtcDrak, I think: > > Modify IsSuperMajority to take an (optional) mask field. Bits set in that > mask are zero'd for the purpose of the IsSuperMajority calculation. For > this vote we mask bits 0x20000007. > > Thus to signal support for CLTV/CSV/etc. (on Core, XT, or not-XT), you set > bit 4. On Core this would mean a minimum version of 0x8, on XT/not-XT a > minimum version of 0x20000008. > > However the vote is still over whether to enforce BIP 65, 68, etc. for > blocks with nVersion>=4. So when this all clears up we haven't needlessly > wasted an additional bit.
Ah, I see your point now re: wasting bits; my post was a bit incorrect on that point.
So a subtle thing with the IsSuperMajority() mechanism, and the nVersion bits proposal alternative, is one of the main objectives of the latter proposal proposal is to allow forks to *fail* to be adopted cleanly.
To illustrate the problem, consider a hypothetical CLTV soft-fork, implemented with IsSuperMajority() nVersion >= 4. We release Bitcoin Core with that code, call it Bitcoin Core v0.12.0, however some substantial fraction of the mining community refuses to upgrade, believing CLTV to be a bad idea. This forms the community into Bitcoin Core and Bitcoin Not-CLTV camps. The Not-CLTV camp then wants to do a new soft-fork upgrade, say for CHECKSIG2
What now? If CHECKSIG2 is implemented via IsSuperMajority, nVersion >= 5, that'll falsely trigger Core nodes to think the upgrade has gone though. You could safely define >= 5 semantics to be "OP_CLTV is now disabled", but that's pretty ugly and unnecessarily uses up a NOP.
You can avoid this problem by assigning one bit out of nVersion for every soft-fork, but then you can only do ~29 more soft-forks - ugly!
Come to think of it, if you're Really Sure™ the soft-fork will be adopted, you can recycle those bits by using the following rule:
if (IsFlagBitMaskSuperMajority(1 << 4, pindex->pprev) || block.nMedianTime > CLTV_SOFTFORK_DEADLINE) {
flags |= SCRIPT_VERIFY_CLTV;
}
IsFlagBitMaskSuperMajority() is a masked version of the existing IsSuperMajority() logic. CLTV_SOFTFORK_DEADLINE would be set some reasonable amount of time in the future, perhaps 3 years.
This would probably be ok for non-controversial forks - implementing DERSIG this way would have been fine - and an unlimited number of soft-forks can be done this way safely. (even in parallel)
However, this idea still causes problems if forks ever fail to get adoption, something the nVersion bits proposal handles cleanly, albeit at the cost of a significantly more complex implementation. With a sufficiently far off fork deadline in practice that may not be a big issue - nearly everyone would have upgraded their software anyway - but it's still technically creating hard-fork scenarios with respect to older software whenever forks fail to get adoption.
Peter Todd via bitcoin-dev 於 2015-08-19 01:50 寫到:
Quoted text (36 lines)
> > > 2) nVersion mask, with IsSuperMajority() > > In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would > be masked away, prior to applying standard IsSuperMajority() logic: > > block.nVersion & ~0x20000007 > > This means that CLTV/CSV/etc. miners running Bitcoin Core would create > blocks with nVersion=8, 0b1000. From the perspective of the > CLTV/CSV/etc. IsSuperMajority() test, XT/Not-Bitcoin-XT miners would > be > advertising blocks that do not trigger the soft-fork. > > For the perpose of soft-fork warnings, the highest known version can > remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks > as well as a future nVersion bits implementation. Equally, > XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an > unknown bit set. > > When nVersion bits is implemented by the Bitcoin protocol, the plan of > setting the high bits to 0b001 still works. The three lowest bits will > be unusable for some time, but will be eventually recoverable as > XT/Not-Bitcoin-XT mining ceases. > > Equally, further IsSuperMajority() softforks can be accomplished with > the same masking technique. > > This option does complicate the XT-coin protocol implementation in the > future. But that's their problem, and anyway, the maintainers > (Hearn/Andresen) has strenuously argued(5) against the use of > soft-forks > and/or appear to be in favor of a more centralized mandatory update > schedule.(6) >
If you are going to mask bits, would you consider to mask all bits except the 4th bit? So other fork proposals may use other bits for voting concurrently.
And as I understand, the masking is applied only during the voting stage? After the softfork is fully enforced with 95% support, the nVersion will be simply >=8, without any masking?
No, the nVersion would be >= 4, so that we don't waste any version values.
On Thu, Aug 20, 2015 at 10:32 AM, jl2012 via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (50 lines)
> Peter Todd via bitcoin-dev 於 2015-08-19 01:50 寫到: > > >> >> 2) nVersion mask, with IsSuperMajority() >> >> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would >> be masked away, prior to applying standard IsSuperMajority() logic: >> >> block.nVersion & ~0x20000007 >> >> This means that CLTV/CSV/etc. miners running Bitcoin Core would create >> blocks with nVersion=8, 0b1000. From the perspective of the >> CLTV/CSV/etc. IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be >> advertising blocks that do not trigger the soft-fork. >> >> For the perpose of soft-fork warnings, the highest known version can >> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks >> as well as a future nVersion bits implementation. Equally, >> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an >> unknown bit set. >> >> When nVersion bits is implemented by the Bitcoin protocol, the plan of >> setting the high bits to 0b001 still works. The three lowest bits will >> be unusable for some time, but will be eventually recoverable as >> XT/Not-Bitcoin-XT mining ceases. >> >> Equally, further IsSuperMajority() softforks can be accomplished with >> the same masking technique. >> >> This option does complicate the XT-coin protocol implementation in the >> future. But that's their problem, and anyway, the maintainers >> (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks >> and/or appear to be in favor of a more centralized mandatory update >> schedule.(6) >> >> > If you are going to mask bits, would you consider to mask all bits except > the 4th bit? So other fork proposals may use other bits for voting > concurrently. > > And as I understand, the masking is applied only during the voting stage? > After the softfork is fully enforced with 95% support, the nVersion will be > simply >=8, without any masking? > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
I have changed BIPS 112 and 113 to reflect this amended deployment strategy. I'm beginning to think the issues created by Bitcoin XT are so serious it probably deserves converting OPs text into an informational BIP.
On Thu, Aug 20, 2015 at 6:42 PM, Mark Friedenbach via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (62 lines)
> No, the nVersion would be >= 4, so that we don't waste any version values. > > On Thu, Aug 20, 2015 at 10:32 AM, jl2012 via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation•org> wrote: >> >> Peter Todd via bitcoin-dev 於 2015-08-19 01:50 寫到: >> >>> >>> >>> 2) nVersion mask, with IsSuperMajority() >>> >>> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would >>> be masked away, prior to applying standard IsSuperMajority() logic: >>> >>> block.nVersion & ~0x20000007 >>> >>> This means that CLTV/CSV/etc. miners running Bitcoin Core would create >>> blocks with nVersion=8, 0b1000. From the perspective of the >>> CLTV/CSV/etc. IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be >>> advertising blocks that do not trigger the soft-fork. >>> >>> For the perpose of soft-fork warnings, the highest known version can >>> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks >>> as well as a future nVersion bits implementation. Equally, >>> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an >>> unknown bit set. >>> >>> When nVersion bits is implemented by the Bitcoin protocol, the plan of >>> setting the high bits to 0b001 still works. The three lowest bits will >>> be unusable for some time, but will be eventually recoverable as >>> XT/Not-Bitcoin-XT mining ceases. >>> >>> Equally, further IsSuperMajority() softforks can be accomplished with >>> the same masking technique. >>> >>> This option does complicate the XT-coin protocol implementation in the >>> future. But that's their problem, and anyway, the maintainers >>> (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks >>> and/or appear to be in favor of a more centralized mandatory update >>> schedule.(6) >>> >> >> If you are going to mask bits, would you consider to mask all bits except >> the 4th bit? So other fork proposals may use other bits for voting >> concurrently. >> >> And as I understand, the masking is applied only during the voting stage? >> After the softfork is fully enforced with 95% support, the nVersion will be >> simply >=8, without any masking? >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Thu, Aug 27, 2015 at 11:08:32PM +0100, Btc Drak wrote:
Quoted text (6 lines)
> This BIP was assigned number 113. > > I have updated the text accordingly and added credits to Gregory Maxwell. > > Please see the changes in the pull request: > https://github.com/bitcoin/bips/pull/182
On Thu, Aug 27, 2015 at 11:11:10PM +0100, Btc Drak via bitcoin-dev wrote:
Quoted text (4 lines)
> I have changed BIPS 112 and 113 to reflect this amended deployment > strategy. I'm beginning to think the issues created by Bitcoin XT are > so serious it probably deserves converting OPs text into an > informational BIP.
I thought we had decided that the masking thing doesn't work as intended?
To recap, XT nodes are producing blocks with nVersion=0b001...111
You're suggesting that we apply a mask of ~0b001...111 then trigger the soft-fork on nVersion >= 0b0...100 == 4, with miners producing blocks with nVersion=0b0...1000
That will work, but it still uses up a version bit. The reason why is blocks with nVersion=0b001...000 - the intended deployment of the nVersion bits proposal - will be rejected by the nVersion >= 4 rule, hard-forking them off the network. In short, we have in fact "burnt" a version bit unnecessarily.
If you're going to accept hard-forking some people off the network, why not just go with my stateless nVersion bits w/ time-expiration proposal instead? The only case where it leads to a hard-fork is if a soft-fork has been rejected by the time the upgrade deadline is reached. It's easy to set this multiple years into the future, so I think in practice it won't be a major issue for non-controversial soft-forks.
Equally, spending the time to implement the original stateful nVersion bits proposal is possible as well, though higher risk due to the extra complexity of tracking soft-fork state.
I have an ugly solution to this problem, with minimal change to the current softfork logic, and still allows all features described in sipa's Version bits BIP
1. xVersion = nVersion AND 0b10000000000000000000000000011000 2. miners supporting BIP65 will set xVersion = 8 or greater 3. If 750 of the last 1000 blocks are xVersion >= 8, reject invalid xVersion 8 (or greater) blocks 4. If 950 of the last 1000 blocks are xVersion >= 8, reject all blocks with xVersion < 8
So the logic is exactly the same as BIP66, with the AND masking in step 1. After the BIP65 softfork is implied, xVersion may take only one of the following values: 8, 16, 24
This is basically moving the "high bits" in sipa's proposal to the middle of the nVersion field. After the BIP65 softfork, this will still leave 29 available bits for parallel soft forks, just in different position.
This is ugly, but I believe this is the easiest solution
Ref: https://gist.github.com/sipa/bf69659f43e763540550
Peter Todd via bitcoin-dev 於 2015-08-27 19:19 寫到:
Quoted text (48 lines)
> On Thu, Aug 27, 2015 at 11:08:32PM +0100, Btc Drak wrote: >> This BIP was assigned number 113. >> >> I have updated the text accordingly and added credits to Gregory >> Maxwell. >> >> Please see the changes in the pull request: >> https://github.com/bitcoin/bips/pull/182 > > On Thu, Aug 27, 2015 at 11:11:10PM +0100, Btc Drak via bitcoin-dev > wrote: >> I have changed BIPS 112 and 113 to reflect this amended deployment >> strategy. I'm beginning to think the issues created by Bitcoin XT are >> so serious it probably deserves converting OPs text into an >> informational BIP. > > I thought we had decided that the masking thing doesn't work as > intended? > > To recap, XT nodes are producing blocks with nVersion=0b001...111 > > You're suggesting that we apply a mask of ~0b001...111 then trigger the > soft-fork on nVersion >= 0b0...100 == 4, with miners producing blocks > with > nVersion=0b0...1000 > > That will work, but it still uses up a version bit. The reason why is > blocks with nVersion=0b001...000 - the intended deployment of the > nVersion bits proposal - will be rejected by the nVersion >= 4 rule, > hard-forking them off the network. In short, we have in fact "burnt" a > version bit unnecessarily. > > If you're going to accept hard-forking some people off the network, why > not just go with my stateless nVersion bits w/ time-expiration proposal > instead? The only case where it leads to a hard-fork is if a soft-fork > has been rejected by the time the upgrade deadline is reached. It's > easy > to set this multiple years into the future, so I think in practice it > won't be a major issue for non-controversial soft-forks. > > Equally, spending the time to implement the original stateful nVersion > bits proposal is possible as well, though higher risk due to the extra > complexity of tracking soft-fork state. > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev