BIP draft: Hardfork bit

Please feel free to comment, for technical issues and language

BIP: ?? Title: Hardfork bit Author: jl2012 <jl2012@xbt•hk> Status: Draft Type: Standard Track Created: 2015-07-23

Abstract

This document specifies a proposed change to the semantics of the most significant bit of the “version” field in Bitcoin block headers, as a mechanism to indicate a hardfork is deployed. It alleviates certain risks related to a hardfork by introducing an explicit “point of no return” in the blockchain. This is a general mechanism which should be employed by any planned hardfork in the future.

Motivation

Hardforks in Bitcoin are usually considered as difficult and risky, because:

1) Hardforks require not only support of miners, but also, most importantly, supermajority support of the Bitcoin economy. As a result, softfork deployment mechanisms described in BIP 34 or BIP XX “Version bits” (https://gist.github.com/sipa/bf69659f43e763540550) are not enough for introducing hardforks safely.

2) Full nodes and SPV nodes following original consensus rules may not be aware of the deployment of a hardfork. They may stick to an economic-minority fork and unknowingly accept devalued legacy tokens.

3) In the case which the original consensus rules are also valid under the new consensus rules, users following the new chain may unexpectedly reorg back to the original chain if it grows faster than the new one. People may find their confirmed transactions becoming unconfirmed and lose money.

The first issue involves soliciting support for a hardfork proposal, which is more a political topic than a technical one. This proposal aims at alleviating the risks related to the second and third issues. It should be employed by any planned hardfork in the future.

Definitions

See BIP YY “Motivation and deployment of consensus rules changes” https://github.com/jtimon/bips/blob/bip-forks/bip-forks.org

Specification

Hardfork bit: The most significant bit in nVersion is defined as the hardfork bit. Currently, blocks with this header bit setting to 1 are invalid, since BIP34 interprets nVersion as a signed number and requires it to be >=2 (with BIP66, >=3). Among the 640 bits in the block header, this is the only one which is fixed and serves no purpose, and therefore the best way to indicate the deployment of a hardfork.

Flag block: Any planned hardfork must have one and only one flag block which is the “point of no return”. To ensure monotonicity, flag block should be determined by block height, or as the first block with GetMedianTimePast() greater than a threshold. Other mechanisms could be difficult for SPV nodes to follow. The height/time threshold could be a predetermined value or relative to other events (e.g. 1000 blocks / 10 days after 75% of miner support). The exact mechanism is out of the scope of this BIP. No matter what mechanism is used, the threshold is consensus critical. It must be publicly verifiable with only blockchain data and the programme source code, and preferably SPV-friendly.

Flag block is constructed in a way that nodes with the original consensus rules must reject. On the other hand, nodes with the new consensus rules must reject a block if it is not a flag block while it is supposed to be. To achieve these goals, the flag block must 1) have the hardfork bit setting to 1, 2) include a short predetermined unique description of the hardfork anywhere in its coinbase, and 3) follow any other rules required by the hardfork. If these conditions are not fully satisfied, upgraded nodes shall reject the block.

The hardfork bit must be turned off in the decedents of the flag block, until the deployment of the next hardfork. The requirement of coinbase message is also limited to the flag block. In the rare case that multiple hardforks share the same flag block, the coinbase shall include all relevant messages and the order/position of the messages shall not be consensus critical.

Although a hardfork is officially deployed after the flag block, the exact behavioural change is out of the scope of this BIP. For example, a hardfork may not be fully active until certain time after the flag block.

Automatic warning system: When a flag block is found on the network, full nodes and SPV nodes should look into its coinbase. They should alert their users and/or stop accepting incoming transactions if it is an unknown hardfork. It should be noted that the warning system could become a DoS vector if the attacker is willing to give up the block reward. Therefore, the warning may be issued only if a few blocks are built on top of the flag block in a reasonable time frame. This will in turn increase the risk in case of a real planned hardfork so it is up to the wallet programmers to decide the optimal strategy. Human warning system (e.g. the emergency alert system in Bitcoin Core) could fill the gap.

Compatibility

As a mechanism to indicate hardfork deployment, this BIP breaks backward compatibility intentionally. However, without further changes in the block header format, full nodes and SPV nodes could still verify the PoW of a flag block and its descendants.

This proposal is also compatible with the BIP XX “Version bits”. The version bits mechanism could be employed to measure miner support towards a hardfork proposal, and to determine the height or time threshold of the flag block. Also, miners of the flag block may still cast votes for other concurrent softfork or hardfork proposals as normal.

After the flag block is generated, a miner may support either fork but not both. It is not possible for miners in one fork to attack or overtake the other fork because the forks are mutually exclusive.

On Thu, Jul 23, 2015 at 5:23 PM, jl2012 via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:

> 2) Full nodes and SPV nodes following original consensus rules may not be
> aware of the deployment of a hardfork. They may stick to an
> economic-minority fork and unknowingly accept devalued legacy tokens.
>

This change means that they are kicked off the main chain immediately when the fork activates.

The change is itself a hard fork. Clients have be updated to get the benefits.

3) In the case which the original consensus rules are also valid under the

> new consensus rules, users following the new chain may unexpectedly reorg
> back to the original chain if it grows faster than the new one. People may
> find their confirmed transactions becoming unconfirmed and lose money.
>

I don't understand the situation here. Is the assumption of a group of miners suddenly switching (for example, they realise that they didn't intend to support the new rules)?

>
> Flag block is constructed in a way that nodes with the original consensus
> rules must reject. On the other hand, nodes with the new consensus rules
> must reject a block if it is not a flag block while it is supposed to be.
> To achieve these goals, the flag block must 1) have the hardfork bit
> setting to 1, 2) include a short predetermined unique description of the
> hardfork anywhere in its coinbase, and 3) follow any other rules required
> by the hardfork. If these conditions are not fully satisfied, upgraded
> nodes shall reject the block.
>

Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP on github in the coinbase?

Since it is a hard fork, the version field could be completely re-purposed. Set the bit and add the BIP number as the lower bits in the version field. This lets SPV clients check if they know about the hard fork.

There network protocol could be updated to add getdata support for asking for a coinbase only merkleblock. This would allow SPV clients to obtain the coinbase.

Automatic warning system: When a flag block is found on the network, full

> nodes and SPV nodes should look into its coinbase. They should alert their
> users and/or stop accepting incoming transactions if it is an unknown
> hardfork. It should be noted that the warning system could become a DoS
> vector if the attacker is willing to give up the block reward. Therefore,
> the warning may be issued only if a few blocks are built on top of the flag
> block in a reasonable time frame. This will in turn increase the risk in
> case of a real planned hardfork so it is up to the wallet programmers to
> decide the optimal strategy. Human warning system (e.g. the emergency alert
> system in Bitcoin Core) could fill the gap.
>

If the rule was that hard forks only take effect 100 blocks after the flag block, then this problem is eliminated.

Emergency hard forks may still have to take effect immediately though, so it would have to be a custom not a rule.

Quoting Tier Nolan via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org>:

> On Thu, Jul 23, 2015 at 5:23 PM, jl2012 via bitcoin-dev <
> bitcoin-dev@lists•linuxfoundation•org> wrote:
>
>> 2) Full nodes and SPV nodes following original consensus rules may not be
>> aware of the deployment of a hardfork. They may stick to an
>> economic-minority fork and unknowingly accept devalued legacy tokens.
>>
>
> This change means that they are kicked off the main chain immediately when
> the fork activates.
>
> The change is itself a hard fork.  Clients have be updated to get the
> benefits.

I refrain from calling it the "main chain". I use "original chain" and "new chain" instead as I make no assumption about the distribution of mining power. This BIP still works when we have a 50/50 hardfork. The main point is to protect all users on both chains, and allow them to make an informed choice.

> 3) In the case which the original consensus rules are also valid under the
>> new consensus rules, users following the new chain may unexpectedly reorg
>> back to the original chain if it grows faster than the new one. People may
>> find their confirmed transactions becoming unconfirmed and lose money.
>>
>
> I don't understand the situation here.  Is the assumption of a group of
> miners suddenly switching (for example, they realise that they didn't
> intend to support the new rules)?
>

Again, as I make no assumption about the mining power distribution, the new chain may actually have less miner support. Without any protection (AFAIK, for example, BIP100, 101, 102), the weaker new chain will get 51%-attacked by the original chain constantly.

>>
>> Flag block is constructed in a way that nodes with the original consensus
>> rules must reject. On the other hand, nodes with the new consensus rules
>> must reject a block if it is not a flag block while it is supposed to be.
>> To achieve these goals, the flag block must 1) have the hardfork bit
>> setting to 1, 2) include a short predetermined unique description of the
>> hardfork anywhere in its coinbase, and 3) follow any other rules required
>> by the hardfork. If these conditions are not fully satisfied, upgraded
>> nodes shall reject the block.
>>
>
> Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP on
> github in the coinbase?

I guess the git hash is not known until the code is written? (correct me if I'm wrong) As the coinbase message is consensus-critical, it must be part of the source code and therefore you can't use any kind of hash of the code itself (a chicken-and-egg problem)

> Since it is a hard fork, the version field could be completely
> re-purposed.  Set the bit and add the BIP number as the lower bits in the
> version field.  This lets SPV clients check if they know about the hard
> fork.

This may not be compatible with the other version bits voting mechanisms.

> There network protocol could be updated to add getdata support for asking
> for a coinbase only merkleblock.  This would allow SPV clients to obtain
> the coinbase.

Yes

> Automatic warning system: When a flag block is found on the network, full
>> nodes and SPV nodes should look into its coinbase. They should alert their
>> users and/or stop accepting incoming transactions if it is an unknown
>> hardfork. It should be noted that the warning system could become a DoS
>> vector if the attacker is willing to give up the block reward. Therefore,
>> the warning may be issued only if a few blocks are built on top of the flag
>> block in a reasonable time frame. This will in turn increase the risk in
>> case of a real planned hardfork so it is up to the wallet programmers to
>> decide the optimal strategy. Human warning system (e.g. the emergency alert
>> system in Bitcoin Core) could fill the gap.
>>
>
> If the rule was that hard forks only take effect 100 blocks after the flag
> block, then this problem is eliminated.
>
> Emergency hard forks may still have to take effect immediately though, so
> it would have to be a custom not a rule.

The flag block itself is a hardfork already and old miners will not mine on top of the flag block. So your suggestion won't be helpful in this situation.

To make it really meaningful, we need to consume one more bit of the 'version' field ("notice bit"). Supporting miners will turn on the notice bit, and include a message in coinbase ("notice block"). When a full node/SPV node find many notice blocks with the same coinbase message, they could bet that the subsequent flag block is a legit one. However, an attacker may still troll you by injecting an invalid flag block after many legit notice blocks. So I'm not sure if it is worth the added complexity.

>I don't understand the situation here.  Is the assumption of a group of
>miners suddenly switching (for example, they realise that they didn't
>intend to support the new rules)?

Or they're economically rational miners, and a large difficulty decrease on the original chain, without an equally large decrease in the value of original chain tokens, has made it profitable to switch?

It's dangerous to assume all miners will continue to support the side they have initially signaled. They're only invested in the chain they mine for a short time (until coinbase maturity.)

If both sides if a fork retain value you'd expect mining to redistribute itself WRT short term profitability at every difficulty adjustment, irrespective of initially signaled support for the fork.

>> Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP on
>> github in the coinbase?
>
>
> I guess the git hash is not known until the code is written? (correct me if
> I'm wrong) As the coinbase message is consensus-critical, it must be part of
> the source code and therefore you can't use any kind of hash of the code
> itself (a chicken-and-egg problem)
>

I took Tier's comment to mean that the commit hash (taken from git) of the BIP for the particular hard fork being rolled out via your hard fork bit mechanism could be used in the coinbase. The commit for the BIP is separate from the commit for the code changes implementing it.

By using the commit hash of the BIP, it means that the BIP cannot specify itself what to put in the coinbase. I'd suggest that your hard fork bit proposal allows BIP authors to specify, within a 20 byte limit (to let them ripe160 a message or something), the unique value to use. This would be just as well and would allow the specific hard fork BIPs to be updated without having to make code changes (e.g.- for simple grammatical updates, post deployment clarification, etc...).

Perhaps more preferable, in order to keep the coinbases small, or, if you're worried about BIP authors using duplicate values, then just specify in your proposal that the coinbase message include "BIP<NUMBER>" as BIP numbers are not going to be reused. All you are trying to achieve is something that can be uniquely pattern matched.

Are the reasons for your use of the coinbase message 1) to guard against one or more hard forks piggy-backing on another's flag block, without prior collaboration, and 2) to have a nicer message in the alerting system? If so, you might want to spell that out in your proposal. At first, I was thinking that the coinbase thing was not entirely necessary since the possibility of multiple hard forks taking place concurrently is low. Multiple forking changes could be coordinated to all use the same possible voting mechanisms, median timestamp, and thus flag block. But if the concern is about competing hard forks where two or more forks abandon the original chain and attempt to use the same checkpoint block, then I can see why you'd need it.

I was reading this proposal alongside another of your messages (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009520.html) proposing a version 0 or even a >1MB block in the specific case of a block size limit hard fork. I can see the >1MB flag block creating DoS banning problems. That's more of a practical issue than a consensus issue due to the existing mono-culture of full nodes. So, currently a

>1MB flag block is not as appealing as a version 0 or this hard fork

bit proposal. Besides the DoS ban, the >1MB proposal would mean that older nodes do not have the chance to notice a fork is happening (for alerting) as they would with a version 0 or hard fork bit.

A version 0 flag block would not be able to contain any transactions unless the flag block version was assumed to be that of the most recent version at the time. This is because we'd want to enforce the rules of the previous soft forks that had been locked in. Other than that, the version 0 idea seems pretty similar to the hard fork bit proposal because you need more context than just the block itself to determine if it's a valid flag block (and this extra context is probably not great, but I don't have a better idea right now).

Were those reasons part of why you progressed towards this hard fork bit proposal?

Although the chance is very slim, it is possible to have multiple hardforks sharing the same flag block. For example, different proposals may decide the flag time based on voting result and 2 proposals may have the same flag time just by chance. The coinbase message is to preclude any potential ambiguity. It also provides additional info to warning system of non-upgrading nodes.

If we are pretty sure that there won't be other hardfork proposal at the same time, the coinbase message may not be necessary. With some prior collaboration, this may also be avoided (e.g. no sharing flag block allowed as consensus rules of the hardforks)

The "version 0" idea is not compatible with the version bits voting system, so I have this hardfork bit BIP after thinking more carefully. Otherwise they are technically similar.

Michael Ruddy 於 2015-08-01 09:05 寫到:

>>> Ok, so set the bit and then include BIP-GIT-HASH of the canonical BIP
>>> on
>>> github in the coinbase?
>>
>>
>> I guess the git hash is not known until the code is written? (correct
>> me if
>> I'm wrong) As the coinbase message is consensus-critical, it must be
>> part of
>> the source code and therefore you can't use any kind of hash of the
>> code
>> itself (a chicken-and-egg problem)
>>
>
> I took Tier's comment to mean that the commit hash (taken from git) of
> the BIP for the particular hard fork being rolled out via your hard
> fork bit mechanism could be used in the coinbase.
> The commit for the BIP is separate from the commit for the code
> changes implementing it.
>
> By using the commit hash of the BIP, it means that the BIP cannot
> specify itself what to put in the coinbase.
> I'd suggest that your hard fork bit proposal allows BIP authors to
> specify, within a 20 byte limit (to let them ripe160 a message or
> something), the unique value to use.
> This would be just as well and would allow the specific hard fork BIPs
> to be updated without having to make code changes (e.g.- for simple
> grammatical updates, post deployment clarification, etc...).
>
> Perhaps more preferable, in order to keep the coinbases small, or, if
> you're worried about BIP authors using duplicate values, then just
> specify in your proposal that the coinbase message include
> "BIP<NUMBER>" as BIP numbers are not going to be reused. All you are
> trying to achieve is something that can be uniquely pattern matched.
>
> Are the reasons for your use of the coinbase message 1) to guard
> against one or more hard forks piggy-backing on another's flag block,
> without prior collaboration, and 2) to have a nicer message in the
> alerting system? If so, you might want to spell that out in your
> proposal. At first, I was thinking that the coinbase thing was not
> entirely necessary since the possibility of multiple hard forks taking
> place concurrently is low. Multiple forking changes could be
> coordinated to all use the same possible voting mechanisms, median
> timestamp, and thus flag block. But if the concern is about competing
> hard forks where two or more forks abandon the original chain and
> attempt to use the same checkpoint block, then I can see why you'd
> need it.
>
> I was reading this proposal alongside another of your messages
> (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009520.html)
> proposing a version 0 or even a >1MB block in the specific case of a
> block size limit hard fork. I can see the >1MB flag block creating DoS
> banning problems. That's more of a practical issue than a consensus
> issue due to the existing mono-culture of full nodes. So, currently a
>> 1MB flag block is not as appealing as a version 0 or this hard fork
> bit proposal. Besides the DoS ban, the >1MB proposal would mean that
> older nodes do not have the chance to notice a fork is happening (for
> alerting) as they would with a version 0 or hard fork bit.
>
> A version 0 flag block would not be able to contain any transactions
> unless the flag block version was assumed to be that of the most
> recent version at the time. This is because we'd want to enforce the
> rules of the previous soft forks that had been locked in. Other than
> that, the version 0 idea seems pretty similar to the hard fork bit
> proposal because you need more context than just the block itself to
> determine if it's a valid flag block (and this extra context is
> probably not great, but I don't have a better idea right now).
>
> Were those reasons part of why you progressed towards this hard fork
> bit proposal?

I think your "hardfork bit" proposal is clever. It addresses the particular valid concern of re-org facing users of a fork that a small/near/fluctuating majority, or less, of mining power supported. While the "economic majority" argument may be enough on its own in that case, it still has some aspect of being a hand wave. This proposal adds support to those economic actors, which makes it easier for them to switch if/when they choose. That is, it provides a good fallback mechanism that allows them to make a decision and say, "we're doing this". Do you have the latest version up on github, or someplace where it would be easier to collaborate on the specific text?

On Sat, Aug 1, 2015 at 4:23 PM, <jl2012@xbt•hk> wrote:

> Although the chance is very slim, it is possible to have multiple hardforks
> sharing the same flag block. For example, different proposals may decide the
> flag time based on voting result and 2 proposals may have the same flag time
> just by chance. The coinbase message is to preclude any potential ambiguity.
> It also provides additional info to warning system of non-upgrading nodes.
>
> If we are pretty sure that there won't be other hardfork proposal at the
> same time, the coinbase message may not be necessary. With some prior
> collaboration, this may also be avoided (e.g. no sharing flag block allowed
> as consensus rules of the hardforks)
>
> The "version 0" idea is not compatible with the version bits voting system,
> so I have this hardfork bit BIP after thinking more carefully. Otherwise
> they are technically similar.

I have put it on the github: https://github.com/jl2012/bips/blob/master/hardforkbit.mediawiki

I removed the specification of coinbase message to make it simpler. Instead, it requires that a flag block must not be shared by multiple hardfork proposals.

I'm not sure whether it is a Standard, Informational, or Process BIP

I'm also thinking whether we should call it "hardfork bit", "hardfork flag", or with other name.

Michael Ruddy 於 2015-08-02 06:53 寫到:

> I think your "hardfork bit" proposal is clever.
> It addresses the particular valid concern of re-org facing users of a
> fork that a small/near/fluctuating majority, or less, of mining power
> supported.
> While the "economic majority" argument may be enough on its own in
> that case, it still has some aspect of being a hand wave.
> This proposal adds support to those economic actors, which makes it
> easier for them to switch if/when they choose. That is, it provides a
> good fallback mechanism that allows them to make a decision and say,
> "we're doing this".
> Do you have the latest version up on github, or someplace where it
> would be easier to collaborate on the specific text?