Consensus fork activation thresholds: Block.nTime vs median time vs block.nHeight

When it comes to define thresholds for consensus fork activation there are 3 options that I know of and each of them has at least a disadvantage that the other 2 lack:

-Block.nTime: It's not monotonic -median time: You cannot validate it without context (in contrast, nTime is contained in the block header and nHeight in the coinbase) -block.nHeight: You cannot know the exact time when a given height will be reached.

I personally think that nHeight's disadvantage is the less worse, but others will likely disagree. The point is we need to find a solid criteria to decide.

When combining the threshold with a later miner's "voting" (upgrade confirmation) on top of it, not being monotonic may be a real problem. Doing that on top of height seems straight forward: check if (prevBlockIndex.nHeight > threshold && IsSuperMajority(...,prevBlockIndex,...)) With median time, seems safe too: if (prevBlockIndex.CalculateMedianTime > threshold && IsSuperMajority(...,prevBlockIndex,...)) Just a little bit less efficient. It would look more like if (IsSuperMajority(...,prevBlockIndex,...) && GetFirstBlockUsedInVoting(prevBlockIndex).nTime > threshold)

But in some cases (say, an emergency consensus fork) you won't combine the mining confirmation, so you may not have the prevBlockIndex available and you may need to pass the height or medianTime down. If the current block is not accessible from wherever the check is being made, you would need an additional blockTime parameter as well.

Are there any example cases where a rule activation check doesn't have the block available?

Of course, let's consider the following hardfork example:

before the hardfork: consensus_size(tx) = real_size(tx) after the hardfork: after consensus_size(tx) = real_size(tx) + delta_utxo_size(tx)

that would allow miners to create bigger blocks if the transactions help reducing the size of the utxo (and penalize transactions that make the utxo grow by considering bigger when it comes to block inclusion).

Well, at the block validation level (the most important one), you obviously have block.nTime available. But what if you're checking an unconfirmed transaction? It's size (and thus it's validity and the policy relay decisions) depends on whether the hardfork is activated or not. So to check an unconfirmed transaction, you would need the block.nTime of the next block, which is unpredictable (unless you're a miner) because miners set those.

AcceptToMemoryPool already uses the nHeight (in fact, there's nHeight and nSpendHeight there, not sure why we need to of them yet), so this case would be trivial to implement. Calculating the median time there wouldn't be difficult either: even if globals weren't so heavily-used in AcceptToMemoryPool, the prevIndex can always be passed down as parameter.

Some people may think that I'm discussing tiny details, but I would really like that we can chose whatever is more generic for any type of consensus fork and always use that from now on (instead of risking of having to use 2 of them if we find out later that the chosen option is not general enough).

It would be also nice to have only one uniform type of threshold in Consensus::Params, and height seems to be the choice for softforks that have been accepted long ago via miners' voting/upgrade_confirmation, like in : https://github.com/bitcoin/bitcoin/pull/5966/files

That doesn't mean it needs to necessarily be height: in a rebased version of #5966 we could replace consensus.nBIP34Height = 227931 with consensus.nBIP34Time = <something>. But I would really like to have a uniform threshold mechanism instead of using the 3 options depending on the fork.

I had assumed that height was the preferred option for everyone and that's why I used it in http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-June/008936.html But judging from the existing blocksize hardfork proposals (using block.nTime, the option I like less ) I was too fast there and clearly I need to reopen the discussion.

I still think using the version and timestamp fields in the block header are simplest and best.

Advantages:
  Available to SPV nodes with no change to the network protocol
  Available after headers downloaded, before full block data is available
  Once well past a fork, allows all block validation except validation
against the UTXO to happen in parallel, out-of-order, independent of any
other block.

Disadvantages:
  Not monotonically increasing

I think discussion about transactions in the memory pool are just a distraction: no matter what criteria is used (timestamp, height, median time), a blockchain re-organization could mean the validity of transactions you've accepted into the memory pool (if you're accepting transactions that switch from valid to invalid at the consensus change -- Core tries hard not to do that via IsStandard policy) must be re-evaluated.

I don't strongly care if median time or block timestamp is used, I think either will work. I don't like height, there are too many cases where the time is known but the block height isn't (see, for example, the max-outputs-in-a-transaction sanity check computation at line 190 of bitcoin-tx.cpp -- bitcoin-tx.cpp has no idea what the current block height is).

On Thu, Jul 30, 2015 at 8:16 PM, Gavin Andresen <gavinandresen@gmail•com> wrote:

> I still think using the version and timestamp fields in the block header are
> simplest and best.

To be clear, all options can use the version.

> Advantages:
>   Available to SPV nodes with no change to the network protocol
>   Available after headers downloaded, before full block data is available
>   Once well past a fork, allows all block validation except validation
> against the UTXO to happen in parallel, out-of-order, independent of any
> other block.

All advantages shared with the height option.

> Disadvantages:
>   Not monotonically increasing
>
>
> I think discussion about transactions in the memory pool are just a
> distraction: no matter what criteria is used (timestamp, height, median
> time), a blockchain re-organization could mean the validity of transactions
> you've accepted into the memory pool (if you're accepting transactions that
> switch from valid to invalid at the consensus change -- Core tries hard not
> to do that via IsStandard policy) must be re-evaluated.

I'm talking about the non-reorg case. Without reorg, you know what the next height or current median time is, but you don't know what the next block time is.

> I don't strongly care if median time or block timestamp is used, I think
> either will work. I don't like height, there are too many cases where the
> time is known but the block height isn't (see, for example, the
> max-outputs-in-a-transaction sanity check computation at line 190 of
> bitcoin-tx.cpp -- bitcoin-tx.cpp has no idea what the current block height
> is).

How does bitcoin-tx know about the next block time? It doesn't. You would need to use the current time as a proxy for the median time or the block.nTime which you don't know either. Or just keep the sanity check as it is. Note that this case is blocksize-specific: other hardforks (like my previous example, or the code proposal in BIP99) don't share that concern.

One thing I've noticed there seems to be disagreement on is whether miners' upgrade confirmation (aka voting) is necessary for uncontroversial hardforks or not. In BIP99 the advice for uncontroversial hardforks is setting a threshold (based on height, but we can change it) and then wait for 95% of the hashrate to upgrade to enforce the chain. But maybe the "voting" can happen first and then the threshold is added to the "miners' confirmation height/time". I think that may influence which of the three discussed options (height, block time and median time) is better, so maybe we should discuss that first.

On 4 August 2015 16:02:53 GMT-04:00, "Jorge Timón via bitcoin-dev" <bitcoin-dev@lists•linuxfoundation•org> wrote:

>One thing I've noticed there seems to be disagreement on is whether
>miners' upgrade confirmation (aka voting) is necessary for
>uncontroversial hardforks or not.

To be clear, without a strong supermajority of miner support the fork risks attack. Requiring 95% approval - which is actually just a 50% majority vote as the majority can squelch the minority - is an obvious minimum safety requirement.

Another option is Hearn's proposal of using centralised checkpoints to override PoW consensus; obviously that raises serious questions, including legal issues.

For forks without miner approval miners have a number of options to defeat them. For instance, they can make their own fork with a new consensus algorithm that requires miners to prove they're attacking the unwanted chain - Garzik's recent 2MB blocks proposal is a hilarious, and probably accidental, example of such a design, with the original Bitcoin protocol rules having the effect of attacking the Garzik 2MB chain.

I'm not sure how bip102 is less secure than other blocksize proposal but please let's keep defects specific to each proposal in their own threads. In any case, I understand that you agree that 95% confirmation is a good idea for uncontroversial hardforks (like in uncontroversial softforks). I'm not sure if you prefer that to happen before or after the time threshold, but I guess you're fine with doing it after the threshold since you didn't complained about that specifically (you can always clarify your preferences of course).

On Tue, Aug 4, 2015 at 11:29 PM, Peter Todd <pete@petertodd•org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>
>
> On 4 August 2015 16:02:53 GMT-04:00, "Jorge Timón via bitcoin-dev" <bitcoin-dev@lists•linuxfoundation•org> wrote:
>>One thing I've noticed there seems to be disagreement on is whether
>>miners' upgrade confirmation (aka voting) is necessary for
>>uncontroversial hardforks or not.
>
> To be clear, without a strong supermajority of miner support the fork risks attack. Requiring 95% approval - which is actually just a 50% majority vote as the majority can squelch the minority - is an obvious minimum safety requirement.
>
> Another option is Hearn's proposal of using centralised checkpoints to override PoW consensus; obviously that raises serious questions, including legal issues.
>
> For forks without miner approval miners have a number of options to defeat them. For instance, they can make their own fork with a new consensus algorithm that requires miners to prove they're attacking the unwanted chain - Garzik's recent 2MB blocks proposal is a hilarious, and probably accidental, example of such a design, with the original Bitcoin protocol rules having the effect of attacking the Garzik 2MB chain.
>
> -----BEGIN PGP SIGNATURE-----
>
> iQE9BAEBCAAnIBxQZXRlciBUb2RkIDxwZXRlQHBldGVydG9kZC5vcmc+BQJVwS7F
> AAoJEMCF8hzn9Lnc47AH/3926JLE4Rn9Fil+wvfxhfmBqIm0wtfStPDAqsQMDIbh
> kbxOw/Mai/AbqNUkYUWvoM2ZfJ/JNkA6HA977CE6huT1ozYVz8TJQmcqN/p1QXfX
> w1559UsXXop2fepY1dbnyBUwB6w6VwBrfj3awYkJsblgcdHrEsAesYeAHphAkwL/
> kxQ0b+QmttaDCSK76hNloKVcN7AczdCSw1pux2rzmsG9zkwWJrIqR/prAO1nuk9Y
> LgQUCvYkZiMmMD8kNx9ZVRG2Y951uLS6594Qy6ZoAMAdA6QxNsP4qyE7s8M2HAon
> WjdS0UqTRyJuDVqpNav6WX4jTllK/UuHRUAOmBmYaRs=
> =0cKq
> -----END PGP SIGNATURE-----
>