Compatibility requirements for hard or soft forks
13 messages, 28 Oct to 3 Nov 2015. Started by Gavin Andresen.
Justus Ranvier (5), Gavin Andresen (2), Johnson Lau (2), Tier Nolan (2), Rusty Russell, Luke Dashjr
I'm hoping this fits under the moderation rule of "short-term changes to the Bitcoin protcol" (I'm not exactly clear on what is meant by "short-term"; it would be lovely if the moderators would start a thread on bitcoin-discuss to clarify that):
Should it be a requirement that ANY one-megabyte transaction that is valid under the existing rules also be valid under new rules?
Pro: There could be expensive-to-validate transactions created and given a lockTime in the future stored somewhere safe. Their owners may have no other way of spending the funds (they might have thrown away the private keys), and changing validation rules to be more strict so that those transactions are invalid would be an unacceptable confiscation of funds.
Con: It is extremely unlikely there are any such large, timelocked transactions, because the Core code has had a clear policy for years that 100,000-byte transactions are "standard" and are relayed and mined, and larger transactions are not. The requirement should be relaxed so that only valid 100,000-byte transaction under old consensus rules must be valid under new consensus rules (larger transactions may or may not be valid).
I had to wrestle with that question when I implemented BIP101/Bitcoin XT when deciding on a limit for signature hashing (and decided the right answer was to support any "non-attack"1MB transaction; see https://bitcoincore.org/~gavin/ValidationSanity.pdf for more details).
Gavin Andresen via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> writes:
Quoted text (8 lines)
> Should it be a requirement that ANY one-megabyte transaction that is valid > under the existing rules also be valid under new rules? > > Pro: There could be expensive-to-validate transactions created and given a > lockTime in the future stored somewhere safe. Their owners may have no > other way of spending the funds (they might have thrown away the private > keys), and changing validation rules to be more strict so that those > transactions are invalid would be an unacceptable confiscation of funds.
Not just lockTime; potentially any tx locked away in a safe.
Consider low-S enforcement: high chance a non-expert user will be unable to spend an old transaction. They need to compromise their privacy and/or spend time and money. A milder "confiscation" but a more likely one.
By that benchmark, we should aim for "reasonable certainty". A transaction which would never have been generated by any known software is the minimum bar. Adding "...which would have to be deliberately stupid with many redundant OP_CHECKSIG etc" surpasses it. The only extra safeguard I can think of is clear, widespread notification of the change.
Cheers, Rusty.
On 10/30/2015 10:43 PM, Rusty Russell via bitcoin-dev wrote:
Quoted text (6 lines)
> By that benchmark, we should aim for "reasonable certainty". A > transaction which would never have been generated by any known software > is the minimum bar. Adding "...which would have to be deliberately > stupid with many redundant OP_CHECKSIG etc" surpasses it. The only extra > safeguard I can think of is clear, widespread notification of the > change.
If the policy of Bitcoin Core development includes a willingness to makes the utxos created by software other than Bitcoin Core unspendable, then it certainly merits clear, widespread notification.
Even if that is actually a good policy, the reasons why should be made abundantly clear.
My answer is simply "No", you don't have to maintain backward compatibility for non-standard tx.
The same question applies to P2SH. Before the deployment of BIP16, one could have created a time-locked tx with one of the output was in the form of HASH160 <hash> EQUAL. The <hash>, however, is not a hash of a valid serialized script, so the output is now permanently frozen.
It also applies to all the OP codes disabled by Satoshi: one could have created a time-locked tx with those now disabled OP codes.
Same for BIP65 with the use of OP_NOP2. Following your logic, we can't make any softfork related to the script system.
I think it is very important to make it clear that non-standard txs and non-standard scripts may become invalid in the future
Gavin Andresen via bitcoin-dev 於 2015-10-28 10:06 寫到:
Quoted text (50 lines)
> I'm hoping this fits under the moderation rule of "short-term changes > to the Bitcoin protcol" (I'm not exactly clear on what is meant by > "short-term"; it would be lovely if the moderators would start a > thread on bitcoin-discuss to clarify that): > > Should it be a requirement that ANY one-megabyte transaction that is > valid > under the existing rules also be valid under new rules? > > Pro: There could be expensive-to-validate transactions created and > given a > lockTime in the future stored somewhere safe. Their owners may have no > other way of spending the funds (they might have thrown away the > private > keys), and changing validation rules to be more strict so that those > transactions are invalid would be an unacceptable confiscation of > funds. > > Con: It is extremely unlikely there are any such large, timelocked > transactions, because the Core code has had a clear policy for years > that > 100,000-byte transactions are "standard" and are relayed and > mined, and > larger transactions are not. The requirement should be relaxed so that > only > valid 100,000-byte transaction under old consensus rules must be valid > under new consensus rules (larger transactions may or may not be > valid). > > I had to wrestle with that question when I implemented BIP101/Bitcoin > XT > when deciding on a limit for signature hashing (and decided the right > answer was to support any "non-attack"1MB transaction; see > https://bitcoincore.org/~gavin/ValidationSanity.pdf [1] for more > details). > > -- > > -- > Gavin Andresen > > > Links: > ------ > [1] https://bitcoincore.org/~gavin/ValidationSanity.pdf > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
On Sun, Nov 1, 2015 at 5:28 PM, jl2012 via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (3 lines)
> I think it is very important to make it clear that non-standard txs and > non-standard scripts may become invalid in the future >
There can be unavoidable situations which cause locked coins become unspendable.
In an ideal world, soft forks that make UTXOs unspendable should increase the tx version number. BIP-13 should have done that. That would make the change opt-in.
The disabled opcodes like OP_CAT were a DOS/network security change.
Invalidating locked coins is another reason that they shouldn't have been disabled permanently.
It would have been better to disable them for six months, so at least people can get their coins back after that. Inherently, protecting the network required some limitations being added so that nodes couldn't be crashed.
For guidelines
* Transaction version numbers will be increased, if possible * Transactions with unknown/large version numbers are unsafe to use with locktime * Reasonable notice is given that the change is being contemplated * Non-opt-in changes will only be to protect the integrity of the network
Locked transaction that can be validated without excessive load on the network should be safe to use, even if non-standard.
An OP_CAT script that requires TBs of RAM to validate crosses the threshold of reasonableness.
Quoted text (60 lines)
> > Gavin Andresen via bitcoin-dev 於 2015-10-28 10:06 寫到: > >> I'm hoping this fits under the moderation rule of "short-term changes >> to the Bitcoin protcol" (I'm not exactly clear on what is meant by >> "short-term"; it would be lovely if the moderators would start a >> thread on bitcoin-discuss to clarify that): >> >> Should it be a requirement that ANY one-megabyte transaction that is >> valid >> under the existing rules also be valid under new rules? >> >> Pro: There could be expensive-to-validate transactions created and >> given a >> lockTime in the future stored somewhere safe. Their owners may have no >> other way of spending the funds (they might have thrown away the >> private >> keys), and changing validation rules to be more strict so that those >> transactions are invalid would be an unacceptable confiscation of >> funds. >> >> Con: It is extremely unlikely there are any such large, timelocked >> transactions, because the Core code has had a clear policy for years >> that >> 100,000-byte transactions are "standard" and are relayed and >> mined, and >> larger transactions are not. The requirement should be relaxed so that >> only >> valid 100,000-byte transaction under old consensus rules must be valid >> under new consensus rules (larger transactions may or may not be >> valid). >> >> I had to wrestle with that question when I implemented BIP101/Bitcoin >> XT >> when deciding on a limit for signature hashing (and decided the right >> answer was to support any "non-attack"1MB transaction; see >> https://bitcoincore.org/~gavin/ValidationSanity.pdf [1] for more >> details). >> >> -- >> >> -- >> Gavin Andresen >> >> >> Links: >> ------ >> [1] https://bitcoincore.org/~gavin/ValidationSanity.pdf >> >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >> > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On 11/01/2015 05:46 PM, Tier Nolan via bitcoin-dev wrote:
Quoted text (2 lines)
> An OP_CAT script that requires TBs of RAM to validate crosses the > threshold of reasonableness.
Are there actually any OP_CAT scripts currently in the utxo set?
It's one thing to have a theoretical scripting ability that gets removed before anyone actually uses it - that doesn't put anyone in the position of retroactively making decisions about the validity of someone else's money.
Presently the utxo set is about $5 billion worth of other people's money.
It's a lot easier to justify the position: "nobody has the right to change the meaning of someone else's outputs", than it is to justify, "some small group of people gets to decide what's standard and what isn't, and if you choose to use the network in a valid but nonstandard way, that group of people might choose to deny you access to your money in the future"
In other words, how close to the shores of "administrators of a virtual currency" do Bitcoin developers want to sail?
On Monday, November 02, 2015 12:23:27 AM Justus Ranvier via bitcoin-dev wrote:
Quoted text (6 lines)
> It's a lot easier to justify the position: "nobody has the right to > change the meaning of someone else's outputs", than it is to justify, > "some small group of people gets to decide what's standard and what > isn't, and if you choose to use the network in a valid but nonstandard > way, that group of people might choose to deny you access to your money > in the future"
The reality is presently "some small group of people gets to decide how and if you can access your money"... and it's getting worse.
Quoted text (2 lines)
> In other words, how close to the shores of "administrators of a virtual > currency" do Bitcoin developers want to sail?
Bitcoin developers don't make this decision, miners do.
Luke
On Mon, Nov 2, 2015 at 12:23 AM, Justus Ranvier via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (2 lines)
> Are there actually any OP_CAT scripts currently in the utxo set? >
A locked transaction could pay to an OP_CAT script with the private key being lost.
Even if it is only in theory, it is still worth trying to prevent rule changes which permanently prevent outputs being spendable.
Quoted text (7 lines)
> It's a lot easier to justify the position: "nobody has the right to > change the meaning of someone else's outputs", than it is to justify, > "some small group of people gets to decide what's standard and what > isn't, and if you choose to use the network in a valid but nonstandard > way, that group of people might choose to deny you access to your money > in the future" >
If at least one year's notice was given, then people aren't going to lose their money, since they have notice.
Locked transactions could have a difference expectation than non-locked ones.
Quoted text (3 lines)
> In other words, how close to the shores of "administrators of a virtual > currency" do Bitcoin developers want to sail? >
Miners can collectively vote to disable specific UTXOs and change the acceptance rules.
Quoted text (8 lines)
> > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > >
I guess by "locked transaction" you must mean a P2SH output?
If so, that's a rather bizarre use of terms since outputs and transactions are very different things.
On 11/01/2015 07:30 PM, Tier Nolan via bitcoin-dev wrote:
Quoted text (2 lines)
> If at least one year's notice was given, then people aren't going to > lose their money, since they have notice.
So after realizing that I misread substantial portions of this thread due to a lack of attention to detail I'd like to point out this:
Bitcoin nodes have the capability to validate blocks going back to the genesis block, including blocks which would not be valid if mined today under current rules.
Therefore it must be the case that all the old consensus rules are preserved somewhere in the current code bases of the various implementations.
Given that, there shouldn't be any technical barrier to validating input scripts according to the consensus rules that were in effect at the time the input being spent was added to the blockchain.
Maybe dealing with output is more difficult.
Had every consensus rule change (deliberate and accidental) been accompanied by a version number bump, it would have been possible to phase out old versions without invaliding signed-but-unbroadcast transactions by saying "as of block height x, transactions with version y or lower are invalid unless their inputs are exclusively sourced from blocks with heights < x"
If there already have been rule changes which have retroactively invalided unbroadcast transactions which were valid at the time they were signed, those rules could be relaxed to not apply to transactions which exclusively spend inputs that existed before the rule change.
On Sun, Nov 1, 2015 at 6:46 PM, Tier Nolan via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (14 lines)
> For guidelines > > * Transaction version numbers will be increased, if possible > * Transactions with unknown/large version numbers are unsafe to use with > locktime > * Reasonable notice is given that the change is being contemplated > * Non-opt-in changes will only be to protect the integrity of the network > > Locked transaction that can be validated without excessive load on the > network should be safe to use, even if non-standard. > > An OP_CAT script that requires TBs of RAM to validate crosses the > threshold of reasonableness. >
I like those guidelines, although I'm sure there may be lots of arguing over what fits under "protects the integrity of the network" or what constitutes "reasonable notice" (publish a BIP at least 30 days before rolling out a change? 60 days? a year?)
On 02/11/15 14:33, Gavin Andresen via bitcoin-dev wrote:
Quoted text (4 lines)
> I like those guidelines, although I'm sure there may be lots of arguing > over what fits under "protects the integrity of the network" or what > constitutes "reasonable notice" (publish a BIP at least 30 days before > rolling out a change? 60 days? a year?)
If Bitcoin were perfect. then it would be the case that any transaction that was valid at the time it was signed would always remain valid until spent regardless of any protocol changes which occurred in the interim.
Certainly, that property, if it was possible to achieve, would give Bitcoin maximum possible utility compared to alternative properties.
There are cases in which that guarantee can be met, and there are some pathological cases where it can not be met.
It's not possible to know if the pathological cases are actually a real problem in practice, because the possible existence of unbroadcast transactions which would trigger them is unknowable.
A possible lazy/optimistic strategy is to implement as much non-pathological backward compatibility as possible, and treat unhandled cases as outstanding bugs whose resolution is deferred unless and until they are actually triggered.
The other strategy is to have an informational BIP to define "safe" use of Bitcoin.
1. scriptPubKey must be one of the following types: P2PK, P2PKH, P2SH, n-of-m multisig with m < 4 (with or without CLTV or CSV, we should define standard use of CLTV and CSV)
2. For P2SH, the serialized script must be one of the standard type
3. No use of unknown transaction version
4. Tx size < 100k
5. If conditions 1-4 are all satisfied, the locktime must not be longer than 4 years from the creation of the tx
6. If at least one of the conditions 1-4 is not satisfied, the lock time must not be longer than 6 months from the creation of the tx
7. A chain of unconfirmed transactions is unsafe due the malleability
8. Tx created by wallet software last updated 1 year ago is unsafe
9. Permanently deleting a private key is unsafe (that might be safe if stricter practice is followed)
We must not introduce a new rule that may permanently invalidate a safe tx, unless in emergency. Even in emergency, we should try to preserve backward compatibility as much as possible (see the example at the end of my message)
Being unsafe means there is a chance that the tx may become unconfirmable, outputs become unspendable, or funds may be stolen without a private key.
A grace period of 5 years must be given for "soft-fork" type change of any of the rules. For example it is ok to introduce new standard script anytime but not to remove.
----------------------
Back to Gavin's original question. If you want to somehow keep backward compatibility for expensive-to-validate transactions in the future, you may have rules like there could only be at most one expensive-to-validate transaction in every 10 blocks, until year 2025. I know this is over-complicated but it's a possible way to address your concern.
Gavin Andresen via bitcoin-dev 於 2015-11-02 15:33 寫到:
Quoted text (36 lines)
> On Sun, Nov 1, 2015 at 6:46 PM, Tier Nolan via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation•org> wrote: > >> For guidelines >> >> * Transaction version numbers will be increased, if possible >> >> * Transactions with unknown/large version numbers are unsafe to use >> with locktime >> >> * Reasonable notice is given that the change is being contemplated >> >> * Non-opt-in changes will only be to protect the integrity of the >> network >> >> Locked transaction that can be validated without excessive load on >> the network should be safe to use, even if non-standard. >> >> An OP_CAT script that requires TBs of RAM to validate crosses the >> threshold of reasonableness. > > I like those guidelines, although I'm sure there may be lots of > arguing over what fits under "protects the integrity of the network" > or what constitutes "reasonable notice" (publish a BIP at least 30 > days before rolling out a change? 60 days? a year?) > > -- > > -- > Gavin Andresen > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev