BIP62 and future script upgrades
9 messages, 4 to 5 Nov 2014. Started by Pieter Wuille.
Pieter Wuille (4), Peter Todd (2), Mike Hearn, Gavin Andresen, Jeff Garzik
Hi all,
one of the rules in BIP62 is the "clean stack" requirement, which makes passing more inputs to a script than necessary illegal.
Unfortunately, this rule needs an exception for P2SH scripts: the test can only be done after (and not before) the second stage evaluation. Otherwise it would reject all spends from P2SH (which rely on "superfluous" inputs to pass data to the second stage).
I submitted a Pull Request to clarify this in BIP62: https://github.com/bitcoin/bips/pull/115
However, this also leads to the interesting observation that the clean-stack rule is incompatible with future P2SH-like constructs - which would be very useful if we'd ever want to deploy a "Script 2.0". Any such upgrade would suffer from the same problem as P2SH, and require an exception in the clean-stack rule, which - once deployed - is no longer a softfork.
Luke suggested on the pull request to not apply this rule on every transaction with nVersion >= 3, which indeed solves the problem. I believe this can easily be generalized: make the (non mandatory) BIP62 rules only apply to transaction with strict nVersion==3, and not to higher ones. The higher ones are non-standard anyway, and shouldn't be used before there is a rule that applies to them anyway - which could include some or all of BIP62 if wanted at that point still.
Opinions?
This is another problem that only exists because of the desire to soft fork. If "script 2.0" is a hard fork upgrade, you no longer need weird hacks like scripts-which-are-not-scripts.
On Tue, Nov 4, 2014 at 5:38 AM, Mike Hearn <mike@plan99•net> wrote:
Quoted text (3 lines)
> This is another problem that only exists because of the desire to soft fork. > If "script 2.0" is a hard fork upgrade, you no longer need weird hacks like > scripts-which-are-not-scripts.
I agree. I also agree that the desire for softforks sometimes lead to ugly hacks. I also that they are not "nice" philosophically because they reduce the security model of former full nodes to SPV wrt. the new rules without their knowledge. I also agree that hardforks should be possible when they're useful.
But in practice, hardforks have a much larger risk which just isn't justified for everything. Especially when it's about introducing a new transaction type that won't be used before the softfork takes place anyway.
And to keep the option for doing future softforks open, I believe we need to be aware of the effects of changes like this.
On Tue, Nov 4, 2014 at 8:29 AM, Pieter Wuille <pieter.wuille@gmail•com> wrote:
Quoted text (6 lines)
> Luke suggested on the pull request to not apply this rule on every > transaction with nVersion >= 3, which indeed solves the problem. I > believe this can easily be generalized: make the (non mandatory) BIP62 > rules only apply to transaction with strict nVersion==3, and not to > higher ones. >
I agree; soft-forking is a useful way of rolling out upgrades, we shouldn't prohibit it.
On Tue, Nov 04, 2014 at 05:29:46AM -0800, Pieter Wuille wrote:
Quoted text (27 lines)
> one of the rules in BIP62 is the "clean stack" requirement, which > makes passing more inputs to a script than necessary illegal. > > Unfortunately, this rule needs an exception for P2SH scripts: the test > can only be done after (and not before) the second stage evaluation. > Otherwise it would reject all spends from P2SH (which rely on > "superfluous" inputs to pass data to the second stage). > > I submitted a Pull Request to clarify this in BIP62: > https://github.com/bitcoin/bips/pull/115 > > However, this also leads to the interesting observation that the > clean-stack rule is incompatible with future P2SH-like constructs - > which would be very useful if we'd ever want to deploy a "Script 2.0". > Any such upgrade would suffer from the same problem as P2SH, and > require an exception in the clean-stack rule, which - once deployed - > is no longer a softfork. > > Luke suggested on the pull request to not apply this rule on every > transaction with nVersion >= 3, which indeed solves the problem. I > believe this can easily be generalized: make the (non mandatory) BIP62 > rules only apply to transaction with strict nVersion==3, and not to > higher ones. The higher ones are non-standard anyway, and shouldn't be > used before there is a rule that applies to them anyway - which could > include some or all of BIP62 if wanted at that point still. > > Opinions?
I agree with Luke: make the rules only apply to transactions with a strict nVersion==3. If we want to extend that later we can do so in another soft-fork.
On another topic, I'm skeptical of the choice of nVersion==3 - we'll likely end up doing more block.nVersion increases in the future, and there's no reason to think they'll have anything to do with transactions. No sense creating a rule that'll be so quickly broken.
On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd•org> wrote:
Quoted text (4 lines)
> On another topic, I'm skeptical of the choice of nVersion==3 - we'll > likely end up doing more block.nVersion increases in the future, and > there's no reason to think they'll have anything to do with > transactions. No sense creating a rule that'll be so quickly broken.
Moderately agreed.
Earlier in BIP 62 lifetime, I had commented on ambiguity that arose from bumping tx version simply because we were bumping block version. The ambiguity was corrected, but IMO remains symptomatic of potential problems and confusion down the road.
Though I ACK'd the change, my general preference remains to disconnect TX and block version.
On Tue, Nov 4, 2014 at 11:56 AM, Jeff Garzik <jgarzik@bitpay•com> wrote:
Quoted text (15 lines)
> On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd•org> wrote: >> On another topic, I'm skeptical of the choice of nVersion==3 - we'll >> likely end up doing more block.nVersion increases in the future, and >> there's no reason to think they'll have anything to do with >> transactions. No sense creating a rule that'll be so quickly broken. > > Moderately agreed. > > Earlier in BIP 62 lifetime, I had commented on ambiguity that arose > from bumping tx version simply because we were bumping block version. > The ambiguity was corrected, but IMO remains symptomatic of potential > problems and confusion down the road. > > Though I ACK'd the change, my general preference remains to disconnect > TX and block version.
I prefer to see consensus rules as one set of rules (especially because they only really apply to blocks - the part for lone transactions is just policy), and thus have a single numbering. Still, I have no strong opinion about it and have now heard 3 'moderately against' comments. I'm fine with using nVersion==2 for transactions.
On Tue, Nov 04, 2014 at 12:00:43PM -0800, Pieter Wuille wrote:
Quoted text (22 lines)
> On Tue, Nov 4, 2014 at 11:56 AM, Jeff Garzik <jgarzik@bitpay•com> wrote: > > On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd•org> wrote: > >> On another topic, I'm skeptical of the choice of nVersion==3 - we'll > >> likely end up doing more block.nVersion increases in the future, and > >> there's no reason to think they'll have anything to do with > >> transactions. No sense creating a rule that'll be so quickly broken. > > > > Moderately agreed. > > > > Earlier in BIP 62 lifetime, I had commented on ambiguity that arose > > from bumping tx version simply because we were bumping block version. > > The ambiguity was corrected, but IMO remains symptomatic of potential > > problems and confusion down the road. > > > > Though I ACK'd the change, my general preference remains to disconnect > > TX and block version. > > I prefer to see consensus rules as one set of rules (especially > because they only really apply to blocks - the part for lone > transactions is just policy), and thus have a single numbering. Still, > I have no strong opinion about it and have now heard 3 'moderately > against' comments. I'm fine with using nVersion==2 for transactions.
Keep in mind that we may even have a circumstance where we need to introduce *two* different new tx version numbers in a single soft-fork, say because we find an exploit that has two different fixes, each of which breaks something.
I don't think we have any certainty how new features will be added in the future - just look at how we only recently realised new opcodes won't be associated with tx version number bumps - so I'm loath to setup expectations.
Besides, transactions can certainly be verified for correctness in a stand-alone fashion outside a block; CHECKLOCKTIMEVERIFY was specifically designed so that verifying scripts containing it could be done in a self-contained manner only referencing the transaction the script was within.
Ok, addressed these (and a few other things) in https://github.com/bitcoin/bips/pull/117: * Better names for the rules. * Clarify interaction of BIP62 with P2SH. * Clarify that known hashtypes are required, despite not being part of DER. * Use v2 transactions instead of v3 transactions. * Apply the optional rules only to strict v2, and not higher or lower.
On Tue, Nov 4, 2014 at 12:07 PM, Peter Todd <pete@petertodd•org> wrote:
Quoted text (43 lines)
> On Tue, Nov 04, 2014 at 12:00:43PM -0800, Pieter Wuille wrote: >> On Tue, Nov 4, 2014 at 11:56 AM, Jeff Garzik <jgarzik@bitpay•com> wrote: >> > On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd•org> wrote: >> >> On another topic, I'm skeptical of the choice of nVersion==3 - we'll >> >> likely end up doing more block.nVersion increases in the future, and >> >> there's no reason to think they'll have anything to do with >> >> transactions. No sense creating a rule that'll be so quickly broken. >> > >> > Moderately agreed. >> > >> > Earlier in BIP 62 lifetime, I had commented on ambiguity that arose >> > from bumping tx version simply because we were bumping block version. >> > The ambiguity was corrected, but IMO remains symptomatic of potential >> > problems and confusion down the road. >> > >> > Though I ACK'd the change, my general preference remains to disconnect >> > TX and block version. >> >> I prefer to see consensus rules as one set of rules (especially >> because they only really apply to blocks - the part for lone >> transactions is just policy), and thus have a single numbering. Still, >> I have no strong opinion about it and have now heard 3 'moderately >> against' comments. I'm fine with using nVersion==2 for transactions. > > Keep in mind that we may even have a circumstance where we need to > introduce *two* different new tx version numbers in a single soft-fork, > say because we find an exploit that has two different fixes, each of > which breaks something. > > I don't think we have any certainty how new features will be added in > the future - just look at how we only recently realised new opcodes > won't be associated with tx version number bumps - so I'm loath to setup > expectations. > > Besides, transactions can certainly be verified for correctness in a > stand-alone fashion outside a block; CHECKLOCKTIMEVERIFY was > specifically designed so that verifying scripts containing it could be > done in a self-contained manner only referencing the transaction the > script was within. > > -- > 'peter'[:-1]@petertodd.org > 0000000000000000036655c955dd94ba7f9856814f3cb87f003e311566921807