Thoughts on fee bumping

> There's been much talk about fee-bumping lately, and for good reason -
> dynamic fee management is going to be a central part of bitcoin use as
> the mempool fills up (lord willing) and right now fee-bumping is
> fraught with difficulty and pinning peril.
>
> Gloria's recent post on the topic[0] was very lucid and highlights a
> lot of the current issues, as well as some proposals to improve the
> situation.
>
> As others have noted, the post was great. But throughout the course
> of reading it and the ensuing discussion, I became troubled by the
> increasing complexity of both the status quo and some of the
> proposed remedies.
>
> Layering on special cases, more carve-outs, and X and Y percentage
> thresholds is going to make reasoning about the mempool harder than it
> already is. Special consideration for "what should be in the next
> block" and/or the caching of block templates seems like an imposing
> dependency, dragging in a bunch of state and infrastructure to a
> question that should be solely limited to mempool feerate aggregates
> and the feerate of the particular txn package a wallet is concerned
> with.
>
> This is bad enough for protocol designers and Core developers, but
> making the situation any more intractable for "end-users" and wallet
> developers feels wrong.
>
> I thought it might be useful to step back and reframe. Here are a few
> aims that are motivated chiefly by the quality of end-user experience,
> constrained to obey incentive compatibility (i.e. miner reward, DoS
> avoidance). Forgive the abstract dalliance for a moment; I'll talk
> through concretes afterwards.
>
>
> # Purely additive feerate bumps should never be impossible
>
> Any user should always be able to add to the incentive to mine any
> transaction in a purely additive way. The countervailing force here
> ends up being spam prevention (a la min-relay-fee) to prevent someone
> from consuming bandwidth and mempool space with a long series of
> infinitesimal fee-bumps.
>
> A fee bump, naturally, should be given the same per-byte consideration
> as a normal Bitcoin transaction in terms of relay and block space,
> although it would be nice to come up with a more succinct
> representation. This leads to another design principle:
>
>
> # The bandwidth and chain space consumed by a fee-bump should be minimal
>
> Instead of prompting a rebroadcast of the original transaction for
> replacement, which contains a lot of data not new to the network, it
> makes more sense to broadcast the "diff" which is the additive
> contribution towards some txn's feerate.
>
> This dovetails with the idea that...
>
>
> # Special transaction structure should not be required to bump fees
>
> In an ideal design, special structural foresight would not be needed
> in order for a txn's feerate to be improved after broadcast.
>
> Anchor outputs specified solely for CPFP, which amount to many bytes of
> wasted chainspace, are a hack. It's probably uncontroversial at this
> point to say that even RBF itself is kind of a hack - a special
> sequence number should not be necessary for post-broadcast contribution
> toward feerate. Not to mention RBF's seemingly wasteful consumption of
> bandwidth due to the rebroadcast of data the network has already seen.
>
> In a sane design, no structural foresight - and certainly no wasted
> bytes in the form of unused anchor outputs - should be needed in order
> to add to a miner's reward for confirming a given transaction.
>
> Planning for fee-bumps explicitly in transaction structure also often
> winds up locking in which keys are required to bump fees, at odds
> with the idea that...
>
>
> # Feerate bumps should be able to come from anywhere
>
> One of the practical downsides of CPFP that I haven't seen discussed in
> this conversation is that it requires the transaction to pre-specify the
> keys needed to sign for fee bumps. This is problematic if you're, for
> example, using a vault structure that makes use of pre-signed
> transactions.
>
> What if the key you specified n the anchor outputs for a bunch of
> pre-signed txns is compromised? What if you'd like to be able to
> dynamically select the wallet that bumps fees? CPFP does you no favors
> here.
>
> There is of course a tension between allowing fee bumps to come from
> anywhere and the threat of pinning-like attacks. So we should venture
> to remove pinning as a possibility, in line with the first design
> principle I discuss.
>
>
> ---
>
> Coming down to earth, the "tabula rasa" thought experiment above has led
> me to favor an approach like the transaction sponsors design that Jeremy
> proposed in a prior discussion back in 2020[1].
>
> Transaction sponsors allow feerates to be bumped after a transaction's
> broadcast, regardless of the structure of the original transaction.
> No rebroadcast (wasted bandwidth) is required for the original txn data.
> No wasted chainspace on only-maybe-used prophylactic anchor outputs.
>
> The interface for end-users is very straightforward: if you want to bump
> fees, specify a transaction that contributes incrementally to package
> feerate for some txid. Simple.
>
> In the original discussion, there were a few main objections that I noted:
>
> 1. In Jeremy's original proposal, only one sponsor txn per txid is
>    allowed by policy. A malicious actor could execute a pinning-like
>    attack by specifying an only-slightly-helpful feerate sponsor that
>    then precludes other larger bumps.
>
> I think there are some ways around this shortcoming. For example: what
> if, by policy, sponsor txns had additional constraints that
>
>   - each input must be signed {SIGHASH_SINGLE,SIGHASH_NONE}|ANYONECANPAY,
>   - the txn must be specified RBFable,
>   - a replacement for the sponsor txn must raise the sponsor feerate,
>     including ancestors (maybe this is inherent in "is RBFable," but
>     I don't want to conflate absolute feerates into this).
>
> That way, there is still at most a single sponsor txn per txid in the
> mempool, but anyone can "mix in" inputs which bump the effective
> feerate of the sponsor.
>
> This may not be the exact solution we want, but I think it demonstrates
> that the sponsors design has some flexibility and merits some thinking.
>
> The second objection about sponsors was
>
> 2. (from Suhas) sponsors break the classic invariant: "once a valid
>    transaction is created, it should not become invalid later on unless
>    the inputs are double-spent."
>
> This doesn't seem like a huge concern to me if you consider the txid
> being sponsored as a sort of spiritual input to the sponsor. While the
> theoretical objection against broadening where one has to look in a txn
> to determine its dependencies is understandable, I don't see what the
> practical cost here is.
>
> Reorg complexity seems comparable if not identical, especially if we
> broaden sponsor rules to allow blocks to contain sponsor txns that are
> both for txids in the same block _or_ already included in the chain.
>
> This theoretical concession seems preferable to heaping more rules onto
> an already labyrinthine mempool policy that is difficult for both
> implementers and users to reason about practically and conceptually.
>
> A third objection that wasn't posed, IIRC, but almost certainly would
> be:
>
> 3. Transaction sponsors requires a soft-fork.
>
> Soft-forks are no fun, but I'll tell you what also isn't fun: being on
> the hook to model (and sometimes implement) a dizzying potpourri of
> mempool policies and special-cases. Expecting wallet implementers to
> abide by a maze of rules faithfully in order to ensure txn broadcast and
> fee management invites bugs for perpetuity and network behavior that is
> difficult to reason about a priori. Use of CPFP in the long-term also
> risks needless chain waste.
>
> If a soft-fork is the cost of cleaning up this essential process,
> consideration should be given to paying it as a one-time cost. This
> topic merits a separate post, but consider that in the 5 years leading
> up to the 2017 SegWit drama, we averaged about a soft-fork a year.
> Uncontroversial, "safe" changes to the consensus protocol shouldn't be
> out of the question when significant practical benefit is plain to see.
>
> ---
>
> I hope this message has added some framing to the discussion on fees,
> as well prompting other participants to go back and give the
> transaction sponsor proposal a serious look. The sponsors interface is
> about the simplest I can imagine for wallets, and it seems easy to
> reason about for implementers on Core and elsewhere.
>
> I'm not out to propose soft-forks lightly, but the current complexity
> in fee management feels untenable, and as evidenced by all the
> discussion lately, fees are an increasingly crucial part of the system.
>
>
>
> [0]:
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
> [1]:
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-September/018168.html
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

> # Purely additive feerate bumps should never be impossible

> There's been much talk about fee-bumping lately, and for good reason -
> dynamic fee management is going to be a central part of bitcoin use as
> the mempool fills up (lord willing) and right now fee-bumping is
> fraught with difficulty and pinning peril.
>
> Gloria's recent post on the topic[0] was very lucid and highlights a
> lot of the current issues, as well as some proposals to improve the
> situation.
>
> As others have noted, the post was great. But throughout the course
> of reading it and the ensuing discussion, I became troubled by the
> increasing complexity of both the status quo and some of the
> proposed remedies.
>
> Layering on special cases, more carve-outs, and X and Y percentage
> thresholds is going to make reasoning about the mempool harder than it
> already is. Special consideration for "what should be in the next
> block" and/or the caching of block templates seems like an imposing
> dependency, dragging in a bunch of state and infrastructure to a
> question that should be solely limited to mempool feerate aggregates
> and the feerate of the particular txn package a wallet is concerned
> with.
>
> This is bad enough for protocol designers and Core developers, but
> making the situation any more intractable for "end-users" and wallet
> developers feels wrong.
>
> I thought it might be useful to step back and reframe. Here are a few
> aims that are motivated chiefly by the quality of end-user experience,
> constrained to obey incentive compatibility (i.e. miner reward, DoS
> avoidance). Forgive the abstract dalliance for a moment; I'll talk
> through concretes afterwards.
>
> # Purely additive feerate bumps should never be impossible
>
> Any user should always be able to add to the incentive to mine any
> transaction in a purely additive way. The countervailing force here
> ends up being spam prevention (a la min-relay-fee) to prevent someone
> from consuming bandwidth and mempool space with a long series of
> infinitesimal fee-bumps.
>
> A fee bump, naturally, should be given the same per-byte consideration
> as a normal Bitcoin transaction in terms of relay and block space,
> although it would be nice to come up with a more succinct
> representation. This leads to another design principle:
>
> # The bandwidth and chain space consumed by a fee-bump should be minimal
>
> Instead of prompting a rebroadcast of the original transaction for
> replacement, which contains a lot of data not new to the network, it
> makes more sense to broadcast the "diff" which is the additive
> contribution towards some txn's feerate.
>
> This dovetails with the idea that...
>
> # Special transaction structure should not be required to bump fees
>
> In an ideal design, special structural foresight would not be needed
> in order for a txn's feerate to be improved after broadcast.
>
> Anchor outputs specified solely for CPFP, which amount to many bytes of
> wasted chainspace, are a hack. It's probably uncontroversial at this
> point to say that even RBF itself is kind of a hack - a special
> sequence number should not be necessary for post-broadcast contribution
> toward feerate. Not to mention RBF's seemingly wasteful consumption of
> bandwidth due to the rebroadcast of data the network has already seen.
>
> In a sane design, no structural foresight - and certainly no wasted
> bytes in the form of unused anchor outputs - should be needed in order
> to add to a miner's reward for confirming a given transaction.
>
> Planning for fee-bumps explicitly in transaction structure also often
> winds up locking in which keys are required to bump fees, at odds
> with the idea that...
>
> # Feerate bumps should be able to come from anywhere
>
> One of the practical downsides of CPFP that I haven't seen discussed in
> this conversation is that it requires the transaction to pre-specify the
> keys needed to sign for fee bumps. This is problematic if you're, for
> example, using a vault structure that makes use of pre-signed
> transactions.
>
> What if the key you specified n the anchor outputs for a bunch of
> pre-signed txns is compromised? What if you'd like to be able to
> dynamically select the wallet that bumps fees? CPFP does you no favors
> here.
>
> There is of course a tension between allowing fee bumps to come from
> anywhere and the threat of pinning-like attacks. So we should venture
> to remove pinning as a possibility, in line with the first design
> principle I discuss.
>
> ---
>
> Coming down to earth, the "tabula rasa" thought experiment above has led
> me to favor an approach like the transaction sponsors design that Jeremy
> proposed in a prior discussion back in 2020[1].
>
> Transaction sponsors allow feerates to be bumped after a transaction's
> broadcast, regardless of the structure of the original transaction.
> No rebroadcast (wasted bandwidth) is required for the original txn data.
> No wasted chainspace on only-maybe-used prophylactic anchor outputs.
>
> The interface for end-users is very straightforward: if you want to bump
> fees, specify a transaction that contributes incrementally to package
> feerate for some txid. Simple.
>
> In the original discussion, there were a few main objections that I noted:
>
> 1. In Jeremy's original proposal, only one sponsor txn per txid is
> allowed by policy. A malicious actor could execute a pinning-like
> attack by specifying an only-slightly-helpful feerate sponsor that
> then precludes other larger bumps.
>
> I think there are some ways around this shortcoming. For example: what
> if, by policy, sponsor txns had additional constraints that
>
> - each input must be signed {SIGHASH_SINGLE,SIGHASH_NONE}|ANYONECANPAY,
> - the txn must be specified RBFable,
> - a replacement for the sponsor txn must raise the sponsor feerate,
> including ancestors (maybe this is inherent in "is RBFable," but
> I don't want to conflate absolute feerates into this).
>
> That way, there is still at most a single sponsor txn per txid in the
> mempool, but anyone can "mix in" inputs which bump the effective
> feerate of the sponsor.
>
> This may not be the exact solution we want, but I think it demonstrates
> that the sponsors design has some flexibility and merits some thinking.
>
> The second objection about sponsors was
>
> 2. (from Suhas) sponsors break the classic invariant: "once a valid
> transaction is created, it should not become invalid later on unless
> the inputs are double-spent."
>
> This doesn't seem like a huge concern to me if you consider the txid
> being sponsored as a sort of spiritual input to the sponsor. While the
> theoretical objection against broadening where one has to look in a txn
> to determine its dependencies is understandable, I don't see what the
> practical cost here is.
>
> Reorg complexity seems comparable if not identical, especially if we
> broaden sponsor rules to allow blocks to contain sponsor txns that are
> both for txids in the same block _or_ already included in the chain.
>
> This theoretical concession seems preferable to heaping more rules onto
> an already labyrinthine mempool policy that is difficult for both
> implementers and users to reason about practically and conceptually.
>
> A third objection that wasn't posed, IIRC, but almost certainly would
> be:
>
> 3. Transaction sponsors requires a soft-fork.
>
> Soft-forks are no fun, but I'll tell you what also isn't fun: being on
> the hook to model (and sometimes implement) a dizzying potpourri of
> mempool policies and special-cases. Expecting wallet implementers to
> abide by a maze of rules faithfully in order to ensure txn broadcast and
> fee management invites bugs for perpetuity and network behavior that is
> difficult to reason about a priori. Use of CPFP in the long-term also
> risks needless chain waste.
>
> If a soft-fork is the cost of cleaning up this essential process,
> consideration should be given to paying it as a one-time cost. This
> topic merits a separate post, but consider that in the 5 years leading
> up to the 2017 SegWit drama, we averaged about a soft-fork a year.
> Uncontroversial, "safe" changes to the consensus protocol shouldn't be
> out of the question when significant practical benefit is plain to see.
>
> ---
>
> I hope this message has added some framing to the discussion on fees,
> as well prompting other participants to go back and give the
> transaction sponsor proposal a serious look. The sponsors interface is
> about the simplest I can imagine for wallets, and it seems easy to
> reason about for implementers on Core and elsewhere.
>
> I'm not out to propose soft-forks lightly, but the current complexity
> in fee management feels untenable, and as evidenced by all the
> discussion lately, fees are an increasingly crucial part of the system.
>
> [0]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
> [1]: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-September/018168.html

> It's not that simple. As a miner, if i have less than 1vMB of

> # Purely additive feerate bumps should never be impossible
>
> Any user should always be able to add to the incentive to mine any
> transaction in a purely additive way. The countervailing force here
> ends up being spam prevention (a la min-relay-fee) to prevent someone
> from consuming bandwidth and mempool space with a long series of
> infinitesimal fee-bumps.
>
> A fee bump, naturally, should be given the same per-byte consideration
> as a normal Bitcoin transaction in terms of relay and block space,
> although it would be nice to come up with a more succinct
> representation. This leads to another design principle:

 > Special consideration for "what should be in the next
 > block" and/or the caching of block templates seems like an imposing
 > dependency

 > # The bandwidth and chain space consumed by a fee-bump should be minimal
 >
 > Instead of prompting a rebroadcast of the original transaction for
 > replacement, which contains a lot of data not new to the network, it
 > makes more sense to broadcast the "diff" which is the additive
 > contribution towards some txn's feerate.

> # Special transaction structure should not be required to bump fees
>
> In an ideal design, special structural foresight would not be needed
> in order for a txn's feerate to be improved after broadcast.
>
> Anchor outputs specified solely for CPFP, which amount to many bytes of
> wasted chainspace, are a hack. > It's probably uncontroversial at this

> Coming down to earth, the "tabula rasa" thought experiment above has led
> me to favor an approach like the transaction sponsors design that Jeremy
> proposed in a prior discussion back in 2020[1].

> Layering on special cases, more carve-outs, and X and Y percentage
> thresholds is going to make reasoning about the mempool harder than it
> already is.

> The countervailing force here ends up being spam prevention (a la

> to prevent someone from consuming bandwidth and mempool space with a long

> infinitesimal fee-bumps.

> Instead of prompting a rebroadcast of the original transaction for
> replacement, which contains a lot of data not new to the network, it
> makes more sense to broadcast the "diff" which is the additive
> contribution towards some txn's feerate.

> It's probably uncontroversial at this
> point to say that even RBF itself is kind of a hack - a special
> sequence number should not be necessary for post-broadcast contribution
> toward feerate.

> In a sane design, no structural foresight - and certainly no wasted
>bytes in the form of unused anchor outputs - should be needed in order
>to add to a miner's reward for confirming a given transaction.

> One of the practical downsides of CPFP that I haven't seen discussed in
> this conversation is that it requires the transaction to pre-specify the
> keys needed to sign for fee bumps. This is problematic if you're, for
> example, using a vault structure that makes use of pre-signed
> transactions.

> The interface for end-users is very straightforward: if you want to bump
> fees, specify a transaction that contributes incrementally to package
> feerate for some txid. Simple.

> This theoretical concession seems preferable to heaping more rules onto

> If a soft-fork is the cost of cleaning up this essential process,
> consideration should be given to paying it as a one-time cost. This
> topic merits a separate post, but consider that in the 5 years leading
> up to the 2017 SegWit drama, we averaged about a soft-fork a year.
> Uncontroversial, "safe" changes to the consensus protocol shouldn't be
> out of the question when significant practical benefit is plain to see.

> I'm not out to propose soft-forks lightly, but the current complexity
> in fee management feels untenable, and as evidenced by all the
> discussion lately, fees are an increasingly crucial part of the system.

> There's been much talk about fee-bumping lately, and for good reason -
> dynamic fee management is going to be a central part of bitcoin use as
> the mempool fills up (lord willing) and right now fee-bumping is
> fraught with difficulty and pinning peril.
>
> Gloria's recent post on the topic[0] was very lucid and highlights a
> lot of the current issues, as well as some proposals to improve the
> situation.
>
> As others have noted, the post was great. But throughout the course
> of reading it and the ensuing discussion, I became troubled by the
> increasing complexity of both the status quo and some of the
> proposed remedies.
>
> Layering on special cases, more carve-outs, and X and Y percentage
> thresholds is going to make reasoning about the mempool harder than it
> already is. Special consideration for "what should be in the next
> block" and/or the caching of block templates seems like an imposing
> dependency, dragging in a bunch of state and infrastructure to a
> question that should be solely limited to mempool feerate aggregates
> and the feerate of the particular txn package a wallet is concerned
> with.
>
> This is bad enough for protocol designers and Core developers, but
> making the situation any more intractable for "end-users" and wallet
> developers feels wrong.
>
> I thought it might be useful to step back and reframe. Here are a few
> aims that are motivated chiefly by the quality of end-user experience,
> constrained to obey incentive compatibility (i.e. miner reward, DoS
> avoidance). Forgive the abstract dalliance for a moment; I'll talk
> through concretes afterwards.
>
>
> # Purely additive feerate bumps should never be impossible
>
> Any user should always be able to add to the incentive to mine any
> transaction in a purely additive way. The countervailing force here
> ends up being spam prevention (a la min-relay-fee) to prevent someone
> from consuming bandwidth and mempool space with a long series of
> infinitesimal fee-bumps.
>
> A fee bump, naturally, should be given the same per-byte consideration
> as a normal Bitcoin transaction in terms of relay and block space,
> although it would be nice to come up with a more succinct
> representation. This leads to another design principle:
>
>
> # The bandwidth and chain space consumed by a fee-bump should be minimal
>
> Instead of prompting a rebroadcast of the original transaction for
> replacement, which contains a lot of data not new to the network, it
> makes more sense to broadcast the "diff" which is the additive
> contribution towards some txn's feerate.
>
> This dovetails with the idea that...
>
>
> # Special transaction structure should not be required to bump fees
>
> In an ideal design, special structural foresight would not be needed
> in order for a txn's feerate to be improved after broadcast.
>
> Anchor outputs specified solely for CPFP, which amount to many bytes of
> wasted chainspace, are a hack. It's probably uncontroversial at this
> point to say that even RBF itself is kind of a hack - a special
> sequence number should not be necessary for post-broadcast contribution
> toward feerate. Not to mention RBF's seemingly wasteful consumption of
> bandwidth due to the rebroadcast of data the network has already seen.
>
> In a sane design, no structural foresight - and certainly no wasted
> bytes in the form of unused anchor outputs - should be needed in order
> to add to a miner's reward for confirming a given transaction.
>
> Planning for fee-bumps explicitly in transaction structure also often
> winds up locking in which keys are required to bump fees, at odds
> with the idea that...
>
>
> # Feerate bumps should be able to come from anywhere
>
> One of the practical downsides of CPFP that I haven't seen discussed in
> this conversation is that it requires the transaction to pre-specify the
> keys needed to sign for fee bumps. This is problematic if you're, for
> example, using a vault structure that makes use of pre-signed
> transactions.
>
> What if the key you specified n the anchor outputs for a bunch of
> pre-signed txns is compromised? What if you'd like to be able to
> dynamically select the wallet that bumps fees? CPFP does you no favors
> here.
>
> There is of course a tension between allowing fee bumps to come from
> anywhere and the threat of pinning-like attacks. So we should venture
> to remove pinning as a possibility, in line with the first design
> principle I discuss.
>
>
> ---
>
> Coming down to earth, the "tabula rasa" thought experiment above has led
> me to favor an approach like the transaction sponsors design that Jeremy
> proposed in a prior discussion back in 2020[1].
>
> Transaction sponsors allow feerates to be bumped after a transaction's
> broadcast, regardless of the structure of the original transaction.
> No rebroadcast (wasted bandwidth) is required for the original txn data.
> No wasted chainspace on only-maybe-used prophylactic anchor outputs.
>
> The interface for end-users is very straightforward: if you want to bump
> fees, specify a transaction that contributes incrementally to package
> feerate for some txid. Simple.
>
> In the original discussion, there were a few main objections that I noted:
>
> 1. In Jeremy's original proposal, only one sponsor txn per txid is
>    allowed by policy. A malicious actor could execute a pinning-like
>    attack by specifying an only-slightly-helpful feerate sponsor that
>    then precludes other larger bumps.
>
> I think there are some ways around this shortcoming. For example: what
> if, by policy, sponsor txns had additional constraints that
>
>   - each input must be signed {SIGHASH_SINGLE,SIGHASH_NONE}|ANYONECANPAY,
>   - the txn must be specified RBFable,
>   - a replacement for the sponsor txn must raise the sponsor feerate,
>     including ancestors (maybe this is inherent in "is RBFable," but
>     I don't want to conflate absolute feerates into this).
>
> That way, there is still at most a single sponsor txn per txid in the
> mempool, but anyone can "mix in" inputs which bump the effective
> feerate of the sponsor.
>
> This may not be the exact solution we want, but I think it demonstrates
> that the sponsors design has some flexibility and merits some thinking.
>
> The second objection about sponsors was
>
> 2. (from Suhas) sponsors break the classic invariant: "once a valid
>    transaction is created, it should not become invalid later on unless
>    the inputs are double-spent."
>
> This doesn't seem like a huge concern to me if you consider the txid
> being sponsored as a sort of spiritual input to the sponsor. While the
> theoretical objection against broadening where one has to look in a txn
> to determine its dependencies is understandable, I don't see what the
> practical cost here is.
>
> Reorg complexity seems comparable if not identical, especially if we
> broaden sponsor rules to allow blocks to contain sponsor txns that are
> both for txids in the same block _or_ already included in the chain.
>
> This theoretical concession seems preferable to heaping more rules onto
> an already labyrinthine mempool policy that is difficult for both
> implementers and users to reason about practically and conceptually.
>
> A third objection that wasn't posed, IIRC, but almost certainly would
> be:
>
> 3. Transaction sponsors requires a soft-fork.
>
> Soft-forks are no fun, but I'll tell you what also isn't fun: being on
> the hook to model (and sometimes implement) a dizzying potpourri of
> mempool policies and special-cases. Expecting wallet implementers to
> abide by a maze of rules faithfully in order to ensure txn broadcast and
> fee management invites bugs for perpetuity and network behavior that is
> difficult to reason about a priori. Use of CPFP in the long-term also
> risks needless chain waste.
>
> If a soft-fork is the cost of cleaning up this essential process,
> consideration should be given to paying it as a one-time cost. This
> topic merits a separate post, but consider that in the 5 years leading
> up to the 2017 SegWit drama, we averaged about a soft-fork a year.
> Uncontroversial, "safe" changes to the consensus protocol shouldn't be
> out of the question when significant practical benefit is plain to see.
>
> ---
>
> I hope this message has added some framing to the discussion on fees,
> as well prompting other participants to go back and give the
> transaction sponsor proposal a serious look. The sponsors interface is
> about the simplest I can imagine for wallets, and it seems easy to
> reason about for implementers on Core and elsewhere.
>
> I'm not out to propose soft-forks lightly, but the current complexity
> in fee management feels untenable, and as evidenced by all the
> discussion lately, fees are an increasingly crucial part of the system.
>
>
>
> [0]:
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
> [1]:
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-September/018168.html
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

>> It's not that simple. As a miner, if i have less than 1vMB of transactions in my mempool. I don't want a 10sats/vb transaction paying 100000sats by a 100sats/vb transaction paying only 10000sats.
>
> I don't understand why the "<1vMB in the mempool" case is even worth consideration because the miner will just include the entire mempool in the next block regardless of feerate.

> Well because in the example i gave you this decreases the miner's reward.
> The rule of increasing feerate you stated isn't always economically
> rationale.
>
>
> Note how it can also be extended, for instance if the miner only has
> 1.5vMB of txs and is not assured to receive enough transactions to fill 2
> blocks he might be interested in maximizing absolute fees, not feerate.
>
>
> Sure, we could make the argument that long term we need a large backlog of
> transactions anyways.. But that'd be unfortunately not in phase with
> today's reality.
>
>
> -------- Original Message --------
> On Feb 11, 2022, 00:51, James O'Beirne < james.obeirne@gmail•com> wrote:
>
>
> > It's not that simple. As a miner, if i have less than 1vMB of
> transactions in my mempool. I don't want a 10sats/vb transaction paying
> 100000sats by a 100sats/vb transaction paying only 10000sats.
>
> I don't understand why the "<1vMB in the mempool" case is even worth
> consideration because the miner will just include the entire mempool in the
> next block regardless of feerate.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

> This entirely misses the network cost. Yes, sure, we can send
> "diffs", but if you send enough diffs eventually you send a lot of data.

> > In an ideal design, special structural foresight would not be
> > needed in order for a txn's feerate to be improved after broadcast.
> >
> > Anchor outputs specified solely for CPFP, which amount to many
> > bytes of wasted chainspace, are a hack. > It's probably
> > uncontroversial at this
>
> This has nothing to do with fee bumping, though, this is only solved
> with covenants or something in that direction, not different relay
> policy.

> How does this not also fail your above criteria of not wasting block
> space?

> Further, this doesn't solve pinning attacks at all. In lightning we
> want to be able to *replace* something in the mempool (or see it
> confirm soon, but that assumes we know exactly what transaction is in
> "the" mempool). Just being able to sponsor something doesn't help if
> you don't know what that thing is.

> In a distributed system such as the Bitcoin p2p network, you might
> have transaction A and transaction B  broadcast at the same time and
> your peer topology might fluctuate between original send and
> broadcast of the diff, you don't know who's seen what... You might
> inefficiently announce diff A on top of B and diff B on top A. We
> might leverage set reconciliation there a la Erlay, though likely
> with increased round-trips.

> Have you heard about SIGHASH_GROUP [0] ?

> > [me complaining CPFP requires lock-in to keys]
>
> It's true it requires to pre-specify the fee-bumping key. Though note
> the fee-bumping key can be fully separated from the
> "vaults"/"channels" set of main keys and hosted on replicated
> infrastructure such as watchtowers.

> As a L2 transaction issuer you can't be sure the transaction you wish
> to point to is already in the mempool, or have not been replaced by
> your counterparty spending the same shared-utxo, either competitively
> or maliciously. So as a measure of caution, you should broadcast
> sponsor + target transactions in the same package, thus cancelling
> the bandwidth saving (I think).

> I don't think a sponsor is a silver-bullet to solve all the
> L2-related mempool issues. It won't solve the most concerning pinning
> attacks, as I think the bottleneck is replace-by-fee. Neither solve
> the issues encumbered by the L2s by the dust limit.

> In the context of fee bumping, I don't see how this is a criticism
> unique to transaction sponsors, since it also applies to CPFP: if you
> tried to bump fees for transaction A with child txn B, if some mempool
> hasn't seen parent A, it will reject B.

> This still doesn't address the issue I'm talking about, which is if you
> pre-commit to some "fee-bumping" key in your CPFP outputs and that key
> ends up being compromised. This isn't a matter of data availability or
> redundancy.

> As I mentioned in the reply to Matt's message, I'm not quite
> understanding this idea of wanting to bump the fee for something
> without knowing what it is; that doesn't make much sense to me.
> The "bump fee" operation seems contingent on knowing
> what you want to bump.

> I'm not familiar with the L2 dust-limit issues, and I do think that
> "fixing" RBF behavior is *probably* worthwhile.

>  Those issues aside, I
> think the transaction sponsors idea may be closer to a silver bullet
> than you're giving it credit for, because designing specifically for the
> fee-management use case has some big benefits.

> Thanks for your thoughtful reply Antoine.
>
> > In a distributed system such as the Bitcoin p2p network, you might
> > have transaction A and transaction B  broadcast at the same time and
> > your peer topology might fluctuate between original send and
> > broadcast of the diff, you don't know who's seen what... You might
> > inefficiently announce diff A on top of B and diff B on top A. We
> > might leverage set reconciliation there a la Erlay, though likely
> > with increased round-trips.
>
> In the context of fee bumping, I don't see how this is a criticism
> unique to transaction sponsors, since it also applies to CPFP: if you
> tried to bump fees for transaction A with child txn B, if some mempool
> hasn't seen parent A, it will reject B.
>
> > Have you heard about SIGHASH_GROUP [0] ?
>
> I haven't - I'll spend some time reviewing this. Thanks.
>
> > > [me complaining CPFP requires lock-in to keys]
> >
> > It's true it requires to pre-specify the fee-bumping key. Though note
> > the fee-bumping key can be fully separated from the
> > "vaults"/"channels" set of main keys and hosted on replicated
> > infrastructure such as watchtowers.
>
> This still doesn't address the issue I'm talking about, which is if you
> pre-commit to some "fee-bumping" key in your CPFP outputs and that key
> ends up being compromised. This isn't a matter of data availability or
> redundancy.
>
> Note that this failure may be unique to vault use cases, when you're
> pre-generating potentially large numbers of transactions or covenants
> that cannot be altered after the fact. If you generate vault txns that
> assume the use of some key for CPFP-based fee bumping and that key
> winds up being compromised, that puts you in a an uncomfortable
> situation: you can no longer bump fees on unvaulting transactions,
> rendering the vaults possibly unretrievable depending on the fee market.
>
> > As a L2 transaction issuer you can't be sure the transaction you wish
> > to point to is already in the mempool, or have not been replaced by
> > your counterparty spending the same shared-utxo, either competitively
> > or maliciously. So as a measure of caution, you should broadcast
> > sponsor + target transactions in the same package, thus cancelling
> > the bandwidth saving (I think).
>
> As I mentioned in the reply to Matt's message, I'm not quite
> understanding this idea of wanting to bump the fee for something
> without knowing what it is; that doesn't make much sense to me.
> The "bump fee" operation seems contingent on knowing
> what you want to bump.
>
> And if you're, say, trying to broadcast a lightning channel close and
> you know you need to bump the fee right away, before even broadcasting
> it, either you're going to
>
> - reformulate the txn to bring up the fee rate (e.g. add inputs
>   with some yet-undeployed sighash) as you would have done with RBF, or
>
> - you'd have the same "package relay" problem with CPFP that you
>   would with transaction sponsors.
>
> So I don't understand the objection here.
>
> Also, I didn't mean to discourage existing work on package relay or
> fixing RBF, which seem clearly important. Maybe I should have noted
> that explicitly in the original message
>
> > I don't think a sponsor is a silver-bullet to solve all the
> > L2-related mempool issues. It won't solve the most concerning pinning
> > attacks, as I think the bottleneck is replace-by-fee. Neither solve
> > the issues encumbered by the L2s by the dust limit.
>
> I'm not familiar with the L2 dust-limit issues, and I do think that
> "fixing" RBF behavior is *probably* worthwhile. Those issues aside, I
> think the transaction sponsors idea may be closer to a silver bullet
> than you're giving it credit for, because designing specifically for the
> fee-management use case has some big benefits.
>
> For one, it makes migration easier. That is to say: there is none,
> whereas there is existing RBF policy that needs consideration.
>
> But maybe more importantly, transaction sponsors' limited use case also
> allows for specifying much more targeted "replacement" policy since
> sponsors are special-purpose transactions that only exist to
> dynamically bump feerate. E.g. my SIGHASH_{NONE,SINGLE}|ANYONECANPAY
> proposal might make complete sense for the sponsors/fee-management use
> case, and clarify the replacement problem, but obviously wouldn't work
> for more general transaction replacement. In other words, RBF's
> general nature might make it a much harder problem to solve well.
>

>   If you wish to fee-bump transaction X with sponsor, how can you be sure

>> 2. (from Suhas) "once a valid transaction is created, it should not

> This doesn't seem like a huge concern to me

> > In the context of fee bumping, I don't see how this is a criticism
> > unique to transaction sponsors, since it also applies to CPFP: if you
> > tried to bump fees for transaction A with child txn B, if some mempool
> > hasn't seen parent A, it will reject B.
>
> Agree, it's a comment raising the shenanigans of tx-diff-only propagation,
> afaict affecting equally all fee-bumping primitives. It wasn't a criticism
> specific to transaction sponsors, as at that point of your post, sponsors
> are not introduced yet.
>
> > This still doesn't address the issue I'm talking about, which is if you
> > pre-commit to some "fee-bumping" key in your CPFP outputs and that key
> > ends up being compromised. This isn't a matter of data availability or
> > redundancy.
>
> I'm not sure about the real safety risk of the compromise of the anchor
> output key. Of course, if your anchor output key is compromised and the
> bumped package is already public/known, an attacker can extend your package
> with junk to neutralize your carve-out capability (I think). That said,
> this issue sounds solved to me with package relay, as you can always
> broadcast a new version of the package from the root UTXO, without
> attention to the carve-out limitation.
>
> (Side-note: I think we can slowly deprecate the carve-out once package
> relay is deployed, as the fee-bumping flexibility of the latter is a
> superset of the former).
>
> > As I mentioned in the reply to Matt's message, I'm not quite
> > understanding this idea of wanting to bump the fee for something
> > without knowing what it is; that doesn't make much sense to me.
> > The "bump fee" operation seems contingent on knowing
> > what you want to bump.
>
> From your post : "No rebroadcast (wasted bandwidth) is required for the
> original txn data."
>
> I'm objecting to that supposed benefit of a transaction sponsor. If you
> have transaction X and transaction Y spending the same UTXO, both of them
> can be defined as "the original txn data". If you wish to fee-bump
> transaction X with sponsor, how can you be sure that transaction
> Y isn't present in the majority of network nodes, and X has _not_ been
> dropped since your last broadcast ? Otherwise iirc sponsor design, your
> sponsor transaction is going to be rejected.
>
> I think you can't, and thus preventively you should broadcast as a (new
> type) of package the sponsoring/sponsored transaction.
>
> That said, I'm not sure if that issue is equally affecting vaults than
> payment channels. With vaults, the tree of transactions is  known ahead,
> and there is no competition in the spends. Assuming the first broadcast has
> been efficient (and it could be a reasonable assumption thanks to mempool
> rebroadcast), the sponsor should propagate.
>
> So I think here for the sake of sponsor efficiency analysis, we might have
> to class between the protocol with once-for-all-transaction-negotiation
> (vaults) and the ones with off-chain, dynamic re-negotiation (payment
> channels, factories) ?
>
> > I'm not familiar with the L2 dust-limit issues, and I do think that
> > "fixing" RBF behavior is *probably* worthwhile.
>
> Sadly, it sounds that "fixing" RBF behavior is a requirement to eradicate
> the most advanced pinnings... That fix is independent of the fee-bumping
> primitive considered.
>
> >  Those issues aside, I
> > think the transaction sponsors idea may be closer to a silver bullet
> > than you're giving it credit for, because designing specifically for the
> > fee-management use case has some big benefits.
>
> I don't deny the scheme is interesting, though I would argue SIGHASH_GROUP
> is more efficient, while offering more flexibility. In any case, I think we
> should still pursue further the collections of problems and requirements
> (batching, key management, ...) that new fee-bumping primitives should aim
> to solve, before engaging more on the deployment of one of them [0].
>
> [0] In that sense see
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-May/019031.html
>
> Le lun. 14 févr. 2022 à 15:29, James O'Beirne <james.obeirne@gmail•com> a
> écrit :
>
>> Thanks for your thoughtful reply Antoine.
>>
>> > In a distributed system such as the Bitcoin p2p network, you might
>> > have transaction A and transaction B  broadcast at the same time and
>> > your peer topology might fluctuate between original send and
>> > broadcast of the diff, you don't know who's seen what... You might
>> > inefficiently announce diff A on top of B and diff B on top A. We
>> > might leverage set reconciliation there a la Erlay, though likely
>> > with increased round-trips.
>>
>> In the context of fee bumping, I don't see how this is a criticism
>> unique to transaction sponsors, since it also applies to CPFP: if you
>> tried to bump fees for transaction A with child txn B, if some mempool
>> hasn't seen parent A, it will reject B.
>>
>> > Have you heard about SIGHASH_GROUP [0] ?
>>
>> I haven't - I'll spend some time reviewing this. Thanks.
>>
>> > > [me complaining CPFP requires lock-in to keys]
>> >
>> > It's true it requires to pre-specify the fee-bumping key. Though note
>> > the fee-bumping key can be fully separated from the
>> > "vaults"/"channels" set of main keys and hosted on replicated
>> > infrastructure such as watchtowers.
>>
>> This still doesn't address the issue I'm talking about, which is if you
>> pre-commit to some "fee-bumping" key in your CPFP outputs and that key
>> ends up being compromised. This isn't a matter of data availability or
>> redundancy.
>>
>> Note that this failure may be unique to vault use cases, when you're
>> pre-generating potentially large numbers of transactions or covenants
>> that cannot be altered after the fact. If you generate vault txns that
>> assume the use of some key for CPFP-based fee bumping and that key
>> winds up being compromised, that puts you in a an uncomfortable
>> situation: you can no longer bump fees on unvaulting transactions,
>> rendering the vaults possibly unretrievable depending on the fee market.
>>
>> > As a L2 transaction issuer you can't be sure the transaction you wish
>> > to point to is already in the mempool, or have not been replaced by
>> > your counterparty spending the same shared-utxo, either competitively
>> > or maliciously. So as a measure of caution, you should broadcast
>> > sponsor + target transactions in the same package, thus cancelling
>> > the bandwidth saving (I think).
>>
>> As I mentioned in the reply to Matt's message, I'm not quite
>> understanding this idea of wanting to bump the fee for something
>> without knowing what it is; that doesn't make much sense to me.
>> The "bump fee" operation seems contingent on knowing
>> what you want to bump.
>>
>> And if you're, say, trying to broadcast a lightning channel close and
>> you know you need to bump the fee right away, before even broadcasting
>> it, either you're going to
>>
>> - reformulate the txn to bring up the fee rate (e.g. add inputs
>>   with some yet-undeployed sighash) as you would have done with RBF, or
>>
>> - you'd have the same "package relay" problem with CPFP that you
>>   would with transaction sponsors.
>>
>> So I don't understand the objection here.
>>
>> Also, I didn't mean to discourage existing work on package relay or
>> fixing RBF, which seem clearly important. Maybe I should have noted
>> that explicitly in the original message
>>
>> > I don't think a sponsor is a silver-bullet to solve all the
>> > L2-related mempool issues. It won't solve the most concerning pinning
>> > attacks, as I think the bottleneck is replace-by-fee. Neither solve
>> > the issues encumbered by the L2s by the dust limit.
>>
>> I'm not familiar with the L2 dust-limit issues, and I do think that
>> "fixing" RBF behavior is *probably* worthwhile. Those issues aside, I
>> think the transaction sponsors idea may be closer to a silver bullet
>> than you're giving it credit for, because designing specifically for the
>> fee-management use case has some big benefits.
>>
>> For one, it makes migration easier. That is to say: there is none,
>> whereas there is existing RBF policy that needs consideration.
>>
>> But maybe more importantly, transaction sponsors' limited use case also
>> allows for specifying much more targeted "replacement" policy since
>> sponsors are special-purpose transactions that only exist to
>> dynamically bump feerate. E.g. my SIGHASH_{NONE,SINGLE}|ANYONECANPAY
>> proposal might make complete sense for the sponsors/fee-management use
>> case, and clarify the replacement problem, but obviously wouldn't work
>> for more general transaction replacement. In other words, RBF's
>> general nature might make it a much harder problem to solve well.
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

> >> 2. (from Suhas) "once a valid transaction is created, it should not
> become invalid later on unless the inputs are double-spent."
> > This doesn't seem like a huge concern to me
>
> I agree that this shouldn't be a concern. In fact, I've asked numerous
> people in numerous places what practical downside there is to transactions
> that become invalid, and I've heard basically radio silence other than one
> off hand remark by satoshi at the dawn of time which didn't seem to me to
> have good reasoning. I haven't seen any downside whatsoever of transactions
> that can become invalid for anyone waiting the standard 6 confirmations -
> the reorg risks only exists for people not waiting for standard
> finalization. So I don't think we should consider that aspect of a
> sponsorship transaction that can only be mined with the transaction it
> sponsors to be a problem unless a specific practical problem case can be
> identified. Even if a significant such case was identified, an easy
> solution would be to simply allow sponsorship transactions to be mined on
> or after the sponsored transaction is mined.
>

> The downside is that in a 6 block reorg any transaction that is moved
> past its expiration date becomes invalid and all its descendants
> become invalid too.

 > there are technical reasons for sponsors to not be monotone. Mostly that

> The sponsors proposal is a change from Epsilon-Strong Reorgability to

> The current consensus threshold for transactions to become invalid is a

>  I promise to personally build a wallet that always creates transactions

> The difference between sponsors and this issue is more subtle. The issue
> Suhas raised was with a variant of sponsors trying to address a second
> criticism, not sponsors itself, which is secure against this.
>
> I think I can make this clear by defining a few different properties:
>
> Strong Reorgability: The transaction graph can be arbitrarily reorged into
> any series of blocks as long as dependency order/timelocks are respected.
> Simple Existential Reorgability: The transaction graph can be reorged into
> a different series of blocks, and it is not computationally difficult to
> find such an ordering.
> Epsilon-Strong Reorgability: The transaction graph can be arbitrarily
> reorged into any series of blocks as long as dependency order/timelocks are
> respected, up to Epsilon blocks.
> Epsilon: Simple Existential Reorgability: The transaction graph can be
> reorged into a different series of blocks, and it is not computationally
> difficult to find such an ordering, up to epsilon blocks.
> Perfect Reorgability: The transaction graph can be reorged into a
> different series of blocks, but the transactions themselves are already
> locked in.
>
> Perfect Reorgability doesn't exist in Bitcoin because unconfirmed
> transactions can be double spent which invalidates descendants. Notably,
> for a subset of the graph which is CTV Congestion control tree expansions,
> perfect reorg ability would exist, so it's not just a bullshit concept to
> think about :)
>
> The sponsors proposal is a change from Epsilon-Strong Reorgability to
> Epsilon-Weak Reorgability. It's not clear to me that there is any
> functional reason to rely on Strongness when Bitcoin's reorgability is
> already not Perfect, so a reorg generator with malicious intent can already
> disturb the tx graph. Epsion-Weak Reorgability seems to be a sufficient
> property.
>
> Do you disagree with that?
>
> Best,
>
> Jeremy
>
> --
> @JeremyRubin <https://twitter.com/JeremyRubin>
>
> On Tue, Feb 15, 2022 at 12:25 PM Russell O'Connor via bitcoin-dev <
> bitcoin-dev@lists•linuxfoundation•org> wrote:
>
>>
>>
>>> >> 2. (from Suhas) "once a valid transaction is created, it should not
>>> become invalid later on unless the inputs are double-spent."
>>> > This doesn't seem like a huge concern to me
>>>
>>> I agree that this shouldn't be a concern. In fact, I've asked numerous
>>> people in numerous places what practical downside there is to transactions
>>> that become invalid, and I've heard basically radio silence other than one
>>> off hand remark by satoshi at the dawn of time which didn't seem to me to
>>> have good reasoning. I haven't seen any downside whatsoever of transactions
>>> that can become invalid for anyone waiting the standard 6 confirmations -
>>> the reorg risks only exists for people not waiting for standard
>>> finalization. So I don't think we should consider that aspect of a
>>> sponsorship transaction that can only be mined with the transaction it
>>> sponsors to be a problem unless a specific practical problem case can be
>>> identified. Even if a significant such case was identified, an easy
>>> solution would be to simply allow sponsorship transactions to be mined on
>>> or after the sponsored transaction is mined.
>>>
>>
>> The downside is that in a 6 block reorg any transaction that is moved
>> past its expiration date becomes invalid and all its descendants become
>> invalid too.
>>
>> The current consensus threshold for transactions to become invalid is a
>> 100 block reorg, and I see no reason to change this threshold.  I promise
>> to personally build a wallet that always creates transactions on the verge
>> of becoming invalid should anyone ever implement a feature that violates
>> this tx validity principle.
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists•linuxfoundation•org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

> What do you mean by monotone in the context of sponsor transactions?

> And when you say tx-index, do you mean an index for looking up a
> transaction by its ID? Is that not already something nodes do?

> > The current consensus threshold for transactions to become invalid
> > is a 100 block reorg
>
> What do you mean by this? The only 100 block period I'm aware of is
> the coinbase cooldown period.

>  the validity of a sponsor txn is "monotonically" true at any point after

> > What do you mean by monotone in the context of sponsor transactions?
>
> I take this to mean that the validity of a sponsor txn is
> "monotonically" true at any point after the inclusion of the sponsored
> txn in a block.
>
> > And when you say tx-index, do you mean an index for looking up a
> > transaction by its ID? Is that not already something nodes do?
>
> Indeed, not all nodes have this ability. Each bitcoind node has a map
> of unspent coins which can be referenced by outpoint i.e.(txid, index),
> but the same isn't true for all historical transactions. I
> (embarrassingly) forgot this in the prior post.
>
> The map of (txid -> transaction) for all time is a separate index that
> must be enabled via the `-txindex=1` flag; it isn't enabled by default
> because it isn't required for consensus and its growth is unbounded.
>
> > > The current consensus threshold for transactions to become invalid
> > > is a 100 block reorg
> >
> > What do you mean by this? The only 100 block period I'm aware of is
> > the coinbase cooldown period.
>
> If there were a reorg deeper than 100 blocks, it would permanently
> invalidate any transactions spending the recently-matured coinbase
> subsidy in any block between $new_reorg_tip and ($former_tip_height -
> 100). These invalidated spends would not be able to be reorganized
> into a new replacement chain.
>
> How this differs in practice or principle from a "regular" double-spend
> via reorg I'll leave for another message. I'm not sure that I understand
> that myself. Personally I think if we hit a >100 block reorg, we've got
> bigger issues than coinbase invalidation.
>

> This is where *all* the complexity comes from. If our goal is to "ensure a
> bump increases a miner's overall revenue" (thus not wasting relay for
> everyone else), then we precisely *do* need
> > Special consideration for "what should be in the next
> > block" and/or the caching of block templates seems like an imposing
> > dependency
> Whether a transaction increases a miner's revenue depends precisely on
> whether the transaction (package) being replaced is in the next block - if
> it is, you care about the absolute fee of the package and its replacement.

> It's not that simple. As a miner, if i have less than 1vMB of transactions in my mempool. I don't want a 10sats/vb transaction paying 100000sats by a 100sats/vb transaction paying only 10000sats.

> Is it really true that miners do/should care about that?

> I suspect the "economically rational" choice would be to happily trade
> off that immediate loss against even a small chance of a simpler policy
> encouraging higher adoption of bitcoin, _or_ a small chance of more
> on-chain activity due to higher adoption of bitcoin protocols like
> lightning and thus a lower chance of an empty mempool in future.

> Certainly those percentages can be expected to double every four years as
> the block reward halves (assuming we don't also reduce the min relay fee
> and block min tx fee), but I think for both miners and network stability,
> it'd be better to have the mempool backlog increase over time, which
> would both mean there's no/less need to worry about the special case of
> the mempool being empty, and give a better incentive for people to pay
> higher fees for quicker confirmations.

> If you really want to do that
> optimally, I think you have to have a mempool that retains conflicting
> txs and runs a dynamic programming solution to pick the best set, rather
> than today's simple greedy algorithms both for building the block and
> populating the mempool?

> Is there a plausible example where the difference isn't that marginal?

> Always accepting (package/descendent) fee rate increases removes the

> Note that if we did have this policy, you could abuse it to cheaply drain
> people's mempools: if there was a 300MB backlog, you could publish 2980
> 100kB txs paying a fee rate just below the next block fee, meaning you'd
> kick out the previous backlog and your transactions take up all but the
> top 2MB of the mempool; if you then replace them all with perhaps 2980
> 100B txs paying a slightly higher fee rate, the default mempool will be
> left with only 2.3MB, at an ultimate cost to you of only about 30% of a
> block in fees, and you could then fill the mempool back up by spamming
> 300MB of ultra low fee rate txs.

> I think spam prevention at the outbound relay level isn't enough to
> prevent that: an attacker could contact every public node and relay the
> txs directly, clearing out the mempool of most public nodes directly. So
> you'd want some sort of spam prevention on inbound txs too?

> On Thu, Feb 10, 2022 at 07:12:16PM -0500, Matt Corallo via bitcoin-dev
> wrote:
> > This is where *all* the complexity comes from. If our goal is to "ensure
> a
> > bump increases a miner's overall revenue" (thus not wasting relay for
> > everyone else), then we precisely *do* need
> > > Special consideration for "what should be in the next
> > > block" and/or the caching of block templates seems like an imposing
> > > dependency
> > Whether a transaction increases a miner's revenue depends precisely on
> > whether the transaction (package) being replaced is in the next block -
> if
> > it is, you care about the absolute fee of the package and its
> replacement.
>
> On Thu, Feb 10, 2022 at 11:44:38PM +0000, darosior via bitcoin-dev wrote:
> > It's not that simple. As a miner, if i have less than 1vMB of
> transactions in my mempool. I don't want a 10sats/vb transaction paying
> 100000sats by a 100sats/vb transaction paying only 10000sats.
>
> Is it really true that miners do/should care about that?
>
> If you did this particular example, the miner would be losing 90k sats
> in fees, which would be at most 1.44 *millionths* of a percent of the
> block reward with the subsidy at 6.25BTC per block, even if there were
> no other transactions in the mempool. Even cumulatively, 10sats/vb over
> 1MB versus 100sats/vb over 10kB is only a 1.44% loss of block revenue.
>
> I suspect the "economically rational" choice would be to happily trade
> off that immediate loss against even a small chance of a simpler policy
> encouraging higher adoption of bitcoin, _or_ a small chance of more
> on-chain activity due to higher adoption of bitcoin protocols like
> lightning and thus a lower chance of an empty mempool in future.
>
> If the network has an "empty mempool" (say less than 2MvB-10MvB of
> backlog even if you have access to every valid 1+ sat/vB tx on any node
> connected to the network), then I don't think you'll generally have txs
> with fee rates greater than ~20 sat/vB (ie 20x the minimum fee rate),
> which means your maximum loss is about 3% of block revenue, at least
> while the block subsidy remains at 6.25BTC/block.
>
> Certainly those percentages can be expected to double every four years as
> the block reward halves (assuming we don't also reduce the min relay fee
> and block min tx fee), but I think for both miners and network stability,
> it'd be better to have the mempool backlog increase over time, which
> would both mean there's no/less need to worry about the special case of
> the mempool being empty, and give a better incentive for people to pay
> higher fees for quicker confirmations.
>
> If we accept that logic (and assuming we had some additional policy
> to prevent p2p relay spam due to replacement txs), we could make
> the mempool accept policy for replacements just be (something like)
> "[package] feerate is greater than max(descendent fee rate)", which
> seems like it'd be pretty straightforward to deal with in general?
>
>
>
> Thinking about it a little more; I think the decision as to whether
> you want to have a "100kvB at 10sat/vb" tx or a conflicting "1kvB at
> 100sat/vb" tx in your mempool if you're going to take into account
> unrelated, lower fee rate txs that are also in the mempool makes block
> building "more" of an NP-hard problem and makes the greedy solution
> we've currently got much more suboptimal -- if you really want to do that
> optimally, I think you have to have a mempool that retains conflicting
> txs and runs a dynamic programming solution to pick the best set, rather
> than today's simple greedy algorithms both for building the block and
> populating the mempool?
>
> For example, if you had two such replacements come through the network,
> a miner could want to flip from initially accepting the first replacement,
> to unaccepting it:
>
> Initial mempool: two big txs at 100k each, many small transactions at
> 15s/vB and 1s/vB
>
>  [100kvB at 20s/vB] [850kvB at 15s/vB] [100kvB at 12s/vB] [1000kvB at
> 1s/vB]
>    -> 0.148 BTC for 1MvB (100*20 + 850*15 + 50*1)
>
> Replacement for the 20s/vB tx paying a higher fee rate but lower total
> fee; that's worth including:
>
>  [10kvB at 100s/vB] [850kvB at 15s/vB] [100kvB at 12s/vB [1000kvB at 1s/vB]
>    -> 0.1499 BTC for 1MvB (10*100 + 850*15 + 100*12 + 40*1)
>
> Later, replacement for the 12s/vB tx comes in, also paying higher fee
> rate but lower total fee. Worth including, but only if you revert the
> original replacement:
>
>  [100kvB at 20s/vB] [50kvB at 20s/vB] [850kvB at 15s/vB] [1000kvB at 1s/vB]
>    -> 0.16 BTC for 1MvB (150*20 + 850*15)
>
>  [10kvB at 100s/vB] [50kvB at 20s/vB] [850kvB at 15s/vB] [1000kvB at 1s/vB]
>    -> 0.1484 BTC for 1MvB (10*100 + 50*20 + 850*15 + 90*1)
>
> Algorithms/mempool policies you might have, and their results with
> this example:
>
>  * current RBF rules: reject both replacements because they don't
>    increase the absolute fee, thus get the minimum block fees of
>    0.148 BTC
>
>  * reject RBF unless it increases the fee rate, and get 0.1484 BTC in
>    fees
>
>  * reject RBF if it's lower fee rate or immediately decreases the block
>    reward: so, accept the first replacement, but reject the second,
>    getting 0.1499 BTC
>
>  * only discard a conflicting tx when it pays both a lower fee rate and
>    lower absolute fees, and choose amongst conflicting txs optimally
>    via some complicated tx allocation algorithm when generating a block,
>    and get 0.16 BTC
>
> In this example, those techniques give 92.5%, 92.75%, 93.69% and 100% of
> total possible fees you could collect; and 99.813%, 99.819%, 99.84% and
> 100% of the total possible block reward at 6.25BTC/block.
>
> Is there a plausible example where the difference isn't that marginal?
> Seems like the simplest solution of just checking the (package/descendent)
> fee rate increases works well enough here at least.
>
> If 90kvB of unrelated txs at 14s/vB were then added to the mempool, then
> replacing both txs becomes (just barely) optimal, meaning the smartest
> possible algorithm and the dumbest one of just considering the fee rate
> produce the same result, while the others are worse:
>
>  [10kvB at 100s/vB] [50kvB at 20s/vB] [850kvB at 15s/vB] [90kvB at 14s/vB]
>    -> 0.1601 BTC for 1MvB
>    (accepting both)
>
>  [100kvB at 20s/vB] [50kvB at 20s/vB] [850kvB at 15s/vB] [90kvB at 14s/vB]
>    -> 0.1575 BTC for 1MvB
>    (accepting only the second replacement)
>
>  [10kvB at 100s/vB] [850kvB at 15s/vB] [90kvB at 14s/vB] [100kvB at 12s/vB]
>    -> 0.1551 BTC for 1MvB
>    (first replacement only, optimal tx selection: 10*100, 850*15, 50*14,
> 100*12)
>
>  [100kvB at 20s/vB] [850kvB at 15s/vB] [90kvB at 14s/vB] [100kvB at 12s/vB]
>    -> 0.1545 BTC for 1MvB
>    (accepting neither replacement)
>
>  [10kvB at 100s/vB] [850kvB at 15s/vB] [90kvB at 14s/vB] [100kvB at 12s/vB]
>    -> 0.1506 BTC for 1MvB
>    (first replacement only, greedy tx selection: 10*100, 850*15, 90*14,
> 50*1)
>
> Always accepting (package/descendent) fee rate increases removes the
> possibility of pinning entirely, I think -- you still have the problem
> where someone else might get a conflicting transaction confirmed first,
> but they can't get a conflicting tx stuck in the mempool without
> confirming if you're willing to pay enough to get it confirmed.
>
>
>
> Note that if we did have this policy, you could abuse it to cheaply drain
> people's mempools: if there was a 300MB backlog, you could publish 2980
> 100kB txs paying a fee rate just below the next block fee, meaning you'd
> kick out the previous backlog and your transactions take up all but the
> top 2MB of the mempool; if you then replace them all with perhaps 2980
> 100B txs paying a slightly higher fee rate, the default mempool will be
> left with only 2.3MB, at an ultimate cost to you of only about 30% of a
> block in fees, and you could then fill the mempool back up by spamming
> 300MB of ultra low fee rate txs.
>
> I think spam prevention at the outbound relay level isn't enough to
> prevent that: an attacker could contact every public node and relay the
> txs directly, clearing out the mempool of most public nodes directly. So
> you'd want some sort of spam prevention on inbound txs too?
>
> So I think you'd need to carefully think about relay spam before making
> this sort of change.  Also, if we had tx rebroadcast implemented then
> having just a few nodes with large mempools might allow the network to
> recover from this situation automatically.
>
> Cheers,
> aj
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

> I suspect the "economically rational" choice would be to happily trade off that immediate loss against even a small chance of a simpler policy encouraging higher adoption of bitcoin, _or_ a small chance of more on-chain activity due to higher adoption of bitcoin protocols like lightning and thus a lower chance of an empty mempool in future.

> If anyone has any indication that there are miners running forks of bitcoind that change this behavior, I'd be curious to know it.

>> Is it really true that miners do/should care about that?
>
> De facto, any miner running an unmodified version of bitcoind doesn't
> care about anything aside from ancestor fee rate, given that the
> BlockAssembler as-written orders transactions for inclusion by
> descending ancestor fee-rate and then greedily adds them to the block
> template. [0]
>
> If anyone has any indication that there are miners running forks of
> bitcoind that change this behavior, I'd be curious to know it.
>
> Along the lines of what AJ wrote, optimal transaction selection is
> NP-hard (knapsack problem). Any time that a miner spends deciding how
> to assemble the next block is time not spent grinding on the nonce, and
> so I'm skeptical that miners in practice are currently doing anything
> that isn't fast and simple like the default implementation: sorting
> fee-rate in descending order and then greedily packing.
>
> But it would be interesting to hear evidence to the contrary.
>
> ---
>
> You can make the argument that transaction selection is just a function
> of mempool contents, and so mempool maintenance criteria might be the
> thing to look at. Mempool acceptance is gated based on a minimum
> feerate[1]. Mempool eviction (when running low on space) happens on
> the basis of max(self_feerate, descendant_feerate) [2]. So even in the
> mempool we're still talking in terms of fee rates, not absolute fees.
>
> That presents us with the "is/ought" problem: just because the mempool
> *is* currently gating only on fee rate doesn't mean that's optimal. But
> if the whole point of the mempool is to hold transactions that will be
> mined, and if there's good reason that txns are chosen for mining based
> on fee rate (it's quick and good enough), then it seems like fee rate
> is the approximation that should ultimately prevail for txn
> replacement.
>
> [0]:
> https://github.com/bitcoin/bitcoin/blob/master/src/node/miner.cpp#L310-L320
> [1]:
> https://github.com/bitcoin/bitcoin/blob/master/src/txmempool.cpp#L1106
> [2]:
> https://github.com/bitcoin/bitcoin/blob/master/src/txmempool.cpp#L1138-L1144

> Unfortunately, there are technical reasons for sponsors to not be monotone.
> Mostly that it requires the maintenance of an additional permanent
> TX-Index