Full Disclosure: CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 "All your mempool are belong to us"
59 messages, 16 Oct to 17 Nov 2023. Started by Antoine Riard.
Antoine Riard (19), Peter Todd (16), Matt Corallo (6), Matt Morehouse (5), ZmnSCPxj (3), Olaoluwa Osuntokun (2), David A. Harding (2), ziggie1984, +5 more
(cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other multi-party bitcoin apps might be affected)
Hi,
End of last year (December 2022), amid technical discussions on eltoo payment channels and incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack affecting lightning channels was discovered.
After careful analysis, it turns out this attack is practical and immediately exposed lightning routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output channels. A potential exploitation plausibly happening even without network mempools congestion.
Mitigations have been designed, implemented and deployed by all major lightning implementations during the last months.
Please find attached the release numbers, where the mitigations should be present: - LDK: v0.0.118 - CVE-2023 -40231 - Eclair: v0.9.0 - CVE-2023-40232 - LND: v.0.17.0-beta - CVE-2023-40233 - Core-Lightning: v.23.08.01 - CVE-2023-40234
While neither replacement cycling attacks have been observed or reported in the wild since the last ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle).
It is understood that a simple replacement cycling attack does not demand privileged capabilities from an attacker (e.g no low-hashrate power) and only access to basic bitcoin and lightning software. Yet I still think executing such an attack successfully requests a fair amount of bitcoin technical know-how and decent preparation.
From my understanding of those issues, it is yet to be determined if the mitigations deployed are robust enough in face of advanced replacement cycling attackers, especially ones able to combine different classes of transaction-relay jamming such as pinnings or vetted with more privileged capabilities.
Please find a list of potential affected bitcoin applications in this full disclosure report using bitcoin script timelocks or multi-party transactions, albeit no immediate security risk exposure as severe as the ones affecting lightning has been identified. Only cursory review of non-lightning applications has been conducted so far.
There is a paper published summarizing replacement cycling attacks on the lightning network: https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf
## Problem
A lightning node allows HTLCs forwarding (in bolt3's parlance accepted HTLC on incoming link and offered HTLC on outgoing link) should settle the outgoing state with either a success or timeout before the incoming state timelock becomes final and an asymmetric defavorable settlement might happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" section 2.3 for a classical exposition of this lightning security property).
Failure to satisfy this settlement requirement exposes a forwarding hop to a loss of fund risk where the offered HTLC is spent by the outgoing link counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming link counterparty's HTLC-timeout.
The specification mandates the incoming HTLC expiration timelock to be spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this exact interval value being an implementation and node policy setting. As a minimal value, the specification recommends 34 blocks of interval. If the timelock expiration I of the inbound HTLC is equal to 100 from chain tip, the timelock expiration O of the outbound HTLC must be equal to 66 blocks from chain tip, giving a reasonable buffer of reaction to the lightning forwarding node.
In the lack of cooperative off-chain settlement of the HTLC on the outgoing link negotiated with the counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the lightning node should broadcast its commitment transaction. Once the commitment is confirmed (if anchor and the 1 CSV encumbrance is present), the lightning node broadcasts and confirms its HTLC-timeout before I height is reached.
Here enter a replacement cycling attack. A malicious channel counterparty can broadcast its HTLC-preimage transaction with a higher absolute fee and higher feerate than the honest HTLC-timeout of the victim lightning node and triggers a replacement. Both for legacy and anchor output channels, a HTLC-preimage on a counterparty commitment transaction is malleable, i.e additional inputs or outputs can be added. The HTLC-preimage spends an unconfirmed and unrelated to the channel parent transaction M and conflicts its child.
As the HTLC-preimage spends an unconfirmed input that was already included in the unconfirmed and unrelated child transaction (rule 2), pays an absolute higher fee of at least the sum paid by the HTLC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is greater than all directly conflicting transactions (rule 6), the replacement is accepted. The honest HTLC-timeout is evicted out of the mempool.
In an ulterior move, the malicious counterparty can replace the parent transaction itself with another candidate N satisfying the replacement rules, triggering the eviction of the malicious HTLC-preimage from the mempool as it was a child of the parent T.
There is no spending candidate of the offered HTLC output for the current block laying in network mempools.
This replacement cycling tricks can be repeated for each rebroadcast attempt of the HTLC-timeout by the honest lightning node until expiration of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout is broadcast by the counterparty's on the incoming link in collusion with the one on the outgoing link broadcasting its own HTLC-preimage.
The honest Lightning node has been "double-spent" in its HTLC forwarding.
As a notable factor impacting the success of the attack, a lightning node's honest HTLC-timeout might be included in the block template of the miner winning the block race and therefore realizes a spent of the offered output. In practice, a replacement cycling attack might over-connect to miners' mempools and public reachable nodes to succeed in a fast eviction of the HTLC-timeout by its HTLC-preimage. As this latter transaction can come with a better ancestor-score, it should be picked up on the flight by economically competitive miners.
A functional test exercising a simple replacement cycling of a HTLC transaction on bitcoin core mempool is available: https://github.com/ariard/bitcoin/commits/2023-test-mempool
## Deployed LN mitigations
Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being usually broadcast by lightning nodes only once every block, or less the replacement cycling malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases the absolute fee cost for the attacker.
Implemented and deployed by Eclair, Core-Lightning, LND and LDK .
Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a cooperative message or go on-chain with it to claim the accepted HTLC output.
Implemented and deployed by Eclair and LND.
CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid by the attacker, a risk of the HTLC-preimage being detected or discovered by the honest lightning node, or the HTLC-timeout to slip in a winning block template. Bumping the default CLTV delta hardens the odds of success of a simple replacement cycling attack.
Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72.
## Affected Bitcoin Protocols and Applications
From my understanding the following list of Bitcoin protocols and applications could be affected by new denial-of-service vectors under some level of network mempools congestion. Neither tests or advanced review of specifications (when available) has been conducted for each of them: - on-chain DLCs - coinjoins - payjoins - wallets with time-sensitive paths - peerswap and submarine swaps - batch payouts - transaction "accelerators"
Inviting their developers, maintainers and operators to investigate how replacement cycling attacks might disrupt their in-mempool chain of transactions, or fee-bumping flows at the shortest delay. Simple flows and non-multi-party transactions should not be affected to the best of my understanding.
## Open Problems: Package Malleability
Pinning attacks have been known for years as a practical vector to compromise lightning channels funds safety, under different scenarios (cf. current bip331's motivation section). Mitigations at the mempool level have been designed, discussed and are under implementation by the community (ancestor package relay + nverrsion=3 policy). Ideally, they should constraint a pinning attacker to always attach a high feerate package (commitment + CPFP) to replace the honest package, or allow a honest lightning node to overbid a malicious pinning package and get its time-sensitive transaction optimistically included in the chain.
Replacement cycling attack seem to offer a new way to neutralize the design goals of package relay and its companion nversion=3 policy, where an attacker package RBF a honest package out of the mempool to subsequently double-spend its own high-fee child with a transaction unrelated to the channel. As the remaining commitment transaction is pre-signed with a minimal relay fee, it can be evicted out of the mempool.
A functional test exercising a simple replacement cycling of a lightning channel commitment transaction on top of the nversion=3 code branch is available: https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2
## Discovery
In 2018, the issue of static fees for pre-signed lightning transactions is made more widely known, the carve-out exemption in mempool rules to mitigate in-mempool package limits pinning and the anchor output pattern are proposed.
In 2019, bitcoin core 0.19 is released with carve-out support. Continued discussion of the anchor output pattern as a dynamic fee-bumping method.
In 2020, draft of anchor output submitted to the bolts. Initial finding of economic pinning against lightning commitment and second-stage HTLC transactions. Subsequent discussions of a preimage-overlay network or package-relay as mitigations. Public call made to inquiry more on potential other transaction-relay jamming attacks affecting lightning.
In 2021, initial work in bitcoin core 22.0 of package acceptance. Continued discussion of the pinning attacks and shortcomings of current mempool rules during community-wide online workshops. Later the year, in light of all issues for bitcoin second-layers, a proposal is made about killing the mempool.
In 2022, bip proposed for package relay and new proposed v3 policy design proposed for a review and implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and conceptual questions about alignment of mempool rules w.r.t miners incentives are investigated.
Along this year 2022, eltoo lightning channels design are discussed, implemented and reviewed. In this context and after discussions on mempool anti-DoS rules, I discovered this new replacement cycling attack was affecting deployed lightning channels and immediately reported the finding to some bitcoin core developers and lightning maintainers.
## Timeline
- 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Zhao - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, Matt Corallo and Olaoluwa Osuntunkun - 2022-12-23: Sharing to Eugene Siegel (LND) - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning potential affected projects) - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers issuers) and initial proposal of an early public disclosure - 2022-01-19: Collection of analysis if other second-layers and multi-party applications affected. LN mitigations development starts. - 2023-05-04: Sharing to Wilmer Paulino (LDK) - 2023-06-20: LN mitigations implemented and progressively released. Week of the 16 october proposed for full disclosure. - 2023-08-10: CVEs assigned by MITRE - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling attack existence to security@bitcoincore•org. - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks
## Conclusion
Despite the line of mitigations adopted and deployed by current major lightning implementations, I believe replacement cycling attacks are still practical for advanced attackers. Beyond this new attack might come as a way to partially or completely defeat some of the pinning mitigations which have been working for years as a community.
As of today, it is uncertain to me if lightning is not affected by a more severe long-term package malleability critical security issue under current consensus rules, and if any other time-sensitive multi-party protocol, designed or deployed isn't de facto affected too (loss of funds or denial of service).
Assuming analysis on package malleability is correct, it is unclear to me if it can be corrected by changes in replacement / eviction rules or mempool chain of transactions processing strategy. Inviting my technical peers and the bitcoin community to look more on this issue, including to dissent. I'll be the first one pleased if I'm fundamentally wrong on those issues, or if any element has not been weighted with the adequate technical accuracy it deserves.
Do not trust, verify. All mistakes and opinions are my own.
Antoine
"meet with Triumph and Disaster. And treat those two impostors just the same" - K.
On October 16, 2023 6:57:36 PM GMT+02:00, Antoine Riard via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (9 lines)
>(cross-posting mempool issues identified are exposing lightning chan to >loss of funds risks, other multi-party bitcoin apps might be affected) > >As the HTLC-preimage spends an unconfirmed input that was already included >in the unconfirmed and unrelated child transaction (rule 2), pays an >absolute higher fee of at least the sum paid by the HTLC-timeout and child >transaction (rule 3) and the HTLC-preimage feerate is greater than all >directly conflicting transactions (rule 6), the replacement is accepted. >The honest HTLC-timeout is evicted out of the mempool.
I think if you want people to understand this exploit, you need to explain in more detail how we have a situation where two different parties can spend the same HTLC txout, without the first party having the right to spend it via their knowledge of the HTLC-preimage.
On Mon, Oct 16, 2023 at 7:21 PM Peter Todd via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (1 line)
> I think if you want people to understand this exploit, you need to explain in more detail how we have a situation where two different parties can spend the same HTLC txout, without the first party having the right to spend it via their knowledge of the HTLC-preimage.
The two main ways of spending an "offered" HTLC txout: 1) With a presigned multisig covenant transaction paying to the offerer (a.k.a HTLC-timeout transaction) 2) With a preimage and the receiver's signature
Since option 1 uses a presigned covenant held by the offerer, only the offerer can spend via that path. Since option 2 requires the receiver's signature, only the receiver can spend via that path.
The exact script used is here: https://github.com/lightning/bolts/blob/master/03-transactions.md#offered-htlc-outputs.
Hi Antoine,
Thanks for this great write up, and also your diligence in reporting this issue to the various implementations, and game planning with us re mitigations and attack scenarios.
One small clarification: all of lnd's relevant mitigations were in place by lnd v0.16.1-beta [1], which was released on April 24th 2023. Since then we've fixed some performance regressions introduced due to some of the mitigations (mempool watching), and in 0.17.1 we'll start to use the new `gettxspendingprevout` RPC call with bitcoind to further reduce load.
[1]: https://github.com/lightningnetwork/lnd/releases/tag/v0.16.1-beta
-- Laolu
On Mon, Oct 16, 2023 at 10:04 AM Antoine Riard via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (291 lines)
> (cross-posting mempool issues identified are exposing lightning chan to > loss of funds risks, other multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and incentives compatibility of the mempool anti-DoS > rules, a new transaction-relay jamming attack affecting lightning channels > was discovered. > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning routing hops carrying HTLC traffic to loss of > funds security risks, both legacy and anchor output channels. A potential > exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major > lightning implementations during the last months. > > Please find attached the release numbers, where the mitigations should be > present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported > in the wild since the last ~10 months or experimented in real-world > conditions on bitcoin mainet, functional test is available exercising the > affected lightning channel against bitcoin core mempool (26.0 release > cycle). > > It is understood that a simple replacement cycling attack does not demand > privileged capabilities from an attacker (e.g no low-hashrate power) and > only access to basic bitcoin and lightning software. Yet I still think > executing such an attack successfully requests a fair amount of bitcoin > technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the > mitigations deployed are robust enough in face of advanced replacement > cycling attackers, especially ones able to combine different classes of > transaction-relay jamming such as pinnings or vetted with more privileged > capabilities. > > Please find a list of potential affected bitcoin applications in this full > disclosure report using bitcoin script timelocks or multi-party > transactions, albeit no immediate security risk exposure as severe as the > ones affecting lightning has been identified. Only cursory review of > non-lightning applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the > lightning network: > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and offered HTLC on outgoing link) should settle the > outgoing state with either a success or timeout before the incoming state > timelock becomes final and an asymmetric defavorable settlement might > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" > section 2.3 for a classical exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to > a loss of fund risk where the offered HTLC is spent by the outgoing link > counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming > link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC > expiration timelock, this exact interval value being an implementation and > node policy setting. As a minimal value, the specification recommends 34 > blocks of interval. If the timelock expiration I of the inbound HTLC is > equal to 100 from chain tip, the timelock expiration O of the outbound HTLC > must be equal to 66 blocks from chain tip, giving a reasonable buffer of > reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the counterparty (either > `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the > lightning node should broadcast its commitment transaction. Once the > commitment is confirmed (if anchor and the 1 CSV encumbrance is present), > the lightning node broadcasts and confirms its HTLC-timeout before I height > is reached. > > Here enter a replacement cycling attack. A malicious channel counterparty > can broadcast its HTLC-preimage transaction with a higher absolute fee and > higher feerate than the honest HTLC-timeout of the victim lightning node > and triggers a replacement. Both for legacy and anchor output channels, a > HTLC-preimage on a counterparty commitment transaction is malleable, i.e > additional inputs or outputs can be added. The HTLC-preimage spends an > unconfirmed and unrelated to the channel parent transaction M and conflicts > its child. > > As the HTLC-preimage spends an unconfirmed input that was already included > in the unconfirmed and unrelated child transaction (rule 2), pays an > absolute higher fee of at least the sum paid by the HTLC-timeout and child > transaction (rule 3) and the HTLC-preimage feerate is greater than all > directly conflicting transactions (rule 6), the replacement is accepted. > The honest HTLC-timeout is evicted out of the mempool. > > In an ulterior move, the malicious counterparty can replace the parent > transaction itself with another candidate N satisfying the replacement > rules, triggering the eviction of the malicious HTLC-preimage from the > mempool as it was a child of the parent T. > > There is no spending candidate of the offered HTLC output for the current > block laying in network mempools. > > This replacement cycling tricks can be repeated for each rebroadcast > attempt of the HTLC-timeout by the honest lightning node until expiration > of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout > is broadcast by the counterparty's on the incoming link in collusion with > the one on the outgoing link broadcasting its own HTLC-preimage. > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > As a notable factor impacting the success of the attack, a lightning > node's honest HTLC-timeout might be included in the block template of the > miner winning the block race and therefore realizes a spent of the offered > output. In practice, a replacement cycling attack might over-connect to > miners' mempools and public reachable nodes to succeed in a fast eviction > of the HTLC-timeout by its HTLC-preimage. As this latter transaction can > come with a better ancestor-score, it should be picked up on the flight by > economically competitive miners. > > A functional test exercising a simple replacement cycling of a HTLC > transaction on bitcoin core mempool is available: > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits > from the HTLC-timeout being usually broadcast by lightning nodes only once > every block, or less the replacement cycling malicious transactions paid > only equal the sum of the absolute fees paid by the HTLC, adjusted with the > replacement penalty. Rebroadcasting randomly and multiple times before the > next block increases the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in > a simple setup broadcast the HTLC-preimage to all the network mempools, the > honest lightning node is able to catch on the flight the unconfirmed > HTLC-preimage, before its subsequent mempool replacement. The preimage can > be extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a cooperative message or go on-chain with it to > claim the accepted HTLC output. > > Implemented and deployed by Eclair and LND. > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid > by the attacker, a risk of the HTLC-preimage being detected or discovered > by the honest lightning node, or the HTLC-timeout to slip in a winning > block template. Bumping the default CLTV delta hardens the odds of success > of a simple replacement cycling attack. > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > ## Affected Bitcoin Protocols and Applications > > From my understanding the following list of Bitcoin protocols and > applications could be affected by new denial-of-service vectors under some > level of network mempools congestion. Neither tests or advanced review of > specifications (when available) has been conducted for each of them: > - on-chain DLCs > - coinjoins > - payjoins > - wallets with time-sensitive paths > - peerswap and submarine swaps > - batch payouts > - transaction "accelerators" > > Inviting their developers, maintainers and operators to investigate how > replacement cycling attacks might disrupt their in-mempool chain of > transactions, or fee-bumping flows at the shortest delay. Simple flows and > non-multi-party transactions should not be affected to the best of my > understanding. > > ## Open Problems: Package Malleability > > Pinning attacks have been known for years as a practical vector to > compromise lightning channels funds safety, under different scenarios (cf. > current bip331's motivation section). Mitigations at the mempool level have > been designed, discussed and are under implementation by the community > (ancestor package relay + nverrsion=3 policy). Ideally, they should > constraint a pinning attacker to always attach a high feerate package > (commitment + CPFP) to replace the honest package, or allow a honest > lightning node to overbid a malicious pinning package and get its > time-sensitive transaction optimistically included in the chain. > > Replacement cycling attack seem to offer a new way to neutralize the > design goals of package relay and its companion nversion=3 policy, where an > attacker package RBF a honest package out of the mempool to subsequently > double-spend its own high-fee child with a transaction unrelated to the > channel. As the remaining commitment transaction is pre-signed with a > minimal relay fee, it can be evicted out of the mempool. > > A functional test exercising a simple replacement cycling of a lightning > channel commitment transaction on top of the nversion=3 code branch is > available: > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > ## Discovery > > In 2018, the issue of static fees for pre-signed lightning transactions is > made more widely known, the carve-out exemption in mempool rules to > mitigate in-mempool package limits pinning and the anchor output pattern > are proposed. > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued > discussion of the anchor output pattern as a dynamic fee-bumping method. > > In 2020, draft of anchor output submitted to the bolts. Initial finding of > economic pinning against lightning commitment and second-stage HTLC > transactions. Subsequent discussions of a preimage-overlay network or > package-relay as mitigations. Public call made to inquiry more on potential > other transaction-relay jamming attacks affecting lightning. > > In 2021, initial work in bitcoin core 22.0 of package acceptance. > Continued discussion of the pinning attacks and shortcomings of current > mempool rules during community-wide online workshops. Later the year, in > light of all issues for bitcoin second-layers, a proposal is made about > killing the mempool. > > In 2022, bip proposed for package relay and new proposed v3 policy design > proposed for a review and implementation. Mempoolfullrbf is supported in > bitcoin core 24.0 and conceptual questions about alignment of mempool rules > w.r.t miners incentives are investigated. > > Along this year 2022, eltoo lightning channels design are discussed, > implemented and reviewed. In this context and after discussions on mempool > anti-DoS rules, I discovered this new replacement cycling attack was > affecting deployed lightning channels and immediately reported the finding > to some bitcoin core developers and lightning maintainers. > > ## Timeline > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg > Sanders and Gloria Zhao > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, > Matt Corallo and Olaoluwa Osuntunkun > - 2022-12-23: Sharing to Eugene Siegel (LND) > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning > potential affected projects) > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers > issuers) and initial proposal of an early public disclosure > - 2022-01-19: Collection of analysis if other second-layers and > multi-party applications affected. LN mitigations development starts. > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > - 2023-06-20: LN mitigations implemented and progressively released. Week > of the 16 october proposed for full disclosure. > - 2023-08-10: CVEs assigned by MITRE > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling > attack existence to security@bitcoincore•org. > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / > CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks > > ## Conclusion > > Despite the line of mitigations adopted and deployed by current major > lightning implementations, I believe replacement cycling attacks are still > practical for advanced attackers. Beyond this new attack might come as a > way to partially or completely defeat some of the pinning mitigations which > have been working for years as a community. > > As of today, it is uncertain to me if lightning is not affected by a more > severe long-term package malleability critical security issue under current > consensus rules, and if any other time-sensitive multi-party protocol, > designed or deployed isn't de facto affected too (loss of funds or denial > of service). > > Assuming analysis on package malleability is correct, it is unclear to me > if it can be corrected by changes in replacement / eviction rules or > mempool chain of transactions processing strategy. Inviting my technical > peers and the bitcoin community to look more on this issue, including to > dissent. I'll be the first one pleased if I'm fundamentally wrong on those > issues, or if any element has not been weighted with the adequate technical > accuracy it deserves. > > Do not trust, verify. All mistakes and opinions are my own. > > Antoine > > "meet with Triumph and Disaster. And treat those two impostors just the > same" - K. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
Quoted text (1 line)
> I think if you want people to understand this exploit, you need to
explain in more detail how we have a situation where two different parties can spend the same HTLC txout, without the first party having the right to spend it via their knowledge of the HTLC-preimage.
If I'm correctly understanding your question, you're asking why we have a situation where the spend of a HTLC output can be in competition between 2 channel counterparties.
LN commitment transactions have offered HTLC outputs where a counterparty Alice is pledging to her other counterparty Caroll the HTLC amount in exchange of a preimage (and Caroll signature).
After the expiration of the HTLC timelock, if the HTLC has not been claimed on-chain by Caroll, Alice can claim it back with her signature (and the pre-exchanged Caroll signature).
The exploit works actually in Caroll leveraging her HTLC-preimage transaction as a replace-by-fee of Alice's HTLC-timeout _after_ the expiration of the timelock, the HTLC-preimage transaction staying consensus valid.
There is nothing in the mempool policy rules that prevent this Caroll's HTLC-preimage of being replaced subsequently, once Alice's HTLC-timeout has been evicted out the mempool.
The HTLC output does not have any spend candidate remaining for this block. If this replacement can be successfully repeated until an inbound HTLC on another Alice's channel expires, the "forward" HTLC can be double-spent.
Le lun. 16 oct. 2023 à 20:13, Peter Todd <pete@petertodd•org> a écrit :
Quoted text (19 lines)
> > > On October 16, 2023 6:57:36 PM GMT+02:00, Antoine Riard via bitcoin-dev < > bitcoin-dev@lists•linuxfoundation•org> wrote: > >(cross-posting mempool issues identified are exposing lightning chan to > >loss of funds risks, other multi-party bitcoin apps might be affected) > > > >As the HTLC-preimage spends an unconfirmed input that was already included > >in the unconfirmed and unrelated child transaction (rule 2), pays an > >absolute higher fee of at least the sum paid by the HTLC-timeout and child > >transaction (rule 3) and the HTLC-preimage feerate is greater than all > >directly conflicting transactions (rule 6), the replacement is accepted. > >The honest HTLC-timeout is evicted out of the mempool. > > I think if you want people to understand this exploit, you need to explain > in more detail how we have a situation where two different parties can > spend the same HTLC txout, without the first party having the right to > spend it via their knowledge of the HTLC-preimage. >
Quoted text (7 lines)
> ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being usually broadcast by lightning nodes only once every block, or less the replacement cycling malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a cooperative message or go on-chain with it to claim the accepted HTLC output.
Hi Antoine,
thanks for this detailed explanation. This class of pinning attacks sound not too unlikely especially if the attacker targets channels with high capacity and very loose channel policies (allowing the full htlc amount to be the channel capacity). Could you add more details about the attack you observed on mainnet ? How did you monitor the chain, are the some tools available I can run in parallel to my lightning software to record this kind of suspicious behaviour (which did you use)? What's also worth mentioning here is that you do not really have to control 2 neighbouring nodes to target your victim. If you can cycle the attack on the tail side and delay the confirmation of the htlc-timeout covenant the peer at the front (incoming link) of the victim will force-close the channel and claim his timeout-path in the same way (canceling back the initial htlc amount to the attackers initial node).
Apart from that I think one can even introduce some kind of feebumping race between the victim and the attacker on the tail side of the attack making the attack even more costly. I think currently when lightning nodes see the preimage in the mempool (during the time where they already can spend the same output with the timeout-covenant) we are honest and just extract the preimage and don't try to race this tx output. So we maybe should start feebumping this output if we end up in this scenario? If we see the preimage and can also claim this output via the htlc-timeout path, we should aggressively fee-bump (racing this output) our htlc-output in addition to grabbing the preimage and claiming it on the incoming. This is only feasible with anchor channels where we can add fees to the htlc-covenant. This would make the attack more costly for a peer when he knows that we use fees up to 50% of the htlc value. When you cycle this 144 times you will be at a heavy loss trying to steal this htlc.
I would add another mitigation to the list for node runners to restrict the amount and number of HTLCs for big channels to unknown peers. It quickly comes with a loss when the HTLCs the attacker tries to steal are small.
Kind regards,
ziggie
------- Original Message ------- On Monday, October 16th, 2023 at 18:57, Antoine Riard <antoine.riard@gmail•com> wrote:
Quoted text (131 lines)
> (cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo payment channels and incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack affecting lightning channels was discovered. > > After careful analysis, it turns out this attack is practical and immediately exposed lightning routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output channels. A potential exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major lightning implementations during the last months. > > Please find attached the release numbers, where the mitigations should be present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported in the wild since the last ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle). > > It is understood that a simple replacement cycling attack does not demand privileged capabilities from an attacker (e.g no low-hashrate power) and only access to basic bitcoin and lightning software. Yet I still think executing such an attack successfully requests a fair amount of bitcoin technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the mitigations deployed are robust enough in face of advanced replacement cycling attackers, especially ones able to combine different classes of transaction-relay jamming such as pinnings or vetted with more privileged capabilities. > > Please find a list of potential affected bitcoin applications in this full disclosure report using bitcoin script timelocks or multi-party transactions, albeit no immediate security risk exposure as severe as the ones affecting lightning has been identified. Only cursory review of non-lightning applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the lightning network: > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted HTLC on incoming link and offered HTLC on outgoing link) should settle the outgoing state with either a success or timeout before the incoming state timelock becomes final and an asymmetric defavorable settlement might happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" section 2.3 for a classical exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to a loss of fund risk where the offered HTLC is spent by the outgoing link counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this exact interval value being an implementation and node policy setting. As a minimal value, the specification recommends 34 blocks of interval. If the timelock expiration I of the inbound HTLC is equal to 100 from chain tip, the timelock expiration O of the outbound HTLC must be equal to 66 blocks from chain tip, giving a reasonable buffer of reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the outgoing link negotiated with the counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the lightning node should broadcast its commitment transaction. Once the commitment is confirmed (if anchor and the 1 CSV encumbrance is present), the lightning node broadcasts and confirms its HTLC-timeout before I height is reached. > > Here enter a replacement cycling attack. A malicious channel counterparty can broadcast its HTLC-preimage transaction with a higher absolute fee and higher feerate than the honest HTLC-timeout of the victim lightning node and triggers a replacement. Both for legacy and anchor output channels, a HTLC-preimage on a counterparty commitment transaction is malleable, i.e additional inputs or outputs can be added. The HTLC-preimage spends an unconfirmed and unrelated to the channel parent transaction M and conflicts its child. > > As the HTLC-preimage spends an unconfirmed input that was already included in the unconfirmed and unrelated child transaction (rule 2), pays an absolute higher fee of at least the sum paid by the HTLC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is greater than all directly conflicting transactions (rule 6), the replacement is accepted. The honest HTLC-timeout is evicted out of the mempool. > > In an ulterior move, the malicious counterparty can replace the parent transaction itself with another candidate N satisfying the replacement rules, triggering the eviction of the malicious HTLC-preimage from the mempool as it was a child of the parent T. > > There is no spending candidate of the offered HTLC output for the current block laying in network mempools. > > This replacement cycling tricks can be repeated for each rebroadcast attempt of the HTLC-timeout by the honest lightning node until expiration of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout is broadcast by the counterparty's on the incoming link in collusion with the one on the outgoing link broadcasting its own HTLC-preimage. > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > As a notable factor impacting the success of the attack, a lightning node's honest HTLC-timeout might be included in the block template of the miner winning the block race and therefore realizes a spent of the offered output. In practice, a replacement cycling attack might over-connect to miners' mempools and public reachable nodes to succeed in a fast eviction of the HTLC-timeout by its HTLC-preimage. As this latter transaction can come with a better ancestor-score, it should be picked up on the flight by economically competitive miners. > > A functional test exercising a simple replacement cycling of a HTLC transaction on bitcoin core mempool is available: > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being usually broadcast by lightning nodes only once every block, or less the replacement cycling malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a cooperative message or go on-chain with it to claim the accepted HTLC output. > > Implemented and deployed by Eclair and LND. > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid by the attacker, a risk of the HTLC-preimage being detected or discovered by the honest lightning node, or the HTLC-timeout to slip in a winning block template. Bumping the default CLTV delta hardens the odds of success of a simple replacement cycling attack. > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > ## Affected Bitcoin Protocols and Applications > > From my understanding the following list of Bitcoin protocols and applications could be affected by new denial-of-service vectors under some level of network mempools congestion. Neither tests or advanced review of specifications (when available) has been conducted for each of them: > - on-chain DLCs > - coinjoins > - payjoins > - wallets with time-sensitive paths > - peerswap and submarine swaps > - batch payouts > - transaction "accelerators" > > Inviting their developers, maintainers and operators to investigate how replacement cycling attacks might disrupt their in-mempool chain of transactions, or fee-bumping flows at the shortest delay. Simple flows and non-multi-party transactions should not be affected to the best of my understanding. > > ## Open Problems: Package Malleability > > Pinning attacks have been known for years as a practical vector to compromise lightning channels funds safety, under different scenarios (cf. current bip331's motivation section). Mitigations at the mempool level have been designed, discussed and are under implementation by the community (ancestor package relay + nverrsion=3 policy). Ideally, they should constraint a pinning attacker to always attach a high feerate package (commitment + CPFP) to replace the honest package, or allow a honest lightning node to overbid a malicious pinning package and get its time-sensitive transaction optimistically included in the chain. > > Replacement cycling attack seem to offer a new way to neutralize the design goals of package relay and its companion nversion=3 policy, where an attacker package RBF a honest package out of the mempool to subsequently double-spend its own high-fee child with a transaction unrelated to the channel. As the remaining commitment transaction is pre-signed with a minimal relay fee, it can be evicted out of the mempool. > > A functional test exercising a simple replacement cycling of a lightning channel commitment transaction on top of the nversion=3 code branch is available: > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > ## Discovery > > In 2018, the issue of static fees for pre-signed lightning transactions is made more widely known, the carve-out exemption in mempool rules to mitigate in-mempool package limits pinning and the anchor output pattern are proposed. > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued discussion of the anchor output pattern as a dynamic fee-bumping method. > > In 2020, draft of anchor output submitted to the bolts. Initial finding of economic pinning against lightning commitment and second-stage HTLC transactions. Subsequent discussions of a preimage-overlay network or package-relay as mitigations. Public call made to inquiry more on potential other transaction-relay jamming attacks affecting lightning. > > In 2021, initial work in bitcoin core 22.0 of package acceptance. Continued discussion of the pinning attacks and shortcomings of current mempool rules during community-wide online workshops. Later the year, in light of all issues for bitcoin second-layers, a proposal is made about killing the mempool. > > In 2022, bip proposed for package relay and new proposed v3 policy design proposed for a review and implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and conceptual questions about alignment of mempool rules w.r.t miners incentives are investigated. > > Along this year 2022, eltoo lightning channels design are discussed, implemented and reviewed. In this context and after discussions on mempool anti-DoS rules, I discovered this new replacement cycling attack was affecting deployed lightning channels and immediately reported the finding to some bitcoin core developers and lightning maintainers. > > ## Timeline > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Zhao > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, Matt Corallo and Olaoluwa Osuntunkun > - 2022-12-23: Sharing to Eugene Siegel (LND) > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning potential affected projects) > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers issuers) and initial proposal of an early public disclosure > - 2022-01-19: Collection of analysis if other second-layers and multi-party applications affected. LN mitigations development starts. > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > - 2023-06-20: LN mitigations implemented and progressively released. Week of the 16 october proposed for full disclosure. > - 2023-08-10: CVEs assigned by MITRE > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling attack existence to security@bitcoincore•org. > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks > > ## Conclusion > > Despite the line of mitigations adopted and deployed by current major lightning implementations, I believe replacement cycling attacks are still practical for advanced attackers. Beyond this new attack might come as a way to partially or completely defeat some of the pinning mitigations which have been working for years as a community. > > As of today, it is uncertain to me if lightning is not affected by a more severe long-term package malleability critical security issue under current consensus rules, and if any other time-sensitive multi-party protocol, designed or deployed isn't de facto affected too (loss of funds or denial of service). > > Assuming analysis on package malleability is correct, it is unclear to me if it can be corrected by changes in replacement / eviction rules or mempool chain of transactions processing strategy. Inviting my technical peers and the bitcoin community to look more on this issue, including to dissent. I'll be the first one pleased if I'm fundamentally wrong on those issues, or if any element has not been weighted with the adequate technical accuracy it deserves. > > Do not trust, verify. All mistakes and opinions are my own. > > Antoine > > "meet with Triumph and Disaster. And treat those two impostors just the same" - K.
Good morning Antoine et al.,
Let me try to rephrase the core of the attack.
There exists these nodes on the LN (letters `A`, `B`, and `C` are nodes, `==` are channels):
A ===== B ===== C
`A` routes `A->B->C`.
The timelocks, for example, could be:
A->B timeelock = 144 B->C timelock = 100
The above satisfies the LN BOLT requirements, as long as `B` has a `cltv_expiry_delta` of 44 or lower.
After `B` forwards the HTLC `B->C`, C suddenly goes offline, and all the signed transactions --- commitment transaction and HTLC-timeout transactions --- are "stuck" at the feerate at the time.
At block height 100, `B` notices the `B->C` HTLC timelock is expired without `C` having claimed it, so `B` forces the `B====C` channel onchain. However, onchain feerates have risen and the commitment transaction and HTLC-timeout transaction do not confirm.
In the mean time, `A` is still online with `B` and updates the onchain fees of the `A====B` channel pre-signed transactions (commitment tx and HTLC-timeout tx) to the latest.
At block height 144, `B` is still not able to claim the `A->B` HTLC, so `A` drops the `A====B` channel onchain. As the fees are up-to-date, this confirms immediately and `A` is able to recover the HTLC funds. However, the feerates of the `B====C` pre-signed transactions remain at the old, uncompetitive feerates.
At this point, `C` broadcasts an HTLC-success transaction with high feerates that CPFPs the commitment tx. However, it replaces the HTLC-timeout transaction, which is at the old, low feerate. `C` is thus able to get the value of the HTLC, but `B` is now no longer able to use the knowledge of the preimage, as its own incoming HTLC was already confirmed as claimed by `A`.
Is the above restatement accurate?
----
Let me also explain to non-Lightning experts why HTLC-timeout is presigned in this case and why `B` cannot feebump it.
In the Poon-Dryja mechanism, the HTLCs are "infected" by the Poon-Dryja penalty case, and are not plain HTLCs.
A plain HTLC offerred by `B` to `C` would look like this:
(B && OP_CLTV) || (C && OP_HASH160)
However, on the commitment transaction held by `B`, it would be infected by the penalty case in this way:
(B && C && OP_CLTV) || (C && OP_HASH160) || (C && revocation)
There are two changes:
* The addition of a revocation branch `C && revocation`. * The branch claimable by `B` in the "plain" HTLC (`B && OP_CLTV`) also includes `C`.
These are necessary in case `B` tries to cheat and this HTLC is on an old, revoked transaction. If the revoked transaction is *really* old, the `OP_CLTV` would already impose a timelock far in the past. This means that a plain `B && OP_CLTV` branch can be claimed by `B` if it retained this very old revoked transaction.
To prevent that, `C` is added to the `B && OP_CLTV` branch. We also introduce an HTLC-timeout transaction, which spends the `B && C && OP_CLTV` branch, and outputs to:
(B && OP_CSV) || (C && revocation)
Thus, even if `B` held onto a very old revoked commitment transaction and attempts to spend the timelock branch (because the `OP_CLTV` is for an old blockheight), it still has to contend with a new output with a *relative* timelock.
Unfortunately, this means that the HTLC-timeout transaction is pre-signed, and has a specific feerate. In order to change the feerate, both `B` and `C` have to agree to re-sign the HTLC-timeout transaction at the higher feerate.
However, the HTLC-success transaction in this case spends the plain `(C && OP_HASH160)` branch, which only involves `C`. This allows `C` to feebump the HTLC-success transaction arbitrarily even if `B` does not cooperate.
While anchor outputs can be added to the HTLC-timeout transaction as well, `C` has a greater advantage here due to being able to RBF the HTLC-timeout out of the way (1 transaction), while `B` has to get both HTLC-timeout and a CPFP-RBF of the anchor output of the HTLC-timeout transaction (2 transactions). `C` thus requires a smaller fee to achieve a particular feerate due to having to push a smaller number of bytes compared to `B`.
Regards, ZmnSCPxj
Hi Ziggie,
Quoted text (1 line)
> thanks for this detailed explanation. This class of pinning attacks sound
not too unlikely especially if the attacker targets channels with high capacity and very loose channel policies (allowing the full htlc
Quoted text (1 line)
> amount to be the channel capacity). Could you add more details about the
attack you observed on mainnet ? How did you monitor the chain, are the some tools available I can run in parallel to my
Quoted text (1 line)
> lightning software to record this kind of suspicious behaviour (which did
you use)?
Just to give a clarification no _such_ attack has been observed on mainnet, since I and other LN maintainers have been aware of this issue. If there is a confusion on the disclosure mail thanks to point to it, I'll correct it.
We discussed privately to experiment a demo attack in restrained dev circles, like we have done in the past for some LN sec issues. We have conducted one so far, multiple scenarios to look at.
I confirm the risk of exposure if an attacker targets channels with high capacity and loose channel policies. Note there is no way to configure the cap for the total value of outbound HTLC in-flight, which is the flow affected.
If you would like to observe the existence of such an attack happening, look at your mempool logs and the amount of HTLC output being systematically conflicted out with the following sequence (HTLC-timeout - HTLC-preimage - HTLC-timeout - ...).
As an observation note, this is not akin to a pinning attack, as there is no "honest" or "malicious" transaction pinned in network mempools. And it can happen without network mempools congestion.
Quoted text (1 line)
> What's also worth mentioning here is that you do not really have to
control 2 neighbouring nodes to target your victim. If you can cycle the attack on the tail side and delay the confirmation of the htlc- timeout covenant the peer at the front (incoming link) of the victim will force-close the channel and claim his timeout-path in the same way (canceling back the initial htlc amount to the attackers initial node).
I think this is a behavior worthy of testing.
Quoted text (1 line)
> Apart from that I think one can even introduce some kind of feebumping
race between the victim and the attacker on the tail side of the attack making the attack even more costly. I think
Quoted text (1 line)
> currently when lightning nodes see the preimage in the mempool (during
the time where they already can spend the same output with the timeout-covenant) we are honest and just extract
Quoted text (1 line)
> the preimage and don't try to race this tx output.
Local-mempool preimage monitoring has been implemented by Eclair for years as a mitigation against old school pinning attacks on second-stage HTLC transactions.
This mechanism has been implemented by LND in the last months, following the report of replacement cycling attacks. As of today this is not implemented by Core-Lightning or LDK.
Quoted text (1 line)
> So we maybe should start feebumping this output if we end up in this
scenario? If we see the preimage and can also claim this output via the htlc-timeout path, we should aggressively fee-bump (racing this output) our htlc-output in addition to grabbing the preimage and claiming it on the incoming. This is only feasible with anchor channels where we can add fees to the htlc-covenant. This would make the attack more costly for a peer when he knows that we use fees up to 50% of the htlc value. When you cycle this 144 times you will be at a heavy loss trying to steal this htlc.
This is the "defensive fee mitigation" proposed in the paper. Coming with some unknown.
Quoted text (1 line)
> I would add another mitigation to the list for node runners to restrict
the amount and number of HTLCs for big channels to unknown peers. It quickly comes with a loss when the HTLCs the attacker tries to steal are small.
See the point above on the lack of way at the spec-level to negotiate cap on the total value of outbound HTLC in-flight.
Le mar. 17 oct. 2023 à 08:21, ziggie1984 <ziggie1984@protonmail•com> a écrit :
Quoted text (355 lines)
> ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits > from the HTLC-timeout being usually broadcast by lightning nodes only once > every block, or less the replacement cycling malicious transactions paid > only equal the sum of the absolute fees paid by the HTLC, adjusted with the > replacement penalty. Rebroadcasting randomly and multiple times before the > next block increases the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in > a simple setup broadcast the HTLC-preimage to all the network mempools, the > honest lightning node is able to catch on the flight the unconfirmed > HTLC-preimage, before its subsequent mempool replacement. The preimage can > be extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a cooperative message or go on-chain with it to > claim the accepted HTLC output. > > > Hi Antoine, > > thanks for this detailed explanation. This class of pinning attacks sound > not too unlikely especially if the attacker targets channels with high > capacity and very loose channel policies (allowing the full htlc amount to > be the channel capacity). Could you add more details about the attack you > observed on mainnet ? How did you monitor the chain, are the some tools > available I can run in parallel to my lightning software to record this > kind of suspicious behaviour (which did you use)? > What's also worth mentioning here is that you do not really have to > control 2 neighbouring nodes to target your victim. If you can cycle the > attack on the tail side and delay the confirmation of the htlc-timeout > covenant the peer at the front (incoming link) of the victim will > force-close the channel and claim his timeout-path in the same way > (canceling back the initial htlc amount to the attackers initial node). > > Apart from that I think one can even introduce some kind of feebumping > race between the victim and the attacker on the tail side of the attack > making the attack even more costly. I think currently when lightning nodes > see the preimage in the mempool (during the time where they already can > spend the same output with the timeout-covenant) we are honest and just > extract the preimage and don't try to race this tx output. So we maybe > should start feebumping this output if we end up in this scenario? If we > see the preimage and can also claim this output via the htlc-timeout path, > we should aggressively fee-bump (racing this output) our htlc-output in > addition to grabbing the preimage and claiming it on the incoming. This is > only feasible with anchor channels where we can add fees to the > htlc-covenant. This would make the attack more costly for a peer when he > knows that we use fees up to 50% of the htlc value. When you cycle this 144 > times you will be at a heavy loss trying to steal this htlc. > > I would add another mitigation to the list for node runners to restrict > the amount and number of HTLCs for big channels to unknown peers. It > quickly comes with a loss when the HTLCs the attacker tries to steal are > small. > > > Kind regards, > > ziggie > > > ------- Original Message ------- > On Monday, October 16th, 2023 at 18:57, Antoine Riard < > antoine.riard@gmail•com> wrote: > > (cross-posting mempool issues identified are exposing lightning chan to > loss of funds risks, other multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and incentives compatibility of the mempool anti-DoS > rules, a new transaction-relay jamming attack affecting lightning channels > was discovered. > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning routing hops carrying HTLC traffic to loss of > funds security risks, both legacy and anchor output channels. A potential > exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major > lightning implementations during the last months. > > Please find attached the release numbers, where the mitigations should be > present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported > in the wild since the last ~10 months or experimented in real-world > conditions on bitcoin mainet, functional test is available exercising the > affected lightning channel against bitcoin core mempool (26.0 release > cycle). > > It is understood that a simple replacement cycling attack does not demand > privileged capabilities from an attacker (e.g no low-hashrate power) and > only access to basic bitcoin and lightning software. Yet I still think > executing such an attack successfully requests a fair amount of bitcoin > technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the > mitigations deployed are robust enough in face of advanced replacement > cycling attackers, especially ones able to combine different classes of > transaction-relay jamming such as pinnings or vetted with more privileged > capabilities. > > Please find a list of potential affected bitcoin applications in this full > disclosure report using bitcoin script timelocks or multi-party > transactions, albeit no immediate security risk exposure as severe as the > ones affecting lightning has been identified. Only cursory review of > non-lightning applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the > lightning network: > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and offered HTLC on outgoing link) should settle the > outgoing state with either a success or timeout before the incoming state > timelock becomes final and an asymmetric defavorable settlement might > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" > section 2.3 for a classical exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to > a loss of fund risk where the offered HTLC is spent by the outgoing link > counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming > link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC > expiration timelock, this exact interval value being an implementation and > node policy setting. As a minimal value, the specification recommends 34 > blocks of interval. If the timelock expiration I of the inbound HTLC is > equal to 100 from chain tip, the timelock expiration O of the outbound HTLC > must be equal to 66 blocks from chain tip, giving a reasonable buffer of > reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the counterparty (either > `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the > lightning node should broadcast its commitment transaction. Once the > commitment is confirmed (if anchor and the 1 CSV encumbrance is present), > the lightning node broadcasts and confirms its HTLC-timeout before I height > is reached. > > Here enter a replacement cycling attack. A malicious channel counterparty > can broadcast its HTLC-preimage transaction with a higher absolute fee and > higher feerate than the honest HTLC-timeout of the victim lightning node > and triggers a replacement. Both for legacy and anchor output channels, a > HTLC-preimage on a counterparty commitment transaction is malleable, i.e > additional inputs or outputs can be added. The HTLC-preimage spends an > unconfirmed and unrelated to the channel parent transaction M and conflicts > its child. > > As the HTLC-preimage spends an unconfirmed input that was already included > in the unconfirmed and unrelated child transaction (rule 2), pays an > absolute higher fee of at least the sum paid by the HTLC-timeout and child > transaction (rule 3) and the HTLC-preimage feerate is greater than all > directly conflicting transactions (rule 6), the replacement is accepted. > The honest HTLC-timeout is evicted out of the mempool. > > In an ulterior move, the malicious counterparty can replace the parent > transaction itself with another candidate N satisfying the replacement > rules, triggering the eviction of the malicious HTLC-preimage from the > mempool as it was a child of the parent T. > > There is no spending candidate of the offered HTLC output for the current > block laying in network mempools. > > This replacement cycling tricks can be repeated for each rebroadcast > attempt of the HTLC-timeout by the honest lightning node until expiration > of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout > is broadcast by the counterparty's on the incoming link in collusion with > the one on the outgoing link broadcasting its own HTLC-preimage. > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > As a notable factor impacting the success of the attack, a lightning > node's honest HTLC-timeout might be included in the block template of the > miner winning the block race and therefore realizes a spent of the offered > output. In practice, a replacement cycling attack might over-connect to > miners' mempools and public reachable nodes to succeed in a fast eviction > of the HTLC-timeout by its HTLC-preimage. As this latter transaction can > come with a better ancestor-score, it should be picked up on the flight by > economically competitive miners. > > A functional test exercising a simple replacement cycling of a HTLC > transaction on bitcoin core mempool is available: > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits > from the HTLC-timeout being usually broadcast by lightning nodes only once > every block, or less the replacement cycling malicious transactions paid > only equal the sum of the absolute fees paid by the HTLC, adjusted with the > replacement penalty. Rebroadcasting randomly and multiple times before the > next block increases the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in > a simple setup broadcast the HTLC-preimage to all the network mempools, the > honest lightning node is able to catch on the flight the unconfirmed > HTLC-preimage, before its subsequent mempool replacement. The preimage can > be extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a cooperative message or go on-chain with it to > claim the accepted HTLC output. > > Implemented and deployed by Eclair and LND. > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid > by the attacker, a risk of the HTLC-preimage being detected or discovered > by the honest lightning node, or the HTLC-timeout to slip in a winning > block template. Bumping the default CLTV delta hardens the odds of success > of a simple replacement cycling attack. > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > ## Affected Bitcoin Protocols and Applications > > From my understanding the following list of Bitcoin protocols and > applications could be affected by new denial-of-service vectors under some > level of network mempools congestion. Neither tests or advanced review of > specifications (when available) has been conducted for each of them: > - on-chain DLCs > - coinjoins > - payjoins > - wallets with time-sensitive paths > - peerswap and submarine swaps > - batch payouts > - transaction "accelerators" > > Inviting their developers, maintainers and operators to investigate how > replacement cycling attacks might disrupt their in-mempool chain of > transactions, or fee-bumping flows at the shortest delay. Simple flows and > non-multi-party transactions should not be affected to the best of my > understanding. > > ## Open Problems: Package Malleability > > Pinning attacks have been known for years as a practical vector to > compromise lightning channels funds safety, under different scenarios (cf. > current bip331's motivation section). Mitigations at the mempool level have > been designed, discussed and are under implementation by the community > (ancestor package relay + nverrsion=3 policy). Ideally, they should > constraint a pinning attacker to always attach a high feerate package > (commitment + CPFP) to replace the honest package, or allow a honest > lightning node to overbid a malicious pinning package and get its > time-sensitive transaction optimistically included in the chain. > > Replacement cycling attack seem to offer a new way to neutralize the > design goals of package relay and its companion nversion=3 policy, where an > attacker package RBF a honest package out of the mempool to subsequently > double-spend its own high-fee child with a transaction unrelated to the > channel. As the remaining commitment transaction is pre-signed with a > minimal relay fee, it can be evicted out of the mempool. > > A functional test exercising a simple replacement cycling of a lightning > channel commitment transaction on top of the nversion=3 code branch is > available: > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > ## Discovery > > In 2018, the issue of static fees for pre-signed lightning transactions is > made more widely known, the carve-out exemption in mempool rules to > mitigate in-mempool package limits pinning and the anchor output pattern > are proposed. > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued > discussion of the anchor output pattern as a dynamic fee-bumping method. > > In 2020, draft of anchor output submitted to the bolts. Initial finding of > economic pinning against lightning commitment and second-stage HTLC > transactions. Subsequent discussions of a preimage-overlay network or > package-relay as mitigations. Public call made to inquiry more on potential > other transaction-relay jamming attacks affecting lightning. > > In 2021, initial work in bitcoin core 22.0 of package acceptance. > Continued discussion of the pinning attacks and shortcomings of current > mempool rules during community-wide online workshops. Later the year, in > light of all issues for bitcoin second-layers, a proposal is made about > killing the mempool. > > In 2022, bip proposed for package relay and new proposed v3 policy design > proposed for a review and implementation. Mempoolfullrbf is supported in > bitcoin core 24.0 and conceptual questions about alignment of mempool rules > w.r.t miners incentives are investigated. > > Along this year 2022, eltoo lightning channels design are discussed, > implemented and reviewed. In this context and after discussions on mempool > anti-DoS rules, I discovered this new replacement cycling attack was > affecting deployed lightning channels and immediately reported the finding > to some bitcoin core developers and lightning maintainers. > > ## Timeline > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg > Sanders and Gloria Zhao > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, > Matt Corallo and Olaoluwa Osuntunkun > - 2022-12-23: Sharing to Eugene Siegel (LND) > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning > potential affected projects) > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers > issuers) and initial proposal of an early public disclosure > - 2022-01-19: Collection of analysis if other second-layers and > multi-party applications affected. LN mitigations development starts. > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > - 2023-06-20: LN mitigations implemented and progressively released. Week > of the 16 october proposed for full disclosure. > - 2023-08-10: CVEs assigned by MITRE > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling > attack existence to security@bitcoincore•org. > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / > CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks > > ## Conclusion > > Despite the line of mitigations adopted and deployed by current major > lightning implementations, I believe replacement cycling attacks are still > practical for advanced attackers. Beyond this new attack might come as a > way to partially or completely defeat some of the pinning mitigations which > have been working for years as a community. > > As of today, it is uncertain to me if lightning is not affected by a more > severe long-term package malleability critical security issue under current > consensus rules, and if any other time-sensitive multi-party protocol, > designed or deployed isn't de facto affected too (loss of funds or denial > of service). > > Assuming analysis on package malleability is correct, it is unclear to me > if it can be corrected by changes in replacement / eviction rules or > mempool chain of transactions processing strategy. Inviting my technical > peers and the bitcoin community to look more on this issue, including to > dissent. I'll be the first one pleased if I'm fundamentally wrong on those > issues, or if any element has not been weighted with the adequate technical > accuracy it deserves. > > Do not trust, verify. All mistakes and opinions are my own. > > Antoine > > "meet with Triumph and Disaster. And treat those two impostors just the > same" - K. > > >
Hi Zeeman,
Quoted text (1 line)
> At block height 100, `B` notices the `B->C` HTLC timelock is expired
without `C` having claimed it, so `B` forces the `B====C` channel onchain.
Quoted text (1 line)
> However, onchain feerates have risen and the commitment transaction and
HTLC-timeout transaction do not confirm.
This is not that the HTLC-timeout does not confirm. It is replaced in cycles by C's HTLC-preimage which is still valid after `B->C` HTLC timelock has expired. And this HTLC-preimage is subsequently replaced itself.
See the test here: https://github.com/ariard/bitcoin/commit/19d61fa8cf22a5050b51c4005603f43d72f1efcf
Quoted text (1 line)
> At block height 144, `B` is still not able to claim the `A->B` HTLC, so
`A` drops the `A====B` channel onchain.
Quoted text (1 line)
> As the fees are up-to-date, this confirms immediately and `A` is able to
recover the HTLC funds.
Quoted text (1 line)
> However, the feerates of the `B====C` pre-signed transactions remain at
the old, uncompetitive feerates.
This is correct that A tries to recover the HTLC funds on the `A===B` channel.
However, there is no need to consider the fee rates nor mempool congestion as the exploit lays on the replacement mechanism itself (in simple scenario).
Quoted text (1 line)
> At this point, `C` broadcasts an HTLC-success transaction with high
feerates that CPFPs the commitment tx.
Quoted text (1 line)
> However, it replaces the HTLC-timeout transaction, which is at the old,
low feerate.
Quoted text (1 line)
> `C` is thus able to get the value of the HTLC, but `B` is now no longer
able to use the knowledge of the preimage, as its own incoming HTLC was already confirmed as claimed by `A`.
This is correct that `C` broadcasts an HTLC-success transaction at block height 144.
However `C` broadcasts this high feerate transaction at _every block_ between blocks 100 and 144 to replace B's HTLC-timeout transaction.
Quoted text (1 line)
> Let me also explain to non-Lightning experts why HTLC-timeout is
presigned in this case and why `B` cannot feebump it.
Note `B` can feebump the HTLC-timeout for anchor output channels thanks to sighash_single | anyonecanpay on C's signature.
Le mar. 17 oct. 2023 à 11:34, ZmnSCPxj <ZmnSCPxj@protonmail•com> a écrit :
Quoted text (111 lines)
> Good morning Antoine et al., > > Let me try to rephrase the core of the attack. > > There exists these nodes on the LN (letters `A`, `B`, and `C` are nodes, > `==` are channels): > > A ===== B ===== C > > `A` routes `A->B->C`. > > The timelocks, for example, could be: > > A->B timeelock = 144 > B->C timelock = 100 > > The above satisfies the LN BOLT requirements, as long as `B` has a > `cltv_expiry_delta` of 44 or lower. > > After `B` forwards the HTLC `B->C`, C suddenly goes offline, and all the > signed transactions --- commitment transaction and HTLC-timeout > transactions --- are "stuck" at the feerate at the time. > > At block height 100, `B` notices the `B->C` HTLC timelock is expired > without `C` having claimed it, so `B` forces the `B====C` channel onchain. > However, onchain feerates have risen and the commitment transaction and > HTLC-timeout transaction do not confirm. > > In the mean time, `A` is still online with `B` and updates the onchain > fees of the `A====B` channel pre-signed transactions (commitment tx and > HTLC-timeout tx) to the latest. > > At block height 144, `B` is still not able to claim the `A->B` HTLC, so > `A` drops the `A====B` channel onchain. > As the fees are up-to-date, this confirms immediately and `A` is able to > recover the HTLC funds. > However, the feerates of the `B====C` pre-signed transactions remain at > the old, uncompetitive feerates. > > At this point, `C` broadcasts an HTLC-success transaction with high > feerates that CPFPs the commitment tx. > However, it replaces the HTLC-timeout transaction, which is at the old, > low feerate. > `C` is thus able to get the value of the HTLC, but `B` is now no longer > able to use the knowledge of the preimage, as its own incoming HTLC was > already confirmed as claimed by `A`. > > Is the above restatement accurate? > > ---- > > Let me also explain to non-Lightning experts why HTLC-timeout is presigned > in this case and why `B` cannot feebump it. > > In the Poon-Dryja mechanism, the HTLCs are "infected" by the Poon-Dryja > penalty case, and are not plain HTLCs. > > A plain HTLC offerred by `B` to `C` would look like this: > > (B && OP_CLTV) || (C && OP_HASH160) > > However, on the commitment transaction held by `B`, it would be infected > by the penalty case in this way: > > (B && C && OP_CLTV) || (C && OP_HASH160) || (C && revocation) > > There are two changes: > > * The addition of a revocation branch `C && revocation`. > * The branch claimable by `B` in the "plain" HTLC (`B && OP_CLTV`) also > includes `C`. > > These are necessary in case `B` tries to cheat and this HTLC is on an old, > revoked transaction. > If the revoked transaction is *really* old, the `OP_CLTV` would already > impose a timelock far in the past. > This means that a plain `B && OP_CLTV` branch can be claimed by `B` if it > retained this very old revoked transaction. > > To prevent that, `C` is added to the `B && OP_CLTV` branch. > We also introduce an HTLC-timeout transaction, which spends the `B && C && > OP_CLTV` branch, and outputs to: > > (B && OP_CSV) || (C && revocation) > > Thus, even if `B` held onto a very old revoked commitment transaction and > attempts to spend the timelock branch (because the `OP_CLTV` is for an old > blockheight), it still has to contend with a new output with a *relative* > timelock. > > Unfortunately, this means that the HTLC-timeout transaction is pre-signed, > and has a specific feerate. > In order to change the feerate, both `B` and `C` have to agree to re-sign > the HTLC-timeout transaction at the higher feerate. > > However, the HTLC-success transaction in this case spends the plain `(C && > OP_HASH160)` branch, which only involves `C`. > This allows `C` to feebump the HTLC-success transaction arbitrarily even > if `B` does not cooperate. > > While anchor outputs can be added to the HTLC-timeout transaction as well, > `C` has a greater advantage here due to being able to RBF the HTLC-timeout > out of the way (1 transaction), while `B` has to get both HTLC-timeout and > a CPFP-RBF of the anchor output of the HTLC-timeout transaction (2 > transactions). > `C` thus requires a smaller fee to achieve a particular feerate due to > having to push a smaller number of bytes compared to `B`. > > Regards, > ZmnSCPxj >
Quoted text (1 line)
> We have conducted one so far, multiple scenarios to look at.
_*none*_ so far. typo of mine - apologize english is not my native language.
We discussed conducting experiments pre-disclosure in an e-mail of the 11th August 2023.
"If someone is down to setup a "black box" Lightning infra on mainet, I'm game on to exercise the vulnerabilities and mitigations during the coming months and revisit the disclosure date dependent on the learnings."
However as the number of Lightning worldwide experts who have level of knowledge and understandings to take part to experiments is I think restrained to people listed on the disclosure mails _and_ we had other pendings non-disclosed security issues at the time like the ones revealed "fake channel DoS vector" the 23th August 2023, we didn't conduct them.
Le mar. 17 oct. 2023 à 18:47, Antoine Riard <antoine.riard@gmail•com> a écrit :
Quoted text (437 lines)
> Hi Ziggie, > > > thanks for this detailed explanation. This class of pinning attacks > sound not too unlikely especially if the attacker targets channels with > high capacity and very loose channel policies (allowing the full htlc > > amount to be the channel capacity). Could you add more details about the > attack you observed on mainnet ? How did you monitor the chain, are the > some tools available I can run in parallel to my > > lightning software to record this kind of suspicious behaviour (which > did you use)? > > Just to give a clarification no _such_ attack has been observed on > mainnet, since I and other LN maintainers have been aware of this issue. If > there is a confusion on the disclosure mail thanks to point to it, I'll > correct it. > > We discussed privately to experiment a demo attack in restrained dev > circles, like we have done in the past for some LN sec issues. We have > conducted one so far, multiple scenarios to look at. > > I confirm the risk of exposure if an attacker targets channels with high > capacity and loose channel policies. Note there is no way to configure the > cap for the total value of outbound HTLC in-flight, which is the flow > affected. > > If you would like to observe the existence of such an attack happening, > look at your mempool logs and the amount of HTLC output being > systematically conflicted out with the following sequence (HTLC-timeout - > HTLC-preimage - HTLC-timeout - ...). > > As an observation note, this is not akin to a pinning attack, as there is > no "honest" or "malicious" transaction pinned in network mempools. And it > can happen without network mempools congestion. > > > What's also worth mentioning here is that you do not really have to > control 2 neighbouring nodes to target your victim. If you can cycle the > attack on the tail side and delay the confirmation of the htlc- timeout > covenant the peer at the front (incoming link) of the victim will > force-close the channel and claim his timeout-path in the same way > (canceling back the initial htlc amount to the attackers initial node). > > I think this is a behavior worthy of testing. > > > Apart from that I think one can even introduce some kind of feebumping > race between the victim and the attacker on the tail side of the attack > making the attack even more costly. I think > > currently when lightning nodes see the preimage in the mempool (during > the time where they already can spend the same output with the > timeout-covenant) we are honest and just extract > > the preimage and don't try to race this tx output. > > Local-mempool preimage monitoring has been implemented by Eclair for years > as a mitigation against old school pinning attacks on second-stage HTLC > transactions. > > This mechanism has been implemented by LND in the last months, following > the report of replacement cycling attacks. As of today this is not > implemented by Core-Lightning or LDK. > > > So we maybe should start feebumping this output if we end up in this > scenario? If we see the preimage and can also claim this output via the > htlc-timeout path, we should aggressively fee-bump (racing this output) our > htlc-output in addition to grabbing the preimage and claiming it on the > incoming. This is only feasible with anchor channels where we can add fees > to the htlc-covenant. This would make the attack more costly for a peer > when he knows that we use fees up to 50% of the htlc value. When you cycle > this 144 times you will be at a heavy loss trying to steal this htlc. > > This is the "defensive fee mitigation" proposed in the paper. Coming with > some unknown. > > > I would add another mitigation to the list for node runners to restrict > the amount and number of HTLCs for big channels to unknown peers. It > quickly comes with a loss when the HTLCs the attacker tries to steal are > small. > > See the point above on the lack of way at the spec-level to negotiate cap > on the total value of outbound HTLC in-flight. > > Le mar. 17 oct. 2023 à 08:21, ziggie1984 <ziggie1984@protonmail•com> a > écrit : > >> ## Deployed LN mitigations >> >> Aggressive rebroadcasting: As the replacement cycling attacker benefits >> from the HTLC-timeout being usually broadcast by lightning nodes only once >> every block, or less the replacement cycling malicious transactions paid >> only equal the sum of the absolute fees paid by the HTLC, adjusted with the >> replacement penalty. Rebroadcasting randomly and multiple times before the >> next block increases the absolute fee cost for the attacker. >> >> Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >> >> Local-mempool preimage monitoring: As the replacement cycling attacker in >> a simple setup broadcast the HTLC-preimage to all the network mempools, the >> honest lightning node is able to catch on the flight the unconfirmed >> HTLC-preimage, before its subsequent mempool replacement. The preimage can >> be extracted from the second-stage HTLC-preimage and used to fetch the >> off-chain inbound HTLC with a cooperative message or go on-chain with it to >> claim the accepted HTLC output. >> >> >> Hi Antoine, >> >> thanks for this detailed explanation. This class of pinning attacks sound >> not too unlikely especially if the attacker targets channels with high >> capacity and very loose channel policies (allowing the full htlc amount to >> be the channel capacity). Could you add more details about the attack you >> observed on mainnet ? How did you monitor the chain, are the some tools >> available I can run in parallel to my lightning software to record this >> kind of suspicious behaviour (which did you use)? >> What's also worth mentioning here is that you do not really have to >> control 2 neighbouring nodes to target your victim. If you can cycle the >> attack on the tail side and delay the confirmation of the htlc-timeout >> covenant the peer at the front (incoming link) of the victim will >> force-close the channel and claim his timeout-path in the same way >> (canceling back the initial htlc amount to the attackers initial node). >> >> Apart from that I think one can even introduce some kind of feebumping >> race between the victim and the attacker on the tail side of the attack >> making the attack even more costly. I think currently when lightning nodes >> see the preimage in the mempool (during the time where they already can >> spend the same output with the timeout-covenant) we are honest and just >> extract the preimage and don't try to race this tx output. So we maybe >> should start feebumping this output if we end up in this scenario? If we >> see the preimage and can also claim this output via the htlc-timeout path, >> we should aggressively fee-bump (racing this output) our htlc-output in >> addition to grabbing the preimage and claiming it on the incoming. This is >> only feasible with anchor channels where we can add fees to the >> htlc-covenant. This would make the attack more costly for a peer when he >> knows that we use fees up to 50% of the htlc value. When you cycle this 144 >> times you will be at a heavy loss trying to steal this htlc. >> >> I would add another mitigation to the list for node runners to restrict >> the amount and number of HTLCs for big channels to unknown peers. It >> quickly comes with a loss when the HTLCs the attacker tries to steal are >> small. >> >> >> Kind regards, >> >> ziggie >> >> >> ------- Original Message ------- >> On Monday, October 16th, 2023 at 18:57, Antoine Riard < >> antoine.riard@gmail•com> wrote: >> >> (cross-posting mempool issues identified are exposing lightning chan to >> loss of funds risks, other multi-party bitcoin apps might be affected) >> >> Hi, >> >> End of last year (December 2022), amid technical discussions on eltoo >> payment channels and incentives compatibility of the mempool anti-DoS >> rules, a new transaction-relay jamming attack affecting lightning channels >> was discovered. >> >> After careful analysis, it turns out this attack is practical and >> immediately exposed lightning routing hops carrying HTLC traffic to loss of >> funds security risks, both legacy and anchor output channels. A potential >> exploitation plausibly happening even without network mempools congestion. >> >> Mitigations have been designed, implemented and deployed by all major >> lightning implementations during the last months. >> >> Please find attached the release numbers, where the mitigations should be >> present: >> - LDK: v0.0.118 - CVE-2023 -40231 >> - Eclair: v0.9.0 - CVE-2023-40232 >> - LND: v.0.17.0-beta - CVE-2023-40233 >> - Core-Lightning: v.23.08.01 - CVE-2023-40234 >> >> While neither replacement cycling attacks have been observed or reported >> in the wild since the last ~10 months or experimented in real-world >> conditions on bitcoin mainet, functional test is available exercising the >> affected lightning channel against bitcoin core mempool (26.0 release >> cycle). >> >> It is understood that a simple replacement cycling attack does not demand >> privileged capabilities from an attacker (e.g no low-hashrate power) and >> only access to basic bitcoin and lightning software. Yet I still think >> executing such an attack successfully requests a fair amount of bitcoin >> technical know-how and decent preparation. >> >> From my understanding of those issues, it is yet to be determined if the >> mitigations deployed are robust enough in face of advanced replacement >> cycling attackers, especially ones able to combine different classes of >> transaction-relay jamming such as pinnings or vetted with more privileged >> capabilities. >> >> Please find a list of potential affected bitcoin applications in this >> full disclosure report using bitcoin script timelocks or multi-party >> transactions, albeit no immediate security risk exposure as severe as the >> ones affecting lightning has been identified. Only cursory review of >> non-lightning applications has been conducted so far. >> >> There is a paper published summarizing replacement cycling attacks on the >> lightning network: >> >> https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf >> >> ## Problem >> >> A lightning node allows HTLCs forwarding (in bolt3's parlance accepted >> HTLC on incoming link and offered HTLC on outgoing link) should settle the >> outgoing state with either a success or timeout before the incoming state >> timelock becomes final and an asymmetric defavorable settlement might >> happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" >> section 2.3 for a classical exposition of this lightning security property). >> >> Failure to satisfy this settlement requirement exposes a forwarding hop >> to a loss of fund risk where the offered HTLC is spent by the outgoing link >> counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming >> link counterparty's HTLC-timeout. >> >> The specification mandates the incoming HTLC expiration timelock to be >> spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC >> expiration timelock, this exact interval value being an implementation and >> node policy setting. As a minimal value, the specification recommends 34 >> blocks of interval. If the timelock expiration I of the inbound HTLC is >> equal to 100 from chain tip, the timelock expiration O of the outbound HTLC >> must be equal to 66 blocks from chain tip, giving a reasonable buffer of >> reaction to the lightning forwarding node. >> >> In the lack of cooperative off-chain settlement of the HTLC on the >> outgoing link negotiated with the counterparty (either >> `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the >> lightning node should broadcast its commitment transaction. Once the >> commitment is confirmed (if anchor and the 1 CSV encumbrance is present), >> the lightning node broadcasts and confirms its HTLC-timeout before I height >> is reached. >> >> Here enter a replacement cycling attack. A malicious channel counterparty >> can broadcast its HTLC-preimage transaction with a higher absolute fee and >> higher feerate than the honest HTLC-timeout of the victim lightning node >> and triggers a replacement. Both for legacy and anchor output channels, a >> HTLC-preimage on a counterparty commitment transaction is malleable, i.e >> additional inputs or outputs can be added. The HTLC-preimage spends an >> unconfirmed and unrelated to the channel parent transaction M and conflicts >> its child. >> >> As the HTLC-preimage spends an unconfirmed input that was already >> included in the unconfirmed and unrelated child transaction (rule 2), pays >> an absolute higher fee of at least the sum paid by the HTLC-timeout and >> child transaction (rule 3) and the HTLC-preimage feerate is greater than >> all directly conflicting transactions (rule 6), the replacement is >> accepted. The honest HTLC-timeout is evicted out of the mempool. >> >> In an ulterior move, the malicious counterparty can replace the parent >> transaction itself with another candidate N satisfying the replacement >> rules, triggering the eviction of the malicious HTLC-preimage from the >> mempool as it was a child of the parent T. >> >> There is no spending candidate of the offered HTLC output for the current >> block laying in network mempools. >> >> This replacement cycling tricks can be repeated for each rebroadcast >> attempt of the HTLC-timeout by the honest lightning node until expiration >> of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout >> is broadcast by the counterparty's on the incoming link in collusion with >> the one on the outgoing link broadcasting its own HTLC-preimage. >> >> The honest Lightning node has been "double-spent" in its HTLC forwarding. >> >> As a notable factor impacting the success of the attack, a lightning >> node's honest HTLC-timeout might be included in the block template of the >> miner winning the block race and therefore realizes a spent of the offered >> output. In practice, a replacement cycling attack might over-connect to >> miners' mempools and public reachable nodes to succeed in a fast eviction >> of the HTLC-timeout by its HTLC-preimage. As this latter transaction can >> come with a better ancestor-score, it should be picked up on the flight by >> economically competitive miners. >> >> A functional test exercising a simple replacement cycling of a HTLC >> transaction on bitcoin core mempool is available: >> https://github.com/ariard/bitcoin/commits/2023-test-mempool >> >> ## Deployed LN mitigations >> >> Aggressive rebroadcasting: As the replacement cycling attacker benefits >> from the HTLC-timeout being usually broadcast by lightning nodes only once >> every block, or less the replacement cycling malicious transactions paid >> only equal the sum of the absolute fees paid by the HTLC, adjusted with the >> replacement penalty. Rebroadcasting randomly and multiple times before the >> next block increases the absolute fee cost for the attacker. >> >> Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >> >> Local-mempool preimage monitoring: As the replacement cycling attacker in >> a simple setup broadcast the HTLC-preimage to all the network mempools, the >> honest lightning node is able to catch on the flight the unconfirmed >> HTLC-preimage, before its subsequent mempool replacement. The preimage can >> be extracted from the second-stage HTLC-preimage and used to fetch the >> off-chain inbound HTLC with a cooperative message or go on-chain with it to >> claim the accepted HTLC output. >> >> Implemented and deployed by Eclair and LND. >> >> CLTV Expiry Delta: With every jammed block comes an absolute fee cost >> paid by the attacker, a risk of the HTLC-preimage being detected or >> discovered by the honest lightning node, or the HTLC-timeout to slip in a >> winning block template. Bumping the default CLTV delta hardens the odds of >> success of a simple replacement cycling attack. >> >> Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. >> >> ## Affected Bitcoin Protocols and Applications >> >> From my understanding the following list of Bitcoin protocols and >> applications could be affected by new denial-of-service vectors under some >> level of network mempools congestion. Neither tests or advanced review of >> specifications (when available) has been conducted for each of them: >> - on-chain DLCs >> - coinjoins >> - payjoins >> - wallets with time-sensitive paths >> - peerswap and submarine swaps >> - batch payouts >> - transaction "accelerators" >> >> Inviting their developers, maintainers and operators to investigate how >> replacement cycling attacks might disrupt their in-mempool chain of >> transactions, or fee-bumping flows at the shortest delay. Simple flows and >> non-multi-party transactions should not be affected to the best of my >> understanding. >> >> ## Open Problems: Package Malleability >> >> Pinning attacks have been known for years as a practical vector to >> compromise lightning channels funds safety, under different scenarios (cf. >> current bip331's motivation section). Mitigations at the mempool level have >> been designed, discussed and are under implementation by the community >> (ancestor package relay + nverrsion=3 policy). Ideally, they should >> constraint a pinning attacker to always attach a high feerate package >> (commitment + CPFP) to replace the honest package, or allow a honest >> lightning node to overbid a malicious pinning package and get its >> time-sensitive transaction optimistically included in the chain. >> >> Replacement cycling attack seem to offer a new way to neutralize the >> design goals of package relay and its companion nversion=3 policy, where an >> attacker package RBF a honest package out of the mempool to subsequently >> double-spend its own high-fee child with a transaction unrelated to the >> channel. As the remaining commitment transaction is pre-signed with a >> minimal relay fee, it can be evicted out of the mempool. >> >> A functional test exercising a simple replacement cycling of a lightning >> channel commitment transaction on top of the nversion=3 code branch is >> available: >> https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 >> >> ## Discovery >> >> In 2018, the issue of static fees for pre-signed lightning transactions >> is made more widely known, the carve-out exemption in mempool rules to >> mitigate in-mempool package limits pinning and the anchor output pattern >> are proposed. >> >> In 2019, bitcoin core 0.19 is released with carve-out support. Continued >> discussion of the anchor output pattern as a dynamic fee-bumping method. >> >> In 2020, draft of anchor output submitted to the bolts. Initial finding >> of economic pinning against lightning commitment and second-stage HTLC >> transactions. Subsequent discussions of a preimage-overlay network or >> package-relay as mitigations. Public call made to inquiry more on potential >> other transaction-relay jamming attacks affecting lightning. >> >> In 2021, initial work in bitcoin core 22.0 of package acceptance. >> Continued discussion of the pinning attacks and shortcomings of current >> mempool rules during community-wide online workshops. Later the year, in >> light of all issues for bitcoin second-layers, a proposal is made about >> killing the mempool. >> >> In 2022, bip proposed for package relay and new proposed v3 policy design >> proposed for a review and implementation. Mempoolfullrbf is supported in >> bitcoin core 24.0 and conceptual questions about alignment of mempool rules >> w.r.t miners incentives are investigated. >> >> Along this year 2022, eltoo lightning channels design are discussed, >> implemented and reviewed. In this context and after discussions on mempool >> anti-DoS rules, I discovered this new replacement cycling attack was >> affecting deployed lightning channels and immediately reported the finding >> to some bitcoin core developers and lightning maintainers. >> >> ## Timeline >> >> - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg >> Sanders and Gloria Zhao >> - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien >> Teinturier, Matt Corallo and Olaoluwa Osuntunkun >> - 2022-12-23: Sharing to Eugene Siegel (LND) >> - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot >> (non-lightning potential affected projects) >> - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and >> cross-layers issuers) and initial proposal of an early public disclosure >> - 2022-01-19: Collection of analysis if other second-layers and >> multi-party applications affected. LN mitigations development starts. >> - 2023-05-04: Sharing to Wilmer Paulino (LDK) >> - 2023-06-20: LN mitigations implemented and progressively released. Week >> of the 16 october proposed for full disclosure. >> - 2023-08-10: CVEs assigned by MITRE >> - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling >> attack existence to security@bitcoincore•org. >> - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / >> CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks >> >> ## Conclusion >> >> Despite the line of mitigations adopted and deployed by current major >> lightning implementations, I believe replacement cycling attacks are still >> practical for advanced attackers. Beyond this new attack might come as a >> way to partially or completely defeat some of the pinning mitigations which >> have been working for years as a community. >> >> As of today, it is uncertain to me if lightning is not affected by a more >> severe long-term package malleability critical security issue under current >> consensus rules, and if any other time-sensitive multi-party protocol, >> designed or deployed isn't de facto affected too (loss of funds or denial >> of service). >> >> Assuming analysis on package malleability is correct, it is unclear to me >> if it can be corrected by changes in replacement / eviction rules or >> mempool chain of transactions processing strategy. Inviting my technical >> peers and the bitcoin community to look more on this issue, including to >> dissent. I'll be the first one pleased if I'm fundamentally wrong on those >> issues, or if any element has not been weighted with the adequate technical >> accuracy it deserves. >> >> Do not trust, verify. All mistakes and opinions are my own. >> >> Antoine >> >> "meet with Triumph and Disaster. And treat those two impostors just the >> same" - K. >> >> >>
There appears to be some confusion about this issue and the mitigations. To be clear, the deployed mitigations are not expected to fix this issue, its arguable if they provide anything more than a PR statement.
There are two discussed mitigations here - mempool scanning and transaction re-signing/re-broadcasting.
Mempool scanning relies on regularly checking the mempool of a local node to see if we can catch the replacement cycle mid-cycle. It only works if wee see the first transaction before the second transaction replaces it.
Today, a large majority of lightning nodes run on machines with a Bitcoin node on the same IP address, making it very clear what the "local node" of the lightning node is. An attacker can trivially use this information to connect to said local node and do the replacement quickly, preventing the victim from seeing the replacement.
More generally, however, similar discoverability is true for mining pools. An attacker performing this attack is likely to do the replacement attack on a miner's node directly, potentially reducing the reach of the intermediate transaction to only miners, such that the victim can never discover it at all.
The second mitigation is similarly pathetic. Re-signing and re-broadcasting the victim's transaction in an attempt to get it to miners even if its been removed may work, if the attacker is super lazy and didn't finish writing their attack system. If the attacker is connected to a large majority of hashrate (which has historically been fairly doable), they can simply do their replacement in a cycle aggressively and arbitrarily reduce the probability that the victim's transaction gets confirmed.
Now, the above is all true in a spherical cow kinda world, and the P2P network has plenty of slow nodes and strange behavior. Its possible that these mitigations might, by some stroke of luck, happen to catch such an attack and prevent it, because something took longer than the attacker intended or whatever. But, that's a far cry from any kind of material "fix" for the issue.
Ultimately the only fix for this issue will be when miners keep a history of transactions they've seen and try them again after they may be able to enter the mempool because of an attack like this.
Matt
On 10/16/23 12:57 PM, Antoine Riard wrote:
Quoted text (248 lines)
> (cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other > multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo payment channels and > incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack > affecting lightning channels was discovered. > > After careful analysis, it turns out this attack is practical and immediately exposed lightning > routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output > channels. A potential exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major lightning implementations > during the last months. > > Please find attached the release numbers, where the mitigations should be present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported in the wild since the last > ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available > exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle). > > It is understood that a simple replacement cycling attack does not demand privileged capabilities > from an attacker (e.g no low-hashrate power) and only access to basic bitcoin and lightning > software. Yet I still think executing such an attack successfully requests a fair amount of bitcoin > technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the mitigations deployed are > robust enough in face of advanced replacement cycling attackers, especially ones able to combine > different classes of transaction-relay jamming such as pinnings or vetted with more privileged > capabilities. > > Please find a list of potential affected bitcoin applications in this full disclosure report using > bitcoin script timelocks or multi-party transactions, albeit no immediate security risk exposure as > severe as the ones affecting lightning has been identified. Only cursory review of non-lightning > applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the lightning network: > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > <https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf> > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted HTLC on incoming link and > offered HTLC on outgoing link) should settle the outgoing state with either a success or timeout > before the incoming state timelock becomes final and an asymmetric defavorable settlement might > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" section 2.3 for a classical > exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to a loss of fund risk where > the offered HTLC is spent by the outgoing link counterparty's HTLC-preimage and the accepted HTLC is > spent by the incoming link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be spaced out by an interval of > `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this exact interval value being an > implementation and node policy setting. As a minimal value, the specification recommends 34 blocks > of interval. If the timelock expiration I of the inbound HTLC is equal to 100 from chain tip, the > timelock expiration O of the outbound HTLC must be equal to 66 blocks from chain tip, giving a > reasonable buffer of reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the outgoing link negotiated with the > counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the lightning > node should broadcast its commitment transaction. Once the commitment is confirmed (if anchor and > the 1 CSV encumbrance is present), the lightning node broadcasts and confirms its HTLC-timeout > before I height is reached. > > Here enter a replacement cycling attack. A malicious channel counterparty can broadcast its > HTLC-preimage transaction with a higher absolute fee and higher feerate than the honest HTLC-timeout > of the victim lightning node and triggers a replacement. Both for legacy and anchor output channels, > a HTLC-preimage on a counterparty commitment transaction is malleable, i.e additional inputs or > outputs can be added. The HTLC-preimage spends an unconfirmed and unrelated to the channel parent > transaction M and conflicts its child. > > As the HTLC-preimage spends an unconfirmed input that was already included in the unconfirmed and > unrelated child transaction (rule 2), pays an absolute higher fee of at least the sum paid by the > HTLC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is greater than all > directly conflicting transactions (rule 6), the replacement is accepted. The honest HTLC-timeout is > evicted out of the mempool. > > In an ulterior move, the malicious counterparty can replace the parent transaction itself with > another candidate N satisfying the replacement rules, triggering the eviction of the malicious > HTLC-preimage from the mempool as it was a child of the parent T. > > There is no spending candidate of the offered HTLC output for the current block laying in network > mempools. > > This replacement cycling tricks can be repeated for each rebroadcast attempt of the HTLC-timeout by > the honest lightning node until expiration of the inbound HTLC timelock I. Once this height is > reached a HTLC-timeout is broadcast by the counterparty's on the incoming link in collusion with the > one on the outgoing link broadcasting its own HTLC-preimage. > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > As a notable factor impacting the success of the attack, a lightning node's honest HTLC-timeout > might be included in the block template of the miner winning the block race and therefore realizes a > spent of the offered output. In practice, a replacement cycling attack might over-connect to miners' > mempools and public reachable nodes to succeed in a fast eviction of the HTLC-timeout by its > HTLC-preimage. As this latter transaction can come with a better ancestor-score, it should be picked > up on the flight by economically competitive miners. > > A functional test exercising a simple replacement cycling of a HTLC transaction on bitcoin core > mempool is available: > https://github.com/ariard/bitcoin/commits/2023-test-mempool > <https://github.com/ariard/bitcoin/commits/2023-test-mempool> > > ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being > usually broadcast by lightning nodes only once every block, or less the replacement cycling > malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with > the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases > the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast > the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the > flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be > extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a > cooperative message or go on-chain with it to claim the accepted HTLC output. > > Implemented and deployed by Eclair and LND. > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid by the attacker, a risk > of the HTLC-preimage being detected or discovered by the honest lightning node, or the HTLC-timeout > to slip in a winning block template. Bumping the default CLTV delta hardens the odds of success of a > simple replacement cycling attack. > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > ## Affected Bitcoin Protocols and Applications > > From my understanding the following list of Bitcoin protocols and applications could be affected by > new denial-of-service vectors under some level of network mempools congestion. Neither tests or > advanced review of specifications (when available) has been conducted for each of them: > - on-chain DLCs > - coinjoins > - payjoins > - wallets with time-sensitive paths > - peerswap and submarine swaps > - batch payouts > - transaction "accelerators" > > Inviting their developers, maintainers and operators to investigate how replacement cycling attacks > might disrupt their in-mempool chain of transactions, or fee-bumping flows at the shortest delay. > Simple flows and non-multi-party transactions should not be affected to the best of my understanding. > > ## Open Problems: Package Malleability > > Pinning attacks have been known for years as a practical vector to compromise lightning channels > funds safety, under different scenarios (cf. current bip331's motivation section). Mitigations at > the mempool level have been designed, discussed and are under implementation by the community > (ancestor package relay + nverrsion=3 policy). Ideally, they should constraint a pinning attacker to > always attach a high feerate package (commitment + CPFP) to replace the honest package, or allow a > honest lightning node to overbid a malicious pinning package and get its time-sensitive transaction > optimistically included in the chain. > > Replacement cycling attack seem to offer a new way to neutralize the design goals of package relay > and its companion nversion=3 policy, where an attacker package RBF a honest package out of the > mempool to subsequently double-spend its own high-fee child with a transaction unrelated to the > channel. As the remaining commitment transaction is pre-signed with a minimal relay fee, it can be > evicted out of the mempool. > > A functional test exercising a simple replacement cycling of a lightning channel commitment > transaction on top of the nversion=3 code branch is available: > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> > > ## Discovery > > In 2018, the issue of static fees for pre-signed lightning transactions is made more widely known, > the carve-out exemption in mempool rules to mitigate in-mempool package limits pinning and the > anchor output pattern are proposed. > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued discussion of the anchor > output pattern as a dynamic fee-bumping method. > > In 2020, draft of anchor output submitted to the bolts. Initial finding of economic pinning against > lightning commitment and second-stage HTLC transactions. Subsequent discussions of a > preimage-overlay network or package-relay as mitigations. Public call made to inquiry more on > potential other transaction-relay jamming attacks affecting lightning. > > In 2021, initial work in bitcoin core 22.0 of package acceptance. Continued discussion of the > pinning attacks and shortcomings of current mempool rules during community-wide online workshops. > Later the year, in light of all issues for bitcoin second-layers, a proposal is made about killing > the mempool. > > In 2022, bip proposed for package relay and new proposed v3 policy design proposed for a review and > implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and conceptual questions about > alignment of mempool rules w.r.t miners incentives are investigated. > > Along this year 2022, eltoo lightning channels design are discussed, implemented and reviewed. In > this context and after discussions on mempool anti-DoS rules, I discovered this new replacement > cycling attack was affecting deployed lightning channels and immediately reported the finding to > some bitcoin core developers and lightning maintainers. > > ## Timeline > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Zhao > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, Matt Corallo and Olaoluwa > Osuntunkun > - 2022-12-23: Sharing to Eugene Siegel (LND) > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning potential affected projects) > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers issuers) and initial > proposal of an early public disclosure > - 2022-01-19: Collection of analysis if other second-layers and multi-party applications affected. > LN mitigations development starts. > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > - 2023-06-20: LN mitigations implemented and progressively released. Week of the 16 october proposed > for full disclosure. > - 2023-08-10: CVEs assigned by MITRE > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling attack existence to > security@bitcoincore•org <mailto:security@bitcoincore.org>. > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 > and replacement cycling attacks > > ## Conclusion > > Despite the line of mitigations adopted and deployed by current major lightning implementations, I > believe replacement cycling attacks are still practical for advanced attackers. Beyond this new > attack might come as a way to partially or completely defeat some of the pinning mitigations which > have been working for years as a community. > > As of today, it is uncertain to me if lightning is not affected by a more severe long-term package > malleability critical security issue under current consensus rules, and if any other time-sensitive > multi-party protocol, designed or deployed isn't de facto affected too (loss of funds or denial of > service). > > Assuming analysis on package malleability is correct, it is unclear to me if it can be corrected by > changes in replacement / eviction rules or mempool chain of transactions processing strategy. > Inviting my technical peers and the bitcoin community to look more on this issue, including to > dissent. I'll be the first one pleased if I'm fundamentally wrong on those issues, or if any element > has not been weighted with the adequate technical accuracy it deserves. > > Do not trust, verify. All mistakes and opinions are my own. > > Antoine > > "meet with Triumph and Disaster. And treat those two impostors just the same" - K. > > _______________________________________________ > Lightning-dev mailing list > Lightning-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
The disclosure mails noted a 3rd mitigation beyond mempool scanning and transaction re-signing / re-broadcasting, namely bumping CLTV delta.
Generally bumping CLTV delta is a basic line of mitigations for a lot of lightning attacks, as it gives opportunity to node operators to intervene and re-broadcast their time-sensitive transactions on other interfaces (e.g a secondary full-node if the first one is eclipsed).
About the second mitigation transaction re-signing, if done correctly at least sounds to put an economic cost (denominated in fees / feerates) on the attack. This is unclear to me if the game-theory of this cost holds.
One thing which sounds to me making the attack harder is stratum v2 deployment, as you're increasing the number of miners which might do their own block templates, and therefore the number of miners' mempools where an attacker has to successfully continuously replace in cycles channels counterparties transactions.
A replacement buffer or history of transactions at the mempool level might be a mitigation to this attack. I believe this is yet to be seen if it can be made robust enough.
I don't know if folks like tadge or rusty who have been involved in the early design of lightning have more ideas of mitigations. Fees was noted as a hard issue in the original paper.
Le mer. 18 oct. 2023 à 01:17, Matt Corallo <lf-lists@mattcorallo•com> a écrit :
Quoted text (430 lines)
> There appears to be some confusion about this issue and the mitigations. > To be clear, the deployed > mitigations are not expected to fix this issue, its arguable if they > provide anything more than a PR > statement. > > There are two discussed mitigations here - mempool scanning and > transaction re-signing/re-broadcasting. > > Mempool scanning relies on regularly checking the mempool of a local node > to see if we can catch the > replacement cycle mid-cycle. It only works if wee see the first > transaction before the second > transaction replaces it. > > Today, a large majority of lightning nodes run on machines with a Bitcoin > node on the same IP > address, making it very clear what the "local node" of the lightning node > is. An attacker can > trivially use this information to connect to said local node and do the > replacement quickly, > preventing the victim from seeing the replacement. > > More generally, however, similar discoverability is true for mining pools. > An attacker performing > this attack is likely to do the replacement attack on a miner's node > directly, potentially reducing > the reach of the intermediate transaction to only miners, such that the > victim can never discover it > at all. > > The second mitigation is similarly pathetic. Re-signing and > re-broadcasting the victim's transaction > in an attempt to get it to miners even if its been removed may work, if > the attacker is super lazy > and didn't finish writing their attack system. If the attacker is > connected to a large majority of > hashrate (which has historically been fairly doable), they can simply do > their replacement in a > cycle aggressively and arbitrarily reduce the probability that the > victim's transaction gets confirmed. > > Now, the above is all true in a spherical cow kinda world, and the P2P > network has plenty of slow > nodes and strange behavior. Its possible that these mitigations might, by > some stroke of luck, > happen to catch such an attack and prevent it, because something took > longer than the attacker > intended or whatever. But, that's a far cry from any kind of material > "fix" for the issue. > > Ultimately the only fix for this issue will be when miners keep a history > of transactions they've > seen and try them again after they may be able to enter the mempool > because of an attack like this. > > Matt > > On 10/16/23 12:57 PM, Antoine Riard wrote: > > (cross-posting mempool issues identified are exposing lightning chan to > loss of funds risks, other > > multi-party bitcoin apps might be affected) > > > > Hi, > > > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and > > incentives compatibility of the mempool anti-DoS rules, a new > transaction-relay jamming attack > > affecting lightning channels was discovered. > > > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning > > routing hops carrying HTLC traffic to loss of funds security risks, both > legacy and anchor output > > channels. A potential exploitation plausibly happening even without > network mempools congestion. > > > > Mitigations have been designed, implemented and deployed by all major > lightning implementations > > during the last months. > > > > Please find attached the release numbers, where the mitigations should > be present: > > - LDK: v0.0.118 - CVE-2023 -40231 > > - Eclair: v0.9.0 - CVE-2023-40232 > > - LND: v.0.17.0-beta - CVE-2023-40233 > > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > > > While neither replacement cycling attacks have been observed or reported > in the wild since the last > > ~10 months or experimented in real-world conditions on bitcoin mainet, > functional test is available > > exercising the affected lightning channel against bitcoin core mempool > (26.0 release cycle). > > > > It is understood that a simple replacement cycling attack does not > demand privileged capabilities > > from an attacker (e.g no low-hashrate power) and only access to basic > bitcoin and lightning > > software. Yet I still think executing such an attack successfully > requests a fair amount of bitcoin > > technical know-how and decent preparation. > > > > From my understanding of those issues, it is yet to be determined if > the mitigations deployed are > > robust enough in face of advanced replacement cycling attackers, > especially ones able to combine > > different classes of transaction-relay jamming such as pinnings or > vetted with more privileged > > capabilities. > > > > Please find a list of potential affected bitcoin applications in this > full disclosure report using > > bitcoin script timelocks or multi-party transactions, albeit no > immediate security risk exposure as > > severe as the ones affecting lightning has been identified. Only cursory > review of non-lightning > > applications has been conducted so far. > > > > There is a paper published summarizing replacement cycling attacks on > the lightning network: > > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > < > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > > > > > ## Problem > > > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and > > offered HTLC on outgoing link) should settle the outgoing state with > either a success or timeout > > before the incoming state timelock becomes final and an asymmetric > defavorable settlement might > > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" > section 2.3 for a classical > > exposition of this lightning security property). > > > > Failure to satisfy this settlement requirement exposes a forwarding hop > to a loss of fund risk where > > the offered HTLC is spent by the outgoing link counterparty's > HTLC-preimage and the accepted HTLC is > > spent by the incoming link counterparty's HTLC-timeout. > > > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of > > `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this > exact interval value being an > > implementation and node policy setting. As a minimal value, the > specification recommends 34 blocks > > of interval. If the timelock expiration I of the inbound HTLC is equal > to 100 from chain tip, the > > timelock expiration O of the outbound HTLC must be equal to 66 blocks > from chain tip, giving a > > reasonable buffer of reaction to the lightning forwarding node. > > > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the > > counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O > is reached, the lightning > > node should broadcast its commitment transaction. Once the commitment is > confirmed (if anchor and > > the 1 CSV encumbrance is present), the lightning node broadcasts and > confirms its HTLC-timeout > > before I height is reached. > > > > Here enter a replacement cycling attack. A malicious channel > counterparty can broadcast its > > HTLC-preimage transaction with a higher absolute fee and higher feerate > than the honest HTLC-timeout > > of the victim lightning node and triggers a replacement. Both for legacy > and anchor output channels, > > a HTLC-preimage on a counterparty commitment transaction is malleable, > i.e additional inputs or > > outputs can be added. The HTLC-preimage spends an unconfirmed and > unrelated to the channel parent > > transaction M and conflicts its child. > > > > As the HTLC-preimage spends an unconfirmed input that was already > included in the unconfirmed and > > unrelated child transaction (rule 2), pays an absolute higher fee of at > least the sum paid by the > > HTLC-timeout and child transaction (rule 3) and the HTLC-preimage > feerate is greater than all > > directly conflicting transactions (rule 6), the replacement is accepted. > The honest HTLC-timeout is > > evicted out of the mempool. > > > > In an ulterior move, the malicious counterparty can replace the parent > transaction itself with > > another candidate N satisfying the replacement rules, triggering the > eviction of the malicious > > HTLC-preimage from the mempool as it was a child of the parent T. > > > > There is no spending candidate of the offered HTLC output for the > current block laying in network > > mempools. > > > > This replacement cycling tricks can be repeated for each rebroadcast > attempt of the HTLC-timeout by > > the honest lightning node until expiration of the inbound HTLC timelock > I. Once this height is > > reached a HTLC-timeout is broadcast by the counterparty's on the > incoming link in collusion with the > > one on the outgoing link broadcasting its own HTLC-preimage. > > > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > > > As a notable factor impacting the success of the attack, a lightning > node's honest HTLC-timeout > > might be included in the block template of the miner winning the block > race and therefore realizes a > > spent of the offered output. In practice, a replacement cycling attack > might over-connect to miners' > > mempools and public reachable nodes to succeed in a fast eviction of the > HTLC-timeout by its > > HTLC-preimage. As this latter transaction can come with a better > ancestor-score, it should be picked > > up on the flight by economically competitive miners. > > > > A functional test exercising a simple replacement cycling of a HTLC > transaction on bitcoin core > > mempool is available: > > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > <https://github.com/ariard/bitcoin/commits/2023-test-mempool> > > > > ## Deployed LN mitigations > > > > Aggressive rebroadcasting: As the replacement cycling attacker benefits > from the HTLC-timeout being > > usually broadcast by lightning nodes only once every block, or less the > replacement cycling > > malicious transactions paid only equal the sum of the absolute fees paid > by the HTLC, adjusted with > > the replacement penalty. Rebroadcasting randomly and multiple times > before the next block increases > > the absolute fee cost for the attacker. > > > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > > > Local-mempool preimage monitoring: As the replacement cycling attacker > in a simple setup broadcast > > the HTLC-preimage to all the network mempools, the honest lightning node > is able to catch on the > > flight the unconfirmed HTLC-preimage, before its subsequent mempool > replacement. The preimage can be > > extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a > > cooperative message or go on-chain with it to claim the accepted HTLC > output. > > > > Implemented and deployed by Eclair and LND. > > > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost > paid by the attacker, a risk > > of the HTLC-preimage being detected or discovered by the honest > lightning node, or the HTLC-timeout > > to slip in a winning block template. Bumping the default CLTV delta > hardens the odds of success of a > > simple replacement cycling attack. > > > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > > > ## Affected Bitcoin Protocols and Applications > > > > From my understanding the following list of Bitcoin protocols and > applications could be affected by > > new denial-of-service vectors under some level of network mempools > congestion. Neither tests or > > advanced review of specifications (when available) has been conducted > for each of them: > > - on-chain DLCs > > - coinjoins > > - payjoins > > - wallets with time-sensitive paths > > - peerswap and submarine swaps > > - batch payouts > > - transaction "accelerators" > > > > Inviting their developers, maintainers and operators to investigate how > replacement cycling attacks > > might disrupt their in-mempool chain of transactions, or fee-bumping > flows at the shortest delay. > > Simple flows and non-multi-party transactions should not be affected to > the best of my understanding. > > > > ## Open Problems: Package Malleability > > > > Pinning attacks have been known for years as a practical vector to > compromise lightning channels > > funds safety, under different scenarios (cf. current bip331's motivation > section). Mitigations at > > the mempool level have been designed, discussed and are under > implementation by the community > > (ancestor package relay + nverrsion=3 policy). Ideally, they should > constraint a pinning attacker to > > always attach a high feerate package (commitment + CPFP) to replace the > honest package, or allow a > > honest lightning node to overbid a malicious pinning package and get its > time-sensitive transaction > > optimistically included in the chain. > > > > Replacement cycling attack seem to offer a new way to neutralize the > design goals of package relay > > and its companion nversion=3 policy, where an attacker package RBF a > honest package out of the > > mempool to subsequently double-spend its own high-fee child with a > transaction unrelated to the > > channel. As the remaining commitment transaction is pre-signed with a > minimal relay fee, it can be > > evicted out of the mempool. > > > > A functional test exercising a simple replacement cycling of a lightning > channel commitment > > transaction on top of the nversion=3 code branch is available: > > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> > > > > ## Discovery > > > > In 2018, the issue of static fees for pre-signed lightning transactions > is made more widely known, > > the carve-out exemption in mempool rules to mitigate in-mempool package > limits pinning and the > > anchor output pattern are proposed. > > > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued > discussion of the anchor > > output pattern as a dynamic fee-bumping method. > > > > In 2020, draft of anchor output submitted to the bolts. Initial finding > of economic pinning against > > lightning commitment and second-stage HTLC transactions. Subsequent > discussions of a > > preimage-overlay network or package-relay as mitigations. Public call > made to inquiry more on > > potential other transaction-relay jamming attacks affecting lightning. > > > > In 2021, initial work in bitcoin core 22.0 of package acceptance. > Continued discussion of the > > pinning attacks and shortcomings of current mempool rules during > community-wide online workshops. > > Later the year, in light of all issues for bitcoin second-layers, a > proposal is made about killing > > the mempool. > > > > In 2022, bip proposed for package relay and new proposed v3 policy > design proposed for a review and > > implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and > conceptual questions about > > alignment of mempool rules w.r.t miners incentives are investigated. > > > > Along this year 2022, eltoo lightning channels design are discussed, > implemented and reviewed. In > > this context and after discussions on mempool anti-DoS rules, I > discovered this new replacement > > cycling attack was affecting deployed lightning channels and immediately > reported the finding to > > some bitcoin core developers and lightning maintainers. > > > > ## Timeline > > > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, > Greg Sanders and Gloria Zhao > > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien > Teinturier, Matt Corallo and Olaoluwa > > Osuntunkun > > - 2022-12-23: Sharing to Eugene Siegel (LND) > > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot > (non-lightning potential affected projects) > > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and > cross-layers issuers) and initial > > proposal of an early public disclosure > > - 2022-01-19: Collection of analysis if other second-layers and > multi-party applications affected. > > LN mitigations development starts. > > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > > - 2023-06-20: LN mitigations implemented and progressively released. > Week of the 16 october proposed > > for full disclosure. > > - 2023-08-10: CVEs assigned by MITRE > > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling > attack existence to > > security@bitcoincore•org <mailto:security@bitcoincore.org>. > > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / > CVE-2023-40233 / CVE-2023-40234 > > and replacement cycling attacks > > > > ## Conclusion > > > > Despite the line of mitigations adopted and deployed by current major > lightning implementations, I > > believe replacement cycling attacks are still practical for advanced > attackers. Beyond this new > > attack might come as a way to partially or completely defeat some of the > pinning mitigations which > > have been working for years as a community. > > > > As of today, it is uncertain to me if lightning is not affected by a > more severe long-term package > > malleability critical security issue under current consensus rules, and > if any other time-sensitive > > multi-party protocol, designed or deployed isn't de facto affected too > (loss of funds or denial of > > service). > > > > Assuming analysis on package malleability is correct, it is unclear to > me if it can be corrected by > > changes in replacement / eviction rules or mempool chain of transactions > processing strategy. > > Inviting my technical peers and the bitcoin community to look more on > this issue, including to > > dissent. I'll be the first one pleased if I'm fundamentally wrong on > those issues, or if any element > > has not been weighted with the adequate technical accuracy it deserves. > > > > Do not trust, verify. All mistakes and opinions are my own. > > > > Antoine > > > > "meet with Triumph and Disaster. And treat those two impostors just the > same" - K. > > > > _______________________________________________ > > Lightning-dev mailing list > > Lightning-dev@lists•linuxfoundation•org > > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev >
Thanks Antoine for your work on this issue.
I confirm that eclair v0.9.0 contains the migitations described.
Eclair has been watching the mempool for preimages since its very early versions (years ago), relying on Bitcoin Core's ZMQ notifications for incoming transactions. I believe this guarantees that we see the HTLC success transaction, even if it is immediately replaced (as long as we don't overflow the ZMQ limits).
I agree with Matt though that more fundamental work most likely needs to happen at the bitcoin layer to allow L2 protocols to be more robust against that class of attacks.
Thanks, Bastien
Le mer. 18 oct. 2023 à 11:07, Antoine Riard via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> a écrit :
Quoted text (465 lines)
> The disclosure mails noted a 3rd mitigation beyond mempool scanning and > transaction re-signing / re-broadcasting, namely bumping CLTV delta. > > Generally bumping CLTV delta is a basic line of mitigations for a lot of > lightning attacks, as it gives opportunity to node operators to intervene > and re-broadcast their time-sensitive transactions on other interfaces (e.g > a secondary full-node if the first one is eclipsed). > > About the second mitigation transaction re-signing, if done correctly at > least sounds to put an economic cost (denominated in fees / feerates) on > the attack. This is unclear to me if the game-theory of this cost holds. > > One thing which sounds to me making the attack harder is stratum v2 > deployment, as you're increasing the number of miners which might do their > own block templates, and therefore the number of miners' mempools where an > attacker has to successfully continuously replace in cycles channels > counterparties transactions. > > A replacement buffer or history of transactions at the mempool level might > be a mitigation to this attack. I believe this is yet to be seen if it can > be made robust enough. > > I don't know if folks like tadge or rusty who have been involved in the > early design of lightning have more ideas of mitigations. Fees was noted as > a hard issue in the original paper. > > Le mer. 18 oct. 2023 à 01:17, Matt Corallo <lf-lists@mattcorallo•com> a > écrit : > >> There appears to be some confusion about this issue and the mitigations. >> To be clear, the deployed >> mitigations are not expected to fix this issue, its arguable if they >> provide anything more than a PR >> statement. >> >> There are two discussed mitigations here - mempool scanning and >> transaction re-signing/re-broadcasting. >> >> Mempool scanning relies on regularly checking the mempool of a local node >> to see if we can catch the >> replacement cycle mid-cycle. It only works if wee see the first >> transaction before the second >> transaction replaces it. >> >> Today, a large majority of lightning nodes run on machines with a Bitcoin >> node on the same IP >> address, making it very clear what the "local node" of the lightning node >> is. An attacker can >> trivially use this information to connect to said local node and do the >> replacement quickly, >> preventing the victim from seeing the replacement. >> >> More generally, however, similar discoverability is true for mining >> pools. An attacker performing >> this attack is likely to do the replacement attack on a miner's node >> directly, potentially reducing >> the reach of the intermediate transaction to only miners, such that the >> victim can never discover it >> at all. >> >> The second mitigation is similarly pathetic. Re-signing and >> re-broadcasting the victim's transaction >> in an attempt to get it to miners even if its been removed may work, if >> the attacker is super lazy >> and didn't finish writing their attack system. If the attacker is >> connected to a large majority of >> hashrate (which has historically been fairly doable), they can simply do >> their replacement in a >> cycle aggressively and arbitrarily reduce the probability that the >> victim's transaction gets confirmed. >> >> Now, the above is all true in a spherical cow kinda world, and the P2P >> network has plenty of slow >> nodes and strange behavior. Its possible that these mitigations might, by >> some stroke of luck, >> happen to catch such an attack and prevent it, because something took >> longer than the attacker >> intended or whatever. But, that's a far cry from any kind of material >> "fix" for the issue. >> >> Ultimately the only fix for this issue will be when miners keep a history >> of transactions they've >> seen and try them again after they may be able to enter the mempool >> because of an attack like this. >> >> Matt >> >> On 10/16/23 12:57 PM, Antoine Riard wrote: >> > (cross-posting mempool issues identified are exposing lightning chan to >> loss of funds risks, other >> > multi-party bitcoin apps might be affected) >> > >> > Hi, >> > >> > End of last year (December 2022), amid technical discussions on eltoo >> payment channels and >> > incentives compatibility of the mempool anti-DoS rules, a new >> transaction-relay jamming attack >> > affecting lightning channels was discovered. >> > >> > After careful analysis, it turns out this attack is practical and >> immediately exposed lightning >> > routing hops carrying HTLC traffic to loss of funds security risks, >> both legacy and anchor output >> > channels. A potential exploitation plausibly happening even without >> network mempools congestion. >> > >> > Mitigations have been designed, implemented and deployed by all major >> lightning implementations >> > during the last months. >> > >> > Please find attached the release numbers, where the mitigations should >> be present: >> > - LDK: v0.0.118 - CVE-2023 -40231 >> > - Eclair: v0.9.0 - CVE-2023-40232 >> > - LND: v.0.17.0-beta - CVE-2023-40233 >> > - Core-Lightning: v.23.08.01 - CVE-2023-40234 >> > >> > While neither replacement cycling attacks have been observed or >> reported in the wild since the last >> > ~10 months or experimented in real-world conditions on bitcoin mainet, >> functional test is available >> > exercising the affected lightning channel against bitcoin core mempool >> (26.0 release cycle). >> > >> > It is understood that a simple replacement cycling attack does not >> demand privileged capabilities >> > from an attacker (e.g no low-hashrate power) and only access to basic >> bitcoin and lightning >> > software. Yet I still think executing such an attack successfully >> requests a fair amount of bitcoin >> > technical know-how and decent preparation. >> > >> > From my understanding of those issues, it is yet to be determined if >> the mitigations deployed are >> > robust enough in face of advanced replacement cycling attackers, >> especially ones able to combine >> > different classes of transaction-relay jamming such as pinnings or >> vetted with more privileged >> > capabilities. >> > >> > Please find a list of potential affected bitcoin applications in this >> full disclosure report using >> > bitcoin script timelocks or multi-party transactions, albeit no >> immediate security risk exposure as >> > severe as the ones affecting lightning has been identified. Only >> cursory review of non-lightning >> > applications has been conducted so far. >> > >> > There is a paper published summarizing replacement cycling attacks on >> the lightning network: >> > >> https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf >> > < >> https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf >> > >> > >> > ## Problem >> > >> > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted >> HTLC on incoming link and >> > offered HTLC on outgoing link) should settle the outgoing state with >> either a success or timeout >> > before the incoming state timelock becomes final and an asymmetric >> defavorable settlement might >> > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" >> section 2.3 for a classical >> > exposition of this lightning security property). >> > >> > Failure to satisfy this settlement requirement exposes a forwarding hop >> to a loss of fund risk where >> > the offered HTLC is spent by the outgoing link counterparty's >> HTLC-preimage and the accepted HTLC is >> > spent by the incoming link counterparty's HTLC-timeout. >> > >> > The specification mandates the incoming HTLC expiration timelock to be >> spaced out by an interval of >> > `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this >> exact interval value being an >> > implementation and node policy setting. As a minimal value, the >> specification recommends 34 blocks >> > of interval. If the timelock expiration I of the inbound HTLC is equal >> to 100 from chain tip, the >> > timelock expiration O of the outbound HTLC must be equal to 66 blocks >> from chain tip, giving a >> > reasonable buffer of reaction to the lightning forwarding node. >> > >> > In the lack of cooperative off-chain settlement of the HTLC on the >> outgoing link negotiated with the >> > counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when >> O is reached, the lightning >> > node should broadcast its commitment transaction. Once the commitment >> is confirmed (if anchor and >> > the 1 CSV encumbrance is present), the lightning node broadcasts and >> confirms its HTLC-timeout >> > before I height is reached. >> > >> > Here enter a replacement cycling attack. A malicious channel >> counterparty can broadcast its >> > HTLC-preimage transaction with a higher absolute fee and higher feerate >> than the honest HTLC-timeout >> > of the victim lightning node and triggers a replacement. Both for >> legacy and anchor output channels, >> > a HTLC-preimage on a counterparty commitment transaction is malleable, >> i.e additional inputs or >> > outputs can be added. The HTLC-preimage spends an unconfirmed and >> unrelated to the channel parent >> > transaction M and conflicts its child. >> > >> > As the HTLC-preimage spends an unconfirmed input that was already >> included in the unconfirmed and >> > unrelated child transaction (rule 2), pays an absolute higher fee of at >> least the sum paid by the >> > HTLC-timeout and child transaction (rule 3) and the HTLC-preimage >> feerate is greater than all >> > directly conflicting transactions (rule 6), the replacement is >> accepted. The honest HTLC-timeout is >> > evicted out of the mempool. >> > >> > In an ulterior move, the malicious counterparty can replace the parent >> transaction itself with >> > another candidate N satisfying the replacement rules, triggering the >> eviction of the malicious >> > HTLC-preimage from the mempool as it was a child of the parent T. >> > >> > There is no spending candidate of the offered HTLC output for the >> current block laying in network >> > mempools. >> > >> > This replacement cycling tricks can be repeated for each rebroadcast >> attempt of the HTLC-timeout by >> > the honest lightning node until expiration of the inbound HTLC timelock >> I. Once this height is >> > reached a HTLC-timeout is broadcast by the counterparty's on the >> incoming link in collusion with the >> > one on the outgoing link broadcasting its own HTLC-preimage. >> > >> > The honest Lightning node has been "double-spent" in its HTLC >> forwarding. >> > >> > As a notable factor impacting the success of the attack, a lightning >> node's honest HTLC-timeout >> > might be included in the block template of the miner winning the block >> race and therefore realizes a >> > spent of the offered output. In practice, a replacement cycling attack >> might over-connect to miners' >> > mempools and public reachable nodes to succeed in a fast eviction of >> the HTLC-timeout by its >> > HTLC-preimage. As this latter transaction can come with a better >> ancestor-score, it should be picked >> > up on the flight by economically competitive miners. >> > >> > A functional test exercising a simple replacement cycling of a HTLC >> transaction on bitcoin core >> > mempool is available: >> > https://github.com/ariard/bitcoin/commits/2023-test-mempool >> > <https://github.com/ariard/bitcoin/commits/2023-test-mempool> >> > >> > ## Deployed LN mitigations >> > >> > Aggressive rebroadcasting: As the replacement cycling attacker benefits >> from the HTLC-timeout being >> > usually broadcast by lightning nodes only once every block, or less the >> replacement cycling >> > malicious transactions paid only equal the sum of the absolute fees >> paid by the HTLC, adjusted with >> > the replacement penalty. Rebroadcasting randomly and multiple times >> before the next block increases >> > the absolute fee cost for the attacker. >> > >> > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >> > >> > Local-mempool preimage monitoring: As the replacement cycling attacker >> in a simple setup broadcast >> > the HTLC-preimage to all the network mempools, the honest lightning >> node is able to catch on the >> > flight the unconfirmed HTLC-preimage, before its subsequent mempool >> replacement. The preimage can be >> > extracted from the second-stage HTLC-preimage and used to fetch the >> off-chain inbound HTLC with a >> > cooperative message or go on-chain with it to claim the accepted HTLC >> output. >> > >> > Implemented and deployed by Eclair and LND. >> > >> > CLTV Expiry Delta: With every jammed block comes an absolute fee cost >> paid by the attacker, a risk >> > of the HTLC-preimage being detected or discovered by the honest >> lightning node, or the HTLC-timeout >> > to slip in a winning block template. Bumping the default CLTV delta >> hardens the odds of success of a >> > simple replacement cycling attack. >> > >> > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. >> > >> > ## Affected Bitcoin Protocols and Applications >> > >> > From my understanding the following list of Bitcoin protocols and >> applications could be affected by >> > new denial-of-service vectors under some level of network mempools >> congestion. Neither tests or >> > advanced review of specifications (when available) has been conducted >> for each of them: >> > - on-chain DLCs >> > - coinjoins >> > - payjoins >> > - wallets with time-sensitive paths >> > - peerswap and submarine swaps >> > - batch payouts >> > - transaction "accelerators" >> > >> > Inviting their developers, maintainers and operators to investigate how >> replacement cycling attacks >> > might disrupt their in-mempool chain of transactions, or fee-bumping >> flows at the shortest delay. >> > Simple flows and non-multi-party transactions should not be affected to >> the best of my understanding. >> > >> > ## Open Problems: Package Malleability >> > >> > Pinning attacks have been known for years as a practical vector to >> compromise lightning channels >> > funds safety, under different scenarios (cf. current bip331's >> motivation section). Mitigations at >> > the mempool level have been designed, discussed and are under >> implementation by the community >> > (ancestor package relay + nverrsion=3 policy). Ideally, they should >> constraint a pinning attacker to >> > always attach a high feerate package (commitment + CPFP) to replace the >> honest package, or allow a >> > honest lightning node to overbid a malicious pinning package and get >> its time-sensitive transaction >> > optimistically included in the chain. >> > >> > Replacement cycling attack seem to offer a new way to neutralize the >> design goals of package relay >> > and its companion nversion=3 policy, where an attacker package RBF a >> honest package out of the >> > mempool to subsequently double-spend its own high-fee child with a >> transaction unrelated to the >> > channel. As the remaining commitment transaction is pre-signed with a >> minimal relay fee, it can be >> > evicted out of the mempool. >> > >> > A functional test exercising a simple replacement cycling of a >> lightning channel commitment >> > transaction on top of the nversion=3 code branch is available: >> > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 >> > <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> >> > >> > ## Discovery >> > >> > In 2018, the issue of static fees for pre-signed lightning transactions >> is made more widely known, >> > the carve-out exemption in mempool rules to mitigate in-mempool package >> limits pinning and the >> > anchor output pattern are proposed. >> > >> > In 2019, bitcoin core 0.19 is released with carve-out support. >> Continued discussion of the anchor >> > output pattern as a dynamic fee-bumping method. >> > >> > In 2020, draft of anchor output submitted to the bolts. Initial finding >> of economic pinning against >> > lightning commitment and second-stage HTLC transactions. Subsequent >> discussions of a >> > preimage-overlay network or package-relay as mitigations. Public call >> made to inquiry more on >> > potential other transaction-relay jamming attacks affecting lightning. >> > >> > In 2021, initial work in bitcoin core 22.0 of package acceptance. >> Continued discussion of the >> > pinning attacks and shortcomings of current mempool rules during >> community-wide online workshops. >> > Later the year, in light of all issues for bitcoin second-layers, a >> proposal is made about killing >> > the mempool. >> > >> > In 2022, bip proposed for package relay and new proposed v3 policy >> design proposed for a review and >> > implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and >> conceptual questions about >> > alignment of mempool rules w.r.t miners incentives are investigated. >> > >> > Along this year 2022, eltoo lightning channels design are discussed, >> implemented and reviewed. In >> > this context and after discussions on mempool anti-DoS rules, I >> discovered this new replacement >> > cycling attack was affecting deployed lightning channels and >> immediately reported the finding to >> > some bitcoin core developers and lightning maintainers. >> > >> > ## Timeline >> > >> > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, >> Greg Sanders and Gloria Zhao >> > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien >> Teinturier, Matt Corallo and Olaoluwa >> > Osuntunkun >> > - 2022-12-23: Sharing to Eugene Siegel (LND) >> > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot >> (non-lightning potential affected projects) >> > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and >> cross-layers issuers) and initial >> > proposal of an early public disclosure >> > - 2022-01-19: Collection of analysis if other second-layers and >> multi-party applications affected. >> > LN mitigations development starts. >> > - 2023-05-04: Sharing to Wilmer Paulino (LDK) >> > - 2023-06-20: LN mitigations implemented and progressively released. >> Week of the 16 october proposed >> > for full disclosure. >> > - 2023-08-10: CVEs assigned by MITRE >> > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling >> attack existence to >> > security@bitcoincore•org <mailto:security@bitcoincore.org>. >> > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / >> CVE-2023-40233 / CVE-2023-40234 >> > and replacement cycling attacks >> > >> > ## Conclusion >> > >> > Despite the line of mitigations adopted and deployed by current major >> lightning implementations, I >> > believe replacement cycling attacks are still practical for advanced >> attackers. Beyond this new >> > attack might come as a way to partially or completely defeat some of >> the pinning mitigations which >> > have been working for years as a community. >> > >> > As of today, it is uncertain to me if lightning is not affected by a >> more severe long-term package >> > malleability critical security issue under current consensus rules, and >> if any other time-sensitive >> > multi-party protocol, designed or deployed isn't de facto affected too >> (loss of funds or denial of >> > service). >> > >> > Assuming analysis on package malleability is correct, it is unclear to >> me if it can be corrected by >> > changes in replacement / eviction rules or mempool chain of >> transactions processing strategy. >> > Inviting my technical peers and the bitcoin community to look more on >> this issue, including to >> > dissent. I'll be the first one pleased if I'm fundamentally wrong on >> those issues, or if any element >> > has not been weighted with the adequate technical accuracy it deserves. >> > >> > Do not trust, verify. All mistakes and opinions are my own. >> > >> > Antoine >> > >> > "meet with Triumph and Disaster. And treat those two impostors just the >> same" - K. >> > >> > _______________________________________________ >> > Lightning-dev mailing list >> > Lightning-dev@lists•linuxfoundation•org >> > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev >> > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (26 lines)
> > There appears to be some confusion about this issue and the mitigations. To be clear, the deployed > mitigations are not expected to fix this issue, its arguable if they provide anything more than a PR > statement. > > There are two discussed mitigations here - mempool scanning and transaction re-signing/re-broadcasting. > > Mempool scanning relies on regularly checking the mempool of a local node to see if we can catch the > replacement cycle mid-cycle. It only works if wee see the first transaction before the second > transaction replaces it. > > Today, a large majority of lightning nodes run on machines with a Bitcoin node on the same IP > address, making it very clear what the "local node" of the lightning node is. An attacker can > trivially use this information to connect to said local node and do the replacement quickly, > preventing the victim from seeing the replacement. > > More generally, however, similar discoverability is true for mining pools. An attacker performing > this attack is likely to do the replacement attack on a miner's node directly, potentially reducing > the reach of the intermediate transaction to only miners, such that the victim can never discover it > at all. > > The second mitigation is similarly pathetic. Re-signing and re-broadcasting the victim's transaction > in an attempt to get it to miners even if its been removed may work, if the attacker is super lazy > and didn't finish writing their attack system. If the attacker is connected to a large majority of > hashrate (which has historically been fairly doable), they can simply do their replacement in a > cycle aggressively and arbitrarily reduce the probability that the victim's transaction gets confirmed.
What if the honest node aggressively fee-bumps and retransmits the HTLC-timeout as the CLTV delta deadline approaches, as suggested by Ziggie? Say, within 10 blocks of the deadline, the honest node starts increasing the fee by 1/10th the HTLC value for each non-confirmation.
This "scorched earth" approach may cost the honest node considerable fees, but it will cost the attacker even more, since each attacker replacement needs to burn at least as much as the HTLC-timeout fees, and the attacker will need to do a replacement every time the honest node fee bumps.
I think this fee-bumping policy will provide sufficient defense even if the attacker is replacement-cycling directly in miners' mempools and the victim has no visibility into the attack.
Quoted text (264 lines)
> > Now, the above is all true in a spherical cow kinda world, and the P2P network has plenty of slow > nodes and strange behavior. Its possible that these mitigations might, by some stroke of luck, > happen to catch such an attack and prevent it, because something took longer than the attacker > intended or whatever. But, that's a far cry from any kind of material "fix" for the issue. > > Ultimately the only fix for this issue will be when miners keep a history of transactions they've > seen and try them again after they may be able to enter the mempool because of an attack like this. > > Matt > > On 10/16/23 12:57 PM, Antoine Riard wrote: > > (cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other > > multi-party bitcoin apps might be affected) > > > > Hi, > > > > End of last year (December 2022), amid technical discussions on eltoo payment channels and > > incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack > > affecting lightning channels was discovered. > > > > After careful analysis, it turns out this attack is practical and immediately exposed lightning > > routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output > > channels. A potential exploitation plausibly happening even without network mempools congestion. > > > > Mitigations have been designed, implemented and deployed by all major lightning implementations > > during the last months. > > > > Please find attached the release numbers, where the mitigations should be present: > > - LDK: v0.0.118 - CVE-2023 -40231 > > - Eclair: v0.9.0 - CVE-2023-40232 > > - LND: v.0.17.0-beta - CVE-2023-40233 > > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > > > While neither replacement cycling attacks have been observed or reported in the wild since the last > > ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available > > exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle). > > > > It is understood that a simple replacement cycling attack does not demand privileged capabilities > > from an attacker (e.g no low-hashrate power) and only access to basic bitcoin and lightning > > software. Yet I still think executing such an attack successfully requests a fair amount of bitcoin > > technical know-how and decent preparation. > > > > From my understanding of those issues, it is yet to be determined if the mitigations deployed are > > robust enough in face of advanced replacement cycling attackers, especially ones able to combine > > different classes of transaction-relay jamming such as pinnings or vetted with more privileged > > capabilities. > > > > Please find a list of potential affected bitcoin applications in this full disclosure report using > > bitcoin script timelocks or multi-party transactions, albeit no immediate security risk exposure as > > severe as the ones affecting lightning has been identified. Only cursory review of non-lightning > > applications has been conducted so far. > > > > There is a paper published summarizing replacement cycling attacks on the lightning network: > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > <https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf> > > > > ## Problem > > > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted HTLC on incoming link and > > offered HTLC on outgoing link) should settle the outgoing state with either a success or timeout > > before the incoming state timelock becomes final and an asymmetric defavorable settlement might > > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" section 2.3 for a classical > > exposition of this lightning security property). > > > > Failure to satisfy this settlement requirement exposes a forwarding hop to a loss of fund risk where > > the offered HTLC is spent by the outgoing link counterparty's HTLC-preimage and the accepted HTLC is > > spent by the incoming link counterparty's HTLC-timeout. > > > > The specification mandates the incoming HTLC expiration timelock to be spaced out by an interval of > > `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this exact interval value being an > > implementation and node policy setting. As a minimal value, the specification recommends 34 blocks > > of interval. If the timelock expiration I of the inbound HTLC is equal to 100 from chain tip, the > > timelock expiration O of the outbound HTLC must be equal to 66 blocks from chain tip, giving a > > reasonable buffer of reaction to the lightning forwarding node. > > > > In the lack of cooperative off-chain settlement of the HTLC on the outgoing link negotiated with the > > counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the lightning > > node should broadcast its commitment transaction. Once the commitment is confirmed (if anchor and > > the 1 CSV encumbrance is present), the lightning node broadcasts and confirms its HTLC-timeout > > before I height is reached. > > > > Here enter a replacement cycling attack. A malicious channel counterparty can broadcast its > > HTLC-preimage transaction with a higher absolute fee and higher feerate than the honest HTLC-timeout > > of the victim lightning node and triggers a replacement. Both for legacy and anchor output channels, > > a HTLC-preimage on a counterparty commitment transaction is malleable, i.e additional inputs or > > outputs can be added. The HTLC-preimage spends an unconfirmed and unrelated to the channel parent > > transaction M and conflicts its child. > > > > As the HTLC-preimage spends an unconfirmed input that was already included in the unconfirmed and > > unrelated child transaction (rule 2), pays an absolute higher fee of at least the sum paid by the > > HTLC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is greater than all > > directly conflicting transactions (rule 6), the replacement is accepted. The honest HTLC-timeout is > > evicted out of the mempool. > > > > In an ulterior move, the malicious counterparty can replace the parent transaction itself with > > another candidate N satisfying the replacement rules, triggering the eviction of the malicious > > HTLC-preimage from the mempool as it was a child of the parent T. > > > > There is no spending candidate of the offered HTLC output for the current block laying in network > > mempools. > > > > This replacement cycling tricks can be repeated for each rebroadcast attempt of the HTLC-timeout by > > the honest lightning node until expiration of the inbound HTLC timelock I. Once this height is > > reached a HTLC-timeout is broadcast by the counterparty's on the incoming link in collusion with the > > one on the outgoing link broadcasting its own HTLC-preimage. > > > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > > > As a notable factor impacting the success of the attack, a lightning node's honest HTLC-timeout > > might be included in the block template of the miner winning the block race and therefore realizes a > > spent of the offered output. In practice, a replacement cycling attack might over-connect to miners' > > mempools and public reachable nodes to succeed in a fast eviction of the HTLC-timeout by its > > HTLC-preimage. As this latter transaction can come with a better ancestor-score, it should be picked > > up on the flight by economically competitive miners. > > > > A functional test exercising a simple replacement cycling of a HTLC transaction on bitcoin core > > mempool is available: > > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > <https://github.com/ariard/bitcoin/commits/2023-test-mempool> > > > > ## Deployed LN mitigations > > > > Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being > > usually broadcast by lightning nodes only once every block, or less the replacement cycling > > malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with > > the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases > > the absolute fee cost for the attacker. > > > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > > > Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast > > the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the > > flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be > > extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a > > cooperative message or go on-chain with it to claim the accepted HTLC output. > > > > Implemented and deployed by Eclair and LND. > > > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid by the attacker, a risk > > of the HTLC-preimage being detected or discovered by the honest lightning node, or the HTLC-timeout > > to slip in a winning block template. Bumping the default CLTV delta hardens the odds of success of a > > simple replacement cycling attack. > > > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > > > ## Affected Bitcoin Protocols and Applications > > > > From my understanding the following list of Bitcoin protocols and applications could be affected by > > new denial-of-service vectors under some level of network mempools congestion. Neither tests or > > advanced review of specifications (when available) has been conducted for each of them: > > - on-chain DLCs > > - coinjoins > > - payjoins > > - wallets with time-sensitive paths > > - peerswap and submarine swaps > > - batch payouts > > - transaction "accelerators" > > > > Inviting their developers, maintainers and operators to investigate how replacement cycling attacks > > might disrupt their in-mempool chain of transactions, or fee-bumping flows at the shortest delay. > > Simple flows and non-multi-party transactions should not be affected to the best of my understanding. > > > > ## Open Problems: Package Malleability > > > > Pinning attacks have been known for years as a practical vector to compromise lightning channels > > funds safety, under different scenarios (cf. current bip331's motivation section). Mitigations at > > the mempool level have been designed, discussed and are under implementation by the community > > (ancestor package relay + nverrsion=3 policy). Ideally, they should constraint a pinning attacker to > > always attach a high feerate package (commitment + CPFP) to replace the honest package, or allow a > > honest lightning node to overbid a malicious pinning package and get its time-sensitive transaction > > optimistically included in the chain. > > > > Replacement cycling attack seem to offer a new way to neutralize the design goals of package relay > > and its companion nversion=3 policy, where an attacker package RBF a honest package out of the > > mempool to subsequently double-spend its own high-fee child with a transaction unrelated to the > > channel. As the remaining commitment transaction is pre-signed with a minimal relay fee, it can be > > evicted out of the mempool. > > > > A functional test exercising a simple replacement cycling of a lightning channel commitment > > transaction on top of the nversion=3 code branch is available: > > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> > > > > ## Discovery > > > > In 2018, the issue of static fees for pre-signed lightning transactions is made more widely known, > > the carve-out exemption in mempool rules to mitigate in-mempool package limits pinning and the > > anchor output pattern are proposed. > > > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued discussion of the anchor > > output pattern as a dynamic fee-bumping method. > > > > In 2020, draft of anchor output submitted to the bolts. Initial finding of economic pinning against > > lightning commitment and second-stage HTLC transactions. Subsequent discussions of a > > preimage-overlay network or package-relay as mitigations. Public call made to inquiry more on > > potential other transaction-relay jamming attacks affecting lightning. > > > > In 2021, initial work in bitcoin core 22.0 of package acceptance. Continued discussion of the > > pinning attacks and shortcomings of current mempool rules during community-wide online workshops. > > Later the year, in light of all issues for bitcoin second-layers, a proposal is made about killing > > the mempool. > > > > In 2022, bip proposed for package relay and new proposed v3 policy design proposed for a review and > > implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and conceptual questions about > > alignment of mempool rules w.r.t miners incentives are investigated. > > > > Along this year 2022, eltoo lightning channels design are discussed, implemented and reviewed. In > > this context and after discussions on mempool anti-DoS rules, I discovered this new replacement > > cycling attack was affecting deployed lightning channels and immediately reported the finding to > > some bitcoin core developers and lightning maintainers. > > > > ## Timeline > > > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Zhao > > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, Matt Corallo and Olaoluwa > > Osuntunkun > > - 2022-12-23: Sharing to Eugene Siegel (LND) > > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning potential affected projects) > > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers issuers) and initial > > proposal of an early public disclosure > > - 2022-01-19: Collection of analysis if other second-layers and multi-party applications affected. > > LN mitigations development starts. > > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > > - 2023-06-20: LN mitigations implemented and progressively released. Week of the 16 october proposed > > for full disclosure. > > - 2023-08-10: CVEs assigned by MITRE > > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling attack existence to > > security@bitcoincore•org <mailto:security@bitcoincore.org>. > > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 > > and replacement cycling attacks > > > > ## Conclusion > > > > Despite the line of mitigations adopted and deployed by current major lightning implementations, I > > believe replacement cycling attacks are still practical for advanced attackers. Beyond this new > > attack might come as a way to partially or completely defeat some of the pinning mitigations which > > have been working for years as a community. > > > > As of today, it is uncertain to me if lightning is not affected by a more severe long-term package > > malleability critical security issue under current consensus rules, and if any other time-sensitive > > multi-party protocol, designed or deployed isn't de facto affected too (loss of funds or denial of > > service). > > > > Assuming analysis on package malleability is correct, it is unclear to me if it can be corrected by > > changes in replacement / eviction rules or mempool chain of transactions processing strategy. > > Inviting my technical peers and the bitcoin community to look more on this issue, including to > > dissent. I'll be the first one pleased if I'm fundamentally wrong on those issues, or if any element > > has not been weighted with the adequate technical accuracy it deserves. > > > > Do not trust, verify. All mistakes and opinions are my own. > > > > Antoine > > > > "meet with Triumph and Disaster. And treat those two impostors just the same" - K. > > > > _______________________________________________ > > Lightning-dev mailing list > > Lightning-dev@lists•linuxfoundation•org > > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Hi Matt,
This mitigation is mentioned in the attached paper (see subsection 3.4 defensive fee-rebroadcasting) https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf
As soon as you start to have a bit of a mempool backlog and the defensive fractional fee HTLC-timeout stays stuck, it gives the advantage to the attacker again.
Beyond that, I think an attacker can replace-cycle multiple honest HTLC-timeout with a single malicious HTLC-preimage (with a sequence of replacement, not concurrently) paying the absolute fee, while only encumbering the RBF penalty. I didn't test this specific behavior, though the "fees" math doesn't seem at the advantage of the defenders at first sight.
Best, Antoine
Le jeu. 19 oct. 2023 à 17:23, Matt Morehouse <mattmorehouse@gmail•com> a écrit :
Quoted text (454 lines)
> On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation•org> wrote: > > > > There appears to be some confusion about this issue and the mitigations. > To be clear, the deployed > > mitigations are not expected to fix this issue, its arguable if they > provide anything more than a PR > > statement. > > > > There are two discussed mitigations here - mempool scanning and > transaction re-signing/re-broadcasting. > > > > Mempool scanning relies on regularly checking the mempool of a local > node to see if we can catch the > > replacement cycle mid-cycle. It only works if wee see the first > transaction before the second > > transaction replaces it. > > > > Today, a large majority of lightning nodes run on machines with a > Bitcoin node on the same IP > > address, making it very clear what the "local node" of the lightning > node is. An attacker can > > trivially use this information to connect to said local node and do the > replacement quickly, > > preventing the victim from seeing the replacement. > > > > More generally, however, similar discoverability is true for mining > pools. An attacker performing > > this attack is likely to do the replacement attack on a miner's node > directly, potentially reducing > > the reach of the intermediate transaction to only miners, such that the > victim can never discover it > > at all. > > > > The second mitigation is similarly pathetic. Re-signing and > re-broadcasting the victim's transaction > > in an attempt to get it to miners even if its been removed may work, if > the attacker is super lazy > > and didn't finish writing their attack system. If the attacker is > connected to a large majority of > > hashrate (which has historically been fairly doable), they can simply do > their replacement in a > > cycle aggressively and arbitrarily reduce the probability that the > victim's transaction gets confirmed. > > What if the honest node aggressively fee-bumps and retransmits the > HTLC-timeout as the CLTV delta deadline approaches, as suggested by > Ziggie? Say, within 10 blocks of the deadline, the honest node starts > increasing the fee by 1/10th the HTLC value for each non-confirmation. > > This "scorched earth" approach may cost the honest node considerable > fees, but it will cost the attacker even more, since each attacker > replacement needs to burn at least as much as the HTLC-timeout fees, > and the attacker will need to do a replacement every time the honest > node fee bumps. > > I think this fee-bumping policy will provide sufficient defense even > if the attacker is replacement-cycling directly in miners' mempools > and the victim has no visibility into the attack. > > > > > Now, the above is all true in a spherical cow kinda world, and the P2P > network has plenty of slow > > nodes and strange behavior. Its possible that these mitigations might, > by some stroke of luck, > > happen to catch such an attack and prevent it, because something took > longer than the attacker > > intended or whatever. But, that's a far cry from any kind of material > "fix" for the issue. > > > > Ultimately the only fix for this issue will be when miners keep a > history of transactions they've > > seen and try them again after they may be able to enter the mempool > because of an attack like this. > > > > Matt > > > > On 10/16/23 12:57 PM, Antoine Riard wrote: > > > (cross-posting mempool issues identified are exposing lightning chan > to loss of funds risks, other > > > multi-party bitcoin apps might be affected) > > > > > > Hi, > > > > > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and > > > incentives compatibility of the mempool anti-DoS rules, a new > transaction-relay jamming attack > > > affecting lightning channels was discovered. > > > > > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning > > > routing hops carrying HTLC traffic to loss of funds security risks, > both legacy and anchor output > > > channels. A potential exploitation plausibly happening even without > network mempools congestion. > > > > > > Mitigations have been designed, implemented and deployed by all major > lightning implementations > > > during the last months. > > > > > > Please find attached the release numbers, where the mitigations should > be present: > > > - LDK: v0.0.118 - CVE-2023 -40231 > > > - Eclair: v0.9.0 - CVE-2023-40232 > > > - LND: v.0.17.0-beta - CVE-2023-40233 > > > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > > > > > While neither replacement cycling attacks have been observed or > reported in the wild since the last > > > ~10 months or experimented in real-world conditions on bitcoin mainet, > functional test is available > > > exercising the affected lightning channel against bitcoin core mempool > (26.0 release cycle). > > > > > > It is understood that a simple replacement cycling attack does not > demand privileged capabilities > > > from an attacker (e.g no low-hashrate power) and only access to basic > bitcoin and lightning > > > software. Yet I still think executing such an attack successfully > requests a fair amount of bitcoin > > > technical know-how and decent preparation. > > > > > > From my understanding of those issues, it is yet to be determined if > the mitigations deployed are > > > robust enough in face of advanced replacement cycling attackers, > especially ones able to combine > > > different classes of transaction-relay jamming such as pinnings or > vetted with more privileged > > > capabilities. > > > > > > Please find a list of potential affected bitcoin applications in this > full disclosure report using > > > bitcoin script timelocks or multi-party transactions, albeit no > immediate security risk exposure as > > > severe as the ones affecting lightning has been identified. Only > cursory review of non-lightning > > > applications has been conducted so far. > > > > > > There is a paper published summarizing replacement cycling attacks on > the lightning network: > > > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > > < > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > > > > > > > ## Problem > > > > > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and > > > offered HTLC on outgoing link) should settle the outgoing state with > either a success or timeout > > > before the incoming state timelock becomes final and an asymmetric > defavorable settlement might > > > happen (cf "Flood & Loot: A Systematic Attack on The Lightning > Network" section 2.3 for a classical > > > exposition of this lightning security property). > > > > > > Failure to satisfy this settlement requirement exposes a forwarding > hop to a loss of fund risk where > > > the offered HTLC is spent by the outgoing link counterparty's > HTLC-preimage and the accepted HTLC is > > > spent by the incoming link counterparty's HTLC-timeout. > > > > > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of > > > `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this > exact interval value being an > > > implementation and node policy setting. As a minimal value, the > specification recommends 34 blocks > > > of interval. If the timelock expiration I of the inbound HTLC is equal > to 100 from chain tip, the > > > timelock expiration O of the outbound HTLC must be equal to 66 blocks > from chain tip, giving a > > > reasonable buffer of reaction to the lightning forwarding node. > > > > > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the > > > counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when > O is reached, the lightning > > > node should broadcast its commitment transaction. Once the commitment > is confirmed (if anchor and > > > the 1 CSV encumbrance is present), the lightning node broadcasts and > confirms its HTLC-timeout > > > before I height is reached. > > > > > > Here enter a replacement cycling attack. A malicious channel > counterparty can broadcast its > > > HTLC-preimage transaction with a higher absolute fee and higher > feerate than the honest HTLC-timeout > > > of the victim lightning node and triggers a replacement. Both for > legacy and anchor output channels, > > > a HTLC-preimage on a counterparty commitment transaction is malleable, > i.e additional inputs or > > > outputs can be added. The HTLC-preimage spends an unconfirmed and > unrelated to the channel parent > > > transaction M and conflicts its child. > > > > > > As the HTLC-preimage spends an unconfirmed input that was already > included in the unconfirmed and > > > unrelated child transaction (rule 2), pays an absolute higher fee of > at least the sum paid by the > > > HTLC-timeout and child transaction (rule 3) and the HTLC-preimage > feerate is greater than all > > > directly conflicting transactions (rule 6), the replacement is > accepted. The honest HTLC-timeout is > > > evicted out of the mempool. > > > > > > In an ulterior move, the malicious counterparty can replace the parent > transaction itself with > > > another candidate N satisfying the replacement rules, triggering the > eviction of the malicious > > > HTLC-preimage from the mempool as it was a child of the parent T. > > > > > > There is no spending candidate of the offered HTLC output for the > current block laying in network > > > mempools. > > > > > > This replacement cycling tricks can be repeated for each rebroadcast > attempt of the HTLC-timeout by > > > the honest lightning node until expiration of the inbound HTLC > timelock I. Once this height is > > > reached a HTLC-timeout is broadcast by the counterparty's on the > incoming link in collusion with the > > > one on the outgoing link broadcasting its own HTLC-preimage. > > > > > > The honest Lightning node has been "double-spent" in its HTLC > forwarding. > > > > > > As a notable factor impacting the success of the attack, a lightning > node's honest HTLC-timeout > > > might be included in the block template of the miner winning the block > race and therefore realizes a > > > spent of the offered output. In practice, a replacement cycling attack > might over-connect to miners' > > > mempools and public reachable nodes to succeed in a fast eviction of > the HTLC-timeout by its > > > HTLC-preimage. As this latter transaction can come with a better > ancestor-score, it should be picked > > > up on the flight by economically competitive miners. > > > > > > A functional test exercising a simple replacement cycling of a HTLC > transaction on bitcoin core > > > mempool is available: > > > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > > <https://github.com/ariard/bitcoin/commits/2023-test-mempool> > > > > > > ## Deployed LN mitigations > > > > > > Aggressive rebroadcasting: As the replacement cycling attacker > benefits from the HTLC-timeout being > > > usually broadcast by lightning nodes only once every block, or less > the replacement cycling > > > malicious transactions paid only equal the sum of the absolute fees > paid by the HTLC, adjusted with > > > the replacement penalty. Rebroadcasting randomly and multiple times > before the next block increases > > > the absolute fee cost for the attacker. > > > > > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > > > > > Local-mempool preimage monitoring: As the replacement cycling attacker > in a simple setup broadcast > > > the HTLC-preimage to all the network mempools, the honest lightning > node is able to catch on the > > > flight the unconfirmed HTLC-preimage, before its subsequent mempool > replacement. The preimage can be > > > extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a > > > cooperative message or go on-chain with it to claim the accepted HTLC > output. > > > > > > Implemented and deployed by Eclair and LND. > > > > > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost > paid by the attacker, a risk > > > of the HTLC-preimage being detected or discovered by the honest > lightning node, or the HTLC-timeout > > > to slip in a winning block template. Bumping the default CLTV delta > hardens the odds of success of a > > > simple replacement cycling attack. > > > > > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > > > > > ## Affected Bitcoin Protocols and Applications > > > > > > From my understanding the following list of Bitcoin protocols and > applications could be affected by > > > new denial-of-service vectors under some level of network mempools > congestion. Neither tests or > > > advanced review of specifications (when available) has been conducted > for each of them: > > > - on-chain DLCs > > > - coinjoins > > > - payjoins > > > - wallets with time-sensitive paths > > > - peerswap and submarine swaps > > > - batch payouts > > > - transaction "accelerators" > > > > > > Inviting their developers, maintainers and operators to investigate > how replacement cycling attacks > > > might disrupt their in-mempool chain of transactions, or fee-bumping > flows at the shortest delay. > > > Simple flows and non-multi-party transactions should not be affected > to the best of my understanding. > > > > > > ## Open Problems: Package Malleability > > > > > > Pinning attacks have been known for years as a practical vector to > compromise lightning channels > > > funds safety, under different scenarios (cf. current bip331's > motivation section). Mitigations at > > > the mempool level have been designed, discussed and are under > implementation by the community > > > (ancestor package relay + nverrsion=3 policy). Ideally, they should > constraint a pinning attacker to > > > always attach a high feerate package (commitment + CPFP) to replace > the honest package, or allow a > > > honest lightning node to overbid a malicious pinning package and get > its time-sensitive transaction > > > optimistically included in the chain. > > > > > > Replacement cycling attack seem to offer a new way to neutralize the > design goals of package relay > > > and its companion nversion=3 policy, where an attacker package RBF a > honest package out of the > > > mempool to subsequently double-spend its own high-fee child with a > transaction unrelated to the > > > channel. As the remaining commitment transaction is pre-signed with a > minimal relay fee, it can be > > > evicted out of the mempool. > > > > > > A functional test exercising a simple replacement cycling of a > lightning channel commitment > > > transaction on top of the nversion=3 code branch is available: > > > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > > <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> > > > > > > ## Discovery > > > > > > In 2018, the issue of static fees for pre-signed lightning > transactions is made more widely known, > > > the carve-out exemption in mempool rules to mitigate in-mempool > package limits pinning and the > > > anchor output pattern are proposed. > > > > > > In 2019, bitcoin core 0.19 is released with carve-out support. > Continued discussion of the anchor > > > output pattern as a dynamic fee-bumping method. > > > > > > In 2020, draft of anchor output submitted to the bolts. Initial > finding of economic pinning against > > > lightning commitment and second-stage HTLC transactions. Subsequent > discussions of a > > > preimage-overlay network or package-relay as mitigations. Public call > made to inquiry more on > > > potential other transaction-relay jamming attacks affecting lightning. > > > > > > In 2021, initial work in bitcoin core 22.0 of package acceptance. > Continued discussion of the > > > pinning attacks and shortcomings of current mempool rules during > community-wide online workshops. > > > Later the year, in light of all issues for bitcoin second-layers, a > proposal is made about killing > > > the mempool. > > > > > > In 2022, bip proposed for package relay and new proposed v3 policy > design proposed for a review and > > > implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and > conceptual questions about > > > alignment of mempool rules w.r.t miners incentives are investigated. > > > > > > Along this year 2022, eltoo lightning channels design are discussed, > implemented and reviewed. In > > > this context and after discussions on mempool anti-DoS rules, I > discovered this new replacement > > > cycling attack was affecting deployed lightning channels and > immediately reported the finding to > > > some bitcoin core developers and lightning maintainers. > > > > > > ## Timeline > > > > > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, > Greg Sanders and Gloria Zhao > > > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien > Teinturier, Matt Corallo and Olaoluwa > > > Osuntunkun > > > - 2022-12-23: Sharing to Eugene Siegel (LND) > > > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot > (non-lightning potential affected projects) > > > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and > cross-layers issuers) and initial > > > proposal of an early public disclosure > > > - 2022-01-19: Collection of analysis if other second-layers and > multi-party applications affected. > > > LN mitigations development starts. > > > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > > > - 2023-06-20: LN mitigations implemented and progressively released. > Week of the 16 october proposed > > > for full disclosure. > > > - 2023-08-10: CVEs assigned by MITRE > > > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement > cycling attack existence to > > > security@bitcoincore•org <mailto:security@bitcoincore.org>. > > > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / > CVE-2023-40233 / CVE-2023-40234 > > > and replacement cycling attacks > > > > > > ## Conclusion > > > > > > Despite the line of mitigations adopted and deployed by current major > lightning implementations, I > > > believe replacement cycling attacks are still practical for advanced > attackers. Beyond this new > > > attack might come as a way to partially or completely defeat some of > the pinning mitigations which > > > have been working for years as a community. > > > > > > As of today, it is uncertain to me if lightning is not affected by a > more severe long-term package > > > malleability critical security issue under current consensus rules, > and if any other time-sensitive > > > multi-party protocol, designed or deployed isn't de facto affected too > (loss of funds or denial of > > > service). > > > > > > Assuming analysis on package malleability is correct, it is unclear to > me if it can be corrected by > > > changes in replacement / eviction rules or mempool chain of > transactions processing strategy. > > > Inviting my technical peers and the bitcoin community to look more on > this issue, including to > > > dissent. I'll be the first one pleased if I'm fundamentally wrong on > those issues, or if any element > > > has not been weighted with the adequate technical accuracy it deserves. > > > > > > Do not trust, verify. All mistakes and opinions are my own. > > > > > > Antoine > > > > > > "meet with Triumph and Disaster. And treat those two impostors just > the same" - K. > > > > > > _______________________________________________ > > > Lightning-dev mailing list > > > Lightning-dev@lists•linuxfoundation•org > > > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists•linuxfoundation•org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Thu, Oct 19, 2023 at 5:22 PM Antoine Riard <antoine.riard@gmail•com> wrote:
Quoted text (9 lines)
> > Hi Matt, > > This mitigation is mentioned in the attached paper (see subsection 3.4 defensive fee-rebroadcasting) > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > As soon as you start to have a bit of a mempool backlog and the defensive fractional fee HTLC-timeout stays stuck, it gives the advantage to the attacker again. > > Beyond that, I think an attacker can replace-cycle multiple honest HTLC-timeout with a single malicious HTLC-preimage (with a sequence of replacement, not concurrently) paying the absolute fee, while only encumbering the RBF penalty. I didn't test this specific behavior, though the "fees" math doesn't seem at the advantage of the defenders at first sight.
Right, some replacement cycles can be avoided by the attacker to reduce the cost of the attack. But with the defender implementing a scorched-earth fee bumping policy, eventually either (1) the HTLC-timeout *will* confirm in the next block or (2) the attacker must pay more fees than the HTLC-timeout fees to replace it. As the CLTV delta deadline approaches, the fees in case 2 may be 50%, 80%, even 100% of the HTLC value under such a scorched earth policy. So even if the attacker only has to do one replacement cycle right before the deadline, the attack can be made unprofitable. And in practice, with HTLC values significantly greater than the next-block fee cost, the attacker will need to do multiple replacements as the deadline approaches.
And of course this linear scorched earth policy is just an illustration. We should further tune the fee bumping curve across the full CLTV delta to ensure minimal fees paid when not under attack. But as we approach the deadline, it makes sense to become very aggressive both to get our transaction confirmed during high mempool congestion and to punish replacement-cycling attackers.
Quoted text (314 lines)
> > Best, > Antoine > > Le jeu. 19 oct. 2023 à 17:23, Matt Morehouse <mattmorehouse@gmail•com> a écrit : >> >> On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev >> <bitcoin-dev@lists•linuxfoundation•org> wrote: >> > >> > There appears to be some confusion about this issue and the mitigations. To be clear, the deployed >> > mitigations are not expected to fix this issue, its arguable if they provide anything more than a PR >> > statement. >> > >> > There are two discussed mitigations here - mempool scanning and transaction re-signing/re-broadcasting. >> > >> > Mempool scanning relies on regularly checking the mempool of a local node to see if we can catch the >> > replacement cycle mid-cycle. It only works if wee see the first transaction before the second >> > transaction replaces it. >> > >> > Today, a large majority of lightning nodes run on machines with a Bitcoin node on the same IP >> > address, making it very clear what the "local node" of the lightning node is. An attacker can >> > trivially use this information to connect to said local node and do the replacement quickly, >> > preventing the victim from seeing the replacement. >> > >> > More generally, however, similar discoverability is true for mining pools. An attacker performing >> > this attack is likely to do the replacement attack on a miner's node directly, potentially reducing >> > the reach of the intermediate transaction to only miners, such that the victim can never discover it >> > at all. >> > >> > The second mitigation is similarly pathetic. Re-signing and re-broadcasting the victim's transaction >> > in an attempt to get it to miners even if its been removed may work, if the attacker is super lazy >> > and didn't finish writing their attack system. If the attacker is connected to a large majority of >> > hashrate (which has historically been fairly doable), they can simply do their replacement in a >> > cycle aggressively and arbitrarily reduce the probability that the victim's transaction gets confirmed. >> >> What if the honest node aggressively fee-bumps and retransmits the >> HTLC-timeout as the CLTV delta deadline approaches, as suggested by >> Ziggie? Say, within 10 blocks of the deadline, the honest node starts >> increasing the fee by 1/10th the HTLC value for each non-confirmation. >> >> This "scorched earth" approach may cost the honest node considerable >> fees, but it will cost the attacker even more, since each attacker >> replacement needs to burn at least as much as the HTLC-timeout fees, >> and the attacker will need to do a replacement every time the honest >> node fee bumps. >> >> I think this fee-bumping policy will provide sufficient defense even >> if the attacker is replacement-cycling directly in miners' mempools >> and the victim has no visibility into the attack. >> >> > >> > Now, the above is all true in a spherical cow kinda world, and the P2P network has plenty of slow >> > nodes and strange behavior. Its possible that these mitigations might, by some stroke of luck, >> > happen to catch such an attack and prevent it, because something took longer than the attacker >> > intended or whatever. But, that's a far cry from any kind of material "fix" for the issue. >> > >> > Ultimately the only fix for this issue will be when miners keep a history of transactions they've >> > seen and try them again after they may be able to enter the mempool because of an attack like this. >> > >> > Matt >> > >> > On 10/16/23 12:57 PM, Antoine Riard wrote: >> > > (cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other >> > > multi-party bitcoin apps might be affected) >> > > >> > > Hi, >> > > >> > > End of last year (December 2022), amid technical discussions on eltoo payment channels and >> > > incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack >> > > affecting lightning channels was discovered. >> > > >> > > After careful analysis, it turns out this attack is practical and immediately exposed lightning >> > > routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output >> > > channels. A potential exploitation plausibly happening even without network mempools congestion. >> > > >> > > Mitigations have been designed, implemented and deployed by all major lightning implementations >> > > during the last months. >> > > >> > > Please find attached the release numbers, where the mitigations should be present: >> > > - LDK: v0.0.118 - CVE-2023 -40231 >> > > - Eclair: v0.9.0 - CVE-2023-40232 >> > > - LND: v.0.17.0-beta - CVE-2023-40233 >> > > - Core-Lightning: v.23.08.01 - CVE-2023-40234 >> > > >> > > While neither replacement cycling attacks have been observed or reported in the wild since the last >> > > ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available >> > > exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle). >> > > >> > > It is understood that a simple replacement cycling attack does not demand privileged capabilities >> > > from an attacker (e.g no low-hashrate power) and only access to basic bitcoin and lightning >> > > software. Yet I still think executing such an attack successfully requests a fair amount of bitcoin >> > > technical know-how and decent preparation. >> > > >> > > From my understanding of those issues, it is yet to be determined if the mitigations deployed are >> > > robust enough in face of advanced replacement cycling attackers, especially ones able to combine >> > > different classes of transaction-relay jamming such as pinnings or vetted with more privileged >> > > capabilities. >> > > >> > > Please find a list of potential affected bitcoin applications in this full disclosure report using >> > > bitcoin script timelocks or multi-party transactions, albeit no immediate security risk exposure as >> > > severe as the ones affecting lightning has been identified. Only cursory review of non-lightning >> > > applications has been conducted so far. >> > > >> > > There is a paper published summarizing replacement cycling attacks on the lightning network: >> > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf >> > > <https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf> >> > > >> > > ## Problem >> > > >> > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted HTLC on incoming link and >> > > offered HTLC on outgoing link) should settle the outgoing state with either a success or timeout >> > > before the incoming state timelock becomes final and an asymmetric defavorable settlement might >> > > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" section 2.3 for a classical >> > > exposition of this lightning security property). >> > > >> > > Failure to satisfy this settlement requirement exposes a forwarding hop to a loss of fund risk where >> > > the offered HTLC is spent by the outgoing link counterparty's HTLC-preimage and the accepted HTLC is >> > > spent by the incoming link counterparty's HTLC-timeout. >> > > >> > > The specification mandates the incoming HTLC expiration timelock to be spaced out by an interval of >> > > `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this exact interval value being an >> > > implementation and node policy setting. As a minimal value, the specification recommends 34 blocks >> > > of interval. If the timelock expiration I of the inbound HTLC is equal to 100 from chain tip, the >> > > timelock expiration O of the outbound HTLC must be equal to 66 blocks from chain tip, giving a >> > > reasonable buffer of reaction to the lightning forwarding node. >> > > >> > > In the lack of cooperative off-chain settlement of the HTLC on the outgoing link negotiated with the >> > > counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the lightning >> > > node should broadcast its commitment transaction. Once the commitment is confirmed (if anchor and >> > > the 1 CSV encumbrance is present), the lightning node broadcasts and confirms its HTLC-timeout >> > > before I height is reached. >> > > >> > > Here enter a replacement cycling attack. A malicious channel counterparty can broadcast its >> > > HTLC-preimage transaction with a higher absolute fee and higher feerate than the honest HTLC-timeout >> > > of the victim lightning node and triggers a replacement. Both for legacy and anchor output channels, >> > > a HTLC-preimage on a counterparty commitment transaction is malleable, i.e additional inputs or >> > > outputs can be added. The HTLC-preimage spends an unconfirmed and unrelated to the channel parent >> > > transaction M and conflicts its child. >> > > >> > > As the HTLC-preimage spends an unconfirmed input that was already included in the unconfirmed and >> > > unrelated child transaction (rule 2), pays an absolute higher fee of at least the sum paid by the >> > > HTLC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is greater than all >> > > directly conflicting transactions (rule 6), the replacement is accepted. The honest HTLC-timeout is >> > > evicted out of the mempool. >> > > >> > > In an ulterior move, the malicious counterparty can replace the parent transaction itself with >> > > another candidate N satisfying the replacement rules, triggering the eviction of the malicious >> > > HTLC-preimage from the mempool as it was a child of the parent T. >> > > >> > > There is no spending candidate of the offered HTLC output for the current block laying in network >> > > mempools. >> > > >> > > This replacement cycling tricks can be repeated for each rebroadcast attempt of the HTLC-timeout by >> > > the honest lightning node until expiration of the inbound HTLC timelock I. Once this height is >> > > reached a HTLC-timeout is broadcast by the counterparty's on the incoming link in collusion with the >> > > one on the outgoing link broadcasting its own HTLC-preimage. >> > > >> > > The honest Lightning node has been "double-spent" in its HTLC forwarding. >> > > >> > > As a notable factor impacting the success of the attack, a lightning node's honest HTLC-timeout >> > > might be included in the block template of the miner winning the block race and therefore realizes a >> > > spent of the offered output. In practice, a replacement cycling attack might over-connect to miners' >> > > mempools and public reachable nodes to succeed in a fast eviction of the HTLC-timeout by its >> > > HTLC-preimage. As this latter transaction can come with a better ancestor-score, it should be picked >> > > up on the flight by economically competitive miners. >> > > >> > > A functional test exercising a simple replacement cycling of a HTLC transaction on bitcoin core >> > > mempool is available: >> > > https://github.com/ariard/bitcoin/commits/2023-test-mempool >> > > <https://github.com/ariard/bitcoin/commits/2023-test-mempool> >> > > >> > > ## Deployed LN mitigations >> > > >> > > Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being >> > > usually broadcast by lightning nodes only once every block, or less the replacement cycling >> > > malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with >> > > the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases >> > > the absolute fee cost for the attacker. >> > > >> > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >> > > >> > > Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast >> > > the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the >> > > flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be >> > > extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a >> > > cooperative message or go on-chain with it to claim the accepted HTLC output. >> > > >> > > Implemented and deployed by Eclair and LND. >> > > >> > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid by the attacker, a risk >> > > of the HTLC-preimage being detected or discovered by the honest lightning node, or the HTLC-timeout >> > > to slip in a winning block template. Bumping the default CLTV delta hardens the odds of success of a >> > > simple replacement cycling attack. >> > > >> > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. >> > > >> > > ## Affected Bitcoin Protocols and Applications >> > > >> > > From my understanding the following list of Bitcoin protocols and applications could be affected by >> > > new denial-of-service vectors under some level of network mempools congestion. Neither tests or >> > > advanced review of specifications (when available) has been conducted for each of them: >> > > - on-chain DLCs >> > > - coinjoins >> > > - payjoins >> > > - wallets with time-sensitive paths >> > > - peerswap and submarine swaps >> > > - batch payouts >> > > - transaction "accelerators" >> > > >> > > Inviting their developers, maintainers and operators to investigate how replacement cycling attacks >> > > might disrupt their in-mempool chain of transactions, or fee-bumping flows at the shortest delay. >> > > Simple flows and non-multi-party transactions should not be affected to the best of my understanding. >> > > >> > > ## Open Problems: Package Malleability >> > > >> > > Pinning attacks have been known for years as a practical vector to compromise lightning channels >> > > funds safety, under different scenarios (cf. current bip331's motivation section). Mitigations at >> > > the mempool level have been designed, discussed and are under implementation by the community >> > > (ancestor package relay + nverrsion=3 policy). Ideally, they should constraint a pinning attacker to >> > > always attach a high feerate package (commitment + CPFP) to replace the honest package, or allow a >> > > honest lightning node to overbid a malicious pinning package and get its time-sensitive transaction >> > > optimistically included in the chain. >> > > >> > > Replacement cycling attack seem to offer a new way to neutralize the design goals of package relay >> > > and its companion nversion=3 policy, where an attacker package RBF a honest package out of the >> > > mempool to subsequently double-spend its own high-fee child with a transaction unrelated to the >> > > channel. As the remaining commitment transaction is pre-signed with a minimal relay fee, it can be >> > > evicted out of the mempool. >> > > >> > > A functional test exercising a simple replacement cycling of a lightning channel commitment >> > > transaction on top of the nversion=3 code branch is available: >> > > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 >> > > <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> >> > > >> > > ## Discovery >> > > >> > > In 2018, the issue of static fees for pre-signed lightning transactions is made more widely known, >> > > the carve-out exemption in mempool rules to mitigate in-mempool package limits pinning and the >> > > anchor output pattern are proposed. >> > > >> > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued discussion of the anchor >> > > output pattern as a dynamic fee-bumping method. >> > > >> > > In 2020, draft of anchor output submitted to the bolts. Initial finding of economic pinning against >> > > lightning commitment and second-stage HTLC transactions. Subsequent discussions of a >> > > preimage-overlay network or package-relay as mitigations. Public call made to inquiry more on >> > > potential other transaction-relay jamming attacks affecting lightning. >> > > >> > > In 2021, initial work in bitcoin core 22.0 of package acceptance. Continued discussion of the >> > > pinning attacks and shortcomings of current mempool rules during community-wide online workshops. >> > > Later the year, in light of all issues for bitcoin second-layers, a proposal is made about killing >> > > the mempool. >> > > >> > > In 2022, bip proposed for package relay and new proposed v3 policy design proposed for a review and >> > > implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and conceptual questions about >> > > alignment of mempool rules w.r.t miners incentives are investigated. >> > > >> > > Along this year 2022, eltoo lightning channels design are discussed, implemented and reviewed. In >> > > this context and after discussions on mempool anti-DoS rules, I discovered this new replacement >> > > cycling attack was affecting deployed lightning channels and immediately reported the finding to >> > > some bitcoin core developers and lightning maintainers. >> > > >> > > ## Timeline >> > > >> > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Zhao >> > > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, Matt Corallo and Olaoluwa >> > > Osuntunkun >> > > - 2022-12-23: Sharing to Eugene Siegel (LND) >> > > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning potential affected projects) >> > > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers issuers) and initial >> > > proposal of an early public disclosure >> > > - 2022-01-19: Collection of analysis if other second-layers and multi-party applications affected. >> > > LN mitigations development starts. >> > > - 2023-05-04: Sharing to Wilmer Paulino (LDK) >> > > - 2023-06-20: LN mitigations implemented and progressively released. Week of the 16 october proposed >> > > for full disclosure. >> > > - 2023-08-10: CVEs assigned by MITRE >> > > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling attack existence to >> > > security@bitcoincore•org <mailto:security@bitcoincore.org>. >> > > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 >> > > and replacement cycling attacks >> > > >> > > ## Conclusion >> > > >> > > Despite the line of mitigations adopted and deployed by current major lightning implementations, I >> > > believe replacement cycling attacks are still practical for advanced attackers. Beyond this new >> > > attack might come as a way to partially or completely defeat some of the pinning mitigations which >> > > have been working for years as a community. >> > > >> > > As of today, it is uncertain to me if lightning is not affected by a more severe long-term package >> > > malleability critical security issue under current consensus rules, and if any other time-sensitive >> > > multi-party protocol, designed or deployed isn't de facto affected too (loss of funds or denial of >> > > service). >> > > >> > > Assuming analysis on package malleability is correct, it is unclear to me if it can be corrected by >> > > changes in replacement / eviction rules or mempool chain of transactions processing strategy. >> > > Inviting my technical peers and the bitcoin community to look more on this issue, including to >> > > dissent. I'll be the first one pleased if I'm fundamentally wrong on those issues, or if any element >> > > has not been weighted with the adequate technical accuracy it deserves. >> > > >> > > Do not trust, verify. All mistakes and opinions are my own. >> > > >> > > Antoine >> > > >> > > "meet with Triumph and Disaster. And treat those two impostors just the same" - K. >> > > >> > > _______________________________________________ >> > > Lightning-dev mailing list >> > > Lightning-dev@lists•linuxfoundation•org >> > > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev >> > _______________________________________________ >> > bitcoin-dev mailing list >> > bitcoin-dev@lists•linuxfoundation•org >> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
That certainly helps, yes, and I think many nodes do something akin to this already, but I'm not sure we can say that the problem has been fixed if the victim has to spend way more than the prevailing mempool fees (and potentially burn a large % of their HTLC value) :).
Matt
On 10/19/23 12:23 PM, Matt Morehouse wrote:
Quoted text (308 lines)
> On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation•org> wrote: >> >> There appears to be some confusion about this issue and the mitigations. To be clear, the deployed >> mitigations are not expected to fix this issue, its arguable if they provide anything more than a PR >> statement. >> >> There are two discussed mitigations here - mempool scanning and transaction re-signing/re-broadcasting. >> >> Mempool scanning relies on regularly checking the mempool of a local node to see if we can catch the >> replacement cycle mid-cycle. It only works if wee see the first transaction before the second >> transaction replaces it. >> >> Today, a large majority of lightning nodes run on machines with a Bitcoin node on the same IP >> address, making it very clear what the "local node" of the lightning node is. An attacker can >> trivially use this information to connect to said local node and do the replacement quickly, >> preventing the victim from seeing the replacement. >> >> More generally, however, similar discoverability is true for mining pools. An attacker performing >> this attack is likely to do the replacement attack on a miner's node directly, potentially reducing >> the reach of the intermediate transaction to only miners, such that the victim can never discover it >> at all. >> >> The second mitigation is similarly pathetic. Re-signing and re-broadcasting the victim's transaction >> in an attempt to get it to miners even if its been removed may work, if the attacker is super lazy >> and didn't finish writing their attack system. If the attacker is connected to a large majority of >> hashrate (which has historically been fairly doable), they can simply do their replacement in a >> cycle aggressively and arbitrarily reduce the probability that the victim's transaction gets confirmed. > > What if the honest node aggressively fee-bumps and retransmits the > HTLC-timeout as the CLTV delta deadline approaches, as suggested by > Ziggie? Say, within 10 blocks of the deadline, the honest node starts > increasing the fee by 1/10th the HTLC value for each non-confirmation. > > This "scorched earth" approach may cost the honest node considerable > fees, but it will cost the attacker even more, since each attacker > replacement needs to burn at least as much as the HTLC-timeout fees, > and the attacker will need to do a replacement every time the honest > node fee bumps. > > I think this fee-bumping policy will provide sufficient defense even > if the attacker is replacement-cycling directly in miners' mempools > and the victim has no visibility into the attack. > >> >> Now, the above is all true in a spherical cow kinda world, and the P2P network has plenty of slow >> nodes and strange behavior. Its possible that these mitigations might, by some stroke of luck, >> happen to catch such an attack and prevent it, because something took longer than the attacker >> intended or whatever. But, that's a far cry from any kind of material "fix" for the issue. >> >> Ultimately the only fix for this issue will be when miners keep a history of transactions they've >> seen and try them again after they may be able to enter the mempool because of an attack like this. >> >> Matt >> >> On 10/16/23 12:57 PM, Antoine Riard wrote: >>> (cross-posting mempool issues identified are exposing lightning chan to loss of funds risks, other >>> multi-party bitcoin apps might be affected) >>> >>> Hi, >>> >>> End of last year (December 2022), amid technical discussions on eltoo payment channels and >>> incentives compatibility of the mempool anti-DoS rules, a new transaction-relay jamming attack >>> affecting lightning channels was discovered. >>> >>> After careful analysis, it turns out this attack is practical and immediately exposed lightning >>> routing hops carrying HTLC traffic to loss of funds security risks, both legacy and anchor output >>> channels. A potential exploitation plausibly happening even without network mempools congestion. >>> >>> Mitigations have been designed, implemented and deployed by all major lightning implementations >>> during the last months. >>> >>> Please find attached the release numbers, where the mitigations should be present: >>> - LDK: v0.0.118 - CVE-2023 -40231 >>> - Eclair: v0.9.0 - CVE-2023-40232 >>> - LND: v.0.17.0-beta - CVE-2023-40233 >>> - Core-Lightning: v.23.08.01 - CVE-2023-40234 >>> >>> While neither replacement cycling attacks have been observed or reported in the wild since the last >>> ~10 months or experimented in real-world conditions on bitcoin mainet, functional test is available >>> exercising the affected lightning channel against bitcoin core mempool (26.0 release cycle). >>> >>> It is understood that a simple replacement cycling attack does not demand privileged capabilities >>> from an attacker (e.g no low-hashrate power) and only access to basic bitcoin and lightning >>> software. Yet I still think executing such an attack successfully requests a fair amount of bitcoin >>> technical know-how and decent preparation. >>> >>> From my understanding of those issues, it is yet to be determined if the mitigations deployed are >>> robust enough in face of advanced replacement cycling attackers, especially ones able to combine >>> different classes of transaction-relay jamming such as pinnings or vetted with more privileged >>> capabilities. >>> >>> Please find a list of potential affected bitcoin applications in this full disclosure report using >>> bitcoin script timelocks or multi-party transactions, albeit no immediate security risk exposure as >>> severe as the ones affecting lightning has been identified. Only cursory review of non-lightning >>> applications has been conducted so far. >>> >>> There is a paper published summarizing replacement cycling attacks on the lightning network: >>> https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf >>> <https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf> >>> >>> ## Problem >>> >>> A lightning node allows HTLCs forwarding (in bolt3's parlance accepted HTLC on incoming link and >>> offered HTLC on outgoing link) should settle the outgoing state with either a success or timeout >>> before the incoming state timelock becomes final and an asymmetric defavorable settlement might >>> happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" section 2.3 for a classical >>> exposition of this lightning security property). >>> >>> Failure to satisfy this settlement requirement exposes a forwarding hop to a loss of fund risk where >>> the offered HTLC is spent by the outgoing link counterparty's HTLC-preimage and the accepted HTLC is >>> spent by the incoming link counterparty's HTLC-timeout. >>> >>> The specification mandates the incoming HTLC expiration timelock to be spaced out by an interval of >>> `cltv_expiry_delta` from the outgoing HTLC expiration timelock, this exact interval value being an >>> implementation and node policy setting. As a minimal value, the specification recommends 34 blocks >>> of interval. If the timelock expiration I of the inbound HTLC is equal to 100 from chain tip, the >>> timelock expiration O of the outbound HTLC must be equal to 66 blocks from chain tip, giving a >>> reasonable buffer of reaction to the lightning forwarding node. >>> >>> In the lack of cooperative off-chain settlement of the HTLC on the outgoing link negotiated with the >>> counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the lightning >>> node should broadcast its commitment transaction. Once the commitment is confirmed (if anchor and >>> the 1 CSV encumbrance is present), the lightning node broadcasts and confirms its HTLC-timeout >>> before I height is reached. >>> >>> Here enter a replacement cycling attack. A malicious channel counterparty can broadcast its >>> HTLC-preimage transaction with a higher absolute fee and higher feerate than the honest HTLC-timeout >>> of the victim lightning node and triggers a replacement. Both for legacy and anchor output channels, >>> a HTLC-preimage on a counterparty commitment transaction is malleable, i.e additional inputs or >>> outputs can be added. The HTLC-preimage spends an unconfirmed and unrelated to the channel parent >>> transaction M and conflicts its child. >>> >>> As the HTLC-preimage spends an unconfirmed input that was already included in the unconfirmed and >>> unrelated child transaction (rule 2), pays an absolute higher fee of at least the sum paid by the >>> HTLC-timeout and child transaction (rule 3) and the HTLC-preimage feerate is greater than all >>> directly conflicting transactions (rule 6), the replacement is accepted. The honest HTLC-timeout is >>> evicted out of the mempool. >>> >>> In an ulterior move, the malicious counterparty can replace the parent transaction itself with >>> another candidate N satisfying the replacement rules, triggering the eviction of the malicious >>> HTLC-preimage from the mempool as it was a child of the parent T. >>> >>> There is no spending candidate of the offered HTLC output for the current block laying in network >>> mempools. >>> >>> This replacement cycling tricks can be repeated for each rebroadcast attempt of the HTLC-timeout by >>> the honest lightning node until expiration of the inbound HTLC timelock I. Once this height is >>> reached a HTLC-timeout is broadcast by the counterparty's on the incoming link in collusion with the >>> one on the outgoing link broadcasting its own HTLC-preimage. >>> >>> The honest Lightning node has been "double-spent" in its HTLC forwarding. >>> >>> As a notable factor impacting the success of the attack, a lightning node's honest HTLC-timeout >>> might be included in the block template of the miner winning the block race and therefore realizes a >>> spent of the offered output. In practice, a replacement cycling attack might over-connect to miners' >>> mempools and public reachable nodes to succeed in a fast eviction of the HTLC-timeout by its >>> HTLC-preimage. As this latter transaction can come with a better ancestor-score, it should be picked >>> up on the flight by economically competitive miners. >>> >>> A functional test exercising a simple replacement cycling of a HTLC transaction on bitcoin core >>> mempool is available: >>> https://github.com/ariard/bitcoin/commits/2023-test-mempool >>> <https://github.com/ariard/bitcoin/commits/2023-test-mempool> >>> >>> ## Deployed LN mitigations >>> >>> Aggressive rebroadcasting: As the replacement cycling attacker benefits from the HTLC-timeout being >>> usually broadcast by lightning nodes only once every block, or less the replacement cycling >>> malicious transactions paid only equal the sum of the absolute fees paid by the HTLC, adjusted with >>> the replacement penalty. Rebroadcasting randomly and multiple times before the next block increases >>> the absolute fee cost for the attacker. >>> >>> Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >>> >>> Local-mempool preimage monitoring: As the replacement cycling attacker in a simple setup broadcast >>> the HTLC-preimage to all the network mempools, the honest lightning node is able to catch on the >>> flight the unconfirmed HTLC-preimage, before its subsequent mempool replacement. The preimage can be >>> extracted from the second-stage HTLC-preimage and used to fetch the off-chain inbound HTLC with a >>> cooperative message or go on-chain with it to claim the accepted HTLC output. >>> >>> Implemented and deployed by Eclair and LND. >>> >>> CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid by the attacker, a risk >>> of the HTLC-preimage being detected or discovered by the honest lightning node, or the HTLC-timeout >>> to slip in a winning block template. Bumping the default CLTV delta hardens the odds of success of a >>> simple replacement cycling attack. >>> >>> Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. >>> >>> ## Affected Bitcoin Protocols and Applications >>> >>> From my understanding the following list of Bitcoin protocols and applications could be affected by >>> new denial-of-service vectors under some level of network mempools congestion. Neither tests or >>> advanced review of specifications (when available) has been conducted for each of them: >>> - on-chain DLCs >>> - coinjoins >>> - payjoins >>> - wallets with time-sensitive paths >>> - peerswap and submarine swaps >>> - batch payouts >>> - transaction "accelerators" >>> >>> Inviting their developers, maintainers and operators to investigate how replacement cycling attacks >>> might disrupt their in-mempool chain of transactions, or fee-bumping flows at the shortest delay. >>> Simple flows and non-multi-party transactions should not be affected to the best of my understanding. >>> >>> ## Open Problems: Package Malleability >>> >>> Pinning attacks have been known for years as a practical vector to compromise lightning channels >>> funds safety, under different scenarios (cf. current bip331's motivation section). Mitigations at >>> the mempool level have been designed, discussed and are under implementation by the community >>> (ancestor package relay + nverrsion=3 policy). Ideally, they should constraint a pinning attacker to >>> always attach a high feerate package (commitment + CPFP) to replace the honest package, or allow a >>> honest lightning node to overbid a malicious pinning package and get its time-sensitive transaction >>> optimistically included in the chain. >>> >>> Replacement cycling attack seem to offer a new way to neutralize the design goals of package relay >>> and its companion nversion=3 policy, where an attacker package RBF a honest package out of the >>> mempool to subsequently double-spend its own high-fee child with a transaction unrelated to the >>> channel. As the remaining commitment transaction is pre-signed with a minimal relay fee, it can be >>> evicted out of the mempool. >>> >>> A functional test exercising a simple replacement cycling of a lightning channel commitment >>> transaction on top of the nversion=3 code branch is available: >>> https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 >>> <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> >>> >>> ## Discovery >>> >>> In 2018, the issue of static fees for pre-signed lightning transactions is made more widely known, >>> the carve-out exemption in mempool rules to mitigate in-mempool package limits pinning and the >>> anchor output pattern are proposed. >>> >>> In 2019, bitcoin core 0.19 is released with carve-out support. Continued discussion of the anchor >>> output pattern as a dynamic fee-bumping method. >>> >>> In 2020, draft of anchor output submitted to the bolts. Initial finding of economic pinning against >>> lightning commitment and second-stage HTLC transactions. Subsequent discussions of a >>> preimage-overlay network or package-relay as mitigations. Public call made to inquiry more on >>> potential other transaction-relay jamming attacks affecting lightning. >>> >>> In 2021, initial work in bitcoin core 22.0 of package acceptance. Continued discussion of the >>> pinning attacks and shortcomings of current mempool rules during community-wide online workshops. >>> Later the year, in light of all issues for bitcoin second-layers, a proposal is made about killing >>> the mempool. >>> >>> In 2022, bip proposed for package relay and new proposed v3 policy design proposed for a review and >>> implementation. Mempoolfullrbf is supported in bitcoin core 24.0 and conceptual questions about >>> alignment of mempool rules w.r.t miners incentives are investigated. >>> >>> Along this year 2022, eltoo lightning channels design are discussed, implemented and reviewed. In >>> this context and after discussions on mempool anti-DoS rules, I discovered this new replacement >>> cycling attack was affecting deployed lightning channels and immediately reported the finding to >>> some bitcoin core developers and lightning maintainers. >>> >>> ## Timeline >>> >>> - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg Sanders and Gloria Zhao >>> - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, Matt Corallo and Olaoluwa >>> Osuntunkun >>> - 2022-12-23: Sharing to Eugene Siegel (LND) >>> - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning potential affected projects) >>> - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers issuers) and initial >>> proposal of an early public disclosure >>> - 2022-01-19: Collection of analysis if other second-layers and multi-party applications affected. >>> LN mitigations development starts. >>> - 2023-05-04: Sharing to Wilmer Paulino (LDK) >>> - 2023-06-20: LN mitigations implemented and progressively released. Week of the 16 october proposed >>> for full disclosure. >>> - 2023-08-10: CVEs assigned by MITRE >>> - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling attack existence to >>> security@bitcoincore•org <mailto:security@bitcoincore.org>. >>> - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / CVE-2023-40233 / CVE-2023-40234 >>> and replacement cycling attacks >>> >>> ## Conclusion >>> >>> Despite the line of mitigations adopted and deployed by current major lightning implementations, I >>> believe replacement cycling attacks are still practical for advanced attackers. Beyond this new >>> attack might come as a way to partially or completely defeat some of the pinning mitigations which >>> have been working for years as a community. >>> >>> As of today, it is uncertain to me if lightning is not affected by a more severe long-term package >>> malleability critical security issue under current consensus rules, and if any other time-sensitive >>> multi-party protocol, designed or deployed isn't de facto affected too (loss of funds or denial of >>> service). >>> >>> Assuming analysis on package malleability is correct, it is unclear to me if it can be corrected by >>> changes in replacement / eviction rules or mempool chain of transactions processing strategy. >>> Inviting my technical peers and the bitcoin community to look more on this issue, including to >>> dissent. I'll be the first one pleased if I'm fundamentally wrong on those issues, or if any element >>> has not been weighted with the adequate technical accuracy it deserves. >>> >>> Do not trust, verify. All mistakes and opinions are my own. >>> >>> Antoine >>> >>> "meet with Triumph and Disaster. And treat those two impostors just the same" - K. >>> >>> _______________________________________________ >>> Lightning-dev mailing list >>> Lightning-dev@lists•linuxfoundation•org >>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Quoted text (3 lines)
> As the CLTV > delta deadline approaches, the fees in case 2 may be 50%, 80%, even > 100% of the HTLC value under such a scorched earth policy.
A replacement-cycling attacker can afford to pay 100% of the HTLC value under the defender scorched earth policy and still realize an economic gain.
Let's say you have Alice, Bob and Caroll all "honest" routing hops targeted by an attacker. They all have 3 independent 10 000 sats HTLC in-flight on their outbound channels.
Under this defensive fee scorched earth policy, Alice broadcasts her HTLC-timeout at T + 1 with 10 000 sats committed as absolute fee.
It is replaced by Mallory at T+2 with a HTLC-preimage X of 200 000 sats (+ rbf penalty 1 sat / vb rule 4). Alice's HTLC-timeout is out of network mempools.
Bob broadcasts her HTLC-timeout of 200 000 sats at T+3. It is replaced by Mallory at T+4 with her HLTC-preimage Y of 200 000 sats (+ rbf penalty 1 sat / vb rule 4 * 2). Bob's HTLC-timeout is out of network mempools. HTLC-preimage Y conflicts with HTLC-preimage X too (here comes the multiplied by 2 of the rbf penalty).
Caroll broadcasts her HTLC-timeout of 200 000 sats at T+5. It is replaced by Mallory at T+6 with her HTLC-preimage Z of 200 000 sats (+rbf penalty 1 sat / vb rule 4 * 3). Caroll's HTLC-timeout is out of network mempools. HTLC-preimage Z conflicts with HTLC-preimage Z too (here comes the multiplied by 3 of rbf penalty).
At any time if Mallory's HTLC-preimage enters into the top mempool feerates group (due to the accumulated rbf penalty), one unconfirmed ancestor can be double-spent to evict out the HTLC-preimage.
If Mallory succeeds the replacement cycling, she might be at loss of 10 000 sats + rbf penalty cost for each rebroadcast attempt of the victim. However she wins Alice + Bob + Carol HTLC value of 200 000 sats each.
Assuming 5 rebroadcasts per block (even on random timers) multiplied by 3 victims, 200 bytes of HTLC-preimage and cltv_delta of 144 blocks, the total attacker cost is 432 000 sats.
The economic gain realized is 168 000 sats. Sounds each additional victim has a cost of 144 000 sats, whatever the HTLC value targeted.
Thanks for checking the fees math and replacement rules, though it sounds correct to me.
(And here without introducing more favorable assumptions to the attacker, like mempool spikes where the "honest" HTLC-timeout transactions can be let floating in network mempools).
Best, Antoine
Le jeu. 19 oct. 2023 à 18:54, Matt Morehouse <mattmorehouse@gmail•com> a écrit :
Quoted text (504 lines)
> On Thu, Oct 19, 2023 at 5:22 PM Antoine Riard <antoine.riard@gmail•com> > wrote: > > > > Hi Matt, > > > > This mitigation is mentioned in the attached paper (see subsection 3.4 > defensive fee-rebroadcasting) > > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > > > As soon as you start to have a bit of a mempool backlog and the > defensive fractional fee HTLC-timeout stays stuck, it gives the advantage > to the attacker again. > > > > Beyond that, I think an attacker can replace-cycle multiple honest > HTLC-timeout with a single malicious HTLC-preimage (with a sequence of > replacement, not concurrently) paying the absolute fee, while only > encumbering the RBF penalty. I didn't test this specific behavior, though > the "fees" math doesn't seem at the advantage of the defenders at first > sight. > > Right, some replacement cycles can be avoided by the attacker to > reduce the cost of the attack. But with the defender implementing a > scorched-earth fee bumping policy, eventually either (1) the > HTLC-timeout *will* confirm in the next block or (2) the attacker must > pay more fees than the HTLC-timeout fees to replace it. As the CLTV > delta deadline approaches, the fees in case 2 may be 50%, 80%, even > 100% of the HTLC value under such a scorched earth policy. So even if > the attacker only has to do one replacement cycle right before the > deadline, the attack can be made unprofitable. And in practice, with > HTLC values significantly greater than the next-block fee cost, the > attacker will need to do multiple replacements as the deadline > approaches. > > And of course this linear scorched earth policy is just an > illustration. We should further tune the fee bumping curve across the > full CLTV delta to ensure minimal fees paid when not under attack. > But as we approach the deadline, it makes sense to become very > aggressive both to get our transaction confirmed during high mempool > congestion and to punish replacement-cycling attackers. > > > > > Best, > > Antoine > > > > Le jeu. 19 oct. 2023 à 17:23, Matt Morehouse <mattmorehouse@gmail•com> > a écrit : > >> > >> On Wed, Oct 18, 2023 at 12:34 AM Matt Corallo via bitcoin-dev > >> <bitcoin-dev@lists•linuxfoundation•org> wrote: > >> > > >> > There appears to be some confusion about this issue and the > mitigations. To be clear, the deployed > >> > mitigations are not expected to fix this issue, its arguable if they > provide anything more than a PR > >> > statement. > >> > > >> > There are two discussed mitigations here - mempool scanning and > transaction re-signing/re-broadcasting. > >> > > >> > Mempool scanning relies on regularly checking the mempool of a local > node to see if we can catch the > >> > replacement cycle mid-cycle. It only works if wee see the first > transaction before the second > >> > transaction replaces it. > >> > > >> > Today, a large majority of lightning nodes run on machines with a > Bitcoin node on the same IP > >> > address, making it very clear what the "local node" of the lightning > node is. An attacker can > >> > trivially use this information to connect to said local node and do > the replacement quickly, > >> > preventing the victim from seeing the replacement. > >> > > >> > More generally, however, similar discoverability is true for mining > pools. An attacker performing > >> > this attack is likely to do the replacement attack on a miner's node > directly, potentially reducing > >> > the reach of the intermediate transaction to only miners, such that > the victim can never discover it > >> > at all. > >> > > >> > The second mitigation is similarly pathetic. Re-signing and > re-broadcasting the victim's transaction > >> > in an attempt to get it to miners even if its been removed may work, > if the attacker is super lazy > >> > and didn't finish writing their attack system. If the attacker is > connected to a large majority of > >> > hashrate (which has historically been fairly doable), they can simply > do their replacement in a > >> > cycle aggressively and arbitrarily reduce the probability that the > victim's transaction gets confirmed. > >> > >> What if the honest node aggressively fee-bumps and retransmits the > >> HTLC-timeout as the CLTV delta deadline approaches, as suggested by > >> Ziggie? Say, within 10 blocks of the deadline, the honest node starts > >> increasing the fee by 1/10th the HTLC value for each non-confirmation. > >> > >> This "scorched earth" approach may cost the honest node considerable > >> fees, but it will cost the attacker even more, since each attacker > >> replacement needs to burn at least as much as the HTLC-timeout fees, > >> and the attacker will need to do a replacement every time the honest > >> node fee bumps. > >> > >> I think this fee-bumping policy will provide sufficient defense even > >> if the attacker is replacement-cycling directly in miners' mempools > >> and the victim has no visibility into the attack. > >> > >> > > >> > Now, the above is all true in a spherical cow kinda world, and the > P2P network has plenty of slow > >> > nodes and strange behavior. Its possible that these mitigations > might, by some stroke of luck, > >> > happen to catch such an attack and prevent it, because something took > longer than the attacker > >> > intended or whatever. But, that's a far cry from any kind of material > "fix" for the issue. > >> > > >> > Ultimately the only fix for this issue will be when miners keep a > history of transactions they've > >> > seen and try them again after they may be able to enter the mempool > because of an attack like this. > >> > > >> > Matt > >> > > >> > On 10/16/23 12:57 PM, Antoine Riard wrote: > >> > > (cross-posting mempool issues identified are exposing lightning > chan to loss of funds risks, other > >> > > multi-party bitcoin apps might be affected) > >> > > > >> > > Hi, > >> > > > >> > > End of last year (December 2022), amid technical discussions on > eltoo payment channels and > >> > > incentives compatibility of the mempool anti-DoS rules, a new > transaction-relay jamming attack > >> > > affecting lightning channels was discovered. > >> > > > >> > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning > >> > > routing hops carrying HTLC traffic to loss of funds security risks, > both legacy and anchor output > >> > > channels. A potential exploitation plausibly happening even without > network mempools congestion. > >> > > > >> > > Mitigations have been designed, implemented and deployed by all > major lightning implementations > >> > > during the last months. > >> > > > >> > > Please find attached the release numbers, where the mitigations > should be present: > >> > > - LDK: v0.0.118 - CVE-2023 -40231 > >> > > - Eclair: v0.9.0 - CVE-2023-40232 > >> > > - LND: v.0.17.0-beta - CVE-2023-40233 > >> > > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > >> > > > >> > > While neither replacement cycling attacks have been observed or > reported in the wild since the last > >> > > ~10 months or experimented in real-world conditions on bitcoin > mainet, functional test is available > >> > > exercising the affected lightning channel against bitcoin core > mempool (26.0 release cycle). > >> > > > >> > > It is understood that a simple replacement cycling attack does not > demand privileged capabilities > >> > > from an attacker (e.g no low-hashrate power) and only access to > basic bitcoin and lightning > >> > > software. Yet I still think executing such an attack successfully > requests a fair amount of bitcoin > >> > > technical know-how and decent preparation. > >> > > > >> > > From my understanding of those issues, it is yet to be determined > if the mitigations deployed are > >> > > robust enough in face of advanced replacement cycling attackers, > especially ones able to combine > >> > > different classes of transaction-relay jamming such as pinnings or > vetted with more privileged > >> > > capabilities. > >> > > > >> > > Please find a list of potential affected bitcoin applications in > this full disclosure report using > >> > > bitcoin script timelocks or multi-party transactions, albeit no > immediate security risk exposure as > >> > > severe as the ones affecting lightning has been identified. Only > cursory review of non-lightning > >> > > applications has been conducted so far. > >> > > > >> > > There is a paper published summarizing replacement cycling attacks > on the lightning network: > >> > > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > >> > > < > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > > >> > > > >> > > ## Problem > >> > > > >> > > A lightning node allows HTLCs forwarding (in bolt3's parlance > accepted HTLC on incoming link and > >> > > offered HTLC on outgoing link) should settle the outgoing state > with either a success or timeout > >> > > before the incoming state timelock becomes final and an asymmetric > defavorable settlement might > >> > > happen (cf "Flood & Loot: A Systematic Attack on The Lightning > Network" section 2.3 for a classical > >> > > exposition of this lightning security property). > >> > > > >> > > Failure to satisfy this settlement requirement exposes a forwarding > hop to a loss of fund risk where > >> > > the offered HTLC is spent by the outgoing link counterparty's > HTLC-preimage and the accepted HTLC is > >> > > spent by the incoming link counterparty's HTLC-timeout. > >> > > > >> > > The specification mandates the incoming HTLC expiration timelock to > be spaced out by an interval of > >> > > `cltv_expiry_delta` from the outgoing HTLC expiration timelock, > this exact interval value being an > >> > > implementation and node policy setting. As a minimal value, the > specification recommends 34 blocks > >> > > of interval. If the timelock expiration I of the inbound HTLC is > equal to 100 from chain tip, the > >> > > timelock expiration O of the outbound HTLC must be equal to 66 > blocks from chain tip, giving a > >> > > reasonable buffer of reaction to the lightning forwarding node. > >> > > > >> > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the > >> > > counterparty (either `update_fulfill_htlc` or `update_fail_htlc`) > when O is reached, the lightning > >> > > node should broadcast its commitment transaction. Once the > commitment is confirmed (if anchor and > >> > > the 1 CSV encumbrance is present), the lightning node broadcasts > and confirms its HTLC-timeout > >> > > before I height is reached. > >> > > > >> > > Here enter a replacement cycling attack. A malicious channel > counterparty can broadcast its > >> > > HTLC-preimage transaction with a higher absolute fee and higher > feerate than the honest HTLC-timeout > >> > > of the victim lightning node and triggers a replacement. Both for > legacy and anchor output channels, > >> > > a HTLC-preimage on a counterparty commitment transaction is > malleable, i.e additional inputs or > >> > > outputs can be added. The HTLC-preimage spends an unconfirmed and > unrelated to the channel parent > >> > > transaction M and conflicts its child. > >> > > > >> > > As the HTLC-preimage spends an unconfirmed input that was already > included in the unconfirmed and > >> > > unrelated child transaction (rule 2), pays an absolute higher fee > of at least the sum paid by the > >> > > HTLC-timeout and child transaction (rule 3) and the HTLC-preimage > feerate is greater than all > >> > > directly conflicting transactions (rule 6), the replacement is > accepted. The honest HTLC-timeout is > >> > > evicted out of the mempool. > >> > > > >> > > In an ulterior move, the malicious counterparty can replace the > parent transaction itself with > >> > > another candidate N satisfying the replacement rules, triggering > the eviction of the malicious > >> > > HTLC-preimage from the mempool as it was a child of the parent T. > >> > > > >> > > There is no spending candidate of the offered HTLC output for the > current block laying in network > >> > > mempools. > >> > > > >> > > This replacement cycling tricks can be repeated for each > rebroadcast attempt of the HTLC-timeout by > >> > > the honest lightning node until expiration of the inbound HTLC > timelock I. Once this height is > >> > > reached a HTLC-timeout is broadcast by the counterparty's on the > incoming link in collusion with the > >> > > one on the outgoing link broadcasting its own HTLC-preimage. > >> > > > >> > > The honest Lightning node has been "double-spent" in its HTLC > forwarding. > >> > > > >> > > As a notable factor impacting the success of the attack, a > lightning node's honest HTLC-timeout > >> > > might be included in the block template of the miner winning the > block race and therefore realizes a > >> > > spent of the offered output. In practice, a replacement cycling > attack might over-connect to miners' > >> > > mempools and public reachable nodes to succeed in a fast eviction > of the HTLC-timeout by its > >> > > HTLC-preimage. As this latter transaction can come with a better > ancestor-score, it should be picked > >> > > up on the flight by economically competitive miners. > >> > > > >> > > A functional test exercising a simple replacement cycling of a HTLC > transaction on bitcoin core > >> > > mempool is available: > >> > > https://github.com/ariard/bitcoin/commits/2023-test-mempool > >> > > <https://github.com/ariard/bitcoin/commits/2023-test-mempool> > >> > > > >> > > ## Deployed LN mitigations > >> > > > >> > > Aggressive rebroadcasting: As the replacement cycling attacker > benefits from the HTLC-timeout being > >> > > usually broadcast by lightning nodes only once every block, or less > the replacement cycling > >> > > malicious transactions paid only equal the sum of the absolute fees > paid by the HTLC, adjusted with > >> > > the replacement penalty. Rebroadcasting randomly and multiple times > before the next block increases > >> > > the absolute fee cost for the attacker. > >> > > > >> > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > >> > > > >> > > Local-mempool preimage monitoring: As the replacement cycling > attacker in a simple setup broadcast > >> > > the HTLC-preimage to all the network mempools, the honest lightning > node is able to catch on the > >> > > flight the unconfirmed HTLC-preimage, before its subsequent mempool > replacement. The preimage can be > >> > > extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a > >> > > cooperative message or go on-chain with it to claim the accepted > HTLC output. > >> > > > >> > > Implemented and deployed by Eclair and LND. > >> > > > >> > > CLTV Expiry Delta: With every jammed block comes an absolute fee > cost paid by the attacker, a risk > >> > > of the HTLC-preimage being detected or discovered by the honest > lightning node, or the HTLC-timeout > >> > > to slip in a winning block template. Bumping the default CLTV delta > hardens the odds of success of a > >> > > simple replacement cycling attack. > >> > > > >> > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > >> > > > >> > > ## Affected Bitcoin Protocols and Applications > >> > > > >> > > From my understanding the following list of Bitcoin protocols and > applications could be affected by > >> > > new denial-of-service vectors under some level of network mempools > congestion. Neither tests or > >> > > advanced review of specifications (when available) has been > conducted for each of them: > >> > > - on-chain DLCs > >> > > - coinjoins > >> > > - payjoins > >> > > - wallets with time-sensitive paths > >> > > - peerswap and submarine swaps > >> > > - batch payouts > >> > > - transaction "accelerators" > >> > > > >> > > Inviting their developers, maintainers and operators to investigate > how replacement cycling attacks > >> > > might disrupt their in-mempool chain of transactions, or > fee-bumping flows at the shortest delay. > >> > > Simple flows and non-multi-party transactions should not be > affected to the best of my understanding. > >> > > > >> > > ## Open Problems: Package Malleability > >> > > > >> > > Pinning attacks have been known for years as a practical vector to > compromise lightning channels > >> > > funds safety, under different scenarios (cf. current bip331's > motivation section). Mitigations at > >> > > the mempool level have been designed, discussed and are under > implementation by the community > >> > > (ancestor package relay + nverrsion=3 policy). Ideally, they should > constraint a pinning attacker to > >> > > always attach a high feerate package (commitment + CPFP) to replace > the honest package, or allow a > >> > > honest lightning node to overbid a malicious pinning package and > get its time-sensitive transaction > >> > > optimistically included in the chain. > >> > > > >> > > Replacement cycling attack seem to offer a new way to neutralize > the design goals of package relay > >> > > and its companion nversion=3 policy, where an attacker package RBF > a honest package out of the > >> > > mempool to subsequently double-spend its own high-fee child with a > transaction unrelated to the > >> > > channel. As the remaining commitment transaction is pre-signed with > a minimal relay fee, it can be > >> > > evicted out of the mempool. > >> > > > >> > > A functional test exercising a simple replacement cycling of a > lightning channel commitment > >> > > transaction on top of the nversion=3 code branch is available: > >> > > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > >> > > <https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2> > >> > > > >> > > ## Discovery > >> > > > >> > > In 2018, the issue of static fees for pre-signed lightning > transactions is made more widely known, > >> > > the carve-out exemption in mempool rules to mitigate in-mempool > package limits pinning and the > >> > > anchor output pattern are proposed. > >> > > > >> > > In 2019, bitcoin core 0.19 is released with carve-out support. > Continued discussion of the anchor > >> > > output pattern as a dynamic fee-bumping method. > >> > > > >> > > In 2020, draft of anchor output submitted to the bolts. Initial > finding of economic pinning against > >> > > lightning commitment and second-stage HTLC transactions. Subsequent > discussions of a > >> > > preimage-overlay network or package-relay as mitigations. Public > call made to inquiry more on > >> > > potential other transaction-relay jamming attacks affecting > lightning. > >> > > > >> > > In 2021, initial work in bitcoin core 22.0 of package acceptance. > Continued discussion of the > >> > > pinning attacks and shortcomings of current mempool rules during > community-wide online workshops. > >> > > Later the year, in light of all issues for bitcoin second-layers, a > proposal is made about killing > >> > > the mempool. > >> > > > >> > > In 2022, bip proposed for package relay and new proposed v3 policy > design proposed for a review and > >> > > implementation. Mempoolfullrbf is supported in bitcoin core 24.0 > and conceptual questions about > >> > > alignment of mempool rules w.r.t miners incentives are investigated. > >> > > > >> > > Along this year 2022, eltoo lightning channels design are > discussed, implemented and reviewed. In > >> > > this context and after discussions on mempool anti-DoS rules, I > discovered this new replacement > >> > > cycling attack was affecting deployed lightning channels and > immediately reported the finding to > >> > > some bitcoin core developers and lightning maintainers. > >> > > > >> > > ## Timeline > >> > > > >> > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony > Towns, Greg Sanders and Gloria Zhao > >> > > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien > Teinturier, Matt Corallo and Olaoluwa > >> > > Osuntunkun > >> > > - 2022-12-23: Sharing to Eugene Siegel (LND) > >> > > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot > (non-lightning potential affected projects) > >> > > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and > cross-layers issuers) and initial > >> > > proposal of an early public disclosure > >> > > - 2022-01-19: Collection of analysis if other second-layers and > multi-party applications affected. > >> > > LN mitigations development starts. > >> > > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > >> > > - 2023-06-20: LN mitigations implemented and progressively > released. Week of the 16 october proposed > >> > > for full disclosure. > >> > > - 2023-08-10: CVEs assigned by MITRE > >> > > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement > cycling attack existence to > >> > > security@bitcoincore•org <mailto:security@bitcoincore.org>. > >> > > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / > CVE-2023-40233 / CVE-2023-40234 > >> > > and replacement cycling attacks > >> > > > >> > > ## Conclusion > >> > > > >> > > Despite the line of mitigations adopted and deployed by current > major lightning implementations, I > >> > > believe replacement cycling attacks are still practical for > advanced attackers. Beyond this new > >> > > attack might come as a way to partially or completely defeat some > of the pinning mitigations which > >> > > have been working for years as a community. > >> > > > >> > > As of today, it is uncertain to me if lightning is not affected by > a more severe long-term package > >> > > malleability critical security issue under current consensus rules, > and if any other time-sensitive > >> > > multi-party protocol, designed or deployed isn't de facto affected > too (loss of funds or denial of > >> > > service). > >> > > > >> > > Assuming analysis on package malleability is correct, it is unclear > to me if it can be corrected by > >> > > changes in replacement / eviction rules or mempool chain of > transactions processing strategy. > >> > > Inviting my technical peers and the bitcoin community to look more > on this issue, including to > >> > > dissent. I'll be the first one pleased if I'm fundamentally wrong > on those issues, or if any element > >> > > has not been weighted with the adequate technical accuracy it > deserves. > >> > > > >> > > Do not trust, verify. All mistakes and opinions are my own. > >> > > > >> > > Antoine > >> > > > >> > > "meet with Triumph and Disaster. And treat those two impostors just > the same" - K. > >> > > > >> > > _______________________________________________ > >> > > Lightning-dev mailing list > >> > > Lightning-dev@lists•linuxfoundation•org > >> > > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev > >> > _______________________________________________ > >> > bitcoin-dev mailing list > >> > bitcoin-dev@lists•linuxfoundation•org > >> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
Hi,
After writing the mail reply on the economics of sequential malicious replacement of honest HTLC-timeout, I did write one more test to verify the behavior on core mempool, and it works as expected.
https://github.com/ariard/bitcoin/commit/30f5d5b270e4ff195e8dcb9ef6b7ddcc5f6a1bf2
Responsible disclosure process has followed the lines of hardware issues affecting operating system, as documented for the Linux kernel, while adapted to the bitcoin ecosystem:
https://docs.kernel.org/6.1/process/embargoed-hardware-issues.html
Effective now, I'm halting my involvement with the development of the lightning network and its implementations, including coordinating the handling of security issues at the protocol level (I informed some senior lightning devs in that sense before).
Closed the very old issue which was affected to me at this purpose on the bolt repository:
https://github.com/lightning/bolts/pull/772
I think this new class of replacement cycling attacks puts lightning in a very perilous position, where only a sustainable fix can happen at the base-layer, e.g adding a memory-intensive history of all-seen transactions or some consensus upgrade. Deployed mitigations are worth something in face of simple attacks, though I don't think they're stopping advanced attackers as said in the first full disclosure mail.
Those types of changes are the ones necessitating the utmost transparency and buy-in of the community as a whole, as we're altering the full-nodes processing requirements or the security architecture of the decentralized bitcoin ecosystem in its integrality.
On the other hand fully explaining why such changes would be warranted for the sake of lightning and for designing them well, we might need to lay out in complete state practical and critical attacks on a ~5 355 public BTC ecosystem.
Hard dilemma.
There might be a lesson in terms of bitcoin protocol deployment, we might have to get them right at first try. Little second chance to fix them in flight.
I'll be silent on those issues on public mailing lists until the week of the 30 oct. Enough material has been published and other experts are available. Then I'll be back focusing more on bitcoin core.
Best, Antoine
Le lun. 16 oct. 2023 à 17:57, Antoine Riard <antoine.riard@gmail•com> a écrit :
Quoted text (287 lines)
> (cross-posting mempool issues identified are exposing lightning chan to > loss of funds risks, other multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and incentives compatibility of the mempool anti-DoS > rules, a new transaction-relay jamming attack affecting lightning channels > was discovered. > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning routing hops carrying HTLC traffic to loss of > funds security risks, both legacy and anchor output channels. A potential > exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major > lightning implementations during the last months. > > Please find attached the release numbers, where the mitigations should be > present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported > in the wild since the last ~10 months or experimented in real-world > conditions on bitcoin mainet, functional test is available exercising the > affected lightning channel against bitcoin core mempool (26.0 release > cycle). > > It is understood that a simple replacement cycling attack does not demand > privileged capabilities from an attacker (e.g no low-hashrate power) and > only access to basic bitcoin and lightning software. Yet I still think > executing such an attack successfully requests a fair amount of bitcoin > technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the > mitigations deployed are robust enough in face of advanced replacement > cycling attackers, especially ones able to combine different classes of > transaction-relay jamming such as pinnings or vetted with more privileged > capabilities. > > Please find a list of potential affected bitcoin applications in this full > disclosure report using bitcoin script timelocks or multi-party > transactions, albeit no immediate security risk exposure as severe as the > ones affecting lightning has been identified. Only cursory review of > non-lightning applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the > lightning network: > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and offered HTLC on outgoing link) should settle the > outgoing state with either a success or timeout before the incoming state > timelock becomes final and an asymmetric defavorable settlement might > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" > section 2.3 for a classical exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to > a loss of fund risk where the offered HTLC is spent by the outgoing link > counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming > link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC > expiration timelock, this exact interval value being an implementation and > node policy setting. As a minimal value, the specification recommends 34 > blocks of interval. If the timelock expiration I of the inbound HTLC is > equal to 100 from chain tip, the timelock expiration O of the outbound HTLC > must be equal to 66 blocks from chain tip, giving a reasonable buffer of > reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the counterparty (either > `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the > lightning node should broadcast its commitment transaction. Once the > commitment is confirmed (if anchor and the 1 CSV encumbrance is present), > the lightning node broadcasts and confirms its HTLC-timeout before I height > is reached. > > Here enter a replacement cycling attack. A malicious channel counterparty > can broadcast its HTLC-preimage transaction with a higher absolute fee and > higher feerate than the honest HTLC-timeout of the victim lightning node > and triggers a replacement. Both for legacy and anchor output channels, a > HTLC-preimage on a counterparty commitment transaction is malleable, i.e > additional inputs or outputs can be added. The HTLC-preimage spends an > unconfirmed and unrelated to the channel parent transaction M and conflicts > its child. > > As the HTLC-preimage spends an unconfirmed input that was already included > in the unconfirmed and unrelated child transaction (rule 2), pays an > absolute higher fee of at least the sum paid by the HTLC-timeout and child > transaction (rule 3) and the HTLC-preimage feerate is greater than all > directly conflicting transactions (rule 6), the replacement is accepted. > The honest HTLC-timeout is evicted out of the mempool. > > In an ulterior move, the malicious counterparty can replace the parent > transaction itself with another candidate N satisfying the replacement > rules, triggering the eviction of the malicious HTLC-preimage from the > mempool as it was a child of the parent T. > > There is no spending candidate of the offered HTLC output for the current > block laying in network mempools. > > This replacement cycling tricks can be repeated for each rebroadcast > attempt of the HTLC-timeout by the honest lightning node until expiration > of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout > is broadcast by the counterparty's on the incoming link in collusion with > the one on the outgoing link broadcasting its own HTLC-preimage. > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > As a notable factor impacting the success of the attack, a lightning > node's honest HTLC-timeout might be included in the block template of the > miner winning the block race and therefore realizes a spent of the offered > output. In practice, a replacement cycling attack might over-connect to > miners' mempools and public reachable nodes to succeed in a fast eviction > of the HTLC-timeout by its HTLC-preimage. As this latter transaction can > come with a better ancestor-score, it should be picked up on the flight by > economically competitive miners. > > A functional test exercising a simple replacement cycling of a HTLC > transaction on bitcoin core mempool is available: > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits > from the HTLC-timeout being usually broadcast by lightning nodes only once > every block, or less the replacement cycling malicious transactions paid > only equal the sum of the absolute fees paid by the HTLC, adjusted with the > replacement penalty. Rebroadcasting randomly and multiple times before the > next block increases the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in > a simple setup broadcast the HTLC-preimage to all the network mempools, the > honest lightning node is able to catch on the flight the unconfirmed > HTLC-preimage, before its subsequent mempool replacement. The preimage can > be extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a cooperative message or go on-chain with it to > claim the accepted HTLC output. > > Implemented and deployed by Eclair and LND. > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid > by the attacker, a risk of the HTLC-preimage being detected or discovered > by the honest lightning node, or the HTLC-timeout to slip in a winning > block template. Bumping the default CLTV delta hardens the odds of success > of a simple replacement cycling attack. > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > ## Affected Bitcoin Protocols and Applications > > From my understanding the following list of Bitcoin protocols and > applications could be affected by new denial-of-service vectors under some > level of network mempools congestion. Neither tests or advanced review of > specifications (when available) has been conducted for each of them: > - on-chain DLCs > - coinjoins > - payjoins > - wallets with time-sensitive paths > - peerswap and submarine swaps > - batch payouts > - transaction "accelerators" > > Inviting their developers, maintainers and operators to investigate how > replacement cycling attacks might disrupt their in-mempool chain of > transactions, or fee-bumping flows at the shortest delay. Simple flows and > non-multi-party transactions should not be affected to the best of my > understanding. > > ## Open Problems: Package Malleability > > Pinning attacks have been known for years as a practical vector to > compromise lightning channels funds safety, under different scenarios (cf. > current bip331's motivation section). Mitigations at the mempool level have > been designed, discussed and are under implementation by the community > (ancestor package relay + nverrsion=3 policy). Ideally, they should > constraint a pinning attacker to always attach a high feerate package > (commitment + CPFP) to replace the honest package, or allow a honest > lightning node to overbid a malicious pinning package and get its > time-sensitive transaction optimistically included in the chain. > > Replacement cycling attack seem to offer a new way to neutralize the > design goals of package relay and its companion nversion=3 policy, where an > attacker package RBF a honest package out of the mempool to subsequently > double-spend its own high-fee child with a transaction unrelated to the > channel. As the remaining commitment transaction is pre-signed with a > minimal relay fee, it can be evicted out of the mempool. > > A functional test exercising a simple replacement cycling of a lightning > channel commitment transaction on top of the nversion=3 code branch is > available: > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > ## Discovery > > In 2018, the issue of static fees for pre-signed lightning transactions is > made more widely known, the carve-out exemption in mempool rules to > mitigate in-mempool package limits pinning and the anchor output pattern > are proposed. > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued > discussion of the anchor output pattern as a dynamic fee-bumping method. > > In 2020, draft of anchor output submitted to the bolts. Initial finding of > economic pinning against lightning commitment and second-stage HTLC > transactions. Subsequent discussions of a preimage-overlay network or > package-relay as mitigations. Public call made to inquiry more on potential > other transaction-relay jamming attacks affecting lightning. > > In 2021, initial work in bitcoin core 22.0 of package acceptance. > Continued discussion of the pinning attacks and shortcomings of current > mempool rules during community-wide online workshops. Later the year, in > light of all issues for bitcoin second-layers, a proposal is made about > killing the mempool. > > In 2022, bip proposed for package relay and new proposed v3 policy design > proposed for a review and implementation. Mempoolfullrbf is supported in > bitcoin core 24.0 and conceptual questions about alignment of mempool rules > w.r.t miners incentives are investigated. > > Along this year 2022, eltoo lightning channels design are discussed, > implemented and reviewed. In this context and after discussions on mempool > anti-DoS rules, I discovered this new replacement cycling attack was > affecting deployed lightning channels and immediately reported the finding > to some bitcoin core developers and lightning maintainers. > > ## Timeline > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg > Sanders and Gloria Zhao > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, > Matt Corallo and Olaoluwa Osuntunkun > - 2022-12-23: Sharing to Eugene Siegel (LND) > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning > potential affected projects) > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers > issuers) and initial proposal of an early public disclosure > - 2022-01-19: Collection of analysis if other second-layers and > multi-party applications affected. LN mitigations development starts. > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > - 2023-06-20: LN mitigations implemented and progressively released. Week > of the 16 october proposed for full disclosure. > - 2023-08-10: CVEs assigned by MITRE > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling > attack existence to security@bitcoincore•org. > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / > CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks > > ## Conclusion > > Despite the line of mitigations adopted and deployed by current major > lightning implementations, I believe replacement cycling attacks are still > practical for advanced attackers. Beyond this new attack might come as a > way to partially or completely defeat some of the pinning mitigations which > have been working for years as a community. > > As of today, it is uncertain to me if lightning is not affected by a more > severe long-term package malleability critical security issue under current > consensus rules, and if any other time-sensitive multi-party protocol, > designed or deployed isn't de facto affected too (loss of funds or denial > of service). > > Assuming analysis on package malleability is correct, it is unclear to me > if it can be corrected by changes in replacement / eviction rules or > mempool chain of transactions processing strategy. Inviting my technical > peers and the bitcoin community to look more on this issue, including to > dissent. I'll be the first one pleased if I'm fundamentally wrong on those > issues, or if any element has not been weighted with the adequate technical > accuracy it deserves. > > Do not trust, verify. All mistakes and opinions are my own. > > Antoine > > "meet with Triumph and Disaster. And treat those two impostors just the > same" - K. >
On Tue, Oct 17, 2023 at 10:34:04AM +0000, ZmnSCPxj via bitcoin-dev wrote:
Quoted text (21 lines)
> Good morning Antoine et al., > > Let me try to rephrase the core of the attack. > > There exists these nodes on the LN (letters `A`, `B`, and `C` are nodes, `==` are channels): > > A ===== B ===== C > > `A` routes `A->B->C`. > > The timelocks, for example, could be: > > A->B timeelock = 144 > B->C timelock = 100 > > The above satisfies the LN BOLT requirements, as long as `B` has a `cltv_expiry_delta` of 44 or lower. > > After `B` forwards the HTLC `B->C`, C suddenly goes offline, and all the signed transactions --- commitment transaction and HTLC-timeout transactions --- are "stuck" at the feerate at the time. > > At block height 100, `B` notices the `B->C` HTLC timelock is expired without `C` having claimed it, so `B` forces the `B====C` channel onchain. > However, onchain feerates have risen and the commitment transaction and HTLC-timeout transaction do not confirm.
The problem here is we're failing to use RBF.
As I have suggested before, the correct way to do pre-signed transactions is to pre-sign enough *different* transactions to cover all reasonable needs for bumping fees. Even if you just increase the fee by 2x each time, pre-signing 10 different replacement transactions covers a fee range of 1024x. And you obviously can improve on this by increasing the multiplier towards the end of the range.
Increasing per-tx (temporary) storage and bandwidth costs by ~10x or even ~100x is not a big deal in the context of a highly scalable protocol like Lightning.
There is zero reason why the B->C transactions should be getting stuck. This is a major failing of the Lightning protocol that should be fixed. And of course, this fix should be applied to other aspects of the lightning protocol, such as channel opens, etc.
On Tue, Oct 17, 2023 at 02:11:20AM +0100, Antoine Riard wrote:
Quoted text (8 lines)
> > I think if you want people to understand this exploit, you need to > explain in more detail how we have a situation where two different parties > can spend the same HTLC txout, without the first party having the right to > spend it via their knowledge of the HTLC-preimage. > > If I'm correctly understanding your question, you're asking why we have a > situation where the spend of a HTLC output can be in competition between 2 > channel counterparties.
No, you are not correctly understanding it.
It's obvious that an HTLC output can be in competition between 2 different parties. Obviously, the HTLC-preimage doesn't expire. The problem is you haven't explained why the party with the HTLC pre-image should not *remain* the party with the *right* to spend that output, even after the timeout branch becomes another possible way to spend it.
Quoted text (12 lines)
> LN commitment transactions have offered HTLC outputs where a counterparty > Alice is pledging to her other counterparty Caroll the HTLC amount in > exchange of a preimage (and Caroll signature). > > After the expiration of the HTLC timelock, if the HTLC has not been claimed > on-chain by Caroll, Alice can claim it back with her signature (and the > pre-exchanged Caroll signature). > > The exploit works actually in Caroll leveraging her HTLC-preimage > transaction as a replace-by-fee of Alice's HTLC-timeout _after_ the > expiration of the timelock, the HTLC-preimage transaction staying consensus > valid.
That's precisely my point re: you not properly explaining the problem. If Caroll has the HTLC-preimage, she has the right to spend it. You need to explain why her right to spend that HTLC-preimage output should expire.
If anything, the way you've explained it sounds like Bob has stolen the output from Caroll by virtue of the fact that Caroll wasn't able to spend the HTLC-preimage output in time.
On Fri, Oct 20, 2023 at 10:31:03AM +0000, Peter Todd via bitcoin-dev wrote:
Quoted text (6 lines)
> As I have suggested before, the correct way to do pre-signed transactions is to > pre-sign enough *different* transactions to cover all reasonable needs for > bumping fees. Even if you just increase the fee by 2x each time, pre-signing 10 > different replacement transactions covers a fee range of 1024x. And you > obviously can improve on this by increasing the multiplier towards the end of > the range.
To be clear, when I say "increasing the multiplier", I mean, starting with a smaller multiplier at the beginning of the range, and ending with a bigger one.
Eg feebumping with fee increases pre-signed for something like:
1.1
1.2
1.4
1.8
2.6
4.2
7.4
etc.
That would use most of the range for smaller bumps, as a %, with larger % bumps reserved for the end where our strategy is changing to something more "scorched-earth"
And of course, applying this idea properly to commitment transactions will mean that the replacements may have HTLCs removed, when their value drops below the fees necessary to get those outputs mined.
Note too that we can sign simultaneous variants of transactions that deduct the fees from different party's outputs. Eg Alice can give Bob the ability to broadcast higher and higher fee txs, taking the fees from Bob's output(s), and Bob can give Alice the same ability, taking the fees from Alice's output(s). I haven't thought through how this would work with musig. But you can certainly do that with plain old OP_CheckMultisig.
I found the original explanation a bit confusing. As I understand it, the attack starts by double-spending the timeout HTLC transaction of the victim with a pre-image revealing HTLC transaction. This itself is not an attack: the victim can then use the pre-image to receive its incoming HTLC safely, because its timeout hasn't expired yet. The trick is now that the attacker double-spends their own transaction before it hits the chain (the third transaction only double-spends some attacker controlled input used also by the pre-image HTLC transaction). In ideal condition, the pre-image transaction is never seen by the victim and the victim still doesn't know the pre-image. The attacker may only attack the mempool of the mining nodes. The victim may not even know that their transaction was replaced and are only confused why it didn't get mined.
On Fri, 20 Oct 2023 at 12:47, Peter Todd via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (46 lines)
> > On Tue, Oct 17, 2023 at 02:11:20AM +0100, Antoine Riard wrote: > > > I think if you want people to understand this exploit, you need to > > explain in more detail how we have a situation where two different parties > > can spend the same HTLC txout, without the first party having the right to > > spend it via their knowledge of the HTLC-preimage. > > > > If I'm correctly understanding your question, you're asking why we have a > > situation where the spend of a HTLC output can be in competition between 2 > > channel counterparties. > > No, you are not correctly understanding it. > > It's obvious that an HTLC output can be in competition between 2 different > parties. Obviously, the HTLC-preimage doesn't expire. The problem is you > haven't explained why the party with the HTLC pre-image should not *remain* the > party with the *right* to spend that output, even after the timeout branch > becomes another possible way to spend it. > > > LN commitment transactions have offered HTLC outputs where a counterparty > > Alice is pledging to her other counterparty Caroll the HTLC amount in > > exchange of a preimage (and Caroll signature). > > > > After the expiration of the HTLC timelock, if the HTLC has not been claimed > > on-chain by Caroll, Alice can claim it back with her signature (and the > > pre-exchanged Caroll signature). > > > > The exploit works actually in Caroll leveraging her HTLC-preimage > > transaction as a replace-by-fee of Alice's HTLC-timeout _after_ the > > expiration of the timelock, the HTLC-preimage transaction staying consensus > > valid. > > That's precisely my point re: you not properly explaining the problem. If > Caroll has the HTLC-preimage, she has the right to spend it. You need to > explain why her right to spend that HTLC-preimage output should expire. > > If anything, the way you've explained it sounds like Bob has stolen the output > from Caroll by virtue of the fact that Caroll wasn't able to spend the > HTLC-preimage output in time. > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
I think if we apply this presigned fee multiplier idea to HTLC spends, we can prevent replacement cycles from happening.
We could modify HTLC scripts so that *both* parties can only spend the HTLC via presigned second-stage transactions, and we can always sign those with SIGHASH_ALL. This will prevent the attacker from adding inputs to their presigned transaction, so (AFAICT) a replacement cycling attack becomes impossible.
The tradeoff is more bookkeeping and less fee granularity when claiming HTLCs on chain.
On Fri, Oct 20, 2023 at 11:04 AM Peter Todd via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (45 lines)
> > On Fri, Oct 20, 2023 at 10:31:03AM +0000, Peter Todd via bitcoin-dev wrote: > > As I have suggested before, the correct way to do pre-signed transactions is to > > pre-sign enough *different* transactions to cover all reasonable needs for > > bumping fees. Even if you just increase the fee by 2x each time, pre-signing 10 > > different replacement transactions covers a fee range of 1024x. And you > > obviously can improve on this by increasing the multiplier towards the end of > > the range. > > To be clear, when I say "increasing the multiplier", I mean, starting with a > smaller multiplier at the beginning of the range, and ending with a bigger one. > > Eg feebumping with fee increases pre-signed for something like: > > 1.1 > 1.2 > 1.4 > 1.8 > 2.6 > 4.2 > 7.4 > > etc. > > That would use most of the range for smaller bumps, as a %, with larger % bumps > reserved for the end where our strategy is changing to something more > "scorched-earth" > > And of course, applying this idea properly to commitment transactions will mean > that the replacements may have HTLCs removed, when their value drops below the > fees necessary to get those outputs mined. > > Note too that we can sign simultaneous variants of transactions that deduct the > fees from different party's outputs. Eg Alice can give Bob the ability to > broadcast higher and higher fee txs, taking the fees from Bob's output(s), and > Bob can give Alice the same ability, taking the fees from Alice's output(s). I > haven't thought through how this would work with musig. But you can certainly > do that with plain old OP_CheckMultisig. > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Sadly this only is really viable for pre-anchor channels. With anchor channels the attack can be performed by either side of the closure, as the HTLCs are now, at max, only signed SIGHASH_SINGLE|ANYONECANPAY, allowing you to add more inputs and perform this attack even as the broadcaster.
I don't think its really viable to walk that change back to fix this, as it also fixed plenty of other issues with channel usability and important edge-cases.
I'll highlight that fixing this issue on the lightning end isn't really the right approach generally - we're talking about a case where a lightning counterparty engineered a transaction broadcast ordering such that miners are *not* including the optimal set of transactions for fee revenue. Given such a scenario exists (and its not unrealistic to think someone might wish to engineer such a situation), the fix ultimately needs to lie with Bitcoin Core (or other parts of the mining stack).
Now, fixing this in the Bitcoin Core stack is no trivial deal - the reason for this attack is to keep enough history to fix it Bitcoin Core would need unbounded memory. However, its not hard to imagine a simple external piece of software which monitors the mempool for transactions which were replaced out but which may be able to re-enter the mempool later with other replacements and store them on disk. From there, this software could optimize the revenue of block template selection, while also accidentally fixing this issue.
Matt
On 10/20/23 2:35 PM, Matt Morehouse via bitcoin-dev wrote:
Quoted text (63 lines)
> I think if we apply this presigned fee multiplier idea to HTLC spends, > we can prevent replacement cycles from happening. > > We could modify HTLC scripts so that *both* parties can only spend the > HTLC via presigned second-stage transactions, and we can always sign > those with SIGHASH_ALL. This will prevent the attacker from adding > inputs to their presigned transaction, so (AFAICT) a replacement > cycling attack becomes impossible. > > The tradeoff is more bookkeeping and less fee granularity when > claiming HTLCs on chain. > > On Fri, Oct 20, 2023 at 11:04 AM Peter Todd via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation•org> wrote: >> >> On Fri, Oct 20, 2023 at 10:31:03AM +0000, Peter Todd via bitcoin-dev wrote: >>> As I have suggested before, the correct way to do pre-signed transactions is to >>> pre-sign enough *different* transactions to cover all reasonable needs for >>> bumping fees. Even if you just increase the fee by 2x each time, pre-signing 10 >>> different replacement transactions covers a fee range of 1024x. And you >>> obviously can improve on this by increasing the multiplier towards the end of >>> the range. >> >> To be clear, when I say "increasing the multiplier", I mean, starting with a >> smaller multiplier at the beginning of the range, and ending with a bigger one. >> >> Eg feebumping with fee increases pre-signed for something like: >> >> 1.1 >> 1.2 >> 1.4 >> 1.8 >> 2.6 >> 4.2 >> 7.4 >> >> etc. >> >> That would use most of the range for smaller bumps, as a %, with larger % bumps >> reserved for the end where our strategy is changing to something more >> "scorched-earth" >> >> And of course, applying this idea properly to commitment transactions will mean >> that the replacements may have HTLCs removed, when their value drops below the >> fees necessary to get those outputs mined. >> >> Note too that we can sign simultaneous variants of transactions that deduct the >> fees from different party's outputs. Eg Alice can give Bob the ability to >> broadcast higher and higher fee txs, taking the fees from Bob's output(s), and >> Bob can give Alice the same ability, taking the fees from Alice's output(s). I >> haven't thought through how this would work with musig. But you can certainly >> do that with plain old OP_CheckMultisig. >> >> -- >> https://petertodd.org 'peter'[:-1]@petertodd.org >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists•linuxfoundation•org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
On Mon, Oct 16, 2023 at 05:57:36PM +0100, Antoine Riard via bitcoin-dev wrote:
Quoted text (8 lines)
> Here enter a replacement cycling attack. A malicious channel counterparty > can broadcast its HTLC-preimage transaction with a higher absolute fee and > higher feerate than the honest HTLC-timeout of the victim lightning node > and triggers a replacement. Both for legacy and anchor output channels, a > HTLC-preimage on a counterparty commitment transaction is malleable, i.e > additional inputs or outputs can be added. The HTLC-preimage spends an > unconfirmed and unrelated to the channel parent transaction M and conflicts > its child.
The basic problem here is after the HTLC-timeout path becomes spendable, the HTLC-preimage path remains spendable. That's bad, because in this case we want spending the HTLC-preimage - if possible - to have an urgency attached to it to ensure that it happens before the previous HTLC-timeout is mined.
So, why can't we make the HTLC-preimage path expire? Traditionally, we've tried to ensure that transactions - once valid - remain valid forever. We do this because we don't want transactions to become impossible to mine in the event of a large reorganization.
A notable example of this design philosophy is seen in Bitcoin's rules around coinbase outputs: they only become spendable after 100 more blocks have been found; a 100 block reorg is quite unlikely.
Enter the OP_Expire and the Coinbase Bit soft-fork upgrade.
# Coinbase Bit
By redefining a bit of the nVersion field, eg the most significant bit, we can apply coinbase-like txout handling to arbitrary transactions. Such a transaction's outputs would be treated similarly to a coinbase transaction, and would be spendable only after 100 more blocks had been mined. Due to this requirement, these transactions will pose no greater risk to reorg safety than the existing hazard of coinbase transactions themselves becoming invalid.
Note how such a transaction is non-standard right now, ensuring compatibility with existing nodes in a soft-fork upgrade.
# OP_Expire
Redefining an existing OP_Nop opcode, OP_Expire would terminate script evaluation with an error if:
1) the Coinbase Bit was not set; or 2) the stack is empty; or 3) the top item on the stack was >= the block height of the containing block
This is conceptually an AntiCheckLockTimeVerify: where CLTV _allows_ a txout to become spendable in a particular way in the future, Expire _prevents_ a txout from being spent in a particular way.
Since OP_Expire requires the Coinbase Bit to be set, the reorg security of OP_Expire-using transactions is no worse than transactions spending miner coinbases.
# How HTLC's Would Use OP_Expire
Whenever revealing the preimage on-chain is necessary to the secure functioning of the HTLC-using protocol, we simply add an appropriate OP_Expire to the pre-image branch of the script along the lines of:
If
<expiry height> Expire Drop
Hash <digest> EqualVerify
<pubkey> CheckSig
ElseIf
# HTLC Expiration conditions
...
EndIf
Now the party receiving the pre-image has a deadline. Either they get a transaction spending the preimage mined, notifying the other party via the blockchain itself, or they fail to get the preimage mined in time, reverting control to the other party who can spend the HTLC output at their leisure, without strict time constraints.
Since the HTLC-expired branch does *not* execute OP_Expire, the transaction spending the HTLC-expired branch does *not* need to set the Coinbase Bit. Thus it can be spent in a perfectly normal transaction, without restrictions.
# Delta Encoding Expiration
Rather than having a specific Coinbase Bit, it may also be feasible to encode the expiration height as a delta against a block-height nLockTime. In this variant, OP_Expire would work similarly to OP_CheckLockTimeVerify, by checking that the absolute expiration height was <= the requested expiration, allowing multiple HTLC preimage outputs to be spent in one transaction.
If the top 16-bits were used, the maximum period a transaction could be valid would be:
2^16 blocks / 144 blocks/day = 455 days
In this variant, a non-zero expiration delta would enable expiration behavior, as well as the coinbase-like output spending restriction. The remaining 16-bits of nVersion would remain available for other meanings.
Similar to how CLTV and CSV verified nLockTime and nSequence respectively, verifying an expiration height encoded in the nVersion has the advantage of making an expiration height easy to detect without validating scripts.
While Lightning's HTLC-success transactions currently use nLockTime=0, AFAIK there is no reason why they could not set nLockTime to be valid in the next block, allowing delta encoding to be used.
## Reusing Time-Based nLockTime
Reusing time-based nLockTime's prior to some pre-2009 genesis point for expiration is another possibility (similar to how Lightning makes use of time-based nLockTime for signalling). However I believe this is not as desirable as delta encoding or a coinbase bit, as it would prevent transactions from using block nLockTime and expiration at the same time. It would also still require a coinbase bit or nVersion increase to ensure expiration-using transactions are non-standard.
# Mempool Behavior
Obviously, mempool logic will need to handle transactions that can expire differently than non-expiring transactions. One notable consideration is that nodes should require higher minimum relay fees for transactions close to their expiration height to ensure we don't waste bandwidth on transactions that have no potential to be mined. Considering the primary use-case, it is probably acceptable to always require a fee rate high enough to be mined in the next block.
On Fri, Oct 20, 2023 at 05:05:48PM -0400, Matt Corallo wrote:
Quoted text (8 lines)
> Sadly this only is really viable for pre-anchor channels. With anchor > channels the attack can be performed by either side of the closure, as the > HTLCs are now, at max, only signed SIGHASH_SINGLE|ANYONECANPAY, allowing you > to add more inputs and perform this attack even as the broadcaster. > > I don't think its really viable to walk that change back to fix this, as it > also fixed plenty of other issues with channel usability and important > edge-cases.
What are anchor outputs used for other than increasing fees?
Because if we've pre-signed the full fee range, there is simply no need for anchor outputs. Under any circumstance we can broadcast a transaction with a sufficiently high fee to get mined.
Quoted text (9 lines)
> Let's say you have Alice, Bob and Caroll all "honest" routing hops > targeted by an attacker. They all have 3 independent 10 000 sats HTLC > in-flight on their outbound channels. > It is replaced by Mallory at T+2 with a HTLC-preimage X of 200 000 sats (+ > rbf penalty 1 sat / vb rule 4). Alice's HTLC-timeout is out of network > mempools. > Bob broadcasts her HTLC-timeout of 200 000 sats at T+3. It is replaced by > Mallory at T+4 with her HLTC-preimage Y of 200 000 sats (+ rbf penalty 1 > sat / vb rule 4 * 2). Bob's HTLC-timeout is out of network mempools.
IIRC correctly, in your scenario, you're dealing with Carol -> Bob -> Alice. Mallory can only replace an HTLC-timeout transaction if she's directly connected with the peer she's targeting via a direct channel. She cannot unilaterally replace any transaction in the mempool solely with knowledge of the preimage. All HTLC transactions are two party contracts, with the public keys of both participants hard coded into the contract. A third party cannot unilaterally sweep an HTLC given only a preimage.
I think it's worth taking a minute to step back here so we can establish some fundamental context. Once the timelock of an HTLC expires, and the receiver has direct knowledge of the preimage, a bidding war begins. If the receiver is able to confirm their success transaction in time, then they gain the funds, and the sender is able to pipeline the preimage backwards in the route, making all parties whole. If the sender is the one that wins out, then it's as if nothing happened, sans the fees paid at the last mile, all other hops are able to safely cancel their HTLC back on the chain.
As mentioned elsewhere in the thread, most implementations today will watch the mempool for preimages, so they can resolve the incoming HTLC as quickly as possible off chain. The attack described critically relies on the ability of an attacker to pull off very precise transaction replacement globally across "the mempool". If any of the honest parties see the preimage, in the mempool, then the attack ends as they can settle back off chain, and if their timeout is able to confirm first, then the defender has actually gained funds.
In addition, such an attack needs to be executed perfectly, for hours if not days. Within the LN, all nodes set a security parameter, the CLTV delta, that dictates how much time they have before the outgoing and incoming HTLCs expire. Increasing this value makes the attack more difficult, as they must maintain the superposition of: fees low enough to not get mined, but high enough to replace the defender's transaction, but not _too_ high, as then it'll be mined and everything is over. With each iteration, the attacker is also forced to increase the amount of fees they pay, increasing the likely hood that the transaction will be mined. If mined, the attack is moot.
As mentioned in the OP, one thing the attacker can do is try and locate the defender's node to launch the replacement attack directly in their mempool. However, by replacing directly in the mempool of the defender, the defender will learn of the preimage! They can use that to just settle everything back, thereby completing the payment, and stopping the attack short. Even ignoring direct access to the defender's mempool, the attacker needs to somehow iteratively execute the replacement across a real network, with all the network jitter, propagation delay, and geographic heterogeneity that entails. If they're off by milliseconds, then either a confirmation occurs, or the preimage is revealed in the mempool. A canned local regtest env is one thing, the distributed system that is the Internet is another.
Returning back to the bidding war: for anchor channels, the second level HTLCs allow the defender to attach arbitrary inputs for fee bumping purposes. Using this, they can iteratively increase their fee (via RBF) as the expiry deadline approaches. With each step, they further increase the cost for the attacker. On top of that, this attack cannot be launched indiscriminately across the network. Instead, it requires per-node set up by the attacker, as they need to consume UTXOs to create a chain of transactions they'll use to launch the replacement attack. These also need to be maintained in a similar state of non-confirming superposition.
That's all to say that imo, this is a rather fragile attack, which requires: per-node setup, extremely precise timing and execution, non-confirming superposition of all transactions, and instant propagation across the entire network of the replacement transaction (somehow also being obscured from the PoV of the defender). The attack can be launched directly with a miner, or "into" their mempool, but that weakens the attack as if the miner broadcasts any of the preimage replacement transactions, then the defender can once again use that to extract the preimage and settle the incoming HTLC.
-- Laolu
On 10/20/23 8:15 PM, Peter Todd wrote:
Quoted text (15 lines)
> On Fri, Oct 20, 2023 at 05:05:48PM -0400, Matt Corallo wrote: >> Sadly this only is really viable for pre-anchor channels. With anchor >> channels the attack can be performed by either side of the closure, as the >> HTLCs are now, at max, only signed SIGHASH_SINGLE|ANYONECANPAY, allowing you >> to add more inputs and perform this attack even as the broadcaster. >> >> I don't think its really viable to walk that change back to fix this, as it >> also fixed plenty of other issues with channel usability and important >> edge-cases. > > What are anchor outputs used for other than increasing fees? > > Because if we've pre-signed the full fee range, there is simply no need for > anchor outputs. Under any circumstance we can broadcast a transaction with a > sufficiently high fee to get mined.
Indeed, that is what anchor outputs are for. Removing the pre-set feerate solved a number of issues with edge-cases and helped address the fee-inflation attack. Now, just using pre-signed transactions doesn't have to re-introduce those issues - as long as the broadcaster gets to pick which of the possible transactions they broadcast its just another transaction of theirs.
Still, I'm generally really dubious of the multiple pre-signed transaction thing, (a) it would mean more fee overhead (not the end of the world for a force-closure, but it sucks to have all these individual transactions rolling around and be unable to batch), but more importantly (b) its a bunch of overhead to keep track of a ton of variants across a sufficiently granular set of feerates for it to not result in substantially overspending on fees.
Like I mentioned in the previous mail, this is really a policy bug - we're talking about a transaction pattern that might well happen where miners aren't getting the optimal value in transaction fees (potentially by a good bit). This needs to be fixed at the policy/Bitcoin Core layer, not in the lightning world (as much as its pretty resource-intensive to fix in the policy domain, I think).
Matt
On Fri, Oct 20, 2023 at 09:03:49PM -0400, Matt Corallo wrote:
Quoted text (21 lines)
> > What are anchor outputs used for other than increasing fees? > > > > Because if we've pre-signed the full fee range, there is simply no need for > > anchor outputs. Under any circumstance we can broadcast a transaction with a > > sufficiently high fee to get mined. > > > Indeed, that is what anchor outputs are for. Removing the pre-set feerate > solved a number of issues with edge-cases and helped address the > fee-inflation attack. Now, just using pre-signed transactions doesn't have > to re-introduce those issues - as long as the broadcaster gets to pick which > of the possible transactions they broadcast its just another transaction of > theirs. > > Still, I'm generally really dubious of the multiple pre-signed transaction > thing, (a) it would mean more fee overhead (not the end of the world for a > force-closure, but it sucks to have all these individual transactions > rolling around and be unable to batch), but more importantly (b) its a bunch > of overhead to keep track of a ton of variants across a sufficiently > granular set of feerates for it to not result in substantially overspending > on fees.
Quite the contrary. Schnorr signatures are 64 bytes, so in situations like lightning where the transaction form is deterministically derived, signing 100 extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s connection can transfer that in 64ms; latency will still dominate.
RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you use those 100 pre-signed transaction variants to do nothing more than sign every possible minimum incremental relay, you've covered a range of 1sat/vByte to 100sat/vByte. I believe that is sufficient to get mined for any block in Bitcoin's entire modern history.
CPFP meanwhile requires two transactions, and thus extra bytes. Other than edge cases with very large transactions in low-fee environments, there's no circumstance where CPFP beats RBF.
On 10/20/23 9:25 PM, Peter Todd wrote:
Quoted text (27 lines)
> On Fri, Oct 20, 2023 at 09:03:49PM -0400, Matt Corallo wrote: >>> What are anchor outputs used for other than increasing fees? >>> >>> Because if we've pre-signed the full fee range, there is simply no need for >>> anchor outputs. Under any circumstance we can broadcast a transaction with a >>> sufficiently high fee to get mined. >> >> >> Indeed, that is what anchor outputs are for. Removing the pre-set feerate >> solved a number of issues with edge-cases and helped address the >> fee-inflation attack. Now, just using pre-signed transactions doesn't have >> to re-introduce those issues - as long as the broadcaster gets to pick which >> of the possible transactions they broadcast its just another transaction of >> theirs. >> >> Still, I'm generally really dubious of the multiple pre-signed transaction >> thing, (a) it would mean more fee overhead (not the end of the world for a >> force-closure, but it sucks to have all these individual transactions >> rolling around and be unable to batch), but more importantly (b) its a bunch >> of overhead to keep track of a ton of variants across a sufficiently >> granular set of feerates for it to not result in substantially overspending >> on fees. > > Quite the contrary. Schnorr signatures are 64 bytes, so in situations like > lightning where the transaction form is deterministically derived, signing 100 > extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s > connection can transfer that in 64ms; latency will still dominate.
Lightning today isn't all that much data, but multiply it by 100 and we start racking up data enough that people may start to have to store a really material amount of data for larger nodes and dealing with that starts to be a much bigger pain then when we're talking about a GiB or twenty.
Quoted text (9 lines)
> RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you use > those 100 pre-signed transaction variants to do nothing more than sign every > possible minimum incremental relay, you've covered a range of 1sat/vByte to > 100sat/vByte. I believe that is sufficient to get mined for any block in > Bitcoin's entire modern history. > > CPFP meanwhile requires two transactions, and thus extra bytes. Other than edge > cases with very large transactions in low-fee environments, there's no > circumstance where CPFP beats RBF.
What I was referring to is that if we have the SIGHASH_SINGLE|ANYONECANPAY we can combine many HTLC claims into one transaction, vs pre-signing means we're stuck with a ton of individual transactions.
Matt
On Fri, Oct 20, 2023 at 09:55:12PM -0400, Matt Corallo wrote:
Quoted text (9 lines)
> > Quite the contrary. Schnorr signatures are 64 bytes, so in situations like > > lightning where the transaction form is deterministically derived, signing 100 > > extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s > > connection can transfer that in 64ms; latency will still dominate. > > Lightning today isn't all that much data, but multiply it by 100 and we > start racking up data enough that people may start to have to store a really > material amount of data for larger nodes and dealing with that starts to be > a much bigger pain then when we're talking about a GiB or twenty.
We are talking about storing ephemeral data here, HTLC transactions and possibly commitment transactions. Since lightning uses disclosed secrets to invalidate old state, you do not need to keep every signature from your counterparty indefinitely.
Per channel, with the above numbers, you'd need <10KB worth of extra signatures to fee bump the current commitment (with pre-signed txs), and if HTLCs happen to be in flight, say <10KB of extra signatures per HTLC.
Amazon AWS charges $0.125/GB/month for their most expensive SSD volumes. Let's round that up to $1/GB/month to account for RAID and backups. Let's say each channel constantly has 483 HTLC's in flight, and each HTLC is associated with ~10KB of extra data for pre-signed transactions.
That's ~5MB/channel of data you constantly need to store, or $0.005/month.
In what world is that too expensive or too much of a pain to deal with? If you're node is actually that busy, you're probably making that cost back per channel every minute, or better.
Quoted text (13 lines)
> > RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you use > > those 100 pre-signed transaction variants to do nothing more than sign every > > possible minimum incremental relay, you've covered a range of 1sat/vByte to > > 100sat/vByte. I believe that is sufficient to get mined for any block in > > Bitcoin's entire modern history. > > > > CPFP meanwhile requires two transactions, and thus extra bytes. Other than edge > > cases with very large transactions in low-fee environments, there's no > > circumstance where CPFP beats RBF. > > What I was referring to is that if we have the SIGHASH_SINGLE|ANYONECANPAY > we can combine many HTLC claims into one transaction, vs pre-signing means > we're stuck with a ton of individual transactions.
Since SIGHASH_SINGLE requires one output per input, the savings you get by combining multiple SIGHASH_SINGLE transactions together aren't very significant. Just 18 bytes for nVersion, nLockTime, and the txin and txout size fields. The HTLC-timeout transaction is 166.5 vBytes, so that's a savings of just 11%
Of course, if you _do_ need to fee bump and add an additional input, that input takes up space, and you'll probably need a change output. At which point you again would probably have been better off with a pre-signed transaction.
You are also assuming there's lots of HTLC's in flight that need to be spent. That's very often not the case.
On 2023-10-20 14:09, Peter Todd via bitcoin-dev wrote:
Quoted text (9 lines)
> The basic problem here is after the HTLC-timeout path becomes > spendable, the > HTLC-preimage path remains spendable. That's bad, because in this case > we want > spending the HTLC-preimage - if possible - to have an urgency attached > to it to > ensure that it happens before the previous HTLC-timeout is mined. > > So, why can't we make the HTLC-preimage path expire?
If the goal is to ensure the HTLC-preimage should be mined before an upstream HTLC-timeout becomes mineable, then I don't think a consensus change is required. We can just make the HTLC-preimage claimable by anyone some time after the HTLC-timeout becomes mineable.
For example, imagine that Alice offers Bob an HTLC with a timeout at block t+200. Bob offers Carol an HTLC with a timeout at block t+100. The Bob-Carol HTLC script looks like this:
If
# Does someone have the preimage?
Hash <digest> EqualVerify
If
# Carol has the preimage at any time
<Carol key> CheckSig
Else
# Anyone else has the preimage after t+150
<t+150> CLTV
EndIf
Else
# Bob is allowed a refund after t+100
<Bob key> CheckSigVerify
<t+100> CLTV
EndIf
In English:
- At any time, Carol can spend the output by releasing the preimage - After t+100, Bob can spend the output - After t+150, anyone with the preimage can spend the output
Let's consider this in the wider context of the forwarded payment Alice->Bob->Carol:
- If Carol attempts to spend the output by releasing the preimage but pays too low of a feerate to get it confirmed by block t+100, Bob can spend the output in block t+101. He then has 99 blocks to settle (revoke) the Alice-Bob HTLC offchain.
- If Carol releases the preimage to the network in general but prevents Bob from using it (e.g. using a replacement cycling attack), anyone who saw the preimage can take Carol's output at t+150 and, by doing so, will put the preimage in the block chain where Bob will learn about it. He'll then have 49 blocks to settle (revoke) the Alice-Bob HTLC offchain.
- (All the normal cases when the HTLC is settled offchain, or where onchain operations occur in a timely manner)
I think that adequately satisfies the concern about the effect on LN from replacement cycling. Looking at potential complications:
- If all miners acted together[1], they are incentivized to not mine Carol's preimage transaction before t+150 because its fees are less than the HTLC value they can receive at t+150. I think this level of miner centralization would result in a general failure for LN given that miners could be any LN user's counterparty (or bribed by a user's counterparty). E.g., stuff like this: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-June/017997.html
- To allow anyone with the preimage to spend the output after t+150, they need to know the script. For taproot, that means the t+150 tapleaf script needs to follow a standard (e.g. a BOLT) and that any internal merkle nodes needed to connect it to the taproot commitment need to be shown in Carol's preimage transaction (or inferable from it or other data).
- Classic RBF pinning of the t+150 transaction to prevent it from confirming by block t+200 might be an issue. E.g., including it in a 400,000 weight low-feerate transaction.
- Full RBF might be required to ensure the t+150 transaction isn't sent with a low feerate and no opt-in signal.
Deployment considerations:
- No changes are required to full nodes (no consensus change required)
- No changes are required to mining Bitcoin nodes[2]
- At least one well-connected Bitcoin relay node will need to be updated to store preimages and related data, and to send the preimage claim transactions. Data only needs to be kept for a rolling window of a few thousand blocks for the LN case, bounding storage requirements. No changes are required to other relaying Bitcoin nodes
- LN nodes will need to update to new HTLC scripts, but this should be doable without closing/re-opening channels. Both anchor and non-anchor channels can continue to be used
Compared to OP_EXPIRE:
- OP_EXPIRE requires consensus and policy changes; this does not
- OP_EXPIRE does not depend on special software; this depends on at least one person running special software
Although this proposal is an alternative to Peter's proposal and is primarily inspired by his idea, it's also a variation on a previous suggestion of mine: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-April/002664.html
-Dave
[1] Perhaps under block censorship threat from a mining majority or a sub-majority performing selfish mining.
[2] Although miners may want to consider running code that allows them to rewrite any malleable transactions to pay themselve
On Fri, Oct 20, 2023 at 10:58:32PM -1000, David A. Harding wrote:
Quoted text (41 lines)
> On 2023-10-20 14:09, Peter Todd via bitcoin-dev wrote: > > The basic problem here is after the HTLC-timeout path becomes spendable, > > the > > HTLC-preimage path remains spendable. That's bad, because in this case > > we want > > spending the HTLC-preimage - if possible - to have an urgency attached > > to it to > > ensure that it happens before the previous HTLC-timeout is mined. > > > > So, why can't we make the HTLC-preimage path expire? > > If the goal is to ensure the HTLC-preimage should be mined before an > upstream HTLC-timeout becomes mineable, then I don't think a consensus > change is required. We can just make the HTLC-preimage claimable by anyone > some time after the HTLC-timeout becomes mineable. > > For example, imagine that Alice offers Bob an HTLC with a timeout at block > t+200. Bob offers Carol an HTLC with a timeout at block t+100. The > Bob-Carol HTLC script looks like this: > > If > # Does someone have the preimage? > Hash <digest> EqualVerify > If > # Carol has the preimage at any time > <Carol key> CheckSig > Else > # Anyone else has the preimage after t+150 > <t+150> CLTV > EndIf > Else > # Bob is allowed a refund after t+100 > <Bob key> CheckSigVerify > <t+100> CLTV > EndIf > > In English: > > - At any time, Carol can spend the output by releasing the preimage > - After t+100, Bob can spend the output > - After t+150, anyone with the preimage can spend the output
This is a clever idea. But it doesn't prevent this attack.
Think about it this way: where previously there was one Carol who could perform the replacement cycling attack, with the anyone-can-spend branch the situation eventually transforms into a situation where there is an unlimited set of Carols who can perform the attack and, if they are a miner, benefit from it.
Either way Bob is in a position where they might not learn about the preimage in time, and thus fail to redeem the received HTLC output.
From Carol's point of view the situation didn't significantly change. Either they manage to succesfully spend the offered HTLC output after the redeemed HTLC output times out. Or they fail. Whether or not that failure happens because Bob got their refund, or someone else spent the offered HTLC output via the anyone-can-spend path is not relevant to Carol.
Finally, this solution is inferior to OP_Expire in another important way: the anyone-can-spend branch represents a strict deadline for Bob too. With OP_Expire, once HTLC preimage branch has expired, Bob can spend the offered HTLC output at their leisure, as they are the only party with the ability to do that (assuming of course that we're talking about a valid commitment transaction, not an illegitmate revoked once).
Quoted text (2 lines)
> [2] Although miners may want to consider running code that allows them to > rewrite any malleable transactions to pay themselve
With full-RBF _anyone_ can run that code on behalf of miners, modulo edge cases where the replacement isn't possible due to the RBF anti-DoS rules. Indeed, people are apparently already doing this to screw over signature-less ordinal transactions.
I think the presigned transactions should be interleaved fee-wise: 1.1 to Alice 1.2 to Bob 1.3 to Alice 1.4 to Bob ...
Then any such transaction evicts all previous transactions in the chain. It reduces risks of mempool split.
If there are two transactions to Alice and to Bob both with fee 1.1, then it is possible that half of nodes have the transaction to Alice in their mempools and another half of nodes has the transaction to Bob. Though I don't see how exactly this can be used in replacement cycling attacks, I think it is safer to prevent this scenario.
On Fri Oct 20 18:35:26 UTC 2023, Matt Morehouse via bitcoin-dev wrote:
Quoted text (9 lines)
> I think if we apply this presigned fee multiplier idea to HTLC spends, > we can prevent replacement cycles from happening. > We could modify HTLC scripts so that *both* parties can only spend the > HTLC via presigned second-stage transactions, and we can always sign > those with SIGHASH_ALL. This will prevent the attacker from adding > inputs to their presigned transaction, so (AFAICT) a replacement > cycling attack becomes impossible. > The tradeoff is more bookkeeping and less fee granularity when > claiming HTLCs on chain.
Hi,
As I've been shown offline Twitter posts misrepresenting my previous mail, I think it's good to correct them. The security flaws are not "intentional backdoor" or whatever misrepresentation that would question the competence and know-how of the Bitcoin and Lightning development community.
The replacement cycling issue discovered has been known by a small circle of Bitcoin developers since December 2022. As it appears to some experts and it has been commented publicly, changes at the bitcoin base-layer might be the most substantial fixes. Those changes take time and here this is akin to how the linux kernel, bsds and OS vendors are working [0].
All I can say is that we had recently had internal discussion on how to improve coordinated security fixes and patching processes for the coming decades. This is an area of concern where I've always been at the forefront as early as 2020 / 2021 [1].
In the meanwhile, lightning experts have already deployed mitigations which are hardening the lightning ecosystem significantly in face of simple or medium attacks. More advanced attacks can only be mounted if you have sufficient p2p and mempool knowledge as was pointed out by other bitcoin experts like Matt or Peter (which take years to acquire for average bitcoin developers) and the months of preparation to attempt them.
If you're a journalist reporting on the information in mainstream crypto publications, I'll suggest waiting to do so before expert reporters of bitcoin circles who have more in-field knowledge can do so and qualify the technical situation with more distance. As I've already been interviewed by top financial publication years ago for my work on bitcoin, as a journalist you're engaging your own reputation on the information you're reporting. Thanks for being responsible here.
This is the nature of the electronic communication and contemporaneous media that information is extremely fluid and there is no native anti-DoS mechanism to slow down the propagation of sensitive information where mitigations are still in deployment. A reason I'm not on social media of any kind [2]. In the meanwhile, it's good to go to read senecca and marcus aurelius take the situation with stoicism and with a zelt of meditation [3].
All my previous statements are mostly technically correct (even if some could have been written with more clarity once again I'm not an english native [4]). While I wish to wait the week of the 30th Oct o discuss further what is best fix and what are the trade-offs as a community as a wide (give time some laggard lightning implementations ship fixes), though I'll comment further on the mailing list if the flow of information on "social media" is DoSing the ability of the bitcoin community to work on the long-term appropriate fixes in a responsible and constructive fashion.
[0] See meltdown class of vulnerability and how operating systems are handling hardware-sourced vulnerabilities https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability). Most of the time they do their best on the software side and they go to see with hardware vendors how to do the necessary updates.
[1] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-April/003002.html
[2] And for the wider analysis on contemporaneous culture of information propagation and network effect, I can only recommend to read venkatesh rao's ribbonfarm essays http://ribbonfarm.com
[3] There are very good reasons why some executives at top modern technology companies are doing meditation daily, some even hours. "mind illuminated" is a good read.
[4] While my former employer, Chaincode Labs, paid for my english lessons back in 2020. Generally it was a good insight from them to train people on how to communicate in a crisis.
Le ven. 20 oct. 2023 à 07:56, Antoine Riard <antoine.riard@gmail•com> a écrit :
Quoted text (346 lines)
> Hi, > > After writing the mail reply on the economics of sequential malicious > replacement of honest HTLC-timeout, I did write one more test to verify the > behavior on core mempool, and it works as expected. > > > https://github.com/ariard/bitcoin/commit/30f5d5b270e4ff195e8dcb9ef6b7ddcc5f6a1bf2 > > Responsible disclosure process has followed the lines of hardware issues > affecting operating system, as documented for the Linux kernel, while > adapted to the bitcoin ecosystem: > > https://docs.kernel.org/6.1/process/embargoed-hardware-issues.html > > Effective now, I'm halting my involvement with the development of the > lightning network and its implementations, including coordinating the > handling of security issues at the protocol level (I informed some senior > lightning devs in that sense before). > > Closed the very old issue which was affected to me at this purpose on the > bolt repository: > > https://github.com/lightning/bolts/pull/772 > > I think this new class of replacement cycling attacks puts lightning in a > very perilous position, where only a sustainable fix can happen at the > base-layer, e.g adding a memory-intensive history of all-seen transactions > or some consensus upgrade. Deployed mitigations are worth something in face > of simple attacks, though I don't think they're stopping advanced attackers > as said in the first full disclosure mail. > > Those types of changes are the ones necessitating the utmost transparency > and buy-in of the community as a whole, as we're altering the full-nodes > processing requirements or the security architecture of the decentralized > bitcoin ecosystem in its integrality. > > On the other hand fully explaining why such changes would be warranted for > the sake of lightning and for designing them well, we might need to lay out > in complete state practical and critical attacks on a ~5 355 public BTC > ecosystem. > > Hard dilemma. > > There might be a lesson in terms of bitcoin protocol deployment, we might > have to get them right at first try. Little second chance to fix them in > flight. > > I'll be silent on those issues on public mailing lists until the week of > the 30 oct. Enough material has been published and other experts are > available. Then I'll be back focusing more on bitcoin core. > > Best, > Antoine > > Le lun. 16 oct. 2023 à 17:57, Antoine Riard <antoine.riard@gmail•com> a > écrit : > >> (cross-posting mempool issues identified are exposing lightning chan to >> loss of funds risks, other multi-party bitcoin apps might be affected) >> >> Hi, >> >> End of last year (December 2022), amid technical discussions on eltoo >> payment channels and incentives compatibility of the mempool anti-DoS >> rules, a new transaction-relay jamming attack affecting lightning channels >> was discovered. >> >> After careful analysis, it turns out this attack is practical and >> immediately exposed lightning routing hops carrying HTLC traffic to loss of >> funds security risks, both legacy and anchor output channels. A potential >> exploitation plausibly happening even without network mempools congestion. >> >> Mitigations have been designed, implemented and deployed by all major >> lightning implementations during the last months. >> >> Please find attached the release numbers, where the mitigations should be >> present: >> - LDK: v0.0.118 - CVE-2023 -40231 >> - Eclair: v0.9.0 - CVE-2023-40232 >> - LND: v.0.17.0-beta - CVE-2023-40233 >> - Core-Lightning: v.23.08.01 - CVE-2023-40234 >> >> While neither replacement cycling attacks have been observed or reported >> in the wild since the last ~10 months or experimented in real-world >> conditions on bitcoin mainet, functional test is available exercising the >> affected lightning channel against bitcoin core mempool (26.0 release >> cycle). >> >> It is understood that a simple replacement cycling attack does not demand >> privileged capabilities from an attacker (e.g no low-hashrate power) and >> only access to basic bitcoin and lightning software. Yet I still think >> executing such an attack successfully requests a fair amount of bitcoin >> technical know-how and decent preparation. >> >> From my understanding of those issues, it is yet to be determined if the >> mitigations deployed are robust enough in face of advanced replacement >> cycling attackers, especially ones able to combine different classes of >> transaction-relay jamming such as pinnings or vetted with more privileged >> capabilities. >> >> Please find a list of potential affected bitcoin applications in this >> full disclosure report using bitcoin script timelocks or multi-party >> transactions, albeit no immediate security risk exposure as severe as the >> ones affecting lightning has been identified. Only cursory review of >> non-lightning applications has been conducted so far. >> >> There is a paper published summarizing replacement cycling attacks on the >> lightning network: >> >> https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf >> >> ## Problem >> >> A lightning node allows HTLCs forwarding (in bolt3's parlance accepted >> HTLC on incoming link and offered HTLC on outgoing link) should settle the >> outgoing state with either a success or timeout before the incoming state >> timelock becomes final and an asymmetric defavorable settlement might >> happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" >> section 2.3 for a classical exposition of this lightning security property). >> >> Failure to satisfy this settlement requirement exposes a forwarding hop >> to a loss of fund risk where the offered HTLC is spent by the outgoing link >> counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming >> link counterparty's HTLC-timeout. >> >> The specification mandates the incoming HTLC expiration timelock to be >> spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC >> expiration timelock, this exact interval value being an implementation and >> node policy setting. As a minimal value, the specification recommends 34 >> blocks of interval. If the timelock expiration I of the inbound HTLC is >> equal to 100 from chain tip, the timelock expiration O of the outbound HTLC >> must be equal to 66 blocks from chain tip, giving a reasonable buffer of >> reaction to the lightning forwarding node. >> >> In the lack of cooperative off-chain settlement of the HTLC on the >> outgoing link negotiated with the counterparty (either >> `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the >> lightning node should broadcast its commitment transaction. Once the >> commitment is confirmed (if anchor and the 1 CSV encumbrance is present), >> the lightning node broadcasts and confirms its HTLC-timeout before I height >> is reached. >> >> Here enter a replacement cycling attack. A malicious channel counterparty >> can broadcast its HTLC-preimage transaction with a higher absolute fee and >> higher feerate than the honest HTLC-timeout of the victim lightning node >> and triggers a replacement. Both for legacy and anchor output channels, a >> HTLC-preimage on a counterparty commitment transaction is malleable, i.e >> additional inputs or outputs can be added. The HTLC-preimage spends an >> unconfirmed and unrelated to the channel parent transaction M and conflicts >> its child. >> >> As the HTLC-preimage spends an unconfirmed input that was already >> included in the unconfirmed and unrelated child transaction (rule 2), pays >> an absolute higher fee of at least the sum paid by the HTLC-timeout and >> child transaction (rule 3) and the HTLC-preimage feerate is greater than >> all directly conflicting transactions (rule 6), the replacement is >> accepted. The honest HTLC-timeout is evicted out of the mempool. >> >> In an ulterior move, the malicious counterparty can replace the parent >> transaction itself with another candidate N satisfying the replacement >> rules, triggering the eviction of the malicious HTLC-preimage from the >> mempool as it was a child of the parent T. >> >> There is no spending candidate of the offered HTLC output for the current >> block laying in network mempools. >> >> This replacement cycling tricks can be repeated for each rebroadcast >> attempt of the HTLC-timeout by the honest lightning node until expiration >> of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout >> is broadcast by the counterparty's on the incoming link in collusion with >> the one on the outgoing link broadcasting its own HTLC-preimage. >> >> The honest Lightning node has been "double-spent" in its HTLC forwarding. >> >> As a notable factor impacting the success of the attack, a lightning >> node's honest HTLC-timeout might be included in the block template of the >> miner winning the block race and therefore realizes a spent of the offered >> output. In practice, a replacement cycling attack might over-connect to >> miners' mempools and public reachable nodes to succeed in a fast eviction >> of the HTLC-timeout by its HTLC-preimage. As this latter transaction can >> come with a better ancestor-score, it should be picked up on the flight by >> economically competitive miners. >> >> A functional test exercising a simple replacement cycling of a HTLC >> transaction on bitcoin core mempool is available: >> https://github.com/ariard/bitcoin/commits/2023-test-mempool >> >> ## Deployed LN mitigations >> >> Aggressive rebroadcasting: As the replacement cycling attacker benefits >> from the HTLC-timeout being usually broadcast by lightning nodes only once >> every block, or less the replacement cycling malicious transactions paid >> only equal the sum of the absolute fees paid by the HTLC, adjusted with the >> replacement penalty. Rebroadcasting randomly and multiple times before the >> next block increases the absolute fee cost for the attacker. >> >> Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >> >> Local-mempool preimage monitoring: As the replacement cycling attacker in >> a simple setup broadcast the HTLC-preimage to all the network mempools, the >> honest lightning node is able to catch on the flight the unconfirmed >> HTLC-preimage, before its subsequent mempool replacement. The preimage can >> be extracted from the second-stage HTLC-preimage and used to fetch the >> off-chain inbound HTLC with a cooperative message or go on-chain with it to >> claim the accepted HTLC output. >> >> Implemented and deployed by Eclair and LND. >> >> CLTV Expiry Delta: With every jammed block comes an absolute fee cost >> paid by the attacker, a risk of the HTLC-preimage being detected or >> discovered by the honest lightning node, or the HTLC-timeout to slip in a >> winning block template. Bumping the default CLTV delta hardens the odds of >> success of a simple replacement cycling attack. >> >> Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. >> >> ## Affected Bitcoin Protocols and Applications >> >> From my understanding the following list of Bitcoin protocols and >> applications could be affected by new denial-of-service vectors under some >> level of network mempools congestion. Neither tests or advanced review of >> specifications (when available) has been conducted for each of them: >> - on-chain DLCs >> - coinjoins >> - payjoins >> - wallets with time-sensitive paths >> - peerswap and submarine swaps >> - batch payouts >> - transaction "accelerators" >> >> Inviting their developers, maintainers and operators to investigate how >> replacement cycling attacks might disrupt their in-mempool chain of >> transactions, or fee-bumping flows at the shortest delay. Simple flows and >> non-multi-party transactions should not be affected to the best of my >> understanding. >> >> ## Open Problems: Package Malleability >> >> Pinning attacks have been known for years as a practical vector to >> compromise lightning channels funds safety, under different scenarios (cf. >> current bip331's motivation section). Mitigations at the mempool level have >> been designed, discussed and are under implementation by the community >> (ancestor package relay + nverrsion=3 policy). Ideally, they should >> constraint a pinning attacker to always attach a high feerate package >> (commitment + CPFP) to replace the honest package, or allow a honest >> lightning node to overbid a malicious pinning package and get its >> time-sensitive transaction optimistically included in the chain. >> >> Replacement cycling attack seem to offer a new way to neutralize the >> design goals of package relay and its companion nversion=3 policy, where an >> attacker package RBF a honest package out of the mempool to subsequently >> double-spend its own high-fee child with a transaction unrelated to the >> channel. As the remaining commitment transaction is pre-signed with a >> minimal relay fee, it can be evicted out of the mempool. >> >> A functional test exercising a simple replacement cycling of a lightning >> channel commitment transaction on top of the nversion=3 code branch is >> available: >> https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 >> >> ## Discovery >> >> In 2018, the issue of static fees for pre-signed lightning transactions >> is made more widely known, the carve-out exemption in mempool rules to >> mitigate in-mempool package limits pinning and the anchor output pattern >> are proposed. >> >> In 2019, bitcoin core 0.19 is released with carve-out support. Continued >> discussion of the anchor output pattern as a dynamic fee-bumping method. >> >> In 2020, draft of anchor output submitted to the bolts. Initial finding >> of economic pinning against lightning commitment and second-stage HTLC >> transactions. Subsequent discussions of a preimage-overlay network or >> package-relay as mitigations. Public call made to inquiry more on potential >> other transaction-relay jamming attacks affecting lightning. >> >> In 2021, initial work in bitcoin core 22.0 of package acceptance. >> Continued discussion of the pinning attacks and shortcomings of current >> mempool rules during community-wide online workshops. Later the year, in >> light of all issues for bitcoin second-layers, a proposal is made about >> killing the mempool. >> >> In 2022, bip proposed for package relay and new proposed v3 policy design >> proposed for a review and implementation. Mempoolfullrbf is supported in >> bitcoin core 24.0 and conceptual questions about alignment of mempool rules >> w.r.t miners incentives are investigated. >> >> Along this year 2022, eltoo lightning channels design are discussed, >> implemented and reviewed. In this context and after discussions on mempool >> anti-DoS rules, I discovered this new replacement cycling attack was >> affecting deployed lightning channels and immediately reported the finding >> to some bitcoin core developers and lightning maintainers. >> >> ## Timeline >> >> - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg >> Sanders and Gloria Zhao >> - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien >> Teinturier, Matt Corallo and Olaoluwa Osuntunkun >> - 2022-12-23: Sharing to Eugene Siegel (LND) >> - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot >> (non-lightning potential affected projects) >> - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and >> cross-layers issuers) and initial proposal of an early public disclosure >> - 2022-01-19: Collection of analysis if other second-layers and >> multi-party applications affected. LN mitigations development starts. >> - 2023-05-04: Sharing to Wilmer Paulino (LDK) >> - 2023-06-20: LN mitigations implemented and progressively released. Week >> of the 16 october proposed for full disclosure. >> - 2023-08-10: CVEs assigned by MITRE >> - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling >> attack existence to security@bitcoincore•org. >> - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / >> CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks >> >> ## Conclusion >> >> Despite the line of mitigations adopted and deployed by current major >> lightning implementations, I believe replacement cycling attacks are still >> practical for advanced attackers. Beyond this new attack might come as a >> way to partially or completely defeat some of the pinning mitigations which >> have been working for years as a community. >> >> As of today, it is uncertain to me if lightning is not affected by a more >> severe long-term package malleability critical security issue under current >> consensus rules, and if any other time-sensitive multi-party protocol, >> designed or deployed isn't de facto affected too (loss of funds or denial >> of service). >> >> Assuming analysis on package malleability is correct, it is unclear to me >> if it can be corrected by changes in replacement / eviction rules or >> mempool chain of transactions processing strategy. Inviting my technical >> peers and the bitcoin community to look more on this issue, including to >> dissent. I'll be the first one pleased if I'm fundamentally wrong on those >> issues, or if any element has not been weighted with the adequate technical >> accuracy it deserves. >> >> Do not trust, verify. All mistakes and opinions are my own. >> >> Antoine >> >> "meet with Triumph and Disaster. And treat those two impostors just the >> same" - K. >> >
Could this be addressed with an OP_CSV_ALLINPUTS, a covenant opcode that requires *all* inputs to have a matching nSequence, and using `1 OP_CSV_ALLINPUTS` in the HTLC preimage branch?
This would prevent using unconfirmed outputs in the HTLC-preimage-spending transaction entirely, which IIUC should protect it against the replacement cycling attack.
(If desirable, this could alternatively be OP_CSV_OTHERINPUTS to allow the HTLC output itself to be spent immediately via the preimage branch, and only require that the other inputs added for fees are confirmed.)
On Mon, Oct 16, 2023 at 8:03 PM Antoine Riard via bitcoin-dev < bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (291 lines)
> (cross-posting mempool issues identified are exposing lightning chan to > loss of funds risks, other multi-party bitcoin apps might be affected) > > Hi, > > End of last year (December 2022), amid technical discussions on eltoo > payment channels and incentives compatibility of the mempool anti-DoS > rules, a new transaction-relay jamming attack affecting lightning channels > was discovered. > > After careful analysis, it turns out this attack is practical and > immediately exposed lightning routing hops carrying HTLC traffic to loss of > funds security risks, both legacy and anchor output channels. A potential > exploitation plausibly happening even without network mempools congestion. > > Mitigations have been designed, implemented and deployed by all major > lightning implementations during the last months. > > Please find attached the release numbers, where the mitigations should be > present: > - LDK: v0.0.118 - CVE-2023 -40231 > - Eclair: v0.9.0 - CVE-2023-40232 > - LND: v.0.17.0-beta - CVE-2023-40233 > - Core-Lightning: v.23.08.01 - CVE-2023-40234 > > While neither replacement cycling attacks have been observed or reported > in the wild since the last ~10 months or experimented in real-world > conditions on bitcoin mainet, functional test is available exercising the > affected lightning channel against bitcoin core mempool (26.0 release > cycle). > > It is understood that a simple replacement cycling attack does not demand > privileged capabilities from an attacker (e.g no low-hashrate power) and > only access to basic bitcoin and lightning software. Yet I still think > executing such an attack successfully requests a fair amount of bitcoin > technical know-how and decent preparation. > > From my understanding of those issues, it is yet to be determined if the > mitigations deployed are robust enough in face of advanced replacement > cycling attackers, especially ones able to combine different classes of > transaction-relay jamming such as pinnings or vetted with more privileged > capabilities. > > Please find a list of potential affected bitcoin applications in this full > disclosure report using bitcoin script timelocks or multi-party > transactions, albeit no immediate security risk exposure as severe as the > ones affecting lightning has been identified. Only cursory review of > non-lightning applications has been conducted so far. > > There is a paper published summarizing replacement cycling attacks on the > lightning network: > > https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf > > ## Problem > > A lightning node allows HTLCs forwarding (in bolt3's parlance accepted > HTLC on incoming link and offered HTLC on outgoing link) should settle the > outgoing state with either a success or timeout before the incoming state > timelock becomes final and an asymmetric defavorable settlement might > happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" > section 2.3 for a classical exposition of this lightning security property). > > Failure to satisfy this settlement requirement exposes a forwarding hop to > a loss of fund risk where the offered HTLC is spent by the outgoing link > counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming > link counterparty's HTLC-timeout. > > The specification mandates the incoming HTLC expiration timelock to be > spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC > expiration timelock, this exact interval value being an implementation and > node policy setting. As a minimal value, the specification recommends 34 > blocks of interval. If the timelock expiration I of the inbound HTLC is > equal to 100 from chain tip, the timelock expiration O of the outbound HTLC > must be equal to 66 blocks from chain tip, giving a reasonable buffer of > reaction to the lightning forwarding node. > > In the lack of cooperative off-chain settlement of the HTLC on the > outgoing link negotiated with the counterparty (either > `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the > lightning node should broadcast its commitment transaction. Once the > commitment is confirmed (if anchor and the 1 CSV encumbrance is present), > the lightning node broadcasts and confirms its HTLC-timeout before I height > is reached. > > Here enter a replacement cycling attack. A malicious channel counterparty > can broadcast its HTLC-preimage transaction with a higher absolute fee and > higher feerate than the honest HTLC-timeout of the victim lightning node > and triggers a replacement. Both for legacy and anchor output channels, a > HTLC-preimage on a counterparty commitment transaction is malleable, i.e > additional inputs or outputs can be added. The HTLC-preimage spends an > unconfirmed and unrelated to the channel parent transaction M and conflicts > its child. > > As the HTLC-preimage spends an unconfirmed input that was already included > in the unconfirmed and unrelated child transaction (rule 2), pays an > absolute higher fee of at least the sum paid by the HTLC-timeout and child > transaction (rule 3) and the HTLC-preimage feerate is greater than all > directly conflicting transactions (rule 6), the replacement is accepted. > The honest HTLC-timeout is evicted out of the mempool. > > In an ulterior move, the malicious counterparty can replace the parent > transaction itself with another candidate N satisfying the replacement > rules, triggering the eviction of the malicious HTLC-preimage from the > mempool as it was a child of the parent T. > > There is no spending candidate of the offered HTLC output for the current > block laying in network mempools. > > This replacement cycling tricks can be repeated for each rebroadcast > attempt of the HTLC-timeout by the honest lightning node until expiration > of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout > is broadcast by the counterparty's on the incoming link in collusion with > the one on the outgoing link broadcasting its own HTLC-preimage. > > The honest Lightning node has been "double-spent" in its HTLC forwarding. > > As a notable factor impacting the success of the attack, a lightning > node's honest HTLC-timeout might be included in the block template of the > miner winning the block race and therefore realizes a spent of the offered > output. In practice, a replacement cycling attack might over-connect to > miners' mempools and public reachable nodes to succeed in a fast eviction > of the HTLC-timeout by its HTLC-preimage. As this latter transaction can > come with a better ancestor-score, it should be picked up on the flight by > economically competitive miners. > > A functional test exercising a simple replacement cycling of a HTLC > transaction on bitcoin core mempool is available: > https://github.com/ariard/bitcoin/commits/2023-test-mempool > > ## Deployed LN mitigations > > Aggressive rebroadcasting: As the replacement cycling attacker benefits > from the HTLC-timeout being usually broadcast by lightning nodes only once > every block, or less the replacement cycling malicious transactions paid > only equal the sum of the absolute fees paid by the HTLC, adjusted with the > replacement penalty. Rebroadcasting randomly and multiple times before the > next block increases the absolute fee cost for the attacker. > > Implemented and deployed by Eclair, Core-Lightning, LND and LDK . > > Local-mempool preimage monitoring: As the replacement cycling attacker in > a simple setup broadcast the HTLC-preimage to all the network mempools, the > honest lightning node is able to catch on the flight the unconfirmed > HTLC-preimage, before its subsequent mempool replacement. The preimage can > be extracted from the second-stage HTLC-preimage and used to fetch the > off-chain inbound HTLC with a cooperative message or go on-chain with it to > claim the accepted HTLC output. > > Implemented and deployed by Eclair and LND. > > CLTV Expiry Delta: With every jammed block comes an absolute fee cost paid > by the attacker, a risk of the HTLC-preimage being detected or discovered > by the honest lightning node, or the HTLC-timeout to slip in a winning > block template. Bumping the default CLTV delta hardens the odds of success > of a simple replacement cycling attack. > > Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. > > ## Affected Bitcoin Protocols and Applications > > From my understanding the following list of Bitcoin protocols and > applications could be affected by new denial-of-service vectors under some > level of network mempools congestion. Neither tests or advanced review of > specifications (when available) has been conducted for each of them: > - on-chain DLCs > - coinjoins > - payjoins > - wallets with time-sensitive paths > - peerswap and submarine swaps > - batch payouts > - transaction "accelerators" > > Inviting their developers, maintainers and operators to investigate how > replacement cycling attacks might disrupt their in-mempool chain of > transactions, or fee-bumping flows at the shortest delay. Simple flows and > non-multi-party transactions should not be affected to the best of my > understanding. > > ## Open Problems: Package Malleability > > Pinning attacks have been known for years as a practical vector to > compromise lightning channels funds safety, under different scenarios (cf. > current bip331's motivation section). Mitigations at the mempool level have > been designed, discussed and are under implementation by the community > (ancestor package relay + nverrsion=3 policy). Ideally, they should > constraint a pinning attacker to always attach a high feerate package > (commitment + CPFP) to replace the honest package, or allow a honest > lightning node to overbid a malicious pinning package and get its > time-sensitive transaction optimistically included in the chain. > > Replacement cycling attack seem to offer a new way to neutralize the > design goals of package relay and its companion nversion=3 policy, where an > attacker package RBF a honest package out of the mempool to subsequently > double-spend its own high-fee child with a transaction unrelated to the > channel. As the remaining commitment transaction is pre-signed with a > minimal relay fee, it can be evicted out of the mempool. > > A functional test exercising a simple replacement cycling of a lightning > channel commitment transaction on top of the nversion=3 code branch is > available: > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > ## Discovery > > In 2018, the issue of static fees for pre-signed lightning transactions is > made more widely known, the carve-out exemption in mempool rules to > mitigate in-mempool package limits pinning and the anchor output pattern > are proposed. > > In 2019, bitcoin core 0.19 is released with carve-out support. Continued > discussion of the anchor output pattern as a dynamic fee-bumping method. > > In 2020, draft of anchor output submitted to the bolts. Initial finding of > economic pinning against lightning commitment and second-stage HTLC > transactions. Subsequent discussions of a preimage-overlay network or > package-relay as mitigations. Public call made to inquiry more on potential > other transaction-relay jamming attacks affecting lightning. > > In 2021, initial work in bitcoin core 22.0 of package acceptance. > Continued discussion of the pinning attacks and shortcomings of current > mempool rules during community-wide online workshops. Later the year, in > light of all issues for bitcoin second-layers, a proposal is made about > killing the mempool. > > In 2022, bip proposed for package relay and new proposed v3 policy design > proposed for a review and implementation. Mempoolfullrbf is supported in > bitcoin core 24.0 and conceptual questions about alignment of mempool rules > w.r.t miners incentives are investigated. > > Along this year 2022, eltoo lightning channels design are discussed, > implemented and reviewed. In this context and after discussions on mempool > anti-DoS rules, I discovered this new replacement cycling attack was > affecting deployed lightning channels and immediately reported the finding > to some bitcoin core developers and lightning maintainers. > > ## Timeline > > - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, Greg > Sanders and Gloria Zhao > - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien Teinturier, > Matt Corallo and Olaoluwa Osuntunkun > - 2022-12-23: Sharing to Eugene Siegel (LND) > - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot (non-lightning > potential affected projects) > - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and cross-layers > issuers) and initial proposal of an early public disclosure > - 2022-01-19: Collection of analysis if other second-layers and > multi-party applications affected. LN mitigations development starts. > - 2023-05-04: Sharing to Wilmer Paulino (LDK) > - 2023-06-20: LN mitigations implemented and progressively released. Week > of the 16 october proposed for full disclosure. > - 2023-08-10: CVEs assigned by MITRE > - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling > attack existence to security@bitcoincore•org. > - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / > CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks > > ## Conclusion > > Despite the line of mitigations adopted and deployed by current major > lightning implementations, I believe replacement cycling attacks are still > practical for advanced attackers. Beyond this new attack might come as a > way to partially or completely defeat some of the pinning mitigations which > have been working for years as a community. > > As of today, it is uncertain to me if lightning is not affected by a more > severe long-term package malleability critical security issue under current > consensus rules, and if any other time-sensitive multi-party protocol, > designed or deployed isn't de facto affected too (loss of funds or denial > of service). > > Assuming analysis on package malleability is correct, it is unclear to me > if it can be corrected by changes in replacement / eviction rules or > mempool chain of transactions processing strategy. Inviting my technical > peers and the bitcoin community to look more on this issue, including to > dissent. I'll be the first one pleased if I'm fundamentally wrong on those > issues, or if any element has not been weighted with the adequate technical > accuracy it deserves. > > Do not trust, verify. All mistakes and opinions are my own. > > Antoine > > "meet with Triumph and Disaster. And treat those two impostors just the > same" - K. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
Quoted text (1 line)
> By redefining a bit of the nVersion field, eg the most significant bit, we can apply coinbase-like txout handling to arbitrary transactions.
We already have that in OP_CHECKSEQUENCEVERIFY. You can have a system with no coinbase transactions at all, and use only OP_CHECKSEQUENCEVERIFY on the first transaction, and set sequence numbers to put a relative locktime of 100 blocks. Also, if you think some soft-fork is needed anyway, then I recommend building it around already existing OP_CHECKSEQUENCEVERIFY, than reinvent the wheel.
Quoted text (1 line)
> Redefining an existing OP_Nop opcode, OP_Expire would terminate script evaluation with an error
This one is also already there. We have reserved opcodes, like OP_RESERVED. You can do something like "<condition> OP_IF OP_RESERVED OP_ENDIF", and then, if "<condition>" is triggered in the Script, the whole transaction is marked as invalid. But if that condition is false, then everything is fine, and you can continue executing next opcodes. Again, the situation is the same as in previous case: build it around OP_RESERVED, that is just "OP_EXPIRE" with a different name than you want, and then the only thing you need, is soft-forking a proper condition on the stack.
On 2023-10-21 02:09:55 user Peter Todd via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote: On Mon, Oct 16, 2023 at 05:57:36PM +0100, Antoine Riard via bitcoin-dev wrote: > Here enter a replacement cycling attack. A malicious channel counterparty > can broadcast its HTLC-preimage transaction with a higher absolute fee and > higher feerate than the honest HTLC-timeout of the victim lightning node > and triggers a replacement. Both for legacy and anchor output channels, a > HTLC-preimage on a counterparty commitment transaction is malleable, i.e > additional inputs or outputs can be added. The HTLC-preimage spends an > unconfirmed and unrelated to the channel parent transaction M and conflicts > its child. The basic problem here is after the HTLC-timeout path becomes spendable, the HTLC-preimage path remains spendable. That's bad, because in this case we want spending the HTLC-preimage - if possible - to have an urgency attached to it to ensure that it happens before the previous HTLC-timeout is mined. So, why can't we make the HTLC-preimage path expire? Traditionally, we've tried to ensure that transactions - once valid - remain valid forever. We do this because we don't want transactions to become impossible to mine in the event of a large reorganization. A notable example of this design philosophy is seen in Bitcoin's rules around coinbase outputs: they only become spendable after 100 more blocks have been found; a 100 block reorg is quite unlikely. Enter the OP_Expire and the Coinbase Bit soft-fork upgrade. # Coinbase Bit By redefining a bit of the nVersion field, eg the most significant bit, we can apply coinbase-like txout handling to arbitrary transactions. Such a transaction's outputs would be treated similarly to a coinbase transaction, and would be spendable only after 100 more blocks had been mined. Due to this requirement, these transactions will pose no greater risk to reorg safety than the existing hazard of coinbase transactions themselves becoming invalid. Note how such a transaction is non-standard right now, ensuring compatibility with existing nodes in a soft-fork upgrade. # OP_Expire Redefining an existing OP_Nop opcode, OP_Expire would terminate script evaluation with an error if: 1) the Coinbase Bit was not set; or 2) the stack is empty; or 3) the top item on the stack was >= the block height of the containing block This is conceptually an AntiCheckLockTimeVerify: where CLTV _allows_ a txout to become spendable in a particular way in the future, Expire _prevents_ a txout from being spent in a particular way. Since OP_Expire requires the Coinbase Bit to be set, the reorg security of OP_Expire-using transactions is no worse than transactions spending miner coinbases. # How HTLC's Would Use OP_Expire Whenever revealing the preimage on-chain is necessary to the secure functioning of the HTLC-using protocol, we simply add an appropriate OP_Expire to the pre-image branch of the script along the lines of: If Expire Drop Hash EqualVerify CheckSig ElseIf # HTLC Expiration conditions ... EndIf Now the party receiving the pre-image has a deadline. Either they get a transaction spending the preimage mined, notifying the other party via the blockchain itself, or they fail to get the preimage mined in time, reverting control to the other party who can spend the HTLC output at their leisure, without strict time constraints. Since the HTLC-expired branch does *not* execute OP_Expire, the transaction spending the HTLC-expired branch does *not* need to set the Coinbase Bit. Thus it can be spent in a perfectly normal transaction, without restrictions. # Delta Encoding Expiration Rather than having a specific Coinbase Bit, it may also be feasible to encode the expiration height as a delta against a block-height nLockTime. In this variant, OP_Expire would work similarly to OP_CheckLockTimeVerify, by checking that the absolute expiration height was <= the requested expiration, allowing multiple HTLC preimage outputs to be spent in one transaction. If the top 16-bits were used, the maximum period a transaction could be valid would be: 2^16 blocks / 144 blocks/day = 455 days In this variant, a non-zero expiration delta would enable expiration behavior, as well as the coinbase-like output spending restriction. The remaining 16-bits of nVersion would remain available for other meanings. Similar to how CLTV and CSV verified nLockTime and nSequence respectively, verifying an expiration height encoded in the nVersion has the advantage of making an expiration height easy to detect without validating scripts. While Lightning's HTLC-success transactions currently use nLockTime=0, AFAIK there is no reason why they could not set nLockTime to be valid in the next block, allowing delta encoding to be used. ## Reusing Time-Based nLockTime Reusing time-based nLockTime's prior to some pre-2009 genesis point for expiration is another possibility (similar to how Lightning makes use of time-based nLockTime for signalling). However I believe this is not as desirable as delta encoding or a coinbase bit, as it would prevent transactions from using block nLockTime and expiration at the same time. It would also still require a coinbase bit or nVersion increase to ensure expiration-using transactions are non-standard. # Mempool Behavior Obviously, mempool logic will need to handle transactions that can expire differently than non-expiring transactions. One notable consideration is that nodes should require higher minimum relay fees for transactions close to their expiration height to ensure we don't waste bandwidth on transactions that have no potential to be mined. Considering the primary use-case, it is probably acceptable to always require a fee rate high enough to be mined in the next block. -- https://petertodd.org 'peter'[:-1]@petertodd.org _______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists•linuxfoundation•org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
On 2023-10-21 18:49, Nadav Ivgi via bitcoin-dev wrote:
Quoted text (7 lines)
> Could this be addressed with an OP_CSV_ALLINPUTS, a covenant opcode > that requires _all_ inputs to have a matching nSequence, and using `1 > OP_CSV_ALLINPUTS` in the HTLC preimage branch? > > This would prevent using unconfirmed outputs in the > HTLC-preimage-spending transaction entirely, which IIUC should protect > it against the replacement cycling attack.
I don't think that addresses the underlying problem. In Riard's description, a replacement cycle looks like this:
- Bob broadcasts an HTLC-timeout (input A, input B for fees, output X) - Mallory replaces the HTLC-timeout with an HTLC-preimage (input A, input C for fees, output Y) - Mallory replaces the transaction that created input C, removing the HTLC-preimage from the mempool
However, an alternative approach is:
- (Same) Bob broadcasts an HTLC-timeout (input A, input B for fees, output X) - (Same) Mallory replaces the HTLC-timeout with an HTLC-preimage (input A, input C for fees, output Y) - (Different) Mallory uses input C to replace the HTLC-preimage with a transaction that does not include input A, removing the preimage from the mempool
The original scenario requires input C to be from an unconfirmed transaction, so OP_CSV_ALLINPUTS works. The alternative scenario works even if input C comes from a confirmed transaction, so OP_CSV_ALLINPUTS is ineffective.
-Dave
Hi all,
This was discussed partially on the platform formerly known as twitter, but an alternate design goes like this:
* Add an `nExpiryTime` field in taproot annex. * This indicates that the transaction MUST NOT exist in a block at or above the height specified. * Mempool should put txes buckets based on `nExpiryTime`, and if the block is reached, drop all the buckets with `nExpiryTime` less than that block height. * Add an `OP_CHECKEXPIRYTIMEVERIFY` opcode, mostly similar in behavior to `OP_EXPIRE` proposed by Peter Todd: * Check if `nExpiryTime` exists and has value equal or less than the stack top.
The primary difference is simply that while Peter proposes an implicit field for the value that `OP_EXPIRE` will put in `CTransaction`, I propose an explicit field for it in the taproot annex.
The hope is that "explicit is better than implicit" and that the design will be better implemented as well by non-Bitcoin-core implementations, as the use of tx buckets is now explicit in treating the `nExpiryTime` field.
Regards, ZmnSCPxj
On Mon, Oct 23, 2023 at 11:10:56AM +0000, ZmnSCPxj wrote:
Quoted text (5 lines)
> Hi all, > > This was discussed partially on the platform formerly known as twitter, but an alternate design goes like this: > > * Add an `nExpiryTime` field in taproot annex.
I would strongly suggest making it nExpiryHeight, and only offering the option of expiration at a given height.
Time-based anything is sketchy, as it could give miners incentives to lie about the current time in the nTime field. If anything, the fact that nLockTime can in fact be time-based was a design mistake.
Quoted text (3 lines)
> * This indicates that the transaction MUST NOT exist in a block at or above the height specified. > * Mempool should put txes buckets based on `nExpiryTime`, and if the block is reached, drop all the buckets with `nExpiryTime` less than that block height. > * Add an `OP_CHECKEXPIRYTIMEVERIFY` opcode, mostly similar in behavior to `OP_EXPIRE` proposed by Peter Todd:
Note that if we redefine an OP_SuccessX opcode, we do not need _Verify behavior. We can produce a true/false stack element, making either OP_Expire or OP_CheckExpiryTime better names for the opcode.
Quoted text (3 lines)
> * Check if `nExpiryTime` exists and has value equal or less than the stack top. > > The primary difference is simply that while Peter proposes an implicit field for the value that `OP_EXPIRE` will put in `CTransaction`, I propose an explicit field for it in the taproot annex.
To be clear, I also proposed an explicit field too. But I had a brainfart and forgot that we'd added the taproot annex. So I proposed re-using part of nVersion.
Quoted text (1 line)
> The hope is that "explicit is better than implicit" and that the design will be better implemented as well by non-Bitcoin-core implementations, as the use of tx buckets is now explicit in treating the `nExpiryTime` field.
Also, having a nExpiryHeight may be useful in cases where a signature covering the field is sufficient.
On 10/20/23 7:43 PM, Peter Todd wrote:
Quoted text (15 lines)
> On Fri, Oct 20, 2023 at 09:55:12PM -0400, Matt Corallo wrote: >>> Quite the contrary. Schnorr signatures are 64 bytes, so in situations like >>> lightning where the transaction form is deterministically derived, signing 100 >>> extra transactions requires just 6400 extra bytes. Even a very slow 100KB/s >>> connection can transfer that in 64ms; latency will still dominate. >> >> Lightning today isn't all that much data, but multiply it by 100 and we >> start racking up data enough that people may start to have to store a really >> material amount of data for larger nodes and dealing with that starts to be >> a much bigger pain then when we're talking about a GiB or twenty. > > We are talking about storing ephemeral data here, HTLC transactions and > possibly commitment transactions. Since lightning uses disclosed secrets to > invalidate old state, you do not need to keep every signature from your > counterparty indefinitely.
Mmm, fair point, yes.
Quoted text (19 lines)
>>> RBF has a minimum incremental relay fee of 1sat/vByte by default. So if you use >>> those 100 pre-signed transaction variants to do nothing more than sign every >>> possible minimum incremental relay, you've covered a range of 1sat/vByte to >>> 100sat/vByte. I believe that is sufficient to get mined for any block in >>> Bitcoin's entire modern history. >>> >>> CPFP meanwhile requires two transactions, and thus extra bytes. Other than edge >>> cases with very large transactions in low-fee environments, there's no >>> circumstance where CPFP beats RBF. >> >> What I was referring to is that if we have the SIGHASH_SINGLE|ANYONECANPAY >> we can combine many HTLC claims into one transaction, vs pre-signing means >> we're stuck with a ton of individual transactions. > > Since SIGHASH_SINGLE requires one output per input, the savings you get by > combining multiple SIGHASH_SINGLE transactions together aren't very > significant. Just 18 bytes for nVersion, nLockTime, and the txin and txout size > fields. The HTLC-timeout transaction is 166.5 vBytes, so that's a savings of > just 11%
Yep, its not a lot, but for a thing that's inherently super chain-spammy, its still quite nice.
Quoted text (6 lines)
> Of course, if you _do_ need to fee bump and add an additional input, that input > takes up space, and you'll probably need a change output. At which point you > again would probably have been better off with a pre-signed transaction. > > You are also assuming there's lots of HTLC's in flight that need to be spent. > That's very often not the case.
In general, yes, in force-close cases often there's been some failure which is repeated in several HTLCs :).
More generally, I think we're getting lost here - this isn't really a material change for lightning's trust model - its already the case that a peer that is willing to put a lot of work in can probably steal your money, and there's now just one more way they can do that. We really don't need to rush to "fix lightning" here, we can do it right and fix it at the ecosystem level. It shouldn't be the case that a policy restriction results in both screwing up a L2 network *and* results in miners getting paid less. That's a policy bug.
Matt
On Sat, Oct 21, 2023 at 09:05:35PM +0100, Antoine Riard via bitcoin-dev wrote:
Quoted text (6 lines)
> In the meanwhile, lightning experts have already deployed mitigations which > are hardening the lightning ecosystem significantly in face of simple or > medium attacks. More advanced attacks can only be mounted if you have > sufficient p2p and mempool knowledge as was pointed out by other bitcoin > experts like Matt or Peter (which take years to acquire for average bitcoin > developers) and the months of preparation to attempt them.
To be clear, I am not making any claims about how easy this attack is to pull off. Indeed, there are probably even cases where it happens by accident. Eg imagine a node with a HTLC-preimage that happens to be offline and then online at the right time to broadcast a HTLC-preimage redemption transaction with a higher fee than the timeout transaction. If the other node happens to go offline at the right time, after broadcasting the timeout transaction, it may not notice the HTLC-preimage in the mempool, and thus fail to redeem it.
OP_Expire would help avoid this situation, by making it impossible to redeem the HTLC-preimage after the timeout.
Hi list,
As I received a lot of feedback on the full disclosure of the 16th week of October and the following posts, some accurate, I'm taking time to address a few of them.
I think one of the most recurring feedback is the fact that the replacement cycling issue laid out in the initial full disclosure post could have been better explained or presented. Here you have a typical disclosure dilemma encountered by any security researcher, on one hand you wish to explain them as best you can to enlighten the public audience, on the other hand if you come up with a full exploitation rootkit, its manual and a paper with a lot of graphic informing on all the way to exploit, you might expose the public audience funds.
As a reminder, at the time of the full disclosure the 16th, some lightning implementations were still adding up mitigations in their codebase and corresponding release has only been tagged last week. Moreover, I think when you're doing a full disclosure as a security researcher, it is wiser to be conservative in the flow of information disclosed, as you might have missed some observations or insights worsening the severity and practicality of exploitation.
In my reasonable and modest experience, the amount and clarity of information you're revealing in matters of security disclosure is an art rhythmed by contingencies, not a science.
Additionally, there is one "unusual" security risk only affecting Bitcoin second-layers, namely triggering winds of panic among lightning node operators. This group of users started to manually force-close channels and holistically congestion network mempools, opening the door to opportunistic "flood & loot" exploitation, a risk known by lightning experts for years.
E.g bare option anchor commitment transactions are 900 WU (bolt3), max block size is 4_000_000 WU, if you have half of the 80k of _public_ channels, you might have a hour and half of full blocks, which might offer the opportunity of stealing payments from payee, based on historical timelocks.
This last risk is based on experience from previous security coordination work and I did inform the core security list of this concern in date of the 5 october: " if we see an abnormal rate of network mempools congestion from LN node operators manually force-closing their channels with low-trusted peers, this phenomena has already happened during past Lightning security issues disclosures" as it could have provoked disruptions ecosystem-wide beyond lightning.
There have been some voices speaking up on the sudden announcement from my side to step down from lightning development. While there has been complete distorsion of my statement by non-specialized journalists and I've been the first one to complain, howev er it was very deliberate on my side to ring the bell on the very poor state of lightning security and marked the score than this new replacement cycling issue is a far more concerning than other major security risks of lightning known so far. As a friendly reminder, we start to pile up a very serious stack of security risks: pinnings, channel jamming, replacement cycling, time dilation (all those ones could kill lightning if exploited at scale - period) and more minor ones such as dust HTLC exposure, fee or liquidity griefing and other denial-of-service.
Hard things about hard things, every new major security risk discovered somehow constrains lightning developers to go back on the whiteboard and check that any mitigation in development for each risk is not broken or the security properties with. Most of the time, we might think in isolation to ease the mitigation of research and development work. Astute adversaries might not give us those flowers. At the end, technical reality advises that lightning security issues are better solved at the base-layer, and this is where expert time is cruelly missing to make lightning more robust on the very long-term.
On a more ethical plan, there is no wish on my side to stay accountable on the coordinated handling of security issues in the lightning ecosystem, when I have the inner conviction than replacement cycling issues are practical and critical, that they cannot be solved correctly on the lightning-side, that any serious mitigation on the base-layer might take numerous years and their integration back in the lightning-side might take another couple of years. Once again, I warned the whole community of the nest of issues at the mempool-level for lightning very consistently during the past years.
With humility, if those replacement cycling attacks start to be played out at scale by sophisticated attackers during the coming future, I'm not competent to improvise an effective mitigation. And even less competent to coordinate dissemination of such eventual patch of mitigations across an ecosystem of nodes, without losing inter-compatibility between implementations, wallets and releases versions (and here the lack of standardized mechanism like dynamic upgrade bolt pr #1090 is a missing safety feature).
Security of critical software systems is somehow akin to medical practice. If you wish to nurture an adequate clinical treatment, you have to get the diagnosis of the illness severity first, even if it's a painful truth for the patient. In my opinion, here the patient is the community of lightning node operators and lightning users, and if we wish for an adequate technical treatment to overcome the vulnerability severity, building consensus on the severity sounds a necessary step.
As of today, I think the criticality of this replacement cycling attack is still misappreciated by other bitcoin devs, including lightning experts. I have not seen so far discussion of the issue entitled "package malleability" and the corresponding test on bitcoin core mempool, adapting the replacement cycling scenario to a post-package relay world. If I'm correct here though welcoming more expert eyes with pleasure here, dynamic fee-bumping of pre-signed lightning states as we have been working from years can be broken by replacement cycling attacks, extending the types of lightning nodes at risk from forwarding nodes to just any lightning node (and in the lack of dynamic fee-bumping pre-signed states might not propagate to miners template due to the dynamic mempool min fee).
Finally, running a piece of software is a matter of an individual or business decision. While one can as a technical "domain expert" explain the risk model with the utmost care and diligence, at the end of the day the decision belongs to everyone on what to do with their own computing infrastructure and stack of satoshis. Yet if you're servicing customers or users with your infrastructure, please be mindful of the safety of their funds and take time to understand how your technical decisions might affect them. Their risk tolerance is very likely not yours [0].
All that said, after months of working on those issues under embargo, I find it pleasant to see the bitcoin technical community and beyond actually talking about those issues at length with a "don't trust, verify" mindset. I'll take time to answer the public mails on those subjects during the coming weeks and months. I'm still enthusiastic to do a real-world demonstration of a replacement cycling attack on mainnet though I won't have time until the second semester of 2024 at the very best (I'm still late on my announcement of 2020 to do a real-world demonstration of pinning attacks - I'm learning to be very conservative with my public open-source commitment now). Assuming a timeline of a decade or two, I'm still optimistic on seeing one day a lightning ecosystem looking like the p2p cypherpunk version as fleshed out in the original paper.
Cheers, Antoine
[0] "Chancellor on the brink of second bailout of banks" - I think the 2008 crisis and its massive bailout of big financial players by the small guy money worldwide is a good meditation. Hopefully, we're not building a new financial system where the big bitcoin and lightning players are concealing massive security risks from the average end-user, and in the occurrence of the risks flagrant realization will deny any form of accountability by walls of PR statements.
Le sam. 21 oct. 2023 à 21:05, Antoine Riard <antoine.riard@gmail•com> a écrit :
Quoted text (420 lines)
> Hi, > > As I've been shown offline Twitter posts misrepresenting my previous mail, > I think it's good to correct them. The security flaws are not "intentional > backdoor" or whatever misrepresentation that would question the competence > and know-how of the Bitcoin and Lightning development community. > > The replacement cycling issue discovered has been known by a small circle > of Bitcoin developers since December 2022. As it appears to some experts > and it has been commented publicly, changes at the bitcoin base-layer might > be the most substantial fixes. Those changes take time and here this is > akin to how the linux kernel, bsds and OS vendors are working [0]. > > All I can say is that we had recently had internal discussion on how to > improve coordinated security fixes and patching processes for the coming > decades. This is an area of concern where I've always been at the forefront > as early as 2020 / 2021 [1]. > > In the meanwhile, lightning experts have already deployed mitigations > which are hardening the lightning ecosystem significantly in face of simple > or medium attacks. More advanced attacks can only be mounted if you have > sufficient p2p and mempool knowledge as was pointed out by other bitcoin > experts like Matt or Peter (which take years to acquire for average bitcoin > developers) and the months of preparation to attempt them. > > If you're a journalist reporting on the information in mainstream crypto > publications, I'll suggest waiting to do so before expert reporters of > bitcoin circles who have more in-field knowledge can do so and qualify the > technical situation with more distance. As I've already been interviewed by > top financial publication years ago for my work on bitcoin, as a journalist > you're engaging your own reputation on the information you're reporting. > Thanks for being responsible here. > > This is the nature of the electronic communication and contemporaneous > media that information is extremely fluid and there is no native anti-DoS > mechanism to slow down the propagation of sensitive information where > mitigations are still in deployment. A reason I'm not on social media of > any kind [2]. In the meanwhile, it's good to go to read senecca and marcus > aurelius take the situation with stoicism and with a zelt of meditation [3]. > > All my previous statements are mostly technically correct (even if some > could have been written with more clarity once again I'm not an english > native [4]). While I wish to wait the week of the 30th Oct o discuss > further what is best fix and what are the trade-offs as a community as a > wide (give time some laggard lightning implementations ship fixes), though > I'll comment further on the mailing list if the flow of information on > "social media" is DoSing the ability of the bitcoin community to work on > the long-term appropriate fixes in a responsible and constructive fashion. > > [0] See meltdown class of vulnerability and how operating systems are > handling hardware-sourced vulnerabilities > https://en.wikipedia.org/wiki/Meltdown_(security_vulnerability). Most of > the time they do their best on the software side and they go to see with > hardware vendors how to do the necessary updates. > > [1] > https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-April/003002.html > > [2] And for the wider analysis on contemporaneous culture of information > propagation and network effect, I can only recommend to read venkatesh > rao's ribbonfarm essays http://ribbonfarm.com > > [3] There are very good reasons why some executives at top modern > technology companies are doing meditation daily, some even hours. "mind > illuminated" is a good read. > > [4] While my former employer, Chaincode Labs, paid for my english lessons > back in 2020. Generally it was a good insight from them to train people on > how to communicate in a crisis. > > > Le ven. 20 oct. 2023 à 07:56, Antoine Riard <antoine.riard@gmail•com> a > écrit : > >> Hi, >> >> After writing the mail reply on the economics of sequential malicious >> replacement of honest HTLC-timeout, I did write one more test to verify the >> behavior on core mempool, and it works as expected. >> >> >> https://github.com/ariard/bitcoin/commit/30f5d5b270e4ff195e8dcb9ef6b7ddcc5f6a1bf2 >> >> Responsible disclosure process has followed the lines of hardware issues >> affecting operating system, as documented for the Linux kernel, while >> adapted to the bitcoin ecosystem: >> >> https://docs.kernel.org/6.1/process/embargoed-hardware-issues.html >> >> Effective now, I'm halting my involvement with the development of the >> lightning network and its implementations, including coordinating the >> handling of security issues at the protocol level (I informed some senior >> lightning devs in that sense before). >> >> Closed the very old issue which was affected to me at this purpose on the >> bolt repository: >> >> https://github.com/lightning/bolts/pull/772 >> >> I think this new class of replacement cycling attacks puts lightning in a >> very perilous position, where only a sustainable fix can happen at the >> base-layer, e.g adding a memory-intensive history of all-seen transactions >> or some consensus upgrade. Deployed mitigations are worth something in face >> of simple attacks, though I don't think they're stopping advanced attackers >> as said in the first full disclosure mail. >> >> Those types of changes are the ones necessitating the utmost transparency >> and buy-in of the community as a whole, as we're altering the full-nodes >> processing requirements or the security architecture of the decentralized >> bitcoin ecosystem in its integrality. >> >> On the other hand fully explaining why such changes would be warranted >> for the sake of lightning and for designing them well, we might need to lay >> out in complete state practical and critical attacks on a ~5 355 public BTC >> ecosystem. >> >> Hard dilemma. >> >> There might be a lesson in terms of bitcoin protocol deployment, we might >> have to get them right at first try. Little second chance to fix them in >> flight. >> >> I'll be silent on those issues on public mailing lists until the week of >> the 30 oct. Enough material has been published and other experts are >> available. Then I'll be back focusing more on bitcoin core. >> >> Best, >> Antoine >> >> Le lun. 16 oct. 2023 à 17:57, Antoine Riard <antoine.riard@gmail•com> a >> écrit : >> >>> (cross-posting mempool issues identified are exposing lightning chan to >>> loss of funds risks, other multi-party bitcoin apps might be affected) >>> >>> Hi, >>> >>> End of last year (December 2022), amid technical discussions on eltoo >>> payment channels and incentives compatibility of the mempool anti-DoS >>> rules, a new transaction-relay jamming attack affecting lightning channels >>> was discovered. >>> >>> After careful analysis, it turns out this attack is practical and >>> immediately exposed lightning routing hops carrying HTLC traffic to loss of >>> funds security risks, both legacy and anchor output channels. A potential >>> exploitation plausibly happening even without network mempools congestion. >>> >>> Mitigations have been designed, implemented and deployed by all major >>> lightning implementations during the last months. >>> >>> Please find attached the release numbers, where the mitigations should >>> be present: >>> - LDK: v0.0.118 - CVE-2023 -40231 >>> - Eclair: v0.9.0 - CVE-2023-40232 >>> - LND: v.0.17.0-beta - CVE-2023-40233 >>> - Core-Lightning: v.23.08.01 - CVE-2023-40234 >>> >>> While neither replacement cycling attacks have been observed or reported >>> in the wild since the last ~10 months or experimented in real-world >>> conditions on bitcoin mainet, functional test is available exercising the >>> affected lightning channel against bitcoin core mempool (26.0 release >>> cycle). >>> >>> It is understood that a simple replacement cycling attack does not >>> demand privileged capabilities from an attacker (e.g no low-hashrate power) >>> and only access to basic bitcoin and lightning software. Yet I still think >>> executing such an attack successfully requests a fair amount of bitcoin >>> technical know-how and decent preparation. >>> >>> From my understanding of those issues, it is yet to be determined if the >>> mitigations deployed are robust enough in face of advanced replacement >>> cycling attackers, especially ones able to combine different classes of >>> transaction-relay jamming such as pinnings or vetted with more privileged >>> capabilities. >>> >>> Please find a list of potential affected bitcoin applications in this >>> full disclosure report using bitcoin script timelocks or multi-party >>> transactions, albeit no immediate security risk exposure as severe as the >>> ones affecting lightning has been identified. Only cursory review of >>> non-lightning applications has been conducted so far. >>> >>> There is a paper published summarizing replacement cycling attacks on >>> the lightning network: >>> >>> https://github.com/ariard/mempool-research/blob/2023-10-replacement-paper/replacement-cycling.pdf >>> >>> ## Problem >>> >>> A lightning node allows HTLCs forwarding (in bolt3's parlance accepted >>> HTLC on incoming link and offered HTLC on outgoing link) should settle the >>> outgoing state with either a success or timeout before the incoming state >>> timelock becomes final and an asymmetric defavorable settlement might >>> happen (cf "Flood & Loot: A Systematic Attack on The Lightning Network" >>> section 2.3 for a classical exposition of this lightning security property). >>> >>> Failure to satisfy this settlement requirement exposes a forwarding hop >>> to a loss of fund risk where the offered HTLC is spent by the outgoing link >>> counterparty's HTLC-preimage and the accepted HTLC is spent by the incoming >>> link counterparty's HTLC-timeout. >>> >>> The specification mandates the incoming HTLC expiration timelock to be >>> spaced out by an interval of `cltv_expiry_delta` from the outgoing HTLC >>> expiration timelock, this exact interval value being an implementation and >>> node policy setting. As a minimal value, the specification recommends 34 >>> blocks of interval. If the timelock expiration I of the inbound HTLC is >>> equal to 100 from chain tip, the timelock expiration O of the outbound HTLC >>> must be equal to 66 blocks from chain tip, giving a reasonable buffer of >>> reaction to the lightning forwarding node. >>> >>> In the lack of cooperative off-chain settlement of the HTLC on the >>> outgoing link negotiated with the counterparty (either >>> `update_fulfill_htlc` or `update_fail_htlc`) when O is reached, the >>> lightning node should broadcast its commitment transaction. Once the >>> commitment is confirmed (if anchor and the 1 CSV encumbrance is present), >>> the lightning node broadcasts and confirms its HTLC-timeout before I height >>> is reached. >>> >>> Here enter a replacement cycling attack. A malicious channel >>> counterparty can broadcast its HTLC-preimage transaction with a higher >>> absolute fee and higher feerate than the honest HTLC-timeout of the victim >>> lightning node and triggers a replacement. Both for legacy and anchor >>> output channels, a HTLC-preimage on a counterparty commitment transaction >>> is malleable, i.e additional inputs or outputs can be added. The >>> HTLC-preimage spends an unconfirmed and unrelated to the channel parent >>> transaction M and conflicts its child. >>> >>> As the HTLC-preimage spends an unconfirmed input that was already >>> included in the unconfirmed and unrelated child transaction (rule 2), pays >>> an absolute higher fee of at least the sum paid by the HTLC-timeout and >>> child transaction (rule 3) and the HTLC-preimage feerate is greater than >>> all directly conflicting transactions (rule 6), the replacement is >>> accepted. The honest HTLC-timeout is evicted out of the mempool. >>> >>> In an ulterior move, the malicious counterparty can replace the parent >>> transaction itself with another candidate N satisfying the replacement >>> rules, triggering the eviction of the malicious HTLC-preimage from the >>> mempool as it was a child of the parent T. >>> >>> There is no spending candidate of the offered HTLC output for the >>> current block laying in network mempools. >>> >>> This replacement cycling tricks can be repeated for each rebroadcast >>> attempt of the HTLC-timeout by the honest lightning node until expiration >>> of the inbound HTLC timelock I. Once this height is reached a HTLC-timeout >>> is broadcast by the counterparty's on the incoming link in collusion with >>> the one on the outgoing link broadcasting its own HTLC-preimage. >>> >>> The honest Lightning node has been "double-spent" in its HTLC forwarding. >>> >>> As a notable factor impacting the success of the attack, a lightning >>> node's honest HTLC-timeout might be included in the block template of the >>> miner winning the block race and therefore realizes a spent of the offered >>> output. In practice, a replacement cycling attack might over-connect to >>> miners' mempools and public reachable nodes to succeed in a fast eviction >>> of the HTLC-timeout by its HTLC-preimage. As this latter transaction can >>> come with a better ancestor-score, it should be picked up on the flight by >>> economically competitive miners. >>> >>> A functional test exercising a simple replacement cycling of a HTLC >>> transaction on bitcoin core mempool is available: >>> https://github.com/ariard/bitcoin/commits/2023-test-mempool >>> >>> ## Deployed LN mitigations >>> >>> Aggressive rebroadcasting: As the replacement cycling attacker benefits >>> from the HTLC-timeout being usually broadcast by lightning nodes only once >>> every block, or less the replacement cycling malicious transactions paid >>> only equal the sum of the absolute fees paid by the HTLC, adjusted with the >>> replacement penalty. Rebroadcasting randomly and multiple times before the >>> next block increases the absolute fee cost for the attacker. >>> >>> Implemented and deployed by Eclair, Core-Lightning, LND and LDK . >>> >>> Local-mempool preimage monitoring: As the replacement cycling attacker >>> in a simple setup broadcast the HTLC-preimage to all the network mempools, >>> the honest lightning node is able to catch on the flight the unconfirmed >>> HTLC-preimage, before its subsequent mempool replacement. The preimage can >>> be extracted from the second-stage HTLC-preimage and used to fetch the >>> off-chain inbound HTLC with a cooperative message or go on-chain with it to >>> claim the accepted HTLC output. >>> >>> Implemented and deployed by Eclair and LND. >>> >>> CLTV Expiry Delta: With every jammed block comes an absolute fee cost >>> paid by the attacker, a risk of the HTLC-preimage being detected or >>> discovered by the honest lightning node, or the HTLC-timeout to slip in a >>> winning block template. Bumping the default CLTV delta hardens the odds of >>> success of a simple replacement cycling attack. >>> >>> Default setting: Eclair 144, Core-Lightning 34, LND 80 and LDK 72. >>> >>> ## Affected Bitcoin Protocols and Applications >>> >>> From my understanding the following list of Bitcoin protocols and >>> applications could be affected by new denial-of-service vectors under some >>> level of network mempools congestion. Neither tests or advanced review of >>> specifications (when available) has been conducted for each of them: >>> - on-chain DLCs >>> - coinjoins >>> - payjoins >>> - wallets with time-sensitive paths >>> - peerswap and submarine swaps >>> - batch payouts >>> - transaction "accelerators" >>> >>> Inviting their developers, maintainers and operators to investigate how >>> replacement cycling attacks might disrupt their in-mempool chain of >>> transactions, or fee-bumping flows at the shortest delay. Simple flows and >>> non-multi-party transactions should not be affected to the best of my >>> understanding. >>> >>> ## Open Problems: Package Malleability >>> >>> Pinning attacks have been known for years as a practical vector to >>> compromise lightning channels funds safety, under different scenarios (cf. >>> current bip331's motivation section). Mitigations at the mempool level have >>> been designed, discussed and are under implementation by the community >>> (ancestor package relay + nverrsion=3 policy). Ideally, they should >>> constraint a pinning attacker to always attach a high feerate package >>> (commitment + CPFP) to replace the honest package, or allow a honest >>> lightning node to overbid a malicious pinning package and get its >>> time-sensitive transaction optimistically included in the chain. >>> >>> Replacement cycling attack seem to offer a new way to neutralize the >>> design goals of package relay and its companion nversion=3 policy, where an >>> attacker package RBF a honest package out of the mempool to subsequently >>> double-spend its own high-fee child with a transaction unrelated to the >>> channel. As the remaining commitment transaction is pre-signed with a >>> minimal relay fee, it can be evicted out of the mempool. >>> >>> A functional test exercising a simple replacement cycling of a lightning >>> channel commitment transaction on top of the nversion=3 code branch is >>> available: >>> https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 >>> >>> ## Discovery >>> >>> In 2018, the issue of static fees for pre-signed lightning transactions >>> is made more widely known, the carve-out exemption in mempool rules to >>> mitigate in-mempool package limits pinning and the anchor output pattern >>> are proposed. >>> >>> In 2019, bitcoin core 0.19 is released with carve-out support. Continued >>> discussion of the anchor output pattern as a dynamic fee-bumping method. >>> >>> In 2020, draft of anchor output submitted to the bolts. Initial finding >>> of economic pinning against lightning commitment and second-stage HTLC >>> transactions. Subsequent discussions of a preimage-overlay network or >>> package-relay as mitigations. Public call made to inquiry more on potential >>> other transaction-relay jamming attacks affecting lightning. >>> >>> In 2021, initial work in bitcoin core 22.0 of package acceptance. >>> Continued discussion of the pinning attacks and shortcomings of current >>> mempool rules during community-wide online workshops. Later the year, in >>> light of all issues for bitcoin second-layers, a proposal is made about >>> killing the mempool. >>> >>> In 2022, bip proposed for package relay and new proposed v3 policy >>> design proposed for a review and implementation. Mempoolfullrbf is >>> supported in bitcoin core 24.0 and conceptual questions about alignment of >>> mempool rules w.r.t miners incentives are investigated. >>> >>> Along this year 2022, eltoo lightning channels design are discussed, >>> implemented and reviewed. In this context and after discussions on mempool >>> anti-DoS rules, I discovered this new replacement cycling attack was >>> affecting deployed lightning channels and immediately reported the finding >>> to some bitcoin core developers and lightning maintainers. >>> >>> ## Timeline >>> >>> - 2022-12-16: Report of the finding to Suhas Daftuar, Anthony Towns, >>> Greg Sanders and Gloria Zhao >>> - 2022-12-16: Report to LN maintainers: Rusty Russell, Bastien >>> Teinturier, Matt Corallo and Olaoluwa Osuntunkun >>> - 2022-12-23: Sharing to Eugene Siegel (LND) >>> - 2022-12-24: Sharing to James O'Beirne and Antoine Poinsot >>> (non-lightning potential affected projects) >>> - 2022-01-14: Sharing to Gleb Naumenko (miners incentives and >>> cross-layers issuers) and initial proposal of an early public disclosure >>> - 2022-01-19: Collection of analysis if other second-layers and >>> multi-party applications affected. LN mitigations development starts. >>> - 2023-05-04: Sharing to Wilmer Paulino (LDK) >>> - 2023-06-20: LN mitigations implemented and progressively released. >>> Week of the 16 october proposed for full disclosure. >>> - 2023-08-10: CVEs assigned by MITRE >>> - 2023-10-05: Pre-disclosure of LN CVEs numbers and replacement cycling >>> attack existence to security@bitcoincore•org. >>> - 2023-10-16: Full disclosure of CVE-2023-40231 / CVE-2023-40232 / >>> CVE-2023-40233 / CVE-2023-40234 and replacement cycling attacks >>> >>> ## Conclusion >>> >>> Despite the line of mitigations adopted and deployed by current major >>> lightning implementations, I believe replacement cycling attacks are still >>> practical for advanced attackers. Beyond this new attack might come as a >>> way to partially or completely defeat some of the pinning mitigations which >>> have been working for years as a community. >>> >>> As of today, it is uncertain to me if lightning is not affected by a >>> more severe long-term package malleability critical security issue under >>> current consensus rules, and if any other time-sensitive multi-party >>> protocol, designed or deployed isn't de facto affected too (loss of funds >>> or denial of service). >>> >>> Assuming analysis on package malleability is correct, it is unclear to >>> me if it can be corrected by changes in replacement / eviction rules or >>> mempool chain of transactions processing strategy. Inviting my technical >>> peers and the bitcoin community to look more on this issue, including to >>> dissent. I'll be the first one pleased if I'm fundamentally wrong on those >>> issues, or if any element has not been weighted with the adequate technical >>> accuracy it deserves. >>> >>> Do not trust, verify. All mistakes and opinions are my own. >>> >>> Antoine >>> >>> "meet with Triumph and Disaster. And treat those two impostors just the >>> same" - K. >>> >>
Hi Peter,
Quoted text (1 line)
> So, why can't we make the HTLC-preimage path expire? Traditionally, we've
tried
Quoted text (1 line)
> to ensure that transactions - once valid - remain valid forever. We do
this
Quoted text (1 line)
> because we don't want transactions to become impossible to mine in the
event of
Quoted text (1 line)
> a large reorganization.
I don't know if reverse time-lock where a lightning spending path becomes invalid after a block height or epoch point solves the more advanced replacement cycling attacks, where a malicious commitment transaction itself replaces out a honest commitment transaction, and the child-pay-for-parent of this malicious transaction is itself replaced out by the attacker, leading to the automatic trimming of the malicious commitment transaction.
I think this attack scenario is exposed here: https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2
If this scenario is correct, there is not only a need for a solution that expires the htlc-preimage spending path, but also channel commitment ones. I think you have a difficulty as both channel commitments can be legitimately valid under lightning protocol semantics, where both counterparties cannot trust the other one to broadcast a commitment state in a timely fashion, to subsequently claim time-sensitive HTLCs.
Of course, one might come with the observation that the time-sensitive HTLCs might be safeguarded under the new reverse time-lock semantic, though I think you're just switching the security risk from one counterparty to the other one. Now, the forwarding node might receive the preimage off-chain from the payee, and then block any attempt of the payee to broadcast its commitment transaction to claim the inbound HTLC, before the reverse time-lock kicks out.
I believe another line of solution could to remove any counterparty malleability in the setting of a package total fees and have fee-bumping reserves pre-committed, though intuitively this sounds to come with the downside of a high-level of total reserve for each channel.
Best, Antoine
Le sam. 21 oct. 2023 à 01:09, Peter Todd <pete@petertodd•org> a écrit :
Quoted text (166 lines)
> On Mon, Oct 16, 2023 at 05:57:36PM +0100, Antoine Riard via bitcoin-dev > wrote: > > Here enter a replacement cycling attack. A malicious channel counterparty > > can broadcast its HTLC-preimage transaction with a higher absolute fee > and > > higher feerate than the honest HTLC-timeout of the victim lightning node > > and triggers a replacement. Both for legacy and anchor output channels, a > > HTLC-preimage on a counterparty commitment transaction is malleable, i.e > > additional inputs or outputs can be added. The HTLC-preimage spends an > > unconfirmed and unrelated to the channel parent transaction M and > conflicts > > its child. > > The basic problem here is after the HTLC-timeout path becomes spendable, > the > HTLC-preimage path remains spendable. That's bad, because in this case we > want > spending the HTLC-preimage - if possible - to have an urgency attached to > it to > ensure that it happens before the previous HTLC-timeout is mined. > > So, why can't we make the HTLC-preimage path expire? Traditionally, we've > tried > to ensure that transactions - once valid - remain valid forever. We do this > because we don't want transactions to become impossible to mine in the > event of > a large reorganization. > > A notable example of this design philosophy is seen in Bitcoin's rules > around > coinbase outputs: they only become spendable after 100 more blocks have > been > found; a 100 block reorg is quite unlikely. > > Enter the OP_Expire and the Coinbase Bit soft-fork upgrade. > > > # Coinbase Bit > > By redefining a bit of the nVersion field, eg the most significant bit, we > can > apply coinbase-like txout handling to arbitrary transactions. Such a > transaction's outputs would be treated similarly to a coinbase > transaction, and > would be spendable only after 100 more blocks had been mined. Due to this > requirement, these transactions will pose no greater risk to reorg safety > than > the existing hazard of coinbase transactions themselves becoming invalid. > > Note how such a transaction is non-standard right now, ensuring > compatibility > with existing nodes in a soft-fork upgrade. > > > # OP_Expire > > Redefining an existing OP_Nop opcode, OP_Expire would terminate script > evaluation with an error if: > > 1) the Coinbase Bit was not set; or > 2) the stack is empty; or > 3) the top item on the stack was >= the block height of the containing > block > > This is conceptually an AntiCheckLockTimeVerify: where CLTV _allows_ a > txout to > become spendable in a particular way in the future, Expire _prevents_ a > txout > from being spent in a particular way. > > Since OP_Expire requires the Coinbase Bit to be set, the reorg security of > OP_Expire-using transactions is no worse than transactions spending miner > coinbases. > > > # How HTLC's Would Use OP_Expire > > Whenever revealing the preimage on-chain is necessary to the secure > functioning > of the HTLC-using protocol, we simply add an appropriate OP_Expire to the > pre-image branch of the script along the lines of: > > If > <expiry height> Expire Drop > Hash <digest> EqualVerify > <pubkey> CheckSig > ElseIf > # HTLC Expiration conditions > ... > EndIf > > Now the party receiving the pre-image has a deadline. Either they get a > transaction spending the preimage mined, notifying the other party via the > blockchain itself, or they fail to get the preimage mined in time, > reverting > control to the other party who can spend the HTLC output at their leisure, > without strict time constraints. > > Since the HTLC-expired branch does *not* execute OP_Expire, the transaction > spending the HTLC-expired branch does *not* need to set the Coinbase Bit. > Thus > it can be spent in a perfectly normal transaction, without restrictions. > > > # Delta Encoding Expiration > > Rather than having a specific Coinbase Bit, it may also be feasible to > encode > the expiration height as a delta against a block-height nLockTime. In this > variant, OP_Expire would work similarly to OP_CheckLockTimeVerify, by > checking > that the absolute expiration height was <= the requested expiration, > allowing > multiple HTLC preimage outputs to be spent in one transaction. > > If the top 16-bits were used, the maximum period a transaction could be > valid > would be: > > 2^16 blocks / 144 blocks/day = 455 days > > In this variant, a non-zero expiration delta would enable expiration > behavior, > as well as the coinbase-like output spending restriction. The remaining > 16-bits > of nVersion would remain available for other meanings. > > Similar to how CLTV and CSV verified nLockTime and nSequence respectively, > verifying an expiration height encoded in the nVersion has the advantage of > making an expiration height easy to detect without validating scripts. > > While Lightning's HTLC-success transactions currently use nLockTime=0, > AFAIK > there is no reason why they could not set nLockTime to be valid in the next > block, allowing delta encoding to be used. > > > ## Reusing Time-Based nLockTime > > Reusing time-based nLockTime's prior to some pre-2009 genesis point for > expiration is another possibility (similar to how Lightning makes use of > time-based nLockTime for signalling). However I believe this is not as > desirable as delta encoding or a coinbase bit, as it would prevent > transactions > from using block nLockTime and expiration at the same time. It would also > still > require a coinbase bit or nVersion increase to ensure expiration-using > transactions are non-standard. > > > # Mempool Behavior > > Obviously, mempool logic will need to handle transactions that can expire > differently than non-expiring transactions. One notable consideration is > that > nodes should require higher minimum relay fees for transactions close to > their > expiration height to ensure we don't waste bandwidth on transactions that > have > no potential to be mined. Considering the primary use-case, it is probably > acceptable to always require a fee rate high enough to be mined in the next > block. > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org >
On Thu, Nov 02, 2023 at 05:24:36AM +0000, Antoine Riard wrote:
Quoted text (17 lines)
> Hi Peter, > > > So, why can't we make the HTLC-preimage path expire? Traditionally, we've > tried > > to ensure that transactions - once valid - remain valid forever. We do > this > > because we don't want transactions to become impossible to mine in the > event of > > a large reorganization. > > I don't know if reverse time-lock where a lightning spending path becomes > invalid after a block height or epoch point solves the more advanced > replacement cycling attacks, where a malicious commitment transaction > itself replaces out a honest commitment transaction, and the > child-pay-for-parent of this malicious transaction is itself replaced out > by the attacker, leading to the automatic trimming of the malicious > commitment transaction.
To be clear, are you talking about anchor channels or non-anchor channels? Because in anchor channels, all outputs other than the anchor outputs provided for fee bumping can't be spent until the commitment transaction is mined, which means RBF/CPFP isn't relevant.
On Thu, Nov 2, 2023 at 6:27 AM Peter Todd via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:
Quoted text (24 lines)
> > On Thu, Nov 02, 2023 at 05:24:36AM +0000, Antoine Riard wrote: > > Hi Peter, > > > > > So, why can't we make the HTLC-preimage path expire? Traditionally, we've > > tried > > > to ensure that transactions - once valid - remain valid forever. We do > > this > > > because we don't want transactions to become impossible to mine in the > > event of > > > a large reorganization. > > > > I don't know if reverse time-lock where a lightning spending path becomes > > invalid after a block height or epoch point solves the more advanced > > replacement cycling attacks, where a malicious commitment transaction > > itself replaces out a honest commitment transaction, and the > > child-pay-for-parent of this malicious transaction is itself replaced out > > by the attacker, leading to the automatic trimming of the malicious > > commitment transaction. > > To be clear, are you talking about anchor channels or non-anchor channels? > Because in anchor channels, all outputs other than the anchor outputs provided > for fee bumping can't be spent until the commitment transaction is mined, which > means RBF/CPFP isn't relevant.
IIUC, Antoine is talking about a cycling attack of the commitment transaction itself, not the HTLC transactions. It seems possible for future (ephemeral) anchor channels in a world with package relay.
The idea with package relay is that commitment transaction fees will be zero and that fees will always be paid via CPFP on the anchor output.
Consider this scenario: Mallory1 -> Alice -> Mallory2. Mallory2 claims an HTLC from Alice off chain via the preimage. Alice attempts to claim the corresponding HTLC from Mallory1, but Mallory1 refuses to cooperate. So Alice publishes her commitment transaction along with a CPFP on the anchor output. Mallory1 publishes her competing commitment transaction with a higher CPFP fee on the anchor output, thereby replacing Alice's package in the mempool. Mallory1 then replacement-cycles the anchor output child transaction, causing her commitment transaction to lose its CPFP and the package feerate to go to zero, which is below the minimum relay fee. Thus, Mallory1's commitment transaction is also evicted from the mempool. Mallory1 repeats this process every time Alice broadcasts her commitment, until the HTLC timeout expires. At that point the preimage path becomes unspendable, and Mallory1 can claim the HTLC via timeout at her leisure.
Quoted text (8 lines)
> > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation•org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
Quoted text (2 lines)
> To be clear, are you talking about anchor channels or non-anchor channels? > Because in anchor channels, all outputs other than the anchor outputs
provided
Quoted text (1 line)
> for fee bumping can't be spent until the commitment transaction is mined,
which
Quoted text (1 line)
> means RBF/CPFP isn't relevant.
I think the distinction is irrelevant here as pre-anchor channel if I have one spendable HTLC output spend and I gain knowledge of my counterparty commitment transaction from networks mempools, the spend is malleable and can be used as a CPFP. If you assume anchor channels, you have 2 anchor outputs as long both parties have balance outputs or pending HTLCs.
Though pre-anchor, legacy channels the counterparty commitment transaction will have to be attached with a fee under min mempool fee for the replacement cycling to happen, and let network congestion happen.
I think the more interesting case is a future world with package relay deployed at the p2p level and anchor output on the lightning-side. Here the most advanced replacement as illustrated in the test can happen (where commitment has an anchor output - see L125).
Best, Antoine
Le jeu. 2 nov. 2023 à 06:26, Peter Todd <pete@petertodd•org> a écrit :
Quoted text (30 lines)
> On Thu, Nov 02, 2023 at 05:24:36AM +0000, Antoine Riard wrote: > > Hi Peter, > > > > > So, why can't we make the HTLC-preimage path expire? Traditionally, > we've > > tried > > > to ensure that transactions - once valid - remain valid forever. We do > > this > > > because we don't want transactions to become impossible to mine in the > > event of > > > a large reorganization. > > > > I don't know if reverse time-lock where a lightning spending path becomes > > invalid after a block height or epoch point solves the more advanced > > replacement cycling attacks, where a malicious commitment transaction > > itself replaces out a honest commitment transaction, and the > > child-pay-for-parent of this malicious transaction is itself replaced out > > by the attacker, leading to the automatic trimming of the malicious > > commitment transaction. > > To be clear, are you talking about anchor channels or non-anchor channels? > Because in anchor channels, all outputs other than the anchor outputs > provided > for fee bumping can't be spent until the commitment transaction is mined, > which > means RBF/CPFP isn't relevant. > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org >
Quoted text (3 lines)
> The idea with package relay is that commitment transaction fees will > be zero and that fees will always be paid via CPFP on the anchor > output.
Yes, even if multiple commitment transactions are pre-signed with a RBF range of more than zero, an attacker can always select the lowest fees pre-signed states and adjust in consequence the CPFP paid, and then evict out the bumping CPFP.
Le jeu. 2 nov. 2023 à 17:07, Matt Morehouse <mattmorehouse@gmail•com> a écrit :
Quoted text (68 lines)
> On Thu, Nov 2, 2023 at 6:27 AM Peter Todd via bitcoin-dev > <bitcoin-dev@lists•linuxfoundation•org> wrote: > > > > On Thu, Nov 02, 2023 at 05:24:36AM +0000, Antoine Riard wrote: > > > Hi Peter, > > > > > > > So, why can't we make the HTLC-preimage path expire? Traditionally, > we've > > > tried > > > > to ensure that transactions - once valid - remain valid forever. We > do > > > this > > > > because we don't want transactions to become impossible to mine in > the > > > event of > > > > a large reorganization. > > > > > > I don't know if reverse time-lock where a lightning spending path > becomes > > > invalid after a block height or epoch point solves the more advanced > > > replacement cycling attacks, where a malicious commitment transaction > > > itself replaces out a honest commitment transaction, and the > > > child-pay-for-parent of this malicious transaction is itself replaced > out > > > by the attacker, leading to the automatic trimming of the malicious > > > commitment transaction. > > > > To be clear, are you talking about anchor channels or non-anchor > channels? > > Because in anchor channels, all outputs other than the anchor outputs > provided > > for fee bumping can't be spent until the commitment transaction is > mined, which > > means RBF/CPFP isn't relevant. > > IIUC, Antoine is talking about a cycling attack of the commitment > transaction itself, not the HTLC transactions. It seems possible for > future (ephemeral) anchor channels in a world with package relay. > > The idea with package relay is that commitment transaction fees will > be zero and that fees will always be paid via CPFP on the anchor > output. > > Consider this scenario: Mallory1 -> Alice -> Mallory2. > Mallory2 claims an HTLC from Alice off chain via the preimage. Alice > attempts to claim the corresponding HTLC from Mallory1, but Mallory1 > refuses to cooperate. So Alice publishes her commitment transaction > along with a CPFP on the anchor output. Mallory1 publishes her > competing commitment transaction with a higher CPFP fee on the anchor > output, thereby replacing Alice's package in the mempool. Mallory1 > then replacement-cycles the anchor output child transaction, causing > her commitment transaction to lose its CPFP and the package feerate to > go to zero, which is below the minimum relay fee. Thus, Mallory1's > commitment transaction is also evicted from the mempool. Mallory1 > repeats this process every time Alice broadcasts her commitment, until > the HTLC timeout expires. At that point the preimage path becomes > unspendable, and Mallory1 can claim the HTLC via timeout at her > leisure. > > > > > > > -- > > https://petertodd.org 'peter'[:-1]@petertodd.org > > _______________________________________________ > > bitcoin-dev mailing list > > bitcoin-dev@lists•linuxfoundation•org > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >
On Fri, Nov 03, 2023 at 05:25:24AM +0000, Antoine Riard wrote:
Quoted text (16 lines)
> > To be clear, are you talking about anchor channels or non-anchor channels? > > Because in anchor channels, all outputs other than the anchor outputs > provided > > for fee bumping can't be spent until the commitment transaction is mined, > which > > means RBF/CPFP isn't relevant. > > I think the distinction is irrelevant here as pre-anchor channel if I have > one spendable HTLC output spend and I gain knowledge of my counterparty > commitment transaction from networks mempools, the spend is malleable and > can be used as a CPFP. If you assume anchor channels, you have 2 anchor > outputs as long both parties have balance outputs or pending HTLCs. > > Though pre-anchor, legacy channels the counterparty commitment transaction > will have to be attached with a fee under min mempool fee for the > replacement cycling to happen, and let network congestion happen.
I think you are misunderstanding a key point to my OP_Expire proposal: because the ability to spend the preimage branch of the HTLC goes away when the refund branch becomes available, replacing cycling or any similar technique becomes entirely irrelevant.
The situation where Carol prevents Bob from learning about the preimage in time simply can't happen: either Carol collects the HTLC with knowledge of the preimage, by spending it in a transaction mined prior to the expiration time and ensuring that Bob learns the preimage from the blockchain itself. Or the HTLC expires and Bob can use the refund branch at his leisure.
Quoted text (4 lines)
> I think the more interesting case is a future world with package relay > deployed at the p2p level and anchor output on the lightning-side. Here the > most advanced replacement as illustrated in the test can happen (where > commitment has an anchor output - see L125).
Again, with OP_Expire, whether or not package relay or anything similar exists is irrelevant. Replacement cycling is totally useless because there is a defined time window in which the HTLC can be spent with the preimage, after which only the refund branch can be used.
Indeed, with OP_Expire Lightning nodes will no longer need to monitor mempools for preimages at all. If the preimage is used, it is guaranteed to end up in the chain, and the Lightning node is guaranteed to see it provided they have access to up-to-date blockchain data.
Quoted text (1 line)
> I think you are misunderstanding a key point to my OP_Expire proposal:
because
Quoted text (1 line)
> the ability to spend the preimage branch of the HTLC goes away when the
refund
Quoted text (1 line)
> branch becomes available, replacing cycling or any similar technique
becomes
Quoted text (2 lines)
> entirely irrelevant. > The situation where Carol prevents Bob from learning about the preimage
in time
Quoted text (2 lines)
> simply can't happen: either Carol collects the HTLC with knowledge of the > preimage, by spending it in a transaction mined prior to the expiration
time
Quoted text (1 line)
> and ensuring that Bob learns the preimage from the blockchain itself. Or
the
Quoted text (1 line)
> HTLC expires and Bob can use the refund branch at his leisure.
I think I understand the semantic of the OP_Expire proposal overall correctly, however I'm not sure it prevents replacing cycling or any similar adversarial technique, as the forwarding node might be the attacker in some scenario.
Consider the following: you have Alice, Bob, Caroll sharing lightning channels.
Alice forwards a HTLC of 1 BTC to Caroll by the intermediary of Bob.
On the Bob-Caroll link, the HTLC expires at block 100.
According to OP_Expire semantics, Caroll shouldn't be able to claim the htlc-preimage spends on the Bob-Caroll link, after block 100.
However, this situation offers the ability to Bob the routing node to steal HTLC payment between Alice and Caroll.
Once the HTLC is committed on the Bob-Caroll link, Caroll releases the preimage off-chain to Bob with an `update_fulfill_htlc` message, though Bob does _not_ send back his signature for the updated channel state.
Some blocks before 100, Caroll goes on-chain to claim the inbound HTLC output with the preimage. Her commitment transaction propagation in network mempools is systematically "replaced cycled out" by Bob.
At block 100, Caroll cannot claim the payment sent to her by Alice.
Bob claims the htlc-refund path on the Bob-Caroll link and claims the htlc-preimage path on the Alice-Bob link, as such making a gain of 1 BTC.
If Caroll is a lightning mobile client, it is easy for Bob to claim publicly that the lack of success in signature exchange to update channels state is a liveliness mistake of her own.
Assuming this advanced scenario is correct, I'm not sure the OP_Expire proposal is substantially fixing all the adversarial replacement cycling situations.
Best, Antoine
Le sam. 4 nov. 2023 à 07:26, Peter Todd <pete@petertodd•org> a écrit :
Quoted text (62 lines)
> On Fri, Nov 03, 2023 at 05:25:24AM +0000, Antoine Riard wrote: > > > To be clear, are you talking about anchor channels or non-anchor > channels? > > > Because in anchor channels, all outputs other than the anchor outputs > > provided > > > for fee bumping can't be spent until the commitment transaction is > mined, > > which > > > means RBF/CPFP isn't relevant. > > > > I think the distinction is irrelevant here as pre-anchor channel if I > have > > one spendable HTLC output spend and I gain knowledge of my counterparty > > commitment transaction from networks mempools, the spend is malleable and > > can be used as a CPFP. If you assume anchor channels, you have 2 anchor > > outputs as long both parties have balance outputs or pending HTLCs. > > > > Though pre-anchor, legacy channels the counterparty commitment > transaction > > will have to be attached with a fee under min mempool fee for the > > replacement cycling to happen, and let network congestion happen. > > I think you are misunderstanding a key point to my OP_Expire proposal: > because > the ability to spend the preimage branch of the HTLC goes away when the > refund > branch becomes available, replacing cycling or any similar technique > becomes > entirely irrelevant. > > The situation where Carol prevents Bob from learning about the preimage in > time > simply can't happen: either Carol collects the HTLC with knowledge of the > preimage, by spending it in a transaction mined prior to the expiration > time > and ensuring that Bob learns the preimage from the blockchain itself. Or > the > HTLC expires and Bob can use the refund branch at his leisure. > > > I think the more interesting case is a future world with package relay > > deployed at the p2p level and anchor output on the lightning-side. Here > the > > most advanced replacement as illustrated in the test can happen (where > > commitment has an anchor output - see L125). > > Again, with OP_Expire, whether or not package relay or anything similar > exists > is irrelevant. Replacement cycling is totally useless because there is a > defined time window in which the HTLC can be spent with the preimage, after > which only the refund branch can be used. > > Indeed, with OP_Expire Lightning nodes will no longer need to monitor > mempools > for preimages at all. If the preimage is used, it is guaranteed to end up > in > the chain, and the Lightning node is guaranteed to see it provided they > have > access to up-to-date blockchain data. > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org >
Good morning Antoine,
Quoted text (3 lines)
> Once the HTLC is committed on the Bob-Caroll link, Caroll releases the preimage off-chain to Bob with an `update_fulfill_htlc` message, though Bob does _not_ send back his signature for the updated channel state. > > Some blocks before 100, Caroll goes on-chain to claim the inbound HTLC output with the preimage. Her commitment transaction propagation in network mempools is systematically "replaced cycled out" by Bob.
I think this is impossible?
In this scenario, there is an HTLC offered by Bob to Carol.
Prior to block 100, only Carol can actually create an HTLC-success transaction. Bob cannot propagate an HTLC-timeout transaction because the HTLC timelock says "wait till block 100".
Neither can Bob replace-recycle out the commitment transaction itself, because the commitment transaction is a single-input transaction, whose sole input requires a signature from Bob and a signature from Carol --- obviously Carol will not cooperate on an attack on herself.
So as long as Carol is able to get the HTLC-success transaction confirmed before block 100, Bob cannot attack. Of course, once block 100 is reached, `OP_EXPIRE` will then mean that Carol cannot claim the fund anymore.
Regards, ZmnSCPxj
Hi Zeeman,
Quoted text (1 line)
> Neither can Bob replace-recycle out the commitment transaction itself,
because the commitment transaction is a single-input transaction, whose sole input requires a signature from
Quoted text (1 line)
> Bob and a signature from Carol --- obviously Carol will not cooperate on
an attack on herself.
The replacement cycling happens on the commitment transaction spend itself, not the second stage, which is effectively locked until block 100.
If the commitment transaction is pre-signed with 0 sat / vb and all the feerate / absolute fee is provided by a CPFP on one of the anchor outputs, Bob can replace the CPFP itself. After replacement of its child, the commitment transaction has a package feerate of 0 sat / vb and it will be trimmed out of the mempool.
This is actually the scenario tested here on the nversion = 3 new mempool policy branch (non-deployed yet): https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2
As of today commitment transactions might not propagate if dynamic mempool min fee is above pre-signed commitment transaction, which is itself unsafe. I think this behavior can currently be opportunistically exploited by attackers.
In a post-package relay world, I think this is possible. And that replacement cycling attacks are breaking future dynamic fee-bumping of pre-signed transactions concerns me a lot.
Best, Antoine
Le mar. 7 nov. 2023 à 11:12, ZmnSCPxj <ZmnSCPxj@protonmail•com> a écrit :
Quoted text (33 lines)
> Good morning Antoine, > > > > Once the HTLC is committed on the Bob-Caroll link, Caroll releases the > preimage off-chain to Bob with an `update_fulfill_htlc` message, though Bob > does _not_ send back his signature for the updated channel state. > > > > Some blocks before 100, Caroll goes on-chain to claim the inbound HTLC > output with the preimage. Her commitment transaction propagation in network > mempools is systematically "replaced cycled out" by Bob. > > I think this is impossible? > > In this scenario, there is an HTLC offered by Bob to Carol. > > Prior to block 100, only Carol can actually create an HTLC-success > transaction. > Bob cannot propagate an HTLC-timeout transaction because the HTLC timelock > says "wait till block 100". > > Neither can Bob replace-recycle out the commitment transaction itself, > because the commitment transaction is a single-input transaction, whose > sole input requires a signature from Bob and a signature from Carol --- > obviously Carol will not cooperate on an attack on herself. > > So as long as Carol is able to get the HTLC-success transaction confirmed > before block 100, Bob cannot attack. > Of course, once block 100 is reached, `OP_EXPIRE` will then mean that > Carol cannot claim the fund anymore. > > Regards, > ZmnSCPxj >
On Mon, Nov 06, 2023 at 06:45:21PM +0000, Antoine Riard wrote:
Quoted text (21 lines)
> > I think you are misunderstanding a key point to my OP_Expire proposal: > because > > the ability to spend the preimage branch of the HTLC goes away when the > refund > > branch becomes available, replacing cycling or any similar technique > becomes > > entirely irrelevant. > > > The situation where Carol prevents Bob from learning about the preimage > in time > > simply can't happen: either Carol collects the HTLC with knowledge of the > > preimage, by spending it in a transaction mined prior to the expiration > time > > and ensuring that Bob learns the preimage from the blockchain itself. Or > the > > HTLC expires and Bob can use the refund branch at his leisure. > > I think I understand the semantic of the OP_Expire proposal overall > correctly, however I'm not sure it prevents replacing cycling or any > similar adversarial technique, as the forwarding node might be the attacker > in some scenario.
<snip>
Quoted text (3 lines)
> Assuming this advanced scenario is correct, I'm not sure the OP_Expire > proposal is substantially fixing all the adversarial replacement cycling > situations.
What you are describing here is a completely different problem than what OP_Expire aims to solve.
The problem that OP_Expire aims to solve is the fact that Carol could prevent Bob from learning about the preimage in time, while still getting a chance to use the preimage herself. OP_Expire thoroughly solves that problem by ensuring that the preimage is either broadcast in the blockchain in a timely fashion, or becomes useless.
The problem you are describing, as Zmm points out below, doesn't actually exist in Bitcoin right now. But it could exist if we adopted v3 package relay, which as I've pointed out elsewhere, is inferior to using RBF.
On Tue, Nov 07, 2023 at 03:44:21PM +0000, Antoine Riard wrote:
Quoted text (25 lines)
> Hi Zeeman, > > > Neither can Bob replace-recycle out the commitment transaction itself, > because the commitment transaction is a single-input transaction, whose > sole input requires a signature from > > Bob and a signature from Carol --- obviously Carol will not cooperate on > an attack on herself. > > The replacement cycling happens on the commitment transaction spend itself, > not the second stage, which is effectively locked until block 100. > > If the commitment transaction is pre-signed with 0 sat / vb and all the > feerate / absolute fee is provided by a CPFP on one of the anchor outputs, > Bob can replace the CPFP itself. After replacement of its child, the > commitment transaction has a package feerate of 0 sat / vb and it will be > trimmed out of the mempool. > > This is actually the scenario tested here on the nversion = 3 new mempool > policy branch (non-deployed yet): > https://github.com/ariard/bitcoin/commits/2023-10-test-mempool-2 > > As of today commitment transactions might not propagate if dynamic mempool > min fee is above pre-signed commitment transaction, which is itself unsafe. > I think this behavior can currently be opportunistically exploited by > attackers.
Yes, obviously it can be. For starters, you can easily end up in a situation where the commitment transaction simply can't get mined, an obvious problem.
Quoted text (3 lines)
> In a post-package relay world, I think this is possible. And that > replacement cycling attacks are breaking future dynamic fee-bumping of > pre-signed transactions concerns me a lot.
Well the answer here is pretty clear: v3 package relay with anchors is broken.
My suggestion of pre-signing RBF replacements, without anchor outputs, and with all outputs rendered unspendable with 1 CSV, is clearly superior: there are zero degrees of freedom to the attacker, other than the possibility of increasing the fee paid or broadcasting a revoked commitment. The latter of course risks the other party punishing the fraud.
This does have the practical problem that a set of refund transactions will also need to be signed for each fee variant. But, eg, signing 10x of each isn't so burdensome. And in the future that problem could be avoided with SIGHASH_NOINPUT, or replacing the pre-signed refund transaction mechanism with a covenant.
Using RBF rather than CPFP with package relay also has the advantage of being more efficient, as no blockspace needs to be consumed by the anchor outputs or transactions spending them. Of course, there are special circumstances where BIP125 rules can cause CPFP to be cheaper. But we can easily fix that, eg by reducing the replacement relay fee, or by delta-encoding transaction updates.
As SIGHASH_NOINPUT is desirable for LN-Symmetry, a softfork containing both it and OP_Expire could make sense.
On Wed, Nov 08, 2023 at 12:51:31AM +0000, Peter Todd via bitcoin-dev wrote:
Quoted text (5 lines)
> > In a post-package relay world, I think this is possible. And that > > replacement cycling attacks are breaking future dynamic fee-bumping of > > pre-signed transactions concerns me a lot. > > Well the answer here is pretty clear: v3 package relay with anchors is broken.
BTW a subtlety of this that may not be obvious is that in v3 package relay, with zero value outputs, the outputs must be spent in the same package. Thus _unlike_ existing anchor-using transactions, there would be only one anchor output on the commitment transaction.
In existing anchor output transactions, this type of attack wouldn't work as when broadcasting the transaction, Alice would be spending her anchor output, which Bob can't double spend. But that doesn't work in v3, which intends to limit UTXO growth by requiring that anchors be spent in the same package. Thus unlike existing anchor outputs, an anchor would be truly a OP_1 output without a signature, and thus belong to either Alice nor Bob uniquely.
Your two latest mails.
Quoted text (1 line)
> The problem that OP_Expire aims to solve is the fact that Carol could
prevent
Quoted text (1 line)
> Bob from learning about the preimage in time, while still getting a
chance to
Quoted text (1 line)
> use the preimage herself. OP_Expire thoroughly solves that problem by
ensuring
Quoted text (1 line)
> that the preimage is either broadcast in the blockchain in a timely
fashion, or
Quoted text (1 line)
> becomes useless.
I respectfully disagree - There is a more general underlying issue for outdated states in multi-party off-chain constructions, where any "revoked" or "updated" consensus-valid state can be used to jam the latest off-chain agreed-on, through techniques like replacement cycling or pinning.
Quoted text (1 line)
> My suggestion of pre-signing RBF replacements, without anchor outputs,
and with
Quoted text (1 line)
> all outputs rendered unspendable with 1 CSV, is clearly superior: there
are
Quoted text (2 lines)
> zero degrees of freedom to the attacker, other than the possibility of > increasing the fee paid or broadcasting a revoked commitment. The latter
of
Quoted text (1 line)
> course risks the other party punishing the fraud.
Assuming the max RBF replacement is pre-signed at 200 sats / vb, with commitment transaction of ~268 vbytes and at least one second-stage HTLC transaction of ~175 vbytes including witness size, a channel counterparty must keep at worst a fee-bumping reserve of 35 268 sats, whatever payment value. As of today, any payment under $13 has to become trimmed HTLCs. Trimmed HTLCs are coming with their own wormhole of issues, notably making them a target to be stolen by low-hashrate capabilities attackers [0].
[0] https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-May/002714.html
Quoted text (1 line)
> This does have the practical problem that a set of refund transactions
will
Quoted text (1 line)
> also need to be signed for each fee variant. But, eg, signing 10x of each
isn't
Quoted text (2 lines)
> so burdensome. And in the future that problem could be avoided with > SIGHASH_NOINPUT, or replacing the pre-signed refund transaction mechanism
with
Quoted text (1 line)
> a covenant.
I think if you wish to be safe against fees griefing games between counterparties, both counterparties have to maintain their own fee-bumping reserves, which make channel usage less capital efficient, rather than being drawn from a common reserve.
Quoted text (1 line)
> Using RBF rather than CPFP with package relay also has the advantage of
being
Quoted text (1 line)
> more efficient, as no blockspace needs to be consumed by the anchor
outputs or
Quoted text (1 line)
> transactions spending them. Of course, there are special circumstances
where
Quoted text (1 line)
> BIP125 rules can cause CPFP to be cheaper. But we can easily fix that, eg
by
Quoted text (1 line)
> reducing the replacement relay fee, or by delta-encoding transaction
updates.
It is left as an exercise to the reader how to break the RBF approach for LN channels as proposed.
Quoted text (1 line)
> As SIGHASH_NOINPUT is desirable for LN-Symmetry, a softfork containing
both it
Quoted text (1 line)
> and OP_Expire could make sense.
I think there is one obvious issue of pre-signing RBF replacements combined with LN-symmetry, namely every state has to pre-commit to fee values attached and such states might spend each other in chain. So now you would need `max-rbf-replacement` * `max-theoretical-number-of-states` of fee-bumping reserves unless you can pipe fee value with some covenant magic, I think.
Quoted text (1 line)
> In existing anchor output transactions, this type of attack wouldn't work
as
Quoted text (1 line)
> when broadcasting the transaction, Alice would be spending her anchor
output,
Quoted text (1 line)
> which Bob can't double spend.
However Bob can double-spend Alice's commitment transaction with his own commitment transaction and a CPFP, as long as it's a better ancestor feerate and absolute fee package, then double-spend his own CPFP. Which is exactly what my test is doing so I don't think your statement of saying this type of advanced replacement cycling attack wouldn't work isn't correct.
Best, Antoine
Le mer. 8 nov. 2023 à 02:06, Peter Todd <pete@petertodd•org> a écrit :
Quoted text (28 lines)
> On Wed, Nov 08, 2023 at 12:51:31AM +0000, Peter Todd via bitcoin-dev wrote: > > > In a post-package relay world, I think this is possible. And that > > > replacement cycling attacks are breaking future dynamic fee-bumping of > > > pre-signed transactions concerns me a lot. > > > > Well the answer here is pretty clear: v3 package relay with anchors is > broken. > > BTW a subtlety of this that may not be obvious is that in v3 package relay, > with zero value outputs, the outputs must be spent in the same package. > Thus > _unlike_ existing anchor-using transactions, there would be only one anchor > output on the commitment transaction. > > In existing anchor output transactions, this type of attack wouldn't work > as > when broadcasting the transaction, Alice would be spending her anchor > output, > which Bob can't double spend. But that doesn't work in v3, which intends to > limit UTXO growth by requiring that anchors be spent in the same package. > Thus > unlike existing anchor outputs, an anchor would be truly a OP_1 output > without > a signature, and thus belong to either Alice nor Bob uniquely. > > -- > https://petertodd.org 'peter'[:-1]@petertodd.org >
On Mon, Nov 13, 2023 at 02:18:16AM +0000, Antoine Riard wrote:
Quoted text (16 lines)
> Your two latest mails. > > > The problem that OP_Expire aims to solve is the fact that Carol could > prevent > > Bob from learning about the preimage in time, while still getting a > chance to > > use the preimage herself. OP_Expire thoroughly solves that problem by > ensuring > > that the preimage is either broadcast in the blockchain in a timely > fashion, or > > becomes useless. > > I respectfully disagree - There is a more general underlying issue for > outdated states in multi-party off-chain constructions, where any "revoked" > or "updated" consensus-valid state can be used to jam the latest off-chain > agreed-on, through techniques like replacement cycling or pinning.
No, that's not a general underlying issue. You've found two separate issues.
Furthermore, revoked states are clearly different than HTLCs: they're fraudulent, and thus in punishment-using protocols they are always associated with high risks of loss if they do in fact get detected or mined. There's probably tweaks we can do to improve this security. But the general principle there is certainly true.
Quoted text (14 lines)
> > My suggestion of pre-signing RBF replacements, without anchor outputs, > and with > > all outputs rendered unspendable with 1 CSV, is clearly superior: there > are > > zero degrees of freedom to the attacker, other than the possibility of > > increasing the fee paid or broadcasting a revoked commitment. The latter > of > > course risks the other party punishing the fraud. > > Assuming the max RBF replacement is pre-signed at 200 sats / vb, with > commitment transaction of ~268 vbytes and at least one second-stage HTLC > transaction of ~175 vbytes including witness size, a channel counterparty > must keep at worst a fee-bumping reserve of 35 268 sats, whatever payment > value.
For a lightning channel to be economical at all in a general routing environment, the highest likely fee has to be small enough for it to represent a small percentage of the total value tied up in the Lightning channel. Tying up a small percentage of the overall capacity for future fee usage is not a significant expense.
Quoted text (6 lines)
> As of today, any payment under $13 has to become trimmed HTLCs. > Trimmed HTLCs are coming with their own wormhole of issues, notably making > them a target to be stolen by low-hashrate capabilities attackers [0]. > > [0] > https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-May/002714.html
That attack doesn't make sense. HTLCs go to fees at a certain feerate. In a normal environment where there is a constant supply of fee paying transactions, the profit for the miner is not the total HTLC value, but the increase in feerate compared to the transactions they had to give up to mine the commitment transaction.
Second, it's obvious that the total trimmed HTLCs should be limited to what would be a reasonable transaction fee. A situation where you have 80% of the channel value going to fees due to a bunch of small HTLCs is obviously ridiculous, and to the extent that existing implementations have this issue, should be fixed.
For RBF fee bumping, obviously you can take the increased channel fees from the party choosing to broadcast the commitment transaction.
Quoted text (13 lines)
> > This does have the practical problem that a set of refund transactions > will > > also need to be signed for each fee variant. But, eg, signing 10x of each > isn't > > so burdensome. And in the future that problem could be avoided with > > SIGHASH_NOINPUT, or replacing the pre-signed refund transaction mechanism > with > > a covenant. > > I think if you wish to be safe against fees griefing games between > counterparties, both counterparties have to maintain their own fee-bumping > reserves, which make channel usage less capital efficient, rather than > being drawn from a common reserve.
Yes, obviously. But as I said above, it just doesn't make sense for channels to be in a situation where closing them costs a significant % of the channel value in fees, so we're not changing the status quo much.
Quoted text (13 lines)
> > Using RBF rather than CPFP with package relay also has the advantage of > being > > more efficient, as no blockspace needs to be consumed by the anchor > outputs or > > transactions spending them. Of course, there are special circumstances > where > > BIP125 rules can cause CPFP to be cheaper. But we can easily fix that, eg > by > > reducing the replacement relay fee, or by delta-encoding transaction > updates. > > It is left as an exercise to the reader how to break the RBF approach for > LN channels as proposed.
Do you have a concrete attack?
Quoted text (10 lines)
> > As SIGHASH_NOINPUT is desirable for LN-Symmetry, a softfork containing > both it > > and OP_Expire could make sense. > > I think there is one obvious issue of pre-signing RBF replacements combined > with LN-symmetry, namely every state has to pre-commit to fee values > attached and such states might spend each other in chain. So now you would > need `max-rbf-replacement` * `max-theoretical-number-of-states` of > fee-bumping reserves unless you can pipe fee value with some covenant > magic, I think.
No, you are missing the point. RBF replacements can use SIGHASH_NOINPUT to sign HTLC refund transactions, removing the need for a set of different HTLC refund transactions for each different feerate of the commitment transaction.
I'm making no comment on how to do RBF replacements with LN-Symmetry, which I consider to be a broken idea in non-trusted situations anyway. Removing justice from Lightning is always going to be hopelessly insecure when you can't at least somewhat trust your counterparty. If your usage of LN-Symmetry is sufficiently secure, you probably don't have to worry about them playing fee games with you either.
Quoted text (1 line)
> No, that's not a general underlying issue. You've found two separate
issues.
Quoted text (2 lines)
> Furthermore, revoked states are clearly different than HTLCs: they're > fraudulent, and thus in punishment-using protocols they are always
associated
Quoted text (2 lines)
> with high risks of loss if they do in fact get detected or mined. There's > probably tweaks we can do to improve this security. But the general
principle
Quoted text (1 line)
> there is certainly true.
I see your point in saying we have two separate issues, one the on-chain inclusion of "expired" off-chain output spend (e.g the HTLC-preimage), the other one the usage of revoked or updated or asymmetric states to jam the confirmation of a valid off-chain state. I do think the second one would bring a solution to the first one, as you will be always sure that your counterparty cannot "replay" an "expired" off-chain output at its advantage.
Even for solving the first issue, I'm not sure op_expire is enough, you need to expire the worthiness of an off-chain revealed witness secret too (here the preimage). E.g using short-lived proofs, i.e after a specified period of time, the proof is no longer convincing.
I still think op_exire is interesting on its own, beyond solving any security issue. E.g for Discreet Log Contract, one can build a time-bounded sequential claim of the same output fund among a set of counterparties.
Quoted text (2 lines)
> For a lightning channel to be economical at all in a general routing > environment, the highest likely fee has to be small enough for it to
represent
Quoted text (1 line)
> a small percentage of the total value tied up in the Lightning channel.
Tying
Quoted text (1 line)
> up a small percentage of the overall capacity for future fee usage is not
a
Quoted text (1 line)
> significant expense.
Sure, I still think this introduces the corollary for lightning nodes that any payment under the highest likely fee now has a probabilistic security, where the lightning node should make guesses on the worst-level of mempools feerate that can happen between the timelock duration of said payment.
Quoted text (1 line)
> That attack doesn't make sense. HTLCs go to fees at a certain feerate. In
a
Quoted text (1 line)
> normal environment where there is a constant supply of fee paying
transactions,
Quoted text (2 lines)
> the profit for the miner is not the total HTLC value, but the increase in > feerate compared to the transactions they had to give up to mine the
commitment
Quoted text (1 line)
> transaction.
The attack makes sense in an environment where the level of HTLC trimmed as fees on the commitment transaction renders the feerates of this transaction more interesting than the marginal known transaction in a miner block template. If there is an environment where you're always guaranteed there is a constant supply of fee paying transactions paying a better feerate than the highest-fee rate that trimmed HTLCs can be a commitment transaction, of course the attack wouldn't be plausible.
In a world where you have a dynamic blockspace demand market and asymmetries of information, Lightning routing nodes will be always exposed to such attacks.
Quoted text (1 line)
> Second, it's obvious that the total trimmed HTLCs should be limited to
what
Quoted text (1 line)
> would be a reasonable transaction fee. A situation where you have 80% of
the
Quoted text (2 lines)
> channel value going to fees due to a bunch of small HTLCs is obviously > ridiculous, and to the extent that existing implementations have this
issue,
Quoted text (1 line)
> should be fixed.
This is the hard thing, the existence of asymmetries of information in what is a reasonable transaction fee and what is the level of mempools fee rates at time of broadcast. One could imagine a consensus change where trimmed HTLCs not worthy at the last X blocks of feerates are automatically aggregated or trimmed (think median-time-past though for median fee rates over X blocks).
Quoted text (1 line)
> Yes, obviously. But as I said above, it just doesn't make sense for
channels to
Quoted text (1 line)
> be in a situation where closing them costs a significant % of the channel
value
Quoted text (1 line)
> in fees, so we're not changing the status quo much.
Evaluation of the significant % of the channel value burned in fees in the worst-case at time of off-chain state commitment is the hard thing.
Quoted text (1 line)
> Do you have a concrete attack?
I don't have a concrete attack with sufficient testing to say with a satisfying level of certainty that I have a concrete attack.
Quoted text (1 line)
> No, you are missing the point. RBF replacements can use SIGHASH_NOINPUT
to sign
Quoted text (1 line)
> HTLC refund transactions, removing the need for a set of different HTLC
refund
Quoted text (1 line)
> transactions for each different feerate of the commitment transaction.
See above, I think this solution with RBF replacement is robust on the assumption you cannot use the commitment transaction to jam the HTLC-preimage until your HTLC-refund transaction is valid (under nLocktime). Though my point here was only about the LN-symmetry states, not second-stage transactions on top of them.
Quoted text (1 line)
> I'm making no comment on how to do RBF replacements with LN-Symmetry,
which I
Quoted text (2 lines)
> consider to be a broken idea in non-trusted situations anyway > Removing justice from Lightning is always going to be hopelessly insecure
when you can't at
Quoted text (2 lines)
> least somewhat trust your counterparty. If your usage of LN-Symmetry is > sufficiently secure, you probably don't have to worry about them playing
fee
Quoted text (1 line)
> games with you either.
I agree here that LN-symmetry would be better if you can conserve the punishment ability.
See https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-July/002064.html
On the lack of worries about playing fees games with you, see concern above if your counterparty is a miner, even one with low-hashrate capability. Here low-hashrate capability can be understood as "do you have a non-null chance to mine a block as long as a HTLC-output is committed on an off-chain state only known among off-chain counterparties e.g".
Quoted text (1 line)
> IIRC correctly, in your scenario, you're dealing with Carol -> Bob ->
Alice.
Quoted text (3 lines)
> Mallory can only replace an HTLC-timeout transaction if she's directly > connected with the peer she's targeting via a direct channel. She cannot > unilaterally replace any transaction in the mempool solely with knowledge
of
Quoted text (1 line)
> the preimage. All HTLC transactions are two party contracts, with the
public
Quoted text (1 line)
> keys of both participants hard coded into the contract. A third party
cannot
Quoted text (4 lines)
> unilaterally sweep an HTLC given only a preimage. > I think it's worth taking a minute to step back here so we can establish > some fundamental context. Once the timelock of an HTLC expires, and the > receiver has direct knowledge of the preimage, a bidding war begins. If
the
Quoted text (2 lines)
> receiver is able to confirm their success transaction in time, then they > gain the funds, and the sender is able to pipeline the preimage backwards
in
Quoted text (1 line)
> the route, making all parties whole. If the sender is the one that wins
out,
Quoted text (3 lines)
> then it's as if nothing happened, sans the fees paid at the last mile, all > other hops are able to safely cancel their HTLC back on the chain. > As mentioned elsewhere in the thread, most implementations today will
watch
Quoted text (1 line)
> the mempool for preimages, so they can resolve the incoming HTLC as
quickly
Quoted text (1 line)
> as possible off chain. The attack described critically relies on the
ability
Quoted text (2 lines)
> of an attacker to pull off very precise transaction replacement globally > across "the mempool". If any of the honest parties see the preimage, in
the
Quoted text (3 lines)
> mempool, then the attack ends as they can settle back off chain, and if > their timeout is able to confirm first, then the defender has actually > gained funds.
Correct mostly so far, a part of the globally across "the mempool".
Quoted text (1 line)
> In addition, such an attack needs to be executed perfectly, for hours if
not
Quoted text (2 lines)
> days. Within the LN, all nodes set a security parameter, the CLTV delta, > that dictates how much time they have before the outgoing and incoming
HTLCs
Quoted text (1 line)
> expire. Increasing this value makes the attack more difficult, as they
must
Quoted text (3 lines)
> maintain the superposition of: fees low enough to not get mined, but high > enough to replace the defender's transaction, but not _too_ high, as then > it'll be mined and everything is over. With each iteration, the attacker
is
Quoted text (2 lines)
> also forced to increase the amount of fees they pay, increasing the likely > hood that the transaction will be mined. If mined, the attack is moot.
The attacker just has to overbid the parent of the HTLC-preimage with a higher fee replacement parent. This replacement parent can be confirmed at any time, even if the attacker realizes a higher economic gain if this replacement transaction is delayed as much as possible. Or the absolute fee consumed to batch attack multiple targets. So I think your assumption on the attack difficulty is not correct.
Quoted text (1 line)
> As mentioned in the OP, one thing the attacker can do is try and locate
the
Quoted text (1 line)
> defender's node to launch the replacement attack directly in their
mempool.
Quoted text (1 line)
> However, by replacing directly in the mempool of the defender, the
defender
Quoted text (4 lines)
> will learn of the preimage! They can use that to just settle everything > back, thereby completing the payment, and stopping the attack short. Even > ignoring direct access to the defender's mempool, the attacker needs to > somehow iteratively execute the replacement across a real network, with
all
Quoted text (2 lines)
> the network jitter, propagation delay, and geographic heterogeneity that > entails. If they're off by milliseconds, then either a confirmation
occurs,
Quoted text (2 lines)
> or the preimage is revealed in the mempool. A canned local regtest env is > one thing, the distributed system that is the Internet is another.
The attacker can blind the defender mempool by locating the associated full-node with its lightning node, then broadcast a conflicting parent of the HTLC-preimage, partitioning the mempool from the rest of the network. The HTLC-preimage will never enter the defender's mempool, as some spent inputs are missing.
All the considerations on network jitter, propagation delay and geographic heterogeneity are irrelevant as network mempools are in practice a distributed system of logically equivalent state machines to favor propagation of high-fees paying transactions, on top of raw TCP/IP connections. Propagation works quite deterministically on average if you have ever used a bitcoin wallet to pay with zero-conf and see the transaction showing up as unconfirmed on the payment processor. Policy divergences across implementations and versions does not matter here.
The p2p stack of Bitcoin Core has some propagation delay in the order of seconds for outbound / inbound connections transaction-relay connections. I think they're at the advantage of the attacker, as they can be bypassed to quickly evict the honest HTLC-timeout bypassing them with mass connections.
The p2p transaction-relay is a public network of nodes running common open software, where their inbound slots can be occupied at low-cost and their network address might be publicly announced. All at the advantage of the attacker.
Quoted text (6 lines)
> Returning back to the bidding war: for anchor channels, the second level > HTLCs allow the defender to attach arbitrary inputs for fee bumping > purposes. Using this, they can iteratively increase their fee (via RBF) as > the expiry deadline approaches. With each step, they further increase the > cost for the attacker. On top of that, this attack cannot be launched > indiscriminately across the network. Instead, it requires a per-node set
up by
Quoted text (3 lines)
> the attacker, as they need to consume UTXOs to create a chain of > transactions they'll use to launch the replacement attack. These also need > to be maintained in a similar state of non-confirming superposition.
As said elsewhere, I'll maintain this is a practical and critical attack on lightning, I don't doubt you have the level of expertise in this attack by yourself and re-think if it's a fragile one or bring new insights to the conversation.
There are no requirements of an "extremely precise timing and execution", the most adversarial event that an attacker might hit is a block being mined in the order of few seconds between a HTLC-timeout hit the block template and a block is mined. On average, you have a 10 min block interval so probabilities are in favor of the attacker.
It is correct that miners can broadcast any of the preimage replacement transactions, however this is not at all a deployed mechanism today by the mining ecosystem. DoS-robustness of such a scheme is very unclear.
One more mitigation that can be done by lightning nodes today is duplicating the mempool-monitoring mitigation to their watchtower backends, assuming those ones are already running on top of a full-node. For each new watchtower deployed, this is one more entity to mempool-partition by the attacker and all of them must be well-partitioned along the attack. This can still be neutralized by sophisticated attackers, though it's notably raising the bar to mount a successful replacement attack.
Recommending to LN softwares to adapt this last mitigation, especially if your implementation is used by high-value routing nodes or LSPs.
Best, Antoine
Le sam. 21 oct. 2023 à 01:19, Olaoluwa Osuntokun <laolu32@gmail•com> a écrit :
Quoted text (97 lines)
> > Let's say you have Alice, Bob and Caroll all "honest" routing hops > > targeted by an attacker. They all have 3 independent 10 000 sats HTLC > > in-flight on their outbound channels. > > > It is replaced by Mallory at T+2 with a HTLC-preimage X of 200 000 sats > (+ > > rbf penalty 1 sat / vb rule 4). Alice's HTLC-timeout is out of network > > mempools. > > > Bob broadcasts her HTLC-timeout of 200 000 sats at T+3. It is replaced by > > Mallory at T+4 with her HLTC-preimage Y of 200 000 sats (+ rbf penalty 1 > > sat / vb rule 4 * 2). Bob's HTLC-timeout is out of network mempools. > > IIRC correctly, in your scenario, you're dealing with Carol -> Bob -> > Alice. > Mallory can only replace an HTLC-timeout transaction if she's directly > connected with the peer she's targeting via a direct channel. She cannot > unilaterally replace any transaction in the mempool solely with knowledge > of > the preimage. All HTLC transactions are two party contracts, with the > public > keys of both participants hard coded into the contract. A third party > cannot > unilaterally sweep an HTLC given only a preimage. > > I think it's worth taking a minute to step back here so we can establish > some fundamental context. Once the timelock of an HTLC expires, and the > receiver has direct knowledge of the preimage, a bidding war begins. If > the > receiver is able to confirm their success transaction in time, then they > gain the funds, and the sender is able to pipeline the preimage backwards > in > the route, making all parties whole. If the sender is the one that wins > out, > then it's as if nothing happened, sans the fees paid at the last mile, all > other hops are able to safely cancel their HTLC back on the chain. > > As mentioned elsewhere in the thread, most implementations today will watch > the mempool for preimages, so they can resolve the incoming HTLC as quickly > as possible off chain. The attack described critically relies on the > ability > of an attacker to pull off very precise transaction replacement globally > across "the mempool". If any of the honest parties see the preimage, in the > mempool, then the attack ends as they can settle back off chain, and if > their timeout is able to confirm first, then the defender has actually > gained funds. > > In addition, such an attack needs to be executed perfectly, for hours if > not > days. Within the LN, all nodes set a security parameter, the CLTV delta, > that dictates how much time they have before the outgoing and incoming > HTLCs > expire. Increasing this value makes the attack more difficult, as they must > maintain the superposition of: fees low enough to not get mined, but high > enough to replace the defender's transaction, but not _too_ high, as then > it'll be mined and everything is over. With each iteration, the attacker is > also forced to increase the amount of fees they pay, increasing the likely > hood that the transaction will be mined. If mined, the attack is moot. > > As mentioned in the OP, one thing the attacker can do is try and locate the > defender's node to launch the replacement attack directly in their mempool. > However, by replacing directly in the mempool of the defender, the defender > will learn of the preimage! They can use that to just settle everything > back, thereby completing the payment, and stopping the attack short. Even > ignoring direct access to the defender's mempool, the attacker needs to > somehow iteratively execute the replacement across a real network, with all > the network jitter, propagation delay, and geographic heterogeneity that > entails. If they're off by milliseconds, then either a confirmation occurs, > or the preimage is revealed in the mempool. A canned local regtest env is > one thing, the distributed system that is the Internet is another. > > Returning back to the bidding war: for anchor channels, the second level > HTLCs allow the defender to attach arbitrary inputs for fee bumping > purposes. Using this, they can iteratively increase their fee (via RBF) as > the expiry deadline approaches. With each step, they further increase the > cost for the attacker. On top of that, this attack cannot be launched > indiscriminately across the network. Instead, it requires per-node set up > by > the attacker, as they need to consume UTXOs to create a chain of > transactions they'll use to launch the replacement attack. These also need > to be maintained in a similar state of non-confirming superposition. > > That's all to say that imo, this is a rather fragile attack, which > requires: > per-node setup, extremely precise timing and execution, non-confirming > superposition of all transactions, and instant propagation across the > entire > network of the replacement transaction (somehow also being obscured from > the > PoV of the defender). The attack can be launched directly with a miner, or > "into" their mempool, but that weakens the attack as if the miner > broadcasts > any of the preimage replacement transactions, then the defender can once > again use that to extract the preimage and settle the incoming HTLC. > > -- Laolu >