BIP70 message delivery reliability

I spoke briefly with Peter (sipa). He recommend I forward this post to the mailing list for further discussion.

My apologies if this has been discussed before, but I was curious about some things re BIP70 message delivery. In particular, I don't clearly see the value of the PaymentACK message. Allow me to explain...

The current BIP70 workflow designates PaymentACK as the final message in a payment exchange. However, it doesn't appear that any mention is made of what happens if that delivery fails. I assume that re-delivery is left as a detail to the implementation, actually.

For sake of argument, let's assume that PaymentACK is never delivered either because of a network outage or a malicious merchant or incompatible software between wallets or a bug. I ask myself: what would be necessary for sufficient proof of payment, say, to an arbiter? I presume the receipt R=(PaymentRequest,[transactions]) would suffice. Am I correct there?

But if the PaymentRequest and broadcasted transactions are enough to prove payment, what's the point of the Payment message? The merchant never has to verify the Payment message, possibly maliciously ignoring it. In the well-behaved case, I presume the point is to help the merchant associate some arbitrary data with the purchase as well as provide a refunding address for the customer. If that's the case, couldn't this protocol be slightly improved like so:

Required steps: 1. Customer clicks "pay now" 2. Merchant sends PaymentRequest/PaymentDetails, which should be signed 3. Customer builds a set of transactions and sends a new PaymentApprovalRequest message which includes a refund address and the unsigned transactions and their associated fully-signed transaction hash, the whole message signed with the private key of the refund address. 4. Merchant responds with PaymentApproved message, signing the PaymentApprovalRequest message with the same key from step 2.

Optional steps: 5. The customer can send a Payment message, which is only a set of signed transactions. 6. The merchant can respond with a PaymentACK message.

In Step 4, the merchant is acknowledging that if the transactions provided PaymentApprovalRequest are broadcast, then payment is complete and no other steps are required. Steps 5 and 6 aren't required but are considered considerate:)

After step 4, all the merchant needs is to do is watch for the transactions that were listed in PaymentApprovalRequest. The (PaymentApproved,[signed transactions]) pair is the customer's proof of payment and this proof of payment includes a refund address that the merchant has agreed to prior to payment, instead of after. Step 3 & 4 also allow the merchant to verify transactions, providing an extra layer of redundancy. The merchant will also be able to ack on fees, time lock (time sensitive purchases?), sequence numbers, etc.

In Step 3, it's critical the customer sign the message with the private key of the refund address, so that the merchant can be confident the refund address is correct.

In each step along the way until step 5, if a message delivery fails nobody is harmed because the purchase is incomplete.

Thoughts?

Chuck

Hi Chuck,

Both Bitcoin Core and bitcoinj are about to ship with the protocol as-is, so any changes from this point on have to be backwards compatible.

On Thu, Jan 30, 2014 at 6:47 AM, Chuck <chuck+bitcoindev@borboggle•com>wrote:

> I presume the receipt R=(PaymentRequest,[transactions]) would suffice.
>

That's all you need to prove payment, yes.

> In the well-behaved case, I presume the point is to help the
> merchant associate some arbitrary data with the purchase as well as
> provide a refunding address for the customer.

That's right (+memo). And to provide an additional hook for future features, like recurring billing, ECDH key agreements etc.

> In Step 3, it's critical the customer sign the message with the private
> key of the refund address, so that the merchant can be confident the
> refund address is correct.
>

Refund addresses as specced currently are optional. For instance bitcoinj currently doesn't use them and won't until HD wallets support is done.

Let's get some practical experience with what we've got so far. We can evolve PaymentRequest/Payment/PaymentACK in the right direction with backwards compatible upgrades, I am hoping.

Hi Mike. Thanks for replying.

On 1/30/2014 5:49 PM, Mike Hearn wrote:

> Both Bitcoin Core and bitcoinj are about to ship with the protocol
> as-is, so any changes from this point on have to be backwards compatible.

Then I think it's critically important to talk about failure situations now, rather than trying to patch on solutions later; it's going to be very hard to wedge/"hack" in fixes for potential problems when they could be addressed now with minor changes.

> Let's get some practical experience with what we've got so far. We can
> evolve PaymentRequest/Payment/PaymentACK in the right direction with
> backwards compatible upgrades, I am hoping.

I think what I'm trying to discuss or find out here is whether the current PP description is defunct or incomplete in some manner, thus making any experience we gain from the current implementation moot.

It seems the largest hole in the implementation is delivery of the Payment message, but I'm happy to accept that maybe I'm just missing something. A malicious merchant could claim he never received the Payment message, or a faulty network connection could cause the message to never be delivered. In arbitration the merchant could argue the transactions seen on the network were insufficient.

To me, this could be a problem.

Cheers,

Chuck

On Thu, Jan 30, 2014 at 12:15 PM, Chuck <chuck+bitcoindev@borboggle•com>wrote:

> In arbitration the merchant could argue the transactions seen on the
> network were insufficient.
>

The arbitrator would presumably have some rules about what is or isn't an acceptable form of payment.

HTTP has response codes for submission of the Payment message. We could add signing to PaymentACK and other things in future, if that turns out to be insufficient in practice.

On 1/30/2014 6:31 PM, Mike Hearn wrote:

> The arbitrator would presumably have some rules about what is or isn't
> an acceptable form of payment.

Do you think this puts unnecessary trust into a third party? If the merchant instead signed and agreed to the unsigned transactions before they were broadcast (as in my OP), these arbitration concerns disappear.

> HTTP has response codes for submission of the Payment message. We
> could add signing to PaymentACK and other things in future, if that
> turns out to be insufficient in practice.

HTTP isn't the only message delivery mechanism. Merchants can also lie: reply with 200 OK and an empty body. Or, reply with 404 not found and broadcast transactions anyway.

Cheers,

Chuck

On Thu, Jan 30, 2014 at 12:15 PM, Chuck <chuck+bitcoindev@borboggle•com> wrote:

> Hi Mike.  Thanks for replying.
>
> On 1/30/2014 5:49 PM, Mike Hearn wrote:
>> Both Bitcoin Core and bitcoinj are about to ship with the protocol
>> as-is, so any changes from this point on have to be backwards compatible.
> Then I think it's critically important to talk about failure situations
> now, rather than trying to patch on solutions later; it's going to be
> very hard to wedge/"hack" in fixes for potential problems when they
> could be addressed now with minor changes.
>> Let's get some practical experience with what we've got so far. We can
>> evolve PaymentRequest/Payment/PaymentACK in the right direction with
>> backwards compatible upgrades, I am hoping.
> I think what I'm trying to discuss or find out here is whether the
> current PP description is defunct or incomplete in some manner, thus
> making any experience we gain from the current implementation moot.
>
> It seems the largest hole in the implementation is delivery of the
> Payment message, but I'm happy to accept that maybe I'm just missing
> something.  A malicious merchant could claim he never received the
> Payment message, or a faulty network connection could cause the message
> to never be delivered. In arbitration the merchant could argue the
> transactions seen on the network were insufficient.

You don't even have to assume malicious intent. A payment message could just fail to arrive because the server is unreachable. As the specification currently doesn't even suggest retrying, there is no way the merchant can rely at all on the memo and refund address being delivered, which makes them in my opinion useless.

Your proposal makes the whole protocol more atomic, which may be a step too far at this point (though I like the idea very much), but I really think the specification should do everything possible to prevent transactions confirming without the payment message ever being delivered (i.e., store them in the sender's client, retry when necessary, exponential backoff, ...).

With the way it works in bitcoinj, the tx is only committed to the wallet if the server accepts the Payment message and ACKs it. So the tx would not be retried if there's a failure submitting or some kind of internal server error, and the UI would show that the payment failed. That seems straightforward and how I'd expect things to work as a user.

On Thu, Jan 30, 2014 at 12:46 PM, Pieter Wuille <pieter.wuille@gmail•com>wrote:

> On Thu, Jan 30, 2014 at 12:15 PM, Chuck <chuck+bitcoindev@borboggle•com>
> wrote:
> > Hi Mike.  Thanks for replying.
> >
> > On 1/30/2014 5:49 PM, Mike Hearn wrote:
> >> Both Bitcoin Core and bitcoinj are about to ship with the protocol
> >> as-is, so any changes from this point on have to be backwards
> compatible.
> > Then I think it's critically important to talk about failure situations
> > now, rather than trying to patch on solutions later; it's going to be
> > very hard to wedge/"hack" in fixes for potential problems when they
> > could be addressed now with minor changes.
> >> Let's get some practical experience with what we've got so far. We can
> >> evolve PaymentRequest/Payment/PaymentACK in the right direction with
> >> backwards compatible upgrades, I am hoping.
> > I think what I'm trying to discuss or find out here is whether the
> > current PP description is defunct or incomplete in some manner, thus
> > making any experience we gain from the current implementation moot.
> >
> > It seems the largest hole in the implementation is delivery of the
> > Payment message, but I'm happy to accept that maybe I'm just missing
> > something.  A malicious merchant could claim he never received the
> > Payment message, or a faulty network connection could cause the message
> > to never be delivered. In arbitration the merchant could argue the
> > transactions seen on the network were insufficient.
>
> You don't even have to assume malicious intent. A payment message
> could just fail to arrive because the server is unreachable. As the
> specification currently doesn't even suggest retrying, there is no way
> the merchant can rely at all on the memo and refund address being
> delivered, which makes them in my opinion useless.
>
> Your proposal makes the whole protocol more atomic, which may be a
> step too far at this point (though I like the idea very much), but I
> really think the specification should do everything possible to
> prevent transactions confirming without the payment message ever being
> delivered (i.e., store them in the sender's client, retry when
> necessary, exponential backoff, ...).
>
> --
> Pieter
>

On Thu, Jan 30, 2014 at 12:59 PM, Mike Hearn <mike@plan99•net> wrote:

> With the way it works in bitcoinj, the tx is only committed to the wallet if
> the server accepts the Payment message and ACKs it. So the tx would not be
> retried if there's a failure submitting or some kind of internal server
> error, and the UI would show that the payment failed. That seems
> straightforward and how I'd expect things to work as a user.

That's one right way to do it imho, but not what is suggested or required by the specification, and not what bitcoin core master currently implements.

On 1/30/2014 7:02 PM, Pieter Wuille wrote:

> On Thu, Jan 30, 2014 at 12:59 PM, Mike Hearn <mike@plan99•net> wrote:
>> With the way it works in bitcoinj, the tx is only committed to the wallet if
>> the server accepts the Payment message and ACKs it. So the tx would not be
>> retried if there's a failure submitting or some kind of internal server
>> error, and the UI would show that the payment failed. That seems
>> straightforward and how I'd expect things to work as a user.
> That's one right way to do it imho, but not what is suggested or
> required by the specification, and not what bitcoin core master
> currently implements.
>

If you sent the Payment message and the server goes silent after receiving it, you retry to delivery. However, the merchant can broadcast the transactions and force them into your wallet anyway. You could, quite likely, pay and never get an ACK.

On Thu, Jan 30, 2014 at 07:03:57PM +0700, Chuck wrote:

> On 1/30/2014 7:02 PM, Pieter Wuille wrote:
> > On Thu, Jan 30, 2014 at 12:59 PM, Mike Hearn <mike@plan99•net> wrote:
> >> With the way it works in bitcoinj, the tx is only committed to the wallet if
> >> the server accepts the Payment message and ACKs it. So the tx would not be
> >> retried if there's a failure submitting or some kind of internal server
> >> error, and the UI would show that the payment failed. That seems
> >> straightforward and how I'd expect things to work as a user.
> > That's one right way to do it imho, but not what is suggested or
> > required by the specification, and not what bitcoin core master
> > currently implements.
> >
> If you sent the Payment message and the server goes silent after
> receiving it, you retry to delivery.  However, the merchant can
> broadcast the transactions and force them into your wallet anyway. You
> could, quite likely, pay and never get an ACK.

I think in that case, you absolultely have to invalidate all the transactions in the Payment message by broadcasting a send-to-self transaction as soon as possible; until that point your wallet balance is indeterminate. Otherwise what will happen if the merchant did in fact receive the Payment message, and then processes it (and broadcasts the transaction) after some delay?

Then what the user will see is: an apparently failed attempt to pay, leaving their wallet balance unchanged. Then, perhaps many hours later, their wallet balance will appear to spontaneously decrement.

roy

>
> If you sent the Payment message and the server goes silent after receiving
> it, you retry to delivery.  However, the merchant can broadcast the
> transactions and force them into your wallet anyway. You could, quite
> likely, pay and never get an ACK.
>

No retries. If there's a timeout the wallet will consider the payment not made, and if the merchant broadcasts anyway, the wallet will see the transactions and sync with them correctly. The ACK is not particularly important in the current design, so that's no big deal.

If we see this situation crop up routinely we can take measures to improve things. I doubt we will.