Payment Protocol Proposal: Invoices/Payments/Receipts

This is the next big "lets all agree to do things the same way" thing I think we should tackle. I'm particularly looking for feedback from other bitcoin client developers, even if it is just a quick "looks reasonable, if everybody else is going to do it then I will (eventually) too..."

Thanks to Pieter Wuille and Mike Hearn for lots of feedback and suggestions and brainstorming.

This document is online at https://gist.github.com/4120476

If you respond to this message, please be considerate of people who subscribe to the digest version of this mailing list and trim your response.

Invoices, Payments and Receipts for Bitcoin Transactions ========================================================

This document proposes protocol buffer-based formats for signed, authenticated "invoices" and "receipts" -- requests for payment, and proof-of-payment.

Separate documents propose an extension to the Bitcoin URI syntax and new MIME types to support them.

Motivation ==========

The idea of a "payment protocol" to improve on Bitcoin addresses has been around for over a year. Users have been asking for some features in this proposal (like the ability to provide a refund address so overpayments or refunds can be returned to customers without the need to ask them for their address) for two or three years, and have started to work around shortcomings in the Bitcoin payment process with creative (but inefficient) uses of transactions.

The key features of this proposal are:

+ Requests for payment (Invoices) are tied to authenticated identities using the only widely-deployed identity authentication system we have right now (X.509 certificates signed by root certificate authorities) + Invoices include a user-friendly description of what the payment is for + Payments include where refunds should be sent + At the end of the payment process, the customer holds a cryptographically signed Receipt that can be used as proof-of-payment if there is any dispute with the merchant.

Specification =============

Invoice/SignedInvoice ---------------------

An Invoice is a request for payment from a merchant to a customer:

::

    message Output {
	optional uint64 amount = 1;
	required bytes script = 2;
    }

amount: Number of satoshis (0.00000001 BTC) to be paid. If not given or zero, then the customer will be asked how much to pay.

script: a "TxOut" script to which the customer should direct payment. This will normally be one of the standard Bitcoin transaction script (e.g. pubkey OP_CHECKSIG).

::

    message Invoice {
        repeated bytes x509chain = 1;
        repeated Output outputs = 2;
        required uint64 time = 3;
        optional uint64 expires = 4;
        optional bool single_use = 5 [default = true];
        optional string memo = 6;
        optional string receiptURI = 7;
        optional bytes merchant_data = 8;
    }

outputs: one or more outputs where Bitcoins are to be sent.

x509chain: one or more DER-encoded X.509 certificates that identifies the merchant. See the "Certificates" section below for details.

time: Unix timestamp (seconds since 1-Jan-1970) when the Invoice was created.

expires: Unix timestamp after which the Invoice should be considered invalid. If not given, the Invoice may be re-used until the earliest certificate expiration date in the X509chain.

single_use: If true, this Invoice should be used for only one payment. If false, it may be added to the user's address book and used repeatedly until it expires (e.g. for donations or a recurring payment).

memo: UTF-8 encoded, plain-text (no formatting) note that should be displayed to the customer, explaining what this Invoice is for.

receiptURI: Secure (https) URI where a Payment message (see below) may be sent to obtain a SignedReceipt as proof-of-payment.

merchant_data : Arbitrary data ignored by the client that may be used by the merchant to identify the Invoice.

::

    message SignedInvoice {
        required Invoice invoice = 1;
        required bytes signature = 2;
    }

A SignedInvoice is an Invoice signed using the private key corresponding to the public key in the first certificate in the x509chain and the HMAC SHA-256 algorithm.

When a Bitcoin client receives a SignedInvoice, it must authorize payment by doing the following:

1. Validate the x509chain certificate chain up to it's list of root certificate authorities 2. Validate that the time on the customer's system is before Invoice.expires 3. Display the "Common Name" (CN) string from the first x509chain certificate and ask the customer if they would like to submit payment

Payment -------

::

    message Payment {
        required Invoice invoice = 1;
        repeated bytes transactions = 2;
        repeated Output refund_to = 3;
        optional string memo = 4;
    }

invoice : the invoice received from the merchant. A merchant must validate the Invoice and may reject the Payment if the Invoice was altered by the customer.

transactions : One or more valid, signed Bitcoin transactions that fully pay the Invoice

refund_to : One or more outputs where the merchant may return funds, if necessary.

memo : UTF-8 encoded, plain-text note from the customer to the merchant.

If the customer authorizes payment, then the Bitcoin client:

1. Creates and signs a transaction with one output sending the Invoice.script 2. If there is no Invoice.receiptURI, then the transaction is broadcast on the Bitcoin p2p network. 3. Else POST a Payment message to Invoice.receiptURI and expect a SignedReceipt in response.

Invoice.receiptURI must be secure against man-in-the-middle attacks that might alter Payment.refund_to.

*Note: an alternative would be a SignedPayment message that ties the signatures in Payment.transactions to a signature for the entire Payment message. Spending multisig inputs that may be controlled by more than one person or spending arbitrary non-standard transactions makes that non-trivial.*

Receipt/SignedReceipt ---------------------

::

    message Receipt {
        required Payment payment = 1;
        required bool accepted = 2;
        optional string memo = 3;
    }

accepted : true if the Payment is accepted and will be broadcast on the Bitcoin p2p network.

memo : UTF-8 encoded note that should be displayed to the customer indicating that the transaction is complete.

::

    message SignedReceipt {
        required Receipt receipt = 1;
        required bytes signature = 3;
    }

A SignedReceipt is a Receipt signed using the private key corresponding to the public key in the first certificate in the Receipt->Payment->Invoice.x509chain and the HMAC SHA-256 algorithm.

Upon receiving a SignedReceipt, a Bitcoin client should validate the signature and, if valid, display the Receipt.memo and store the SignedReceipt as proof-of-payment.

If a SignedReceipt is not received for any reason (timeout, error) and Payment.transactions has not been broadcast by the merchant on the Bitcoin p2p network, then the Bitcoin client should assume that the payment failed, inform the customer that the payment failed, and return coins involved in the transaction to the customer's wallet.

Certificates ============

The Invoice.x509chain (X.509 Certificate Chain) field contains the X.509 public key certificate or certificate chain [RFC5280] corresponding to the key used to digitally sign the Invoice and Receipt. The certificate or certificate chain is represented as an array of DER [ITU.X690.1994] PKIX certificate value. The certificate containing the public key of the entity that digitally signed the Invoice MUST be the first certificate. This MAY be followed by additional certificates, with each subsequent certificate being the one used to certify the previous one. The recipient MUST verify the certificate chain according to [RFC5280] and reject the payment request if any validation failure occurs.

*What should we say about root certificates and certificate management in general? Any requirements, or leave it up to each Bitcoin client to determine which root CA's are trustworthy, as happens with web browsers? Gavin suggests trusting only (say) ten of the Extended Validation authorities: http://en.wikipedia.org/wiki/Extended_Validation_Certificate#Extended_Validation_certificate_identification *

*X.509 is widely criticised for doing too much. However, it is the Public Key Infrastructure (PKI) system we're stuck with. Do web browsers / certificate authorities support the full X.509 spec, or only a subset? Should Bitcoin clients only support some well-defined subset of X.509 ? More research needed here... *

Use Cases =========

Merchant Payment Service ------------------------

A merchant payment service (like Paysius or bit-pay.com) would use Invoices and Receipts as follows:

1. Merchant pays for a certificate from a certificate authority, and then gives the payment service the certificate and their private key. This could be the same certificate and private key as is used for the merchant's web site, but best security practice would be to purchase a separate certificate for authenticating Invoices. Very successful merchant payment services might act as intermediate certificate authorities, issuing certificates for their merchants. 2. Customer goes through the checkout process on either the merchant's or payment service's web site. 3. At the end of the checkout process, a SignedInvoice is generated and sent to the customer's Bitcoin client. 4. Customer's Bitcoin client displays the Invoice, showing that the payment is for the merchant. 5. On customer approval, a Payment is sent to the payment service's paymentURI. The merchant is notified of the payment, and the customer receives a SignedReceipt as proof-of-payment.

SatoshiDice -----------

SatoshiDice (www.satoshidice.com) is an extremely popular game that uses tiny transactions for some customer/service communications. In particular, customers can add an extra output to their transactions to indicate where winnings should be sent. And SatoshiDice creates tiny transactions to let their customers know that a bet was received, but lost.

Assuming Bitcoin clients upgrade to support this proposal, a bet on SatoshiDice would proceed as follows:

1. Customer clicks on a link on SatoshiDice.com and their Bitcoin client receives a SignedInvoice. 2. Customer authorizes payment, and their Bitcoin client creates a Payment message and submits it directly to https://satoshidice.com/something 3. The SatoshiDice web server checks to make sure the transaction is valid, broadcasts it, and determines whether the customer wins or loses. It returns a SignedReceipt with either a "You win" or "You lost" memo. 4. If the customer won, it broadcasts a transaction to pay them using Payment.refund_to 5. Customer's Bitcoin client displays the win/lose memo, and if they won the winnings appear in their wallet when received over the p2p network.

Multiperson Wallet ------------------

This use case starts with a multi-signature Bitcoin address or wallet, with keys held by two different people (Alice and Bob). Payments from that address/wallet must be authorized by both Alice and Bob, and both are running multi-signature-capable Bitcoin clients.

Alice begins the payment process by getting a SignedInvoice from a merchant that needs to be paid. She authorizes payment and her Bitcoin client creates a Payment message with a partially-signed transaction, which is then sent to Bob any way that is convenient (email attachment, smoke signals...).

Bob's Bitcoin client validates the SignedInvoice and asks Bob to authorize the transaction. He says OK, his Bitcoin client completes the transaction by providing his signature, submits the payment to the merchant, and then sends a message to Alice with the SignedReceipt he received from the merchant, completing the payment process.

Design Notes ============

Why X.509 Certificates? -----------------------

This proposal uses X.509 certificates as the identity system for merchants because most of them will have already purchased a certificate to secure their website and will be familiar with the process of proving their identity to a certificate issuing authority.

Implementing a better global PKI is outside the scope of this proposal. If a better PKI is adopted, the only change to this proposal would be to replace the Invoice.x509chain with whatever that better infrastructure uses to identify entities.

Why not JSON? -------------

Invoice, Payment and Receipt messages could all be JSON-encoded. And the Javascript Object Signing and Encryption (JOSE) working group at the IETF has a draft specification for signing JSON data.

But the spec is non-trivial. Signing JSON data is troublesome because JSON can encode the same data in multiple ways (whitespace is insignificant, characters in strings can be represented escaped or un-escaped, etc.), and the standards committee identified at least one security-related issue that will require special JSON parsers for handling JSON-Web-Signed (JWS) data (duplicate keys must be rejected by the parser, which is more strict than the JSON spec requires).

A binary message format has none of those complicating issues. Which encoding format to pick is largely a matter of taste, but Protocol Buffers is a simple, robust, multi-programming-language, well-documented, easy-to-work-with, extensible format.

What about a merchant-pays-fee feature? ---------------------------------------

It is desireable to allow a merchant to pay the cost of any Bitcoin network transaction processing fees, so if a customer is paying for a 1 BTC item they pay exactly 1 BTC.

One way of accomplishing that is to add a 'maxfee' field to the Invoice, and have the Bitcoin client construct a transaction that pays the merchant (amount-maxfee).

Another way of accomplishing that is to change the transaction selection code used by Bitcoin miners, so that dependent transactions are considered as a group. Then a merchant with several unconfirmed zero-fee transaction from customers can create a pay-to-self transaction with a large enough fee to pay for the set of transactions to be confirmed.

A third way of accomplishing that is for the Bitcoin client to sign Payment.transactions[0] using the SIGHASH_ANYONECANPAY flag, and for the merchant to add an additional, small-BTC-value input to the transaction before broadcasting it. That additional input would go directly to miners as a fee. *Note: Gavin is not sure if he loves or hates this idea.*

Checking for revoked certificates ---------------------------------

The Online Certificate Checking Protocol (OCSP) is supposed to be a quick and easy way for applications to check for revoked certificates.

In practice, it doesn't work very well. Certificate Authorities have no financial incentive to support a robust infrastructure that can handle millions of OCSP validation requests quickly.

Ideally, Bitcoin clients would use OCSP to check certificate statuses every time they received or re-used an Invoice. But if that results in long pauses or lots of false-positive rejections (because an OCSP endpoint is offline or overwhelmed, perhaps) then merchants and customers might revert to just using "never fails" Bitcoin addresses.

References ==========

Public-Key Infrastructure (X.509) working group : http://datatracker.ietf.org/wg/pkix/charter/

RFC 2560, X.509 Internet Public Key Infrastructure Online Certificate Status Protocol - OCSP : http://tools.ietf.org/html/rfc2560

Protocol Buffers : https://developers.google.com/protocol-buffers/

See Also ========

Javascript Object Signing and Encryption working group : http://datatracker.ietf.org/wg/jose/

sipa's payment protocol proposal: https://gist.github.com/1237788

ThomasV's "Signed Aliases" proposal : http://ecdsa.org/bitcoin_URIs.html

Obviously this LGTM :)

Minor caveat, IMHO we should support all CAs used by the popular browsers. This ensures no merchant ever finds that their SSL cert they already own is OK for the web but not for Bitcoin. I don't see a need to be stricter here, given all it achieves is signing some data in a way linked with a domain name.

X.509 is pretty baroque indeed, for our use cases it'd not be hard to do better. In particular, the inability to delegate properly rather defeats the benefits of chained certificates. For the payment processor case what you really want to do is take your keys, then issue a new cert that is specific to signing Bitcoin transactions and give that to the payment processor secure in the knowledge that they cannot MITM your secure connections. Unfortunately X.509 wasn't designed for the web and thus certificates you buy are marked such that they are not allowed to sign for other certs (due to lack of real namespace support).

This leads to the idea of redefining the cert chain part of the protocol like this:

  repeated bytes x509_chain = 1;

becomes

  message Certificate {
    enum Type {
      X509 = 1;
    }
    required Type type = 1;
    required bytes data = 2;
  }

  repeated Certificate cert_chain = 1;

Then if later we want to introduce our own minimal certificate formats which include features we want, we can add new enum types to do so.

Note that if an old client encounters an invoice with a cert type it doesn't recognize, it will abort parsing of the message entirely. So the request to download the invoice should probably include a protocol version number of some kind so the server knows when it's safe to use new invoice features.

On Monday, November 26, 2012 11:02:14 PM Mike Hearn wrote:

> Minor caveat, IMHO we should support all CAs used by the popular
> browsers.

I would prefer using the user-accepted certs at the operating system level...

They could be included as well of course, but from a seller perspective the most important thing is consistency. You have to be able to predict what CAs the user has, otherwise your invoice would appear in the UI as unverified and is subject to manipulation by viruses, etc.

So using the OS cert store would effectively restrict merchants to the intersection of what ships in all the operating systems their users use, which could be unnecessarily restrictive. As far as I know, every browser has its own cert store for that reason.

On Monday, November 26, 2012 11:16:03 PM Mike Hearn wrote:

> They could be included as well of course, but from a seller
> perspective the most important thing is consistency. You have to be
> able to predict what CAs the user has, otherwise your invoice would
> appear in the UI as unverified and is subject to manipulation by
> viruses, etc.

That's expected behaviour - except it's mainly be manipulated by *users*, not viruses (which can just as easily manipulate whatever custom cert store we use). If I don't trust Joe's certs, I don't want Bitcoin overriding that no matter who Joe is or what connections he has.

> So using the OS cert store would effectively restrict merchants to the
> intersection of what ships in all the operating systems their users
> use, which could be unnecessarily restrictive. As far as I know, every
> browser has its own cert store for that reason.

Browsers with this bug are not relevant IMO.

> That's expected behaviour - except it's mainly be manipulated by *users*, not
> viruses (which can just as easily manipulate whatever custom cert store we
> use).

The point of using signed invoices as virus protection isn't to change what the user sees on the infected host. The point is the invoice can be relayed to a second device that isn't also compromised which then independently renders a payment confirmation screen (like your mobile phone), and it has an identifier in it that's useful to people, like bitmit.net instead of an address.

If it was just showing you a Bitcoin address, that doesn't mean anything to you so a virus on your PC could wait until you want to make a large payment somewhere and swap out the address in use. You'd never know it was the wrong address and you'd happily confirm on your second device.

For this to work, the seller has to be able to predict what certs you have in all your devices. If it's up to the OS vendors then it's hard to know and in practice all that'll happen is somebody will compile a list of CAs that are "known good" (ie, present in all deployed mobile and desktop OS') and that'll be the minimal cert list. No different to if it was hard-coded in the spec.

> If I don't trust Joe's certs, I don't want Bitcoin overriding that no
> matter who Joe is or what connections he has.

Nothing says your wallet software can't provide cert management UI like browsers do.

In practice I have a feeling that cert management UI is one of the least used parts of a browser. I've used browsers for years and the only time I've ever had to go into those screens was to manage installation/removal of self signed certs used by various organizations. I never manually revoked a root authority. When it was necessary due to breaches (Comodo/DigiNotar) the browser makers revoked them for me.

On Mon, Nov 26, 2012 at 6:19 PM, Luke-Jr <luke@dashjr•org> wrote:

> On Monday, November 26, 2012 11:16:03 PM Mike Hearn wrote:
>> They could be included as well of course, but from a seller
>> perspective the most important thing is consistency. You have to be
>> able to predict what CAs the user has, otherwise your invoice would
>> appear in the UI as unverified and is subject to manipulation by
>> viruses, etc.
>
> That's expected behaviour - except it's mainly be manipulated by *users*, not
> viruses (which can just as easily manipulate whatever custom cert store we
> use). If I don't trust Joe's certs, I don't want Bitcoin overriding that no
> matter who Joe is or what connections he has.
>
>> So using the OS cert store would effectively restrict merchants to the
>> intersection of what ships in all the operating systems their users
>> use, which could be unnecessarily restrictive. As far as I know, every
>> browser has its own cert store for that reason.
>
> Browsers with this bug are not relevant IMO.

This is messy. It's important to people to know that their cert will be accepted by ~everyone because non-acceptance looks like malice. If the cert system is actually to provide value then false positives need to be low enough that people can start calling in law enforcement, computer investigators, etc.. every time a cert failure happens. Otherwise there is little incentive for an attacker to not _try_.

Obviously the state of the world with browsers is not that good... but in our own UAs we can do better and get closer to that.

Would you find it acceptable if something supported a static whitelist plus a OS provided list minus a user configured blacklist and the ability for sophisticated users to disable the whitelist?

This way people could trust that if their cert is signed via one on the whitelist they'll work for ALL normal users.. and the UI can have very strong behavior that protects people (e.g. no 'click here to disable all security because tldr' button)... but advanced users who can deal with sorting out failure can still have complete control including OS based control.

X.509 has some problems we have recent experience with. I'd prefer to leverage something like DANE which looks to help with assertions around certificates and creates an option around the CAs and x.509 roots.

-rick

On Mon, Nov 26, 2012 at 2:37 PM, Gavin Andresen <gavinandresen@gmail•com> wrote:

> This is the next big "lets all agree to do things the same way" thing
> I think we should tackle. I'm particularly looking for feedback from
> other bitcoin client developers, even if it is just a quick "looks
> reasonable, if everybody else is going to do it then I will
> (eventually) too..."
>
> Thanks to Pieter Wuille and Mike Hearn for lots of feedback and
> suggestions and brainstorming.
>
> This document is online at https://gist.github.com/4120476
>
> If you respond to this message, please be considerate of people who
> subscribe to the digest version of this mailing list and trim your
> response.
>
>
> Invoices, Payments and Receipts for Bitcoin Transactions
> ========================================================
>
> This document proposes protocol buffer-based formats for signed,
> authenticated "invoices" and "receipts" -- requests for payment, and
> proof-of-payment.
>
> Separate documents propose an extension to the Bitcoin URI syntax and
> new MIME types to support them.
>
> Motivation
> ==========
>
> The idea of a "payment protocol" to improve on Bitcoin addresses has
> been around for over a year. Users have been asking for some features
> in this proposal (like the ability to provide a refund address so
> overpayments or refunds can be returned to customers without the need
> to ask them for their address) for two or three years, and have
> started to work around shortcomings in the Bitcoin payment process
> with creative (but inefficient) uses of transactions.
>
> The key features of this proposal are:
>
> + Requests for payment (Invoices) are tied to authenticated identities
> using the only widely-deployed identity authentication system we have
> right now (X.509 certificates signed by root certificate authorities)
> + Invoices include a user-friendly description of what the payment is for
> + Payments include where refunds should be sent
> + At the end of the payment process, the customer holds a
> cryptographically signed Receipt that can be used as proof-of-payment
> if there is any dispute with the merchant.
>
>
> Specification
> =============
>
> Invoice/SignedInvoice
> ---------------------
>
> An Invoice is a request for payment from a merchant to a customer:
>
> ::
>
>     message Output {
>         optional uint64 amount = 1;
>         required bytes script = 2;
>     }
>
> amount: Number of satoshis (0.00000001 BTC) to be paid. If not given
> or zero, then the customer will be asked how much to pay.
>
> script: a "TxOut" script to which the customer should direct payment.
> This will normally be one of the standard Bitcoin transaction script
> (e.g. pubkey OP_CHECKSIG).
>
> ::
>
>     message Invoice {
>         repeated bytes x509chain = 1;
>         repeated Output outputs = 2;
>         required uint64 time = 3;
>         optional uint64 expires = 4;
>         optional bool single_use = 5 [default = true];
>         optional string memo = 6;
>         optional string receiptURI = 7;
>         optional bytes merchant_data = 8;
>     }
>
> outputs: one or more outputs where Bitcoins are to be sent.
>
> x509chain: one or more DER-encoded X.509 certificates that identifies
> the merchant. See the "Certificates" section below for details.
>
> time: Unix timestamp (seconds since 1-Jan-1970) when the Invoice was created.
>
> expires: Unix timestamp after which the Invoice should be considered
> invalid. If not given, the Invoice may be re-used until the earliest
> certificate expiration date in the X509chain.
>
> single_use: If true, this Invoice should be used for only one payment.
> If false, it may be added to the user's address book and used
> repeatedly until it expires (e.g. for donations or a recurring
> payment).
>
> memo: UTF-8 encoded, plain-text (no formatting) note that should be
> displayed to the customer, explaining what this Invoice is for.
>
> receiptURI: Secure (https) URI where a Payment message (see below) may
> be sent to obtain a SignedReceipt as proof-of-payment.
>
> merchant_data : Arbitrary data ignored by the client that may be used
> by the merchant to identify the Invoice.
>
> ::
>
>     message SignedInvoice {
>         required Invoice invoice = 1;
>         required bytes signature = 2;
>     }
>
> A SignedInvoice is an Invoice signed using the private key
> corresponding to the public key in the first certificate in the
> x509chain and the HMAC SHA-256 algorithm.
>
> When a Bitcoin client receives a SignedInvoice, it must authorize
> payment by doing the following:
>
> 1. Validate the x509chain certificate chain up to it's list of root
> certificate authorities
> 2. Validate that the time on the customer's system is before Invoice.expires
> 3. Display the "Common Name" (CN) string from the first x509chain
> certificate and ask the customer if they would like to submit payment
>
> Payment
> -------
>
> ::
>
>     message Payment {
>         required Invoice invoice = 1;
>         repeated bytes transactions = 2;
>         repeated Output refund_to = 3;
>         optional string memo = 4;
>     }
>
> invoice : the invoice received from the merchant. A merchant must
> validate the Invoice and may reject the Payment if the Invoice was
> altered by the customer.
>
> transactions : One or more valid, signed Bitcoin transactions that
> fully pay the Invoice
>
> refund_to : One or more outputs where the merchant may return funds,
> if necessary.
>
> memo : UTF-8 encoded, plain-text note from the customer to the merchant.
>
> If the customer authorizes payment, then the Bitcoin client:
>
> 1. Creates and signs a transaction with one output sending the Invoice.script
> 2. If there is no Invoice.receiptURI, then the transaction is
> broadcast on the Bitcoin p2p network.
> 3. Else POST a Payment message to Invoice.receiptURI and expect a
> SignedReceipt in response.
>
> Invoice.receiptURI must be secure against man-in-the-middle attacks
> that might alter Payment.refund_to.
>
> *Note: an alternative would be a SignedPayment message that ties the
> signatures in Payment.transactions to a signature for the entire
> Payment message. Spending multisig inputs that may be controlled by
> more than one person or spending arbitrary non-standard transactions
> makes that non-trivial.*
>
> Receipt/SignedReceipt
> ---------------------
>
> ::
>
>     message Receipt {
>         required Payment payment = 1;
>         required bool accepted = 2;
>         optional string memo = 3;
>     }
>
> accepted : true if the Payment is accepted and will be broadcast on
> the Bitcoin p2p network.
>
> memo : UTF-8 encoded note that should be displayed to the customer
> indicating that the transaction is complete.
>
> ::
>
>     message SignedReceipt {
>         required Receipt receipt = 1;
>         required bytes signature = 3;
>     }
>
> A SignedReceipt is a Receipt signed using the private key
> corresponding to the public key in the first certificate in the
> Receipt->Payment->Invoice.x509chain and the HMAC SHA-256 algorithm.
>
> Upon receiving a SignedReceipt, a Bitcoin client should validate the
> signature and, if valid, display the Receipt.memo and store the
> SignedReceipt as proof-of-payment.
>
> If a SignedReceipt is not received for any reason (timeout, error) and
> Payment.transactions has not been broadcast by the merchant on the
> Bitcoin p2p network, then the Bitcoin client should assume that the
> payment failed, inform the customer that the payment failed, and
> return coins involved in the transaction to the customer's wallet.
>
>
> Certificates
> ============
>
> The Invoice.x509chain (X.509 Certificate Chain) field contains the
> X.509 public key certificate or certificate chain [RFC5280]
> corresponding to the key used to digitally sign the Invoice and
> Receipt. The certificate or certificate chain is represented as an
> array of DER [ITU.X690.1994] PKIX certificate value. The certificate
> containing the public key of the entity that digitally signed the
> Invoice MUST be the first certificate. This MAY be followed by
> additional certificates, with each subsequent certificate being the
> one used to certify the previous one. The recipient MUST verify the
> certificate chain according to [RFC5280] and reject the payment
> request if any validation failure occurs.
>
> *What should we say about root certificates and certificate management
> in general? Any requirements, or leave it up to each Bitcoin client to
> determine which root CA's are trustworthy, as happens with web
> browsers? Gavin suggests trusting only (say) ten of the Extended
> Validation authorities:
> http://en.wikipedia.org/wiki/Extended_Validation_Certificate#Extended_Validation_certificate_identification
> *
>
> *X.509 is widely criticised for doing too much. However, it is the
> Public Key Infrastructure (PKI) system we're stuck with. Do web
> browsers / certificate authorities support the full X.509 spec, or
> only a subset? Should Bitcoin clients only support some well-defined
> subset of X.509 ? More research needed here... *
>
> Use Cases
> =========
>
> Merchant Payment Service
> ------------------------
>
> A merchant payment service (like Paysius or bit-pay.com) would use
> Invoices and Receipts as follows:
>
> 1. Merchant pays for a certificate from a certificate authority, and
> then gives the payment service the certificate and their private key.
> This could be the same certificate and private key as is used for the
> merchant's web site, but best security practice would be to purchase a
> separate certificate for authenticating Invoices. Very successful
> merchant payment services might act as intermediate certificate
> authorities, issuing certificates for their merchants.
> 2. Customer goes through the checkout process on either the merchant's
> or payment service's web site.
> 3. At the end of the checkout process, a SignedInvoice is generated
> and sent to the customer's Bitcoin client.
> 4. Customer's Bitcoin client displays the Invoice, showing that the
> payment is for the merchant.
> 5. On customer approval, a Payment is sent to the payment service's
> paymentURI. The merchant is notified of the payment, and the customer
> receives a SignedReceipt as proof-of-payment.
>
> SatoshiDice
> -----------
>
> SatoshiDice (www.satoshidice.com) is an extremely popular game that
> uses tiny transactions for some customer/service communications. In
> particular, customers can add an extra output to their transactions to
> indicate where winnings should be sent. And SatoshiDice creates tiny
> transactions to let their customers know that a bet was received, but
> lost.
>
> Assuming Bitcoin clients upgrade to support this proposal, a bet on
> SatoshiDice would proceed as follows:
>
> 1. Customer clicks on a link on SatoshiDice.com and their Bitcoin
> client receives a SignedInvoice.
> 2. Customer authorizes payment, and their Bitcoin client creates a
> Payment message and submits it directly to
> https://satoshidice.com/something
> 3. The SatoshiDice web server checks to make sure the transaction is
> valid, broadcasts it, and determines whether the customer wins or
> loses. It returns a SignedReceipt with either a "You win" or "You
> lost" memo.
> 4. If the customer won, it broadcasts a transaction to pay them using
> Payment.refund_to
> 5. Customer's Bitcoin client displays the win/lose memo, and if they
> won the winnings appear in their wallet when received over the p2p
> network.
>
> Multiperson Wallet
> ------------------
>
> This use case starts with a multi-signature Bitcoin address or wallet,
> with keys held by two different people (Alice and Bob). Payments from
> that address/wallet must be authorized by both Alice and Bob, and both
> are running multi-signature-capable Bitcoin clients.
>
> Alice begins the payment process by getting a SignedInvoice from a
> merchant that needs to be paid. She authorizes payment and her Bitcoin
> client creates a Payment message with a partially-signed transaction,
> which is then sent to Bob any way that is convenient (email
> attachment, smoke signals...).
>
> Bob's Bitcoin client validates the SignedInvoice and asks Bob to
> authorize the transaction. He says OK, his Bitcoin client completes
> the transaction by providing his signature, submits the payment to the
> merchant, and then sends a message to Alice with the SignedReceipt he
> received from the merchant, completing the payment process.
>
>
> Design Notes
> ============
>
> Why X.509 Certificates?
> -----------------------
>
> This proposal uses X.509 certificates as the identity system for
> merchants because most of them will have already purchased a
> certificate to secure their website and will be familiar with the
> process of proving their identity to a certificate issuing authority.
>
> Implementing a better global PKI is outside the scope of this
> proposal. If a better PKI is adopted, the only change to this proposal
> would be to replace the Invoice.x509chain with whatever that better
> infrastructure uses to identify entities.
>
>
> Why not JSON?
> -------------
>
> Invoice, Payment and Receipt messages could all be JSON-encoded. And
> the Javascript Object Signing and Encryption (JOSE) working group at
> the IETF has a draft specification for signing JSON data.
>
> But the spec is non-trivial. Signing JSON data is troublesome because
> JSON can encode the same data in multiple ways (whitespace is
> insignificant, characters in strings can be represented escaped or
> un-escaped, etc.), and the standards committee identified at least one
> security-related issue that will require special JSON parsers for
> handling JSON-Web-Signed (JWS) data (duplicate keys must be rejected
> by the parser, which is more strict than the JSON spec requires).
>
> A binary message format has none of those complicating issues. Which
> encoding format to pick is largely a matter of taste, but Protocol
> Buffers is a simple, robust, multi-programming-language,
> well-documented, easy-to-work-with, extensible format.
>
> What about a merchant-pays-fee feature?
> ---------------------------------------
>
> It is desireable to allow a merchant to pay the cost of any Bitcoin
> network transaction processing fees, so if a customer is paying for a
> 1 BTC item they pay exactly 1 BTC.
>
> One way of accomplishing that is to add a 'maxfee' field to the
> Invoice, and have the Bitcoin client construct a transaction that pays
> the merchant (amount-maxfee).
>
> Another way of accomplishing that is to change the transaction
> selection code used by Bitcoin miners, so that dependent transactions
> are considered as a group. Then a merchant with several unconfirmed
> zero-fee transaction from customers can create a pay-to-self
> transaction with a large enough fee to pay for the set of transactions
> to be confirmed.
>
> A third way of accomplishing that is for the Bitcoin client to sign
> Payment.transactions[0] using the SIGHASH_ANYONECANPAY flag, and for
> the merchant to add an additional, small-BTC-value input to the
> transaction before broadcasting it. That additional input would go
> directly to miners as a fee. *Note: Gavin is not sure if he loves or
> hates this idea.*
>
> Checking for revoked certificates
> ---------------------------------
>
> The Online Certificate Checking Protocol (OCSP) is supposed to be a
> quick and easy way for applications to check for revoked certificates.
>
> In practice, it doesn't work very well. Certificate Authorities have
> no financial incentive to support a robust infrastructure that can
> handle millions of OCSP validation requests quickly.
>
> Ideally, Bitcoin clients would use OCSP to check certificate statuses
> every time they received or re-used an Invoice. But if that results in
> long pauses or lots of false-positive rejections (because an OCSP
> endpoint is offline or overwhelmed, perhaps) then merchants and
> customers might revert to just using "never fails" Bitcoin addresses.
>
>
>
> References
> ==========
>
> Public-Key Infrastructure (X.509) working group :
> http://datatracker.ietf.org/wg/pkix/charter/
>
> RFC 2560, X.509 Internet Public Key Infrastructure Online Certificate
> Status Protocol - OCSP : http://tools.ietf.org/html/rfc2560
>
> Protocol Buffers : https://developers.google.com/protocol-buffers/
>
> See Also
> ========
>
> Javascript Object Signing and Encryption working group :
> http://datatracker.ietf.org/wg/jose/
>
> sipa's payment protocol proposal: https://gist.github.com/1237788
>
> ThomasV's "Signed Aliases" proposal : http://ecdsa.org/bitcoin_URIs.html
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

On Monday, November 26, 2012 11:32:46 PM Gregory Maxwell wrote:

> Obviously the state of the world with browsers is not that good... but
> in our own UAs we can do better and get closer to that.

This effectively centralizes Bitcoin (at least in the eyes of many) and even if each competing client had their own list, you'd be back to the original "problem" of not being sure your CA is on all lists.

> Would you find it acceptable if something supported a static whitelist
> plus a OS provided list minus a user configured blacklist and the
> ability for sophisticated users to disable the whitelist?

How is this whitelist any different from the list of CAs included by default with every OS?

On Mon, Nov 26, 2012 at 5:37 PM, Gavin Andresen <gavinandresen@gmail•com> wrote:

> This is the next big "lets all agree to do things the same way" thing
> I think we should tackle. I'm particularly looking for feedback from
> other bitcoin client developers, even if it is just a quick "looks
> reasonable, if everybody else is going to do it then I will
> (eventually) too..."

Comments:

1) Payment message should include ability to specify the transaction _or_ a transaction id sent via normal means over the network.

2) I think a significant bitcoin userbase will want to operate outside the full root-CA chain. Just look at https:// websites now. Self-signed certs are quite common, because it is easier, while being more secure than http://

So some provision for self-signed certs, a use case in wide use elsewhere, or equivalent thereof, seems reasonable.

I hope you all take a moment to see what DANE leverages with DNSSEC and SelfSigned x.509 certs. DANE provides the capability for any entity to associate a self signed certificate with a domain name. This capability removes the critical path of whitelists and/or Root CA certs.

Another nifty thing is that it can associate a cert to a domain and a payment address, if one were to put said address in the DNS :)

Now I am sure the majority of the bitcoin user-base desires anonymity, but as a merchant I would like to be knowable and wouldn't mind it if my identity and those of my transactions were "known" and associated both with my domains and x.509 cert. In most commercial transactions (which include many of those that leverage invoices) identity is important, at least for the merchant.

-rick

On Mon, Nov 26, 2012 at 3:52 PM, Jeff Garzik <jgarzik@exmulti•com> wrote:

> On Mon, Nov 26, 2012 at 5:37 PM, Gavin Andresen <gavinandresen@gmail•com> wrote:
>> This is the next big "lets all agree to do things the same way" thing
>> I think we should tackle. I'm particularly looking for feedback from
>> other bitcoin client developers, even if it is just a quick "looks
>> reasonable, if everybody else is going to do it then I will
>> (eventually) too..."
>
> Comments:
>
> 1) Payment message should include ability to specify the transaction
> _or_ a transaction id sent via normal means over the network.
>
> 2) I think a significant bitcoin userbase will want to operate outside
> the full root-CA chain.  Just look at https:// websites now.
> Self-signed certs are quite common, because it is easier, while being
> more secure than http://
>
> So some provision for self-signed certs, a use case in wide use
> elsewhere, or equivalent thereof, seems reasonable.
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti•com
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

On Mon, Nov 26, 2012 at 6:44 PM, Luke-Jr <luke@dashjr•org> wrote:

> On Monday, November 26, 2012 11:32:46 PM Gregory Maxwell wrote:
>> Obviously the state of the world with browsers is not that good... but
>> in our own UAs we can do better and get closer to that.
>
> This effectively centralizes Bitcoin (at least in the eyes of many) and even
> if each competing client had their own list, you'd be back to the original
> "problem" of not being sure your CA is on all lists.

Thats the CA model generally. It _is_ a distributed-centralized model in practice.

>> Would you find it acceptable if something supported a static whitelist
>> plus a OS provided list minus a user configured blacklist and the
>> ability for sophisticated users to disable the whitelist?
>
> How is this whitelist any different from the list of CAs included by default
> with every OS?

Because the list is not identical (and of course, couldn't be without centralizing control of all OSes :P ) meaning that the software has to be setup in a way where false-positive authentication failures are a common thing (terrible for user security) or merchants have to waste a bunch of time, probably unsuccessfully, figuring out what certs work sufficiently 'everwhere' and likely end up handing over extortion level fees to the most well established CAs that happen to be included on the oldest and most obscure things.

Taking— say— the intersection of Chrome, Webkit, and Firefox's CA list as of the first of the year every year and putting the result on a whitelist would be a possible nothing-up-my-sleeve approach which is not as limited as having some users subject to the WinXP cert list, which IIRC is very limited (but not in a way that improves security!).

Jeff wrote:

> Self-signed certs are quite common, because it is easier, while being
> more secure than http://

Uhh. Really? Well, I agree with you that they should be (I unsuccessfully lobbied browser vendors to make self-signed https on http URLs JustWork and simply hide all user visible evidence of security), but the really nasty warnings on those sites undermines the security of the sites _and_ of other HTTPS sites because it conditions users to click ignore-ignore-ignore. I don't think they are all that common.

One thing which I think will be hard for us in this discussion is being sensitive to the (quite justified!) concerns that the current CA system is absolute rubbish, both terrible for security, usability, and an unreasonable barrier to entry relative to the provided security— without allowing the discussion to be usurped by everyone's pet replacement, which there are a great many of with varying feasibility and security.

Perhaps we should agree to talk about everything _except_ that first?

> Perhaps we should agree to talk about everything _except_ that first?

Yeah, alternatives to X.509 chains don't interest me right now except in the sense that they should be cleanly implementable with future extensions.

So if you care about DANE or DNSSEC or custom PKI infrastructures or whatever, rather than proposing them as replacements here (DOA), just figure out how you would extend the protocol in Gavins mail in a future extension. If you can't see a clean way to do it then let's discuss that. If you can think of a way to do it then let's table it. Better replacements can come in later BIPs.

On Tuesday, November 27, 2012 12:02:42 AM Rick Wesson wrote:

> Another nifty thing is that it can associate a cert to a domain and a
> payment address, if one were to put said address in the DNS :)
>
> Now I am sure the majority of the bitcoin user-base desires anonymity,
> but as a merchant I would like to be knowable and wouldn't mind it if
> my identity and those of my transactions were "known" and associated
> both with my domains and x.509 cert. In most commercial transactions
> (which include many of those that leverage invoices) identity is
> important, at least for the merchant.

Anonymity isn't a feature we claim to have, nor a goal of the project for the most part. Using a single Bitcoin address has many problems besides non- anonymity: your customers are denied basic privacy and there is no good way to guarantee the user who says he paid you really did (since transaction ids are public record, anyone can claim they sent it).

In short, it is for the most part considered a rule to always use a unique address per transaction or at least per customer.

Luke

On Mon, Nov 26, 2012 at 4:31 PM, Luke-Jr <luke@dashjr•org> wrote:

> On Tuesday, November 27, 2012 12:02:42 AM Rick Wesson wrote:
>> Another nifty thing is that it can associate a cert to a domain and a
>> payment address, if one were to put said address in the DNS :)
>>
>> Now I am sure the majority of the bitcoin user-base desires anonymity,
>> but as a merchant I would like to be knowable and wouldn't mind it if
>> my identity and those of my transactions were "known" and associated
>> both with my domains and x.509 cert. In most commercial transactions
>> (which include many of those that leverage invoices) identity is
>> important, at least for the merchant.
>
> Anonymity isn't a feature we claim to have, nor a goal of the project for the
> most part. Using a single Bitcoin address has many problems besides non-
> anonymity: your customers are denied basic privacy and there is no good way to
> guarantee the user who says he paid you really did (since transaction ids are
> public record, anyone can claim they sent it).
>
> In short, it is for the most part considered a rule to always use a unique
> address per transaction or at least per customer.

putting payment addresses in the DNS does not require that only a single address be used. This is an assumption and a possible use case, but there is no requirement that payment addresses must be 1:1 associated.

-rick

On Tuesday, November 27, 2012 12:16:07 AM Gregory Maxwell wrote:

> On Mon, Nov 26, 2012 at 6:44 PM, Luke-Jr <luke@dashjr•org> wrote:
> > On Monday, November 26, 2012 11:32:46 PM Gregory Maxwell wrote:
> >> Would you find it acceptable if something supported a static whitelist
> >> plus a OS provided list minus a user configured blacklist and the
> >> ability for sophisticated users to disable the whitelist?
> >
> > How is this whitelist any different from the list of CAs included by
> > default with every OS?
>
> Because the list is not identical (and of course, couldn't be without
> centralizing control of all OSes :P ) meaning that the software has to
> be setup in a way where false-positive authentication failures are a
> common thing (terrible for user security) or merchants have to waste a
> bunch of time, probably unsuccessfully, figuring out what certs work
> sufficiently 'everwhere' and likely end up handing over extortion
> level fees to the most well established CAs that happen to be included
> on the oldest and most obscure things.

There is a common subset of CAs which are included in all OSs. That's the "whitelist equivalent". We or someone else could even setup a list of these common CAs for merchants if that is needed.

The fees CAs charge for certs is a flaw in the CA model in general, I don't see that it's important for us to solve it.

On Mon, Nov 26, 2012 at 4:26 PM, Mike Hearn <mike@plan99•net> wrote:

>> Perhaps we should agree to talk about everything _except_ that first?
>
> Yeah, alternatives to X.509 chains don't interest me right now except
> in the sense that they should be cleanly implementable with future
> extensions.
>
> So if you care about DANE or DNSSEC or custom PKI infrastructures or
> whatever, rather than proposing them as replacements here (DOA), just
> figure out how you would extend the protocol in Gavins mail in a
> future extension. If you can't see a clean way to do it then let's
> discuss that. If you can think of a way to do it then let's table it.
> Better replacements can come in later BIPs.

The only part that has an x509 cert associated is in the invoice message.

message Invoice {
//    repeated bytes x509chain = 1;
    optional string domainName =1;
    repeated Output outputs = 2;
    required uint64 time = 3;
    optional uint64 expires = 4;
    optional bool single_use = 5 [default = true];
    optional string memo = 6;
    optional string receiptURI = 7;
    optional bytes merchant_data = 8;
}

Removing that and adding a opaque string called domain name, or identityName would be sufficient to move the conversation forward without the x.509 baggage.

-rick

Supporting DNSSEC/DANE in the future when they are widely deployed is a great idea.

Note that the x509chain field is 'repeated', and any repeated field may have zero entries. So I would suggest supporting other PKI systems in the future by adding optional new fields (for maximum compatibility or security merchants might want to include both a x509chain AND

> This is the next big "lets all agree to do things the same way" thing
> I think we should tackle. I'm particularly looking for feedback from
> other bitcoin client developers, even if it is just a quick "looks
> reasonable, if everybody else is going to do it then I will
> (eventually) too..."

I agree this is a very pertinent subject, and with a bit of looking around it is clear that there is a requirement here for emerging financial ecosystems of many types, certainly not just for the Bitcoin community, which until now seems to have been getting along just about OK despite the current levels of complexity.

That said, I have a number of serious concerns with the proposal.

1. Undue Broadening of Scope: From an architectural perspective, if one accepts the unix mantra of "do one thing and do it well" as reasonable and time-proven doctrine, given that Bitcoin is already trying to be both a commodity and a distributed consensus-based settlement system, does it really make sense to attempt to tack-on business-level functions?

2. X.509: I have read (somewhere or other, recently) that it is generally considered bad form to mandate specific cryptographic systems in new protocols where open support is possible. Given the recent issues with X.509, the security nightmare that already exists with the volume of (sometimes cracked, sometimes government-compromised?) issuers, and the complexity of the scheme, it seems a little strange to singularly mandate X.509, despite its widespread use at present. There are also a swathe of potential issues around DNS interdependence, information leakage within certificates themselves and/or their DNS-interpretation by clients, etc. I would consider suggesting open support with initial support for GPG, as it is apparently preferred as a simple and further decentralized solution by the majority of the open source and cryptographic software development community.

3. Failure to Review Existing Work: I would urge anyone to be wary of adopting any proposal that does not inform itself through reference to existing protocols in the same area. In this area there are a few protocols in current use (chiefly in Europe) such as those listed at http://en.wikipedia.org/wiki/Invoice#Electronic_invoices as well as various hosted platforms such as http://xero.co.nz/ (chiefly Australia/New Zealand). Often, existing work shows its age with after-the-fact alterations that sit poorly with initial assumptions: exactly the kind of situation one can walk in to developing against a proposal before adequately researching the area.

4. Complexity of Metadata: Physical and digital invoicing for businesses operating at scale often requires delivery terms, product classification codes, locale-specific taxation (often at multiple levels), various fees and discounts (sometimes fulfillment-speed linked with multiple tiers/thresholds), and other features that I am skeptical are ever going to be made fully available within a business protocol tacked on to a hybrid digital currency/settlement system (like Bitcoin) as a secondary concern.

5. Non-BTC Currencies/Currency-like Commodities: No approach to non-BTC currencies appears to have been made, which makes the "invoice" of limited utility for almost all businesses, save those willing to accept all of the 'capital risk' (exchange rate fluctuation risk) inherent in a BTC-based fulfilment process with a potential term long enough to justify an invoicing process. (Does this narrow scope actually cover any existing business?)

6. DNS: As already mentioned with regards to X.509: a huge red flag as an area of potential vulnerability, or at least information leakage.

I must now admit that in raising the above I am definitely biased. My employer (Payward, Inc.) and other organizations (OpenCoin, Inc., etc.) have been working with the Internet Engineering Task Force (IETF) on tabling some open proposals within this area under the auspices of the Internet Financial Exchange Project (http://ifex-project.org/). Our hope is to facilitate the requisite standardisation within internet-connected systems to deal with what is perhaps fairly characterised as a relatively heterogeneous outlook on the rise of cryptographic (and other alternate) currencies and commodities, and emerging settlement infrastructures.

Whilst the current Bitcoin proposal is admirable for correctly raising the area as one of immediate concern, I hope that the above points out some of the perhaps as-yet unconsidered complexities and draws in to question whether Bitcoin is in fact the appropriate place to implement a solution, given the hassles that will entail. After all, wouldn't Bitcoin developer time would be better spent improving the core of bitcoin (ie. distributed settlement system and commodity) rather than adding new features?

I would invite parties within the Bitcoin community with an interest in non directly settlement-linked financial transaction negotiation and reporting features to consider contributing to the existing, re-usable efforts at the IFEX Project, rather than supporting the extension of one currency/commodity and settlement infrastructure (ie. Bitcoin) which IMHO is likely to detract from developer time, increase complexity, and perhaps result in a less polished and re-applicable solution overall.

Our proposals: - X-ISO4217-A3 (X-ISO4217-A3). A published proposal that provides a mechanism for the open identification of currencies or currency-like commodities on the internet. (Bitcoin is registered as XBTC). http://www.ifex-project.org/our-proposals/x-iso4217-a3 - Internet IBAN (IIBAN). A published proposal that provides a mechanism for the open identification of financial endpoints on the internet. (IBAN compatible, checksum-included, name-squatting problem avoiding. The registry of entities is IANA-managed, encourages GPG use, and avoids the X.509 requirement.) http://www.ifex-project.org/our-proposals/iiban - Internet MIC (IMIC). A published proposal that provides a mechanism for the open identification of financial markets on the internet. (Such as most Bitcoin exchanges) http://www.ifex-project.org/our-proposals/imic - Internet Financial EXchange (IFEX). A proposal under development that facilitates the negotiation of financial transactions between internet-based financial endpoints. (The area we would love your input) http://www.ifex-project.org/our-proposals/ifex

Sincerely and with the utmost respect for the Bitcoin project's excellent work, Walter Stanish

On Mon, Nov 26, 2012 at 9:16 PM, Walter Stanish <walter@stani•sh> wrote:

> X-ISO4217-A3

I see that draft-stanish-x-iso4217-a3 is not standards track, is there a reason for this?

It also doesn't appear to address ~any of the the targeted items here. Is there another draft I should be looking for which has more overlap with the discussion here?

>> X-ISO4217-A3
>
> I see that draft-stanish-x-iso4217-a3 is not standards track, is there
> a reason for this?

Of the three currently published proposals, all are essentially IANA registry proposals.

We are currently working with IETF staff, with open offers of support from multiple well funded commercial bodies, to transition these proposals through to IANA management.

It appears that the Independent Stream Editor path will be used to transition these through to IANA, at which time the proposals themselves will be converted to Informational status.

(As far as I understand right now, Within the IETF, Standards Track has special meaning and entails relatively large degrees of bureaucracy that are not within the current contributors' resources. It is also worth pointing out that many popular protocols implemented on the majority of systems (IIRC, such as IMAP) never reach formal standardization for this reason. It should be noted that in these cases, this does not make the protocols any less attractive as potential components for system implementation.)

> It also doesn't appear to address ~any of the the targeted items here.
> Is there another draft I should be looking for which has more overlap
> with the discussion here?

As outlined in the previous post:
  - Internet Financial EXchange (IFEX). A proposal under development
that facilitates the negotiation of financial transactions between
internet-based financial endpoints. (The area we would love your
input) http://www.ifex-project.org/our-proposals/ifex

As well as the information linked to above, significant but not particularly well grounded discussions have occurred regarding the IFEX-based paradigm for settlement versus some other proposed paradigms, in particular Ripple (as it appeared some months ago), which can be read here: https://groups.google.com/forum/?fromgroups#!topic/rippleusers/v4bEBZZVEsA[1-25]

Kind regards and with the hopes of combining our efforts as a joint proposal that can benefit other currencies/commodities and settlement systems as well as Bitcoin, Walter Stanish Skype:walter.stanish

On Mon, Nov 26, 2012 at 7:16 PM, Walter Stanish <walter@stani•sh> wrote:

>>> X-ISO4217-A3
>>
>> I see that draft-stanish-x-iso4217-a3 is not standards track, is there
>> a reason for this?
>
> Of the three currently published proposals, all are essentially IANA
> registry proposals.
>
> We are currently working with IETF staff, with open offers of support
> from multiple well funded commercial bodies, to transition these
> proposals through to IANA management.

I hate to inform you that you have been mislead. The IETF and the IANA do not operate as you outlined above. Having spent too many years within ICANN/IETF/IANA I can assure you are mistaken. Your drafts are expired and it appears that there is no support for a "finance" working group in the IETF.

-rick

>> We are currently working with IETF staff, with open offers of support
>> from multiple well funded commercial bodies, to transition these
>> proposals through to IANA management.
>
> I hate to inform you that you have been mislead. The IETF and the IANA
> do not operate as you outlined above. Having spent too many years
> within ICANN/IETF/IANA I can assure you are mistaken.
> Your drafts are expired and it appears that there is no support for a
> "finance" working group in the IETF.

We are not establishing an IETF working group, which is an option that was explored prior to the Paris meeting and has been sidelined at present for depth-of-bureaucracy by the backing commercial entities. Rather, we are establishing a top-level IANA registry group. This is not anticipated by the IETF old-guard working with us to be either (a) controversial or (b) possible to block.

- Walter

>
> We are not establishing an IETF working group, which is an option that
> was explored prior to the Paris meeting and has been sidelined at
> present for depth-of-bureaucracy by the backing commercial entities.
> Rather, we are establishing a top-level IANA registry group. This is
> not anticipated by the IETF old-guard working with us to be either (a)
> controversial or (b) possible to block.

My last note in this sub-thread.

There are no IANA registry groups, there is no such thing, and no way to form one. The IETF can ask the IANA to form a registry but these things take lots of support and take a long time, and these are only created through standards track RFC. ICANN runs the IANA and there is no such framework that you elude to. Review http://www.iana.org/protocols/

If you are applying for a gTLD, good luck with that.

-rick

>> We are not establishing an IETF working group, which is an option that
>> was explored prior to the Paris meeting and has been sidelined at
>> present for depth-of-bureaucracy by the backing commercial entities.
>> Rather, we are establishing a top-level IANA registry group. This is
>> not anticipated by the IETF old-guard working with us to be either (a)
>> controversial or (b) possible to block.
>
> My last note in this sub-thread.

Mine too!

> There are no IANA registry groups, there is no such thing, and no way
> to form one.

Reading between the lines, I believe this phrase, which is not my own but that of experienced IETF staff, refers to the groups visible at http://www.iana.org/protocols/ (which you yourself cited). Whether it is formally used or not is unknown to me.

> The IETF can ask the IANA to form a registry but these
> things take lots of support and take a long time,

Expert opinion estimates six weeks, and by current estimates, we should have an arrival circa February.

> and these are only
> created through standards track RFC. ICANN runs the IANA and there is
> no such framework that you elude to. Review
> http://www.iana.org/protocols/

I would like to suggest that perhaps exactly this sort of banter is an excellent illustration for the Bitcoin community of what we have been up against in this (conceivable simple an public benefit oriented) endeavour. If you also look at the fact that the ISO4217 registry (to take currency/commodity codes as just one example) there is apparently not even a public list of requirements for codepoint issue. This sort of thing is *exactly* why the internet community appears to desperately need an open registry - allowing public internet bodies (IANA) to function to support innovation and interconnectivity for all sectors of the internet's various financial communities so that anyone, including innovators, can obtain interoperability via simple, hassle-free paths, without encountering self-important bureaucrats.

We anticipate victory circa February.

- Walter

Short comments:

* What if the SignedReceipt is not received AND the transactions IS posted on the p2p. Then you have payed for the goods, but you don't have a receipt. This could happen both from malice or system failures. ** Suggestion - sign the invoice with the key to which to send the transaction, the proof of payment, equivalent to a signed receipt is then in the blockchain.

This scheme would work both with or without x509, if you want to include x509, the message in the invoice could simply be signed by the x509 certificate as well.

PRO: Any user can send signed invoices, not only those with a x509 cert. PRO: No limbo situation with no SignedReceipt CON: This disables the use of anything but payment to key/address incl multisig etc.

However, the wast majority of use will anyway be payment to key/address.

Support of general pay to script could be supported through the payment scheme proposed earlier by Mike: No non-fee payments are accepted, except in a group - i.e. it is up to the merchant to generate the final transaction incl the fees, and that one could be to a general script. This also keeps the support of pay to general script needed for a client to a minimum.

Cheers,

Michael

Luke-Jr - common subset of what operating systems ship is fine for me as long as people do due diligence around mobile OS' here. It seems easier to me to just grab a list from a popular browser, on the grounds that SSL is mostly used by them so nobody is going to buy an SSL cert rejected by IE/Firefox/Chrome/etc. But intersecting OS lists is effectively the same.

For my own clients I'd just ship my own copy of the canonical CA certs regardless, because integrating with each operating systems proprietary crypto APIs is a lot of work vs just loading a pem file into OpenSSL. If there are a lot of people who want to use the OS cert management UIs then I guess that can be a point wallet clients compete on.

> Removing that and adding a opaque string called domain name, or
> identityName would be sufficient to move the conversation forward
> without the x.509 baggage.

But it would result in implementations that do not meet the requirements.

Yes, X.509 has problems. It's in the proposal because we can get the effect we want (verifiable domain names in the UI) in about 50 lines of code, today, with the id-verified keys people actually have already bought.

As Gavin says, we can add optional fields later to extend the protocol in a backwards compatible way.

On Tue, Nov 27, 2012 at 9:43 AM, Michael Gronager <gronager@ceptacle•com> wrote:

> * What if the SignedReceipt is not received AND the transactions IS posted on the p2p.

I think this is a problem with confusing terminology rather then the spec itself.

The original formulation had a receipt being something generated purely by the buyer. The signed Invoice message + the Bitcoin transactions paying to the outputs + the merkle branches showing acceptance by the network *is* the receipt.

The SignedReceipt message is useful in the sense that it shows confirmation by the merchant, but if you don't get one, you can still prove you paid the invoice. So from this perspective perhaps SignedReceipt should be renamed to Acceptance or something like that, and then the spec should call out that a signed invoice plus accepted Bitcoin transactions is mathematically a proof of purchase.

>
> The SignedReceipt message is useful in the sense that it shows
> confirmation by the merchant, but if you don't get one, you can still
> prove you paid the invoice. So from this perspective perhaps
> SignedReceipt should be renamed to Acceptance or something like that,
> and then the spec should call out that a signed invoice plus accepted
> Bitcoin transactions is mathematically a proof of purchase.

Which is why I find the "SignedReceipt" somewhat superfluous. If you implement a payment system, like bit-pay/wallet you are likely to double that through some sort of e-mail receipt anyway.

Further, the inclusion of x509 is not really needed in the spec - you don't need to sign the invoice with an x509, you can use the payment key. The proof would still be equally binding, and valid also for non holders of x509 (server) certificates (like normal people). Finally, host certificates does not normally keep in their "purpose" S/MIME Signing. So you are bending the intended use of the x509 certificate anyway.

/M

>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

On Tue, Nov 27, 2012 at 11:42:01AM +0100, Michael Gronager wrote:

> >
> > The SignedReceipt message is useful in the sense that it shows
> > confirmation by the merchant, but if you don't get one, you can still
> > prove you paid the invoice. So from this perspective perhaps
> > SignedReceipt should be renamed to Acceptance or something like that,
> > and then the spec should call out that a signed invoice plus accepted
> > Bitcoin transactions is mathematically a proof of purchase.
>
> Which is why I find the "SignedReceipt" somewhat superfluous. If you implement a payment system, like bit-pay/wallet you are likely to double that through some sort of e-mail receipt anyway.

Gavin's proposal differs in this from my original proposal, where I exactly *didn't* want to couple the receipt with the acceptance of the Bitcoin transaction.

If a merchant/payment processor is willing to take the risk of zero or low confirmation transactions (because they are insured against it, for example), they were allowed to reply "accepted" immediately, and this would be a permanent proof of payment, even if the actual Bitcoin transaction that backs it gets reverted.

For that reason, I also had a separate "pending" state, which means the receiver isn't willing to just accept the current state as irrevocably paid. In this case, the sender was allowed to retry until the receipt sayd "accepted" or "rejected".

The whole point was to avoid that customers/merchants would have to deal with the uncertainty involved in Bitcoin transaction. At some point, someone is going to accept the transaction (whether that is at 0 or at 120 confirmations), and acceptance will at the higher level be considered a boolean anyway - not some "probably, unless reorg".

>
> If a merchant/payment processor is willing to take the risk of zero or
> low confirmation transactions (because they are insured against it,
> for example), they were allowed to reply "accepted" immediately, and
> this would be a permanent proof of payment, even if the actual Bitcoin
> transaction that backs it gets reverted.

I guess that moves the discussion from developers to lawyers ;) Even though you send a signed receipt, if you can proof you didn't get the money, you will never be expected to deliver the goods. (and you can even write that in the the receipt ...)

So the SignedReceipt is legally not worth the bits it is composed of, hence I don't see the point in supporting it.

If you are selling atoms you can usually wait for N confirmations (even though you start shipping I guess you can recall a parcel within 144 blocks). If you are selling bits (like access to a site), you can revoke that access once you discover the transaction did not go through. So I can't find a use case where a Signed Receipt in the proposed form is advantageous.

/M

> Further, the inclusion of x509 is not really needed in the spec - you don't need to sign the invoice with an x509, you can use the payment key.

No, the point of using X509 certs is to get a verified identity (a domain name) on the receipt, this is needed for multi-factor authentication. You can't do that without some kind of third party asserting to an identity.

> No, the point of using X509 certs is to get a verified identity (a
> domain name) on the receipt, this is needed for multi-factor
> authentication. You can't do that without some kind of third party
> asserting to an identity.

Agree that you need a third party to verify identity. But the verification policy of sites is the job for a payment provider not a payment technology. So if you would like verification of the site you could just sign the memo using standard S/MIME - why mix it with the payment protocol?

Further, it is controversial use of the host key to use it for digital signing of documents, and not even within the policy of a host certificate as far as I recall.

The problem you are trying to tackle is that we don't have an ID solution on the internet today for this purpose. Certificates for signing messages are distributed freely and insecurely only based on temporarily having an email from within an organization, and the host certificates are meant for SSL handshakes. Funnily, any CA can issue digital certificates for email signing for any domain, even though they don't own them, and without notifying the owner. DANE actually solves this, but until then using the host certificates is unintended use, it is cryptographically a nice solution, but legally and standard-wise a hack.

/M

RE: SignedReceipt: I agree it is superfluous. I'll remove it from the spec.

RE: "it is controversial use of the host key to use it for digital signing of documents" : The idea of embedding a x509 certificate chain comes from the IETF's JSON Object Signing and Encryption working group "JWS" specification, so I can't be TOO controversial.

RE: the ifex-project and other electronic invoicing standards: Thanks for the pointers, Walter! I'm all for adopting the best ideas that have come before, as long as we end up with something useful and small enough to convince ourselves it is as secure as we can make it. I looked at the ifex spec, and quickly got lost. It would help me if you could write up what our motivating use cases would look like if implemented on top of ifex.

RE: jgarzik's suggestion to allow txids in the Payment: that worries me, because it is trivial to create several different variations of the same transaction (same inputs to same outputs) with different txids (re-signing inputs uses a different signature nonce, which changes the signature/txid, for example).

RE: using self-signed certificates: as Mike said, I assume Bitcoin clients will have some way of managing root certificates, so experts could add trusted self-signed certs.

One more thought:

RE: "Receipt" verus "Acceptance" :

I believe "Receipt" is the right term-- it means "I got your payment", NOT "your payment has cleared." E.g. if I hand a merchant a paper check they'll hand me a receipt, but the check could still bounce. That's the analogy here-- a merchant might give you a receipt, but if the transaction is rejected by the network for whatever reason (Finney attack maybe) you cannot expect to go to court with your invoice/receipt and claim you made a valid payment.

On Monday 26 November 2012 22:37:31 Gavin Andresen wrote:

> x509chain: one or more DER-encoded X.509 certificates that identifies
> the merchant. See the "Certificates" section below for details.

Personally, I'd like to see fewer implicit ties to X509. With X509 as one option. For example, I'd much prefer to see a doorway to the future left open like this:

    message Invoice {
        repeated bytes issuerIdentityType;
        repeated bytes issuerIdentityBytes;

or similar, instead of "x509chain".

In particular two additional identification types:

 - GnuPG (obviously)
 - Hash based

The hash-based system would be there as a method of leveraging an existing trusted connection, without needing to get into the nitty-gritty of certificates. For example, I am paying for something on a web site; I presumably already have a secure connection that I trust to that site. That site can issue me an invoice (which is to be sent to the bitcoin client) _and_ a hash of the certificate on the same page.

I trust that hash because I received it over a secure connection from a trusted source. When my bitcoin client pops up with the received invoice, it shows me the hash of the invoice, and I can be sure that it is from the web site I thought it was from.

Imagine I'm a (very) small business, I have two or three customers. I want to email one of my customers an invoice. I don't want to have to get an X509 certificate, and I don't necessarily know how. However, I can ring my customer up and say "I've generated an invoice with my bitcoin client, it is hashed A7DE-521X-9977. Write that down and confirm it when you get my invoice". Alternatively, I might attach a file called invoice-A7DE-521X-9977.bitinv to a signed GnuPG email. The receipient can easily confirm I sent it because the filename must match the contents and GnuPG protects against tampering.

Andy

> Personally, I'd like to see fewer implicit ties to X509.  With X509 as one
> option.

That's pretty much what we have today - in future other schemes can be proposed as extensions. Protocol buffers are easily extended, they ignore unknown fields. Then you'd wait and see what the invoice request looked like and produce an invoice with the right security bits.

> In particular two additional identification types:
>
>  - GnuPG (obviously)

It's not obvious to me, incidentally. The web of trust has been dead-on-arrival since it was first proposed, and for good reasons. SSL/X.509, for better or worse, has significant usage.

Your case of a small business is a perfect example of people who won't be using GPG. If they don't want to buy an SSL cert, they can just as well put a reference number in the memo field or a "Hey Bob, here is the bill we discussed". The payer does not get the multi-factor auth protection so if their computer has a virus, they may be hosed. But that's good incentive for sellers to get verified. Some CA authorities do it for free these days.

On Tuesday 27 November 2012 17:14:19 Mike Hearn wrote:

> That's pretty much what we have today - in future other schemes can be
> proposed as extensions. Protocol buffers are easily extended, they
> ignore unknown fields. Then you'd wait and see what the invoice
> request looked like and produce an invoice with the right security
> bits.

That's good; I've not done anything with protocol buffers, so wasn't aware it was that simple.

> > In particular two additional identification types:
> >  - GnuPG (obviously)
>
> It's not obvious to me, incidentally. The web of trust has been
> dead-on-arrival since it was first proposed, and for good reasons.
> SSL/X.509, for better or worse, has significant usage.

Sorry, I meant "obviously" in the sense that "obviously that's the other one that everyone will want". The web-of-trust as a universal identity mechanism is, I agree, not useful. However, as a localised, smaller-scale identity verification system it's used by every GnuPG user. You become your own certificate authority. For example, I've set up my whole family with GnuPG; I've set them up to trust me to authenticate (and I doubt any of them has ever added anyone else). Then I take on the responsibility of signing all my family/friends keys and they don't need to worry about it.

There's no reason that a small group of companies wouldn't do exactly the same sort of thing.

> Your case of a small business is a perfect example of people who won't
> be using GPG. If they don't want to buy an SSL cert, they can just as

Bear in mind, I was using that example as an example of a hash protected in a GPG envelope, not a GPG-signed invoice. People who've already got their GPG system in place will appreciate being able to leverage it.

> well put a reference number in the memo field or a "Hey Bob, here is
> the bill we discussed". The payer does not get the multi-factor auth

How can they put a hash of an invoice inside the invoice? In my "hash mode" invoices, it would be a random number (or possibly specifying the hash algorithm) then the SignedInvoice would simply be the original invoice + hash. That hash would then be reported via some secure channel outside of bitcoin's domain.

> protection so if their computer has a virus, they may be hosed. But
> that's good incentive for sellers to get verified. Some CA authorities
> do it for free these days.

I don't understand what the relevance of multi-factor is to invoices? The payment is performed via normal bitcoin mechanisms isn't it -- multi-factor or not? This invoice system has one primary job: to ensure that the target of the payment is who the payer thinks it is -- that's not affected by multi- factor methods of protecting my wallet.

Andy

> That hash would then be reported via some secure channel outside of bitcoin's
> domain.

OK, I see. I guess that could be a reasonable fallback for the case where you have a secure channel.

> I don't understand what the relevance of multi-factor is to invoices?

Yes, exactly. It's about paying who you think you're paying (when you confirm on a second uncompromised device).

Spec updated: https://gist.github.com/4120476

Notable changes are:

+ Removed SignedReceipt

+ Replaced Invoice.x509chain with a "pki_type" and "pki_data" to make using other identity systems cleaner.

+ Added a "Why not an existing electronic invoice standard?" section to the design notes

On Mon, Nov 26, 2012 at 05:37:31PM -0500, Gavin Andresen wrote:

> Why not JSON?
> -------------
>
> Invoice, Payment and Receipt messages could all be JSON-encoded. And
> the Javascript Object Signing and Encryption (JOSE) working group at
> the IETF has a draft specification for signing JSON data.
>
> But the spec is non-trivial. Signing JSON data is troublesome because
> JSON can encode the same data in multiple ways (whitespace is
> insignificant, characters in strings can be represented escaped or
> un-escaped, etc.), and the standards committee identified at least one
> security-related issue that will require special JSON parsers for
> handling JSON-Web-Signed (JWS) data (duplicate keys must be rejected
> by the parser, which is more strict than the JSON spec requires).
>
> A binary message format has none of those complicating issues. Which
> encoding format to pick is largely a matter of taste, but Protocol
> Buffers is a simple, robust, multi-programming-language,
> well-documented, easy-to-work-with, extensible format.

I'm not sure this is actually as much of an advantage as you'd expect. I looked into Google Protocol buffers a while back for a timestamping project and unfortunately there are many ways in which the actual binary encoding of a message can differ even if the meaning of the message is the same, just like JSON.

First of all while the order in which fields are encoded *should* be written sequentially, parsers are also required to accept the fields in any order. There is also a repeated fields feature where the fields can either be serialized as one packed key-list pair, or multiple key-value(s) pairs; in the latter case the payloads are concatenated.

The general case of how to handle a duplicated field that isn't supposed to be repeated seems to be undefined in the standard. Yet at the same time the standard mentions creating messages by concatenating two messages together. Presumably parsers treat that case as an error, but I wouldn't be surprised if that isn't always true.

Implementations differ as well. The current Java and C++ implementations write unknown fields in arbitrary order after the sequentially-ordered known fields, while on the other hand the Python implementation simply drops unknown fields entirely. As far as I know no implementation preserves order for unknown fields.

Finally, while not a Protocol Buffers specific problem, UTF8 encoded text isn't guaranteed to survive a UTF8-UTFx-UTF8 round trip. Multiple code point sequences can be semanticly identical so you can expect some software to convert one to the other. Similarly lots of languages internally store unicode strings by converting to something like UTF16. One solution is to use one of the normalization forms such as NFKD - an idempotent transformation - although I wouldn't be surprised if normalization itself is complex enough that implementation bugs exist, not to mention the fact that the normalization forms have undergone different versions.

I think the best way(1) to handle (most) the above by simply treating the binary message as immutable and never re-serializing a deserialized message, but if you're willing to do that just using JSON isn't unreasonable either.

1) Of course I went off an created Yet Another Binary Serialization for my project, but I'm young and foolish...

The current spec is ambiguous in the case of what to do if the invoice contains one output of a fixed amount and one or more outputs of an unspecified amount. Should the user be prompted once per output? That seems suboptimal. Prompted once for a value that's then randomly distributed between all open-value outputs? It seems this ability of the protocol is somewhat more complex than it appears. The ability to have open outputs is nice for tips though.

You could consider moving pki_type and pki_data into a separate message and making both fields required, then making the pki message optional. Otherwise you can have pki_type set but no data or vice-versa. It doesn't make much difference in the end, just slightly improves the automatic sanity checks produced by the proto compiler.

w.r.t SIGHASH_ANYONECANPAY. I think it's best not to use this routinely as it relaxes the signature checks in ways that may open non-obvious holes when combined with other features. I thought we pretty much had consensus on recursively calculating fees including dependents in the memory pool?

Peter is correct that there are a few degrees of freedom in protobuf serialization, though far fewer than with JSON. I'd like to think upstream would be open to resolving these ambiguities. Re-serialization of an Invoice message in the Payment message is a potential source of mistakes. There's no need to ever concatenate these messages and alternative implementations that don't order serialized fields by tag number are missing an important optimization, so they could be fixed. The main issue is treatment of unknown fields. If/when the Invoice message is extended with other fields that are round-tripped through an old client, the data may get lost. JSON doesn't help resolve that either, of course. There are a few solutions:

1) Change the type of the Invoice field in Payment to be "bytes" and set it to be the hash of the originally received binary Invoice message. Downside, requires merchants to track all outstanding invoices. 2) Ask protobufs upstream to modify the spec/implementations so ordering of unknown fields is specified. The Python implementation could be extended to support them so Python implementors don't end up with accidental message downgrades. 3) Language of the spec could be changed to explicitly state that the received Invoice may not be binary-identical to the one that was sent, in the case of a client that incorrectly downgrades the message. Thus you'd be expected to check what the Invoice was using merchant_data which is opaque and could just be, eg, a database key on your own end. 4) Instead of submitting the entire Invoice back to the merchant, just the merchant_data could be in the Payment message.

Of the four options I prefer the last. What is the use case for resubmitting the entire invoice anyway? Even if protobufs are improved so handling of round-tripping new messages through old [Python] clients is more rigorous, some implementors will probably convert the protobuf objects into some internal forms for whatever reason (or serialize them to a database, etc) and they're very likely to mess up the handling of unknown fields when they do it.

On Wed, Nov 28, 2012 at 11:43:19AM +0100, Mike Hearn wrote:

> Peter is correct that there are a few degrees of freedom in protobuf
> serialization, though far fewer than with JSON.

FWIW I re-read the specs again and turns out my memory was wrong. (I last looked at this about four months ago) Duplicated fields are handled in a defined manner, with the last field seen in the serialization being the one whose value is used. Again, repeated fields are treated as elements of a list, preserving order.

It does raise the interesting question do the implementations that don't preserve order of unknown fields, preserve the order of multiple unknown fields, either repeated or not?

> I'd like to think upstream would be open to resolving these
> ambiguities.

I gotta admit, I suspect they won't be that open. Protocol buffers was designed because Google needed a fast serialization method suitable for many different internal projects. Needing round-trip idempotence seems like a rare requirement to me, especially for internal use.

> Re-serialization of an Invoice message in the Payment message is a
> potential source of mistakes. There's no need to ever concatenate
> these messages and alternative implementations that don't order
> serialized fields by tag number are missing an important optimization,
> so they could be fixed. The main issue is treatment of unknown fields.
> If/when the Invoice message is extended with other fields that are
> round-tripped through an old client, the data may get lost. JSON
> doesn't help resolve that either, of course. There are a few
> solutions:

Well, actually you can take advantage of the message concatination ability of protocol buffers to extend a message by simply appending the new fields to the existing thus either defining new fields, or overriding old values as required. If you want to de-duplicate though you run into the problem all over again.

On the other hand JSON handles this case fine too provided that your JSON implementation supports dictionary objects with arbitrary fields. Just use the object as is and the unknown fields will be re-serialized properly at the other end. Some implementations will have to be careful to handle collisions with existing keys in the namespace. (consider in Python what would happen if you mapped your object to a class instance, and the serialization included the key "__init__")

That said, JSON is quite problematic with numbers. For instance, you have to be careful to keep integers represented as pure integers below what Javascript can handle, the maximum integer exactly representable in a double float, or the JSON won't be parsable in Javascript even if many other languages handle it fine. Protocol buffers is at least pretty explicit about what size integers are.

> 1) Change the type of the Invoice field in Payment to be "bytes" and
> set it to be the hash of the originally received binary Invoice
> message. Downside, requires merchants to track all outstanding
> invoices.
> 2) Ask protobufs upstream to modify the spec/implementations so
> ordering of unknown fields is specified. The Python implementation
> could be extended to support them so Python implementors don't end up
> with accidental message downgrades.
> 3) Language of the spec could be changed to explicitly state that the
> received Invoice may not be binary-identical to the one that was sent,
> in the case of a client that incorrectly downgrades the message. Thus
> you'd be expected to check what the Invoice was using merchant_data
> which is opaque and could just be, eg, a database key on your own end.
> 4) Instead of submitting the entire Invoice back to the merchant, just
> the merchant_data could be in the Payment message.
>
> Of the four options I prefer the last. What is the use case for
> resubmitting the entire invoice anyway? Even if protobufs are improved

Note that I think the SignedInvoice message itself is broken, because protobuf implementations have no reason to guarantee that they can give you the serialized bytes of the Invoice sub-message. It's a quite specific use-case that isn't needed for pretty much anything but crypto. FWIW I took a quick look at the official API's, C++, Java and Python, and as far as I can tell none of them support accessing the binary serialization of a message field other than by re-serializing the message.

Really the invoice field should be declared as bytes serialized_invoice, as inconvenient as that is to work with.

> so handling of round-tripping new messages through old [Python]
> clients is more rigorous, some implementors will probably convert the
> protobuf objects into some internal forms for whatever reason (or
> serialize them to a database, etc) and they're very likely to mess up
> the handling of unknown fields when they do it.

Since the Payment message includes an *untrusted* Invoice that the vendor needs to authenticate the whole invoice no matter what on Payment reception. In many cases that implies they have to keep some sort of database of "quotes" or similar anyway as the client can change anything they want otherwise. Again that leads back to the argument of why not just stick with the merchant_dat as you suggest, which will usually be some short invoice number attached to a database? A vendor that wants to operation a stateless invoicing system can just stuff a HMAC-protected serialized invoice into the merchant_data

I guess you could use a mutable invoice field as a way of achieving some sort of negotiation protocol, but I think it's better to stick to the original concept of just ensuring that the user is really paying the right amount to the right address.

> RE: the ifex-project and other electronic invoicing standards:  Thanks
> for the pointers, Walter! I'm all for adopting the best ideas that
> have come before, as long as we end up with something useful and small
> enough to convince ourselves it is as secure as we can make it.

Fair enough, although I would posit: 0. You can't get more secure than not doing it at all. 1. Small is sometimes beautiful, but sometimes just the 999th crippled and half-featured attempt at a nontrivial problem space (...linked heavily to implementation-time assumptions and/or specific peer systems, and by its very nature destined for the dustbin of time? Well, maybe. Eventually, we all are.) 2. X.509 is not small (or beautiful, unless one is some weird new kind of centralized cryptographic trust fetishist, of some sort you might see with furrowed brows, boozing at midday sporting key & anchor tattoos in old convention polo-shirts, mumbling to themselves about the employee-motivation benefits of single sign-on...)

> looked at the ifex spec, and quickly got lost. It would help me if you
> could write up what our motivating use cases would look like if
> implemented on top of ifex.

Yes. Understandable. Thoughts around an IFEX protocol proposal, unlike the other proposals, are still drafty (err...as a coastal verandah?) and not the clearest. However, we have much research and progress has been achieved in the odd year-or-so since beginning. The fundamental concerns of such a protocol (regarding the establishment of neutral, open and technically-viable proposals for internet-wide currency/commodity, market and financial endpoint identification) have already been reasonably met.

This has opened the door to the potential for faster progress on the
IFEX protocol itself ("a mechanism for the identification,
negotiation, description, execution and management of financial
transactions over their lifetime"), like, right about now. Which is
kind of the same zone of potential functionality (at least in a naive,
linguistic comparison sense) that people are talking about here.
Because of the hope to garner more interest from the Bitcoin community
with this post (do read on!), I spent a bunch of hours today cleaning
up and converting the IFEX Protocol's current breezy-draft form back
from the wiki formatting it had been lazing and grazing (and growing,
albeit slowly) in for the greater part of the year and moving to
Github (forks and issues very much encouraged) over here:
https://github.com/globalcitizen/ifex-protocol
  (Discussion list at http://group.ifex-project.org/ actually has
quite a few members at present, despite appearances to the contrary)

> implemented on top of ifex.

Re: "implemented on top of ifex", this is kind of opposite to how IFEX works.

IFEX's idea is to provide a flexible yet stable protocol that lets individual (potential, ongoing, or completed) transactions on arbitrary (legacy, conventional, emerging, or future) settlement systems (in arbitrary currencies/commodities) to be described and facilitated (executed, routed, monitored, etc.) in real time, by describing accurately the objective properties of each of those systems and components.

So, for example, an end user, requiring a transfer of <x> of <y> currency/commodity from 'point a' to 'point b' would find routes to achieve that, evaluate them in terms of monetary and temporal overheads against their own trust and risk models plus any legal, privacy or other requirements in order to select and effect the most appropriate manner of settlement.

In short, IFEX sees Bitcoin as having such-and-such properties, matches that to a need to transfer some funds, and effects the transfer, monitoring and/or reporting on its state in a normalized fashion throughout its lifetime.

I'll try again to describe the motivating use case: ============== Recognising that Bitcoin is not the only emerging financial community or settlement system facing real world business integration challenges, and recognising the significant complexity of these in common situations (multi-hop transactions, arbitrary currency transactions, foreign exchange automation, liquidity guarantee challenges, settlement latency negotiation, invoicing periods, commercial payment or shipping terms, sovereign (exchange rate fluctuation) and other forms of risk management, potentially simultaneous multi-level fee, tax and discount requirements, product/service coding, line items, complex tax calculations (particularly in the US, and which may be based on both buyer and seller geolocation), legal requirements to include various metadata, etc.), instead of investing valuable developer time on internal implementation (and subsequent maintenance) of a tightly-scoped (==crippled?) business-level protocol extension to Bitcoin that can be perhaps fairly characterised as unlikely to quickly evolve to meet many of these real world requirements, and with as yet unclear real world demand for such from the Bitcoin community, who must already overcome significant complexity hurdles to use Bitcoin at all, Bitcoin developers can instead simply declare this "out of scope" (win!) and focus on Bitcoin's current roles as a digital commodity and settlement system.

This approach to scope limitation has excellent historical support from the unix community - "do one thing and do it well". Bitcoin already does a few things, notably in its triple roles as network-community, currency-like-commodity, and settlement system.

The alternative is that instead of creating load for the Bitcoin developers, who may be ill-equipped and ill-resourced to properly tackle these peripheral requirements, interested parties instead contribute to projects like IFEX that view systems such as Bitcoin as core use cases but are not limited by Bitcoin developer time or project trajectory, and promise re-usability for other conventional, emerging and future currencies/commodities/settlement systems, thus retaining the capacity to attract interest and resources from those communities (and broader, internet-centric interest groups and infrastructure) toward a common goal, while creating a valuable shared platform for interoperability between Bitcoin and those other systems (including legacy financial systems) that allows Bitcoin to objectively showcase its strengths.

Net result: Bitcoin developers can focus on Bitcoin. Meanwhile, business level integration things completely tangential to the core bitcoin codebase get done with a far broader scope and applicability, providing a broader community and potential resource base for moving things forward, and presenting a less "shifting-sands" approach to potential implementers (who are safe in the knowledge that their now standardized infrastructure can support a wide range of currencies/commodities, settlement systems, financial network topologies settlement paradigms, and will not critically rely upon any given component system as a single point of failure). Bitcoin, through the platform IFEX develops, gets to compete at the business level on a fair and even basis with legacy and other emerging systems based upon its highly desirable objective properties (speed, reach, low overheads, rapid connection, lack of wacky X.509 certificate purchasing requirements... yet, etc.), such that a business case *not* to use it in various commercial settings becomes difficult to field. Everyone wins. ==============

Note that the above basically echoes much of the (less verbose? more digestible?) information available at http://ifex-project.org/ where - along with http://tools.ietf.org/ - the full text of existing proposals are also available, but attempts to do so in a more specific fashion for Bitcoin developer community.

Considering the above, and that a single system is never going to meet every person's needs all of the time (yes, even Bitcoin!), I really hope that the Bitcoin developer community will see the benefits of supporting an external rather than internal solution to business level (or at least non directly settlement-facilitating) financial transaction requirements, both for the Bitcoin project itself, its users, and for that broader and longer-term goal (in very much the same spirit) of effecting some sorely-needed, socially positive and lasting change in global financial systems. Nothing can be all things to all people (especially X.509, which can be completely different kinds of pain to all who come in to contact with it) - but at least with an open, platform neutral financial transaction oriented protocol outside of individual settlement system, currency and commodity projects, each new system can stand on its own merit and support the others, deriving shared fruits of increased user base, usability and liquidity through heterogeneous interoperability.

Sincerely, hoping to work together with interested parties to move forward in this area (come issue/fork the github repo!), and with the utmost respect for all of the valuable work of the Bitcoin community (though scratching my head a bit on the lack of an April 1 date or punchline for this X.509 stuff!!), and, and, out of breath, Walter Stanish

RE: Changing SignedInvoice's invoice field to 'bytes serialized_invoice':

Good Idea, I agree it will avoid potential issues. I think it then makes sense to pull the pki_type and pki_data into SignedInvoice, too, and specify that the signature is on the SHA256-HMAC of pki_type, pki_data, and serialized_invoice (being careful to combine them in a way that is secure).

RE: Changing Payment to include just merchant_data and not the entire Invoice:

Agreed, good idea.

RE: Mr. Stanish's suggestion to punt all of this and wait for a Grand Unified Solution:

No, we have problems that need a solution right now. And, having written one (I was the lead author of the ISO/IEC 14772-1 international standard) I'm very pessimistic about your chances for anything like IFEX to actually be adopted.

> If a Receipt is not received for any reason (timeout, error) and
> Payment.transactions has not been broadcast by the merchant on the
> Bitcoin p2p network, then the Bitcoin client should assume that the
> payment failed, inform the customer that the payment failed, and
> return coins involved in the transaction to the customer's wallet.

I'm not sure I understand the rationale for this. In the above scenario the buyer has no way to determine whether the merchant still has a copy of the transaction that they could broadcast in future. Maybe there is simply a systems problem at the merchant which has temporarily delayed the transaction broadcast. Or maybe a dishonest merchant deliberately engineered this situation in an attempt to mislead the buyer as to the status of their payment.

Either way, having the buyer think the coins have been returned to their wallet - only to disappear from their wallet again at some later time - would seriously damage user confidence in Bitcoin IMHO.

It seems to me that the first thing the buyer should do given the protocol as it stands is simply resend the Payment message - if there was a temporary problem then resending the payment message (with the same signed transation) might resolve the sitution.

If after several retries the status of the transaction is still undefined then it's really not clear what to do, but it seems desireable to have the client take steps so that it can return to a state of certainly about its wallet balance as quickly as possible. Two things I can imagine that the buyer might want their client to do at this point are:

 * broadcast the transaction itself, so they are sure the payment
   transaction will make it into the blockchain without any further
   action on their part, or

 * invalidate the transaction by immediately broadcasting a
   pay-to-self transaction that spends one or more of the same outputs
   that the payment transaction spends (and treat the funds as part of
   the unconfirmed balance until this pay-to-self transaction
   confirms).  This ensures the merchant can't subsequently use a
   transaction which the buyer thinks has failed

It seems to me it would be simpler and cleaner if the buyer just always broadcasted the transaction on the p2p network, regardless of whether the Invoice includes a receiptURI. If a receiptURI is included, the buyer's client would also include the transaction in the Purchase message. The merchant then tries to broadcast the transaction as well (unless their bitcoind has already seen it, which may well be the common case). This approach seems to me to have fewer nasty edge cases.s

roy

After doing more thinking, what about letting a spend sign more information associated with the transaction, such as a transaction ID provided by the merchant? This seems to solve a lot of the problems being put forward, with much less complexity.

Hi,

not sure if you already noticed, but I and my friends are actively working on bitcoin hardware wallet. This should be pocket size device with something like 256kB flash and 80 MHz CPU, talking with the computer over USB. User will prepare transaction on the machine, send it to the device, device shows target address on the display and user confirms it by pressing the button.

We're trying to make bitcoin payments safe even on hacked computer. For this reason we're also implementing SPV so device don't need to trust computer with any kind of information. The biggest existing problem is that user cannot be sure that the address displayed on computer screen is correct and he's confirming valid address.

I don't have any solution for this problem yet. I just appreciate an activity in payment protocol area, because it can (with some care) solve this problem and my appeal si to keep all this simple. I'd be very happy with simple payment protocol which can be implemented even on devices like I'm working on, so device with few widely used certificates stored in the memory will be able to display origin of the invoice and confirm its validity.

slush

On Thu, Nov 29, 2012 at 1:30 AM, Watson Ladd <wbl@uchicago•edu> wrote:

> After doing more thinking, what about letting a spend sign more
> information associated with the transaction, such as a transaction ID
> provided by the merchant? This seems to solve a lot of the problems being
> put forward, with much less complexity.
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> VERIFY Test and improve your parallel project with help from experts
> and peers. http://goparallel.sourceforge.net
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

RE: Roy Badami's comments on edge cases around submitting a Payment message to a merchant and then not receiving a timely response:

I agree, it is messy.

I'm hesitant to try to specify One True Way of handling it in the spec; I've got a feeling that this might be a place where different implementations might try different things, with the best implementation winning.

For example, if some future nifty-keen Bitcoin client is re-using an old Invoice to send a monthly subscription payment and they can't contact the paymentURI, then the right thing is probably for it to retry once a day for three or four days and if they all fail then give up and tell the user that the service is no longer in business (or changed their paymentURI without leaving behind a redirect).

If it has a single-use Invoice created a minute or two ago, the right
logic might be:
  + If the paymentURI is completely non-responsive, just error and
tell the user "payment failed"
  + If connected to the paymentURI and payment sent, but disconnected
before receiving a response, then try to send-to-self the coins to
cancel payment.

Again, I'm not at all sure that is the best way to handle it; implementors have the right incentives to give their users the best user experience, so I feel comfortable leaving the spec fuzzy for now.

I'd still like to understand the rationale for having the merchant broadcast the transaction - it seems to add complexity and create edge cases.

How about this as an alternative proposal:

The buyer's client prepares the transaction and computes its txid. It then sends a ValidatePurchase message to the merchant containing the proposed Outputs and a copy of the merchant_data along with the txid.

Assuming the proposed payment is accepted as valid by the merchant, the buyer's client simply broadcasts the pre-prepared transaction in the normal way, and it is the merchant's responsibility to watch for this transaction to arrive over the p2p network/blockchain to complete the purchase. (So if the merchant rejects the purchase at the ValidatePurchase stage, they never get to see the transaction that the buyer prepared, and there's therefore no need for a send-to-self to cancel it.)

An optional RequestReceipt message (perhaps containing the merchant_data and txid) can be sent by the client after the transaction has been broadcast - but by making this explicitly optional it forces the merchant to rely on seeing the bitcoin transaction to 'commit' the payment and not on the RequestReceipt message.

As far as I can see this proposal has no edge cases where the buyer and merchant have differing ideas as to whether the transaction has 'comitted' - or at least, no more edge cases than the standard bitcoin protocol has.

roy

On Thu, Nov 29, 2012 at 12:07 PM, Roy Badami <roy@gnomon•org•uk> wrote:

> I'd still like to understand the rationale for having the merchant
> broadcast the transaction - it seems to add complexity and create edge
> cases.

Mike Hearn has experimented with in-person payments using bluetooth/NFC on a phone, where the merchant has full Internet connectivity but the phone might only be able to connect to the merchant via a Bluetooth/NFC paymentURI.

I think I agree with you, though: if the device DOES have bitcoin-p2p-network-connectivity, then expecting the client to broadcast the transaction might be cleaner.

However, if a connection to the paymentURI is made and the transaction data has been sent, clients have to deal with the case where the merchant also broadcasts the transaction, no matter what the spec says and even if the merchant sends an "accepted : false" response.

> I'd still like to understand the rationale for having the merchant
> broadcast the transaction

There are several reasons for this:

1) P2P network sockets are a limited resource and bringing up connections to the network, whilst somewhat fast today, is not guaranteed to be fast in future. Passing transactions to the merchant for broadcast reduces the load on the P2P nodes because lots of thin clients aren't any longer connecting and disconnecting when sending. They only need to talk to the network when the user has received money.

2) Some users may not have network connectivity at all. For example, this happens quite often whilst traveling at Bitcoin conferences ;) The solution, which Andreas and I prototyped in Berlin together, is for the buyer to communicate only with the seller which can be done over Bluetooth or WiFi Direct or some other mobile radio protocol. Again, send only, but for the common case where you load up your wallet before setting out and then buy things, it works OK.

4) A longer term reason - in time, people may choose to not broadcast transactions at all in some cases. I think how network speed will be funded post-inflation is still an open question. Assuming the simplest arrangement where users pay fees, getting transactions into the chain has a cost. In cases where you trust the sender to not double spend on you, you may keep a fee-less transaction around "in your pocket". Then when it's your turn to pay, you use some unconfirmed transactions to do so. People pass around longer and longer chains of un-broadcast transactions until a payment crosses a trust boundary, at which point the receiver adds on their own transaction that spends back to himself but with a fee, and broadcasts them all together as a unit. In this way only people who genuinely need to fear double spends pay for security.

On Thu, Nov 29, 2012 at 06:31:24PM +0100, Mike Hearn wrote:

> > I'd still like to understand the rationale for having the merchant
> > broadcast the transaction
>
> There are several reasons for this:

[snip]

All good reasons, thanks for the explanation.

Though I still like my idea of a ValidatePurchase message that allows a buyer to ask a merchant "would you accept this payment?" without actually supplying a signed transaction. Make it optional if you care about minimising the number of round trips, e.g. for fast NFC payments.

Having such a message reduces the extent to which you need to trust the merchant not to spend a transaction that they've rejected. (And in the non-Internet connected case this is particularly useful since the client won't have the ability to broadcast a pay-to-self transaction.)

roy

Spec updated: https://gist.github.com/4120476

Changes are:

Version numbers: a couple of people asked privately about adding version numbers to the messages. In general, Protocol Buffers don't need version numbers if later versions add only optional fields.

And best-practice is to know what version of something you're expecting BEFORE you start parsing that something.

So, if a bitcoin client is getting Invoice messages via email or from
a web server, the version will be specified as part of the MIME type;
for example:
   Content-Type: application/x-bitcoin-invoice; version=1
The version= syntax is part of the MIME standard.

Following that best-practice of knowing what you're parsing before you parse it, I added an invoice_version field to the SignedInvoice message. It is now:

message SignedInvoice {
    required bytes pki_data = 1;
    required string pki_type = 2 [default = "x509"];
    required bytes serialized_invoice = 3;
    required uint32 invoice_version = 4 [default = 1];
    required bytes signature = 5;
}

Handling of receiptURI errors:

Following discussion here, I changed the spec to say:

"Clients may handle errors communicating with the receiptURI server however they like, but should assume that if they cannot communicate at all with the server then the Payment should either be retried later or immediately rejected."

and under Receipt added:

"The Bitcoin client must be prepared to handle the case of an evil merchant that returns accepted=false but broadcasts the transactions anyway."

I also added a TODO "Test Vectors" section with base64-encoded examples of everything.

The thing that bugged me most about the original spec was the sole reliance on X.509 - glad to see you've made that optional. I think many people will balk at deferring our identity trust to the existing CA's. I think it's a fine bootstrap method, but I'd really like to see another option that allows for out-of-band trust (based on ECDSA, probably).

It would also be really nice to migrate to textual representations of data structures as opposed to binary ones. The most successful internet standards are based on text, making them that much more accessible for developers to deal with them. JSON would be my preferred candidate.

Why don't we sign the text representation of a (utf8) JSON, rather than some complex encoding standard of JSON? That way the signatures are simple - and you need only retain the original textual representation of a message to validate the signature (as well as the decoded version, if you don't want to alway re-parse the message when writing programs that use it).

On Sat, Dec 1, 2012 at 11:25 AM, Gavin Andresen <gavinandresen@gmail•com>wrote:

> Spec updated: https://gist.github.com/4120476
>
> Changes are:
>
> Version numbers:  a couple of people asked privately about adding
> version numbers to the messages. In general, Protocol Buffers don't
> need version numbers if later versions add only optional fields.
>
> And best-practice is to know what version of something you're
> expecting BEFORE you start parsing that something.
>
> So, if a bitcoin client is getting Invoice messages via email or from
> a web server, the version will be specified as part of the MIME type;
> for example:
>    Content-Type: application/x-bitcoin-invoice; version=1
> The version= syntax is part of the MIME standard.
>
> Following that best-practice of knowing what you're parsing before you
> parse it, I added an invoice_version field to the SignedInvoice
> message. It is now:
>
> message SignedInvoice {
>     required bytes pki_data = 1;
>     required string pki_type = 2 [default = "x509"];
>     required bytes serialized_invoice = 3;
>     required uint32 invoice_version = 4 [default = 1];
>     required bytes signature = 5;
> }
>
>
> Handling of receiptURI errors:
>
> Following discussion here, I changed the spec to say:
>
> "Clients may handle errors communicating with the receiptURI server
> however they like, but should assume that if they cannot communicate
> at all with the server then the Payment should either be retried later
> or immediately rejected."
>
> and under Receipt added:
>
> "The Bitcoin client must be prepared to handle the case of an evil
> merchant that returns accepted=false but broadcasts the transactions
> anyway."
>
>
> I also added a TODO "Test Vectors" section with base64-encoded
> examples of everything.
>
> --
> --
> Gavin Andresen
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

On Mon, Dec 3, 2012 at 2:35 PM, Mike Koss <mike@coinlab•com> wrote:

> Why don't we sign the text representation of a (utf8) JSON, rather than some
> complex encoding standard of JSON?

Because the results from standard JSON parsers are undefined if I give you an "envelope" JSON that has repeated keys.

For example:

{
  "pki_data" : "...hex-or-base64-encoded certificate chain...",
  "signature" : "....hex-or-base64-encoded-signature-bytes",
  "message" : "....string-encoded-utf8-JSON",
  "message" : "....another string-encoded-utf8-JSON",
  "signature" : "....more hex-or-base64-encoded-signature-bytes",
  "pki_data" : "...another certificate chain...",
}

The JSON spec doesn't say what you'll get when you decode that mess. Maybe the first instance of each field, maybe the last, maybe one picked at random...

The JOSE (Javascript Signing and Encryption) spec says "Thou Shalt Use A JSON Parser That Treats Multi-defined-keys As An Error."

I expect that most developers will be lazy and will just use whatever JSON parser is convenient, no matter how much the spec/documentation warns them not to. And that makes me nervous, because I can imagine attackers taking advantage of mismatches between (say) the JSON parsing software used by some back-end server process and a front-end JavaScript web wallet UI.

> It would also be really nice to migrate to textual representations of data
> structures as opposed to binary ones.  The most successful internet
> standards are based on text.

There are lots of successful binary protocols: TCP, IP, PNG, JPEG, MP3, DNS, SSH, SSL, the Bitcoin protocol itself. What's more some other protocols that are text based have suffered serious problems due to that choice. Witness the absurd design of SMTP that means you can't start a paragraph with the word From because that's a new-message marker! Or the fact that file attachments grow by 33% when you send them. Or the various exploits that can exist in web servers thanks to header splitting attacks.

Trying to represent something binary as text doesn't make any sense. If you look at these data structures they consist of keys, signatures, hashes, certificates and other fundamentally binary things. You'd just end up base64 encoding everything anyway, at which point all you've done is design an inefficient binary protocol that masquerades as text. The disadvantages of both with the advantages of neither.

Protocol buffers have a text form that you can print to and parse from, if you so wish, though I only normally see people use that support for debug prints and sometimes because they want to load hand-written config files directly into protobuf generated objects.

On Thu, Nov 29, 2012 at 12:31 PM, Mike Hearn <mike@plan99•net> wrote:

> 4) A longer term reason - in time, people may choose to not broadcast
> transactions at all in some cases. I think how network speed will be
> funded post-inflation is still an open question. Assuming the simplest
> arrangement where users pay fees, getting transactions into the chain
> has a cost. In cases where you trust the sender to not double spend on
> you, you may keep a fee-less transaction around "in your pocket". Then
> when it's your turn to pay, you use some unconfirmed transactions to
> do so.

This brings up an additional point. If we're mutually trusting parties (or secured by some kind of external mechanism), and you've given me a payment which I haven't broadcast for confirmation— and later we make another transactions I should be able to offer you the original unconfirmed txn and ask if you'd instead be willing to write a replacement that combines both payments.

On Mon, Dec 03, 2012 at 10:28:13PM +0100, Mike Hearn wrote:

> Witness the absurd design of SMTP that means you can't
> start a paragraph with the word From because that's a new-message
> marker!

Actually that has absolutely nothing to do with SMTP. It's down to the file format of the standard BSD UNIX mailbox (which uses lines beginning with 'From ' to delimit messages).

roy

On Mon, Dec 3, 2012 at 5:26 PM, Roy Badami <roy@gnomon•org•uk> wrote:

> On Mon, Dec 03, 2012 at 10:28:13PM +0100, Mike Hearn wrote:
>> Witness the absurd design of SMTP that means you can't
>> start a paragraph with the word From because that's a new-message
>> marker!
>
> Actually that has absolutely nothing to do with SMTP.  It's down to
> the file format of the standard BSD UNIX mailbox (which uses lines
> beginning with 'From ' to delimit messages).

His point (pun intended) stands. SMTP uses "." terminator, which also causes annoying confusion.

You shouldn't need to escape and unescape data that is not being interpreted in any way.

On Mon, Dec 03, 2012 at 05:34:12PM -0500, Jeff Garzik wrote:

> You shouldn't need to escape and unescape data that is not being
> interpreted in any way.

Funilly enough pretty much all low-level links that make up the Internet use either bit-stuffing or byte-stuffing to escape a particular bit sequence or byte that terminates an HDLC frame.

I'm not particularly agreeing or disagreeing with you on the suitability for the case at hand, but as an absolute your statement doesn't hold water. The use of a terminator for a variable-length data structure rather than a length prefix is a design desicion that has little-to-nothing to do with the debate of text-versus-binary.

Anyone remember Holerith constants?

roy

> So, if a bitcoin client is getting Invoice messages via email or from
> a web server, the version will be specified as part of the MIME type;
> for example:
>    Content-Type: application/x-bitcoin-invoice; version=1
> The version= syntax is part of the MIME standard.

I think that's OK. However, you should only be getting the version you expect because when you request an invoice, your client should be telling the merchant what protocol version you implement.

Does it make sense to have this spec not include the details of bootstrapping? It's not complicated - we extend the URI spec in a backwards compatible way:

   bitcoin:1AbCdEfG?value=10.0&label=Pay%20for%20Foo&invoice=https://merchant.com/inv/aB425az

When a compatible client sees the invoice param, it ignores the rest of the URI and downloads the URL https://merchant.com/inv/aB425az?ver=1.0

A server on merchant.com sees that the client expects a version 1.0 invoice and vends it. If ver=2.0 or whatever, it knows it can use 2.0 features. If extensions are supported, add new query params.

We should define a simple mechanism for extending the protocol now, so people who want to make proprietary extensions don't conflict. The simplest is to just say, if you want to add new fields to an Invoice message, please update a wiki page with the tag numbers you're going to use, and start from number X. Protobufs have a simple way to formalize this in the language:

   https://developers.google.com/protocol-buffers/docs/proto#extensions

message Invoice {
  extensions 1000 to max;
}

The point of this is to allow you to define new parts of the messages in separate .proto files. It's only a minor convenience but it means if you want to use, say, two extensions that weren't yet folded into the main spec, you can more easily do so without having to do a manual merge of the message definitions together.

For instance, if you wanted to extend the protocol to support specification of recurring billing, you could make a file called recurring-invoices.proto containing:

message Recurrences {
  required uint32 every_seconds = 1;
  optional uint32 start_time = 2;
}

extend Invoice {
  optional Recurrences recurrences = 1005;
}

then you update the wiki page to claim tag number 1005 and apps can easily use your new features. If/when the feature gets standardized via a BIP, the core .proto definition can be extended to include these messages and the extensions can go away.

I've had some push-back on the names of the proposed messages-- e.g. "Invoice" in the accounting world means "I've already given you a product or service, here is what you owe, payment terms, what forms of payment are accepted, etc."

I think there might also be confusion about why we're defining our own Invoice when there are at least three or four other existing standard for electronic invoices.

So unless there is strong objection I'm going to change the names of the messages:

Invoice --> PaymentRequest Payment : ok as-is Receipt --> PaymentACK (payment acknowledgement)

On Tue, Dec 4, 2012 at 12:06 PM, Mike Hearn <mike@plan99•net> wrote:

> Does it make sense to have this spec not include the details of
> bootstrapping? It's not complicated ....

BIP 0001 says: "If in doubt, split your BIP into several well-focussed ones."

I think it makes sense to keep the URI extension separate from the binary message format.

> We should define a simple mechanism for extending the protocol now...
>
> message Invoice {
>   extensions 1000 to max;
> }

Ok.

During/before the Payment Request there should be a method to exchange the public keys to be able to generate a common multisig address. Should this be handled in a different protocol, or be included in this spec? Or is there a method for the customer to verify that the specified BIP16 Output contains his address and the one from an escrow service?

Escrow/multisig is complicated enough to wait for another day. But certainly having a payment protocol is an important step towards it On 6 Dec 2012 07:32, "Andreas Petersson" <andreas@petersson•at> wrote:

> During/before the Payment Request there should be a method to exchange
> the public keys to be able to generate a common multisig address.
> Should this be handled in a different protocol, or be included in this
> spec?
> Or is there a method for the customer to verify that the specified BIP16
> Output contains his address and the one from an escrow service?
>
> --
> Andreas
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Spec updated yet again:
  https://gist.github.com/4120476

Renamed to PaymentRequest/PaymentACK.

Added a 'network' field ("main" or "test") to PaymentRequest so testnet and main network (and alterna-chain) payment requests don't get confused.

Updated description of PaymentRequest.outputs:

outputs: one or more outputs where Bitcoins are to be sent. If the sum of outputs.amount is zero, the customer will be asked how much to pay, and the bitcoin client may choose any or all of the Outputs (if there are more than one) for payment. If the sum of outputs.amount is non-zero, then the customer will be asked to pay the sum, and the payment shall be split among the Outputs with non-zero amounts (if there are more than one; Outputs with zero amounts shall be ignored). -------------

RE: escrow/multisig:

Setting up a multi-person escrow will, I think, need it's own set of messages. I think we should leave that for a future spec.

Thumbnail sketch: escrow service or participant sends around an EscrowProposal, gets EscrowProposalACK's with public keys to use, then sends all participants an EscrowEstablished message with the final multisig script or address. Escrow gets funded by any/all of the participants, and then gets spent using the SignedPaymentRequest/Payment/PaymentACK protocol-- participants will pass around a SignedPaymentRequest and a partially-signed Payment message for all to approve.

When I say "pass around" I'm not thinking of users copying and pasting, that would be a terrible user experience; all of that communication needs to happen automatically behind the scenes. Lets tackle that after we've got the simpler customer-pays-merchant flow working nicely (funded-escrow-pays-merchant is a subset of that, anyway).

Re: the newest spec. Rather than make the signature over the "concatenation of", why not just make it a signature over the serialized protobuf minus the signature field (as I did in my demo code). Otherwise it seems like we'd need more code than really necessary. We can state explicitly tags must be ordered if you want, even though all implementations should do that already.

> Thumbnail sketch:  escrow service or participant sends around an
> EscrowProposal, gets EscrowProposalACK's with public keys to use, then sends
> all participants an EscrowEstablished message with the final multisig script
> or address.

Yeah, that sounds reasonable. Not that we should really design it now, but let's use the term "mediation" rather than "escrow", which has connotations of depositing funds with the escrow service.

I think it's best to see the existing payment messages as structures that'll get filled out with more features over time. So rather than have a separate EscrowProposal message, you would integrate it with payment requests. Older clients that don't understand mediation would just ignore the extra data they don't recognize.

message PaymentRequest {
   ....

  // One per mediator acceptable to the seller.
  repeated MediationProposal mediation_data = 10;
}

message MediationProposal {
  required SignedMediatorIdentity identity = 2;

  // Opaque bytes that the mediator can be asked to turn into a human
readable description
  // of how disputes will be mediated. The merchant sets this to
describe whatever policy it
  // is willing to go along with, so policies may be arbitrarily complicated.
  required bytes policy = 3;
}

message SignedMediatorIdentity {
  required MediatorIdentity identity = 1;
  // If the identity data is signed ...
  optional string pki_type = 2;
  optional bytes pki_data = 3;
  optional bytes pki_signature = 4;
}

message MediatorIdentity {
  // Name of the mediator to be displayed to the user.
  required string friendly_name = 1;
  // PNG image that can be used to represent the mediator to the user.
  optional bytes logo = 2;
  // Some text shown to the user under the name explaining the
mediators policies, why they should be chosen, etc.
  optional string blurb = 3;

  // An HTTP URL where a mediator can be reached to do things like
prove ownership of pubkeys, initiate the protocols, etc.
  required string contact_url = 4;
}

... etc ....

So the user experience would be that when a payment request is received:

- older clients ignore the mediation_data field and do a direct payment as normal - newer clients ask the user to pick a mediator (if they want to) and if mediation is requested, the PaymentRequest is then discarded and the next step of the mediation protocol begins.

The old request has to be discarded because the outputs would have been written on the assumption of no mediation being in use (for backwards compatibility).

Anyway, though I'm awfully guilty, let's not get off track. Just that this is how I imagined new payment features being done - as new extensions to the payment protocol, which would be a living document amended by BIPs.

On Thu, Dec 6, 2012 at 11:56 AM, Gavin Andresen <gavinandresen@gmail•com>wrote:

> When I say "pass around" I'm not thinking of users copying and pasting,
> that would be a terrible user experience; all of that communication needs
> to happen automatically behind the scenes. Lets tackle that after we've got
> the simpler customer-pays-merchant flow working nicely
> (funded-escrow-pays-merchant is a subset of that, anyway).

I think that the "pass around" method needs to happen in addition to the methods of transparent protocols that occur behind the scenes. For one, there's a lot of CONOPs that need to be worked out by getting knowledgeable people using it, and providing feedback about how it could/should/will be used and how it could be improved. The pass-around method is simpler to implement and still usable by the types of users that will be using it in the beginning -- experts. Also, I see that for very large, important multi-sig tx/contracts/escrow, the "manual" method might be preferred -- much the same way many people prefer manual-transmission cars even though automatics are "easier" -- some people/organizations will want the control.

I'm all for protocols that enable higher-level access to this functionality, I'm just saying there should be lower-level access, too.

On Thu, Dec 6, 2012 at 12:55 PM, Mike Hearn <mike@plan99•net> wrote:

> Re: the newest spec. Rather than make the signature over the
> "concatenation of", why not just make it a signature over the
> serialized protobuf minus the signature field (as I did in my demo
> code). Otherwise it seems like we'd need more code than really
> necessary. We can state explicitly tags must be ordered if you want,
> even though all implementations should do that already.

OK. I want to keep the signature field required, though, so how about:

signature: digital signature over a protocol buffer serialized variation of the SignedPaymentRequest message where signature is a zero-byte array and fields are serialized in numerical order (all current protocol buffer implementations serialize fields in numerical order), using the public key in pki_data.

> OK. I want to keep the signature field required, though, so how about:
>
> signature: digital signature over a protocol buffer serialized variation of
> the SignedPaymentRequest message where signature is a zero-byte array and
> fields are serialized in numerical order (all current protocol buffer
> implementations serialize fields in numerical order), using the public key
> in pki_data.

Looks good to me.

Yet more comments (I guess at some point we need to stick a fork in it - or at least move on to implementing a prototype version).

Maybe don't require the payment URI to be HTTPS. If you want to pay a Tor hidden service then HTTPS just adds unnecessary complexity. Just recommend to merchants that they use an encrypted connection and leave it at that.

Though it's not strictly necessary, it'd be nice to have defined behavior for if you want to pay more than the requested amount, for a tip. Perhaps rather than "zero value outputs will be ignored" say, "if some outputs have value and others don't, the user will be given the option of overpaying and the extra money will be split evenly between the zero valued outputs". That way a waitress can have the phone add a zero-valued output to her own wallet and that would prompt the wallet software to display some convenient UI for adding on 10% or whatever.

receiptURI -> receipt_url ? technically it has to be resolvable so "uri" isn't quite right.

"Display the proposed Outputs in as human-friendly a form as possible" .... ??? Surely you'd just display the total amount requested? I don't think it ever makes sense to try and display outputs to the user directly.

Re: the UI TODO - agreed but let's take it out of the BIP and maybe make it an alternative document. Or just replace it with a recommendation that "the user interface should be designed to ensure users understand the difference between an unsigned and signed payment request, for best practices see <here>"

serialized_paymentrequest -> serialized_payment_request? Otherwise languages that use CamelCase will look odd .... setSerializedPaymentrequest()

The question of root CAs still needs resolution. I stick with my recommendation to support all CAs that browsers support. Obviously, it's better for a merchant to obtain an EV cert than a domain verification cert - the UI can reflect the higher level of verification.

I doubt there's a need to specify a max number of certs in a chain. But if you want to, go for something high, like 256. There's no point in trying to put DoS mitigations into something like this, for the same reason HTML doesn't impose a maximum page size. It's in the message builders interest to ensure it gets read by all users. Crashing their clients doesn't achieve anything as long as the crash isn't exploitable.

On Fri, Dec 7, 2012 at 11:45 AM, Mike Hearn <mike@plan99•net> wrote:

>> OK. I want to keep the signature field required, though, so how about:
>>
>> signature: digital signature over a protocol buffer serialized variation of
>> the SignedPaymentRequest message where signature is a zero-byte array and
>> fields are serialized in numerical order (all current protocol buffer
>> implementations serialize fields in numerical order), using the public key
>> in pki_data.
>
> Looks good to me.

On Fri, Dec 7, 2012 at 6:01 AM, Mike Hearn <mike@plan99•net> wrote:

> Yet more comments (I guess at some point we need to stick a fork in it
> - or at least move on to implementing a prototype version).
>

Yes, my next step is prototyping.

Note that this is not a BIP yet: I want to have a working implementation before making this an Official BIP.

> Maybe don't require the payment URI to be HTTPS.

Changed:
   receipt_url: Secure (usually https) location where...

Though it's not strictly necessary, it'd be nice to have defined

> behavior for if you want to pay more than the requested amount, for a
> tip.

yeah... I had similar thoughts on what to do if some Outputs specify an amount and others don't. I'm still waffling on whether or not I like allowing repeated Outputs; a single Output would make the spec a fair bit simpler, and if a merchant wants to split up a payment for some reason they could just generate another transaction.

I want to move on to actually implementing this before creating complicated rules. Maybe the best way to tip a waitress is to get two separate PaymentRequests, one for the restaurant and one that goes directly to the waitress (depends on whether or not the restaurant needs or wants to know how much their employees are getting tipped, I suppose). Maybe it would be best to have a separate "gratuity" Output in the PaymentRequest. That's the kind of detail I think doesn't need to be worked out right now, I'd rather restaurants tell us what they need/want.

> "Display the proposed Outputs in as human-friendly a form as possible"
> .... ??? Surely you'd just display the total amount requested? I don't
> think it ever makes sense to try and display outputs to the user
> directly.
>

This is the case of getting an UNSIGNED payment request; I've changed the wording a little to make that more clear.

If a bitcoin client accepts unsigned payment requests (a couple of people have asked if that would be possible so I think that is desired), then it doesn't have the payer's identity-- all it has is the Outputs that will be paid.

> Re: the UI TODO - agreed but let's take it out of the BIP...

Not a BIP yet....

serialized_paymentrequest -> serialized_payment_request?

Done.

> The question of root CAs still needs resolution.  I stick with
> my recommendation to support all CAs that browsers support.

I still like the idea of only including the root CAs who have jumped through the hoops needed to get the "allowed to issue EV certs" blessing. I'm not suggesting that all bitcoin merchants must get EV certs, but I am suggesting that they must get a certificate from one of the most reputable certificate authorities, and the ability to issue EV certificates is, I think, a good proxy for that.

But, again: Not a BIP yet. Lets get something implemented and then hammer out details (implementing always turns up edge cases you forgot when spec'ing).

> yeah... I had similar thoughts on what to do if some Outputs specify an
> amount and others don't. I'm still waffling on whether or not I like
> allowing repeated Outputs; a single Output would make the spec a fair bit
> simpler

Yes, but at the cost of privacy. Generators of payment requests always have the option of just adding a single output and being done with it. But in future they'll probably want to keep their income in unlinkable chunks of a size that's up to them, and multi-outputs are needed for this (the idea being, the users wallet tries to keep a close-as-possible match between the requested outputs and their own).

OK, let's punt on tipping for now.

> If a bitcoin client accepts unsigned payment requests (a couple of people
> have asked if that would be possible so I think that is desired), then it
> doesn't have the payer's identity-- all it has is the Outputs that will be
> paid.

I see. If I were to implement a wallet I'd just display nothing (except the size of the request). Showing an address doesn't really help the user in any way.

> I still like the idea of only including the root CAs who have jumped through
> the hoops needed to get the "allowed to issue EV certs" blessing.

The hoops only actually apply for EV certs though, they aren't required to do that verification for DV certs.

The main reason to use the browser root CAs is that merchants are guaranteed to be able to re-use their existing certs. Otherwise they might have to buy new ones, which would be annoying.

On 3 December 2012 20:35, Mike Koss <mike@coinlab•com> wrote:

> The thing that bugged me most about the original spec was the sole
> reliance on X.509 - glad to see you've made that optional.  I think many
> people will balk at deferring our identity trust to the existing CA's.  I
> think it's a fine bootstrap method, but I'd really like to see another
> option that allows for out-of-band trust (based on ECDSA, probably).
>
> It would also be really nice to migrate to textual representations of data
> structures as opposed to binary ones.  The most successful internet
> standards are based on text, making them that much more accessible for
> developers to deal with them.   JSON would be my preferred candidate.
>
> Why don't we sign the text representation of a (utf8) JSON, rather than
> some complex encoding standard of JSON?  That way the signatures are simple
> - and you need only retain the original textual representation of a message
> to validate the signature (as well as the decoded version, if you don't
> want to alway re-parse the message when writing programs that use it).
>

Binary formats can be challenging to deal with and convert to other formats. The experiences in the PKI world of ASN.1 have not been great, in terms of interop. It tends to create islands and silos. This is probably one of the reasons why X.509 and GPG are fragmented and why we dont really have a widely deployed web of trust on the net. Another reason is simply lack of developer resources to make tools. In that respect I think JSON offers significant advantages, though I am interested in the security issues raised.

>
> On Sat, Dec 1, 2012 at 11:25 AM, Gavin Andresen <gavinandresen@gmail•com>wrote:
>
>> Spec updated: https://gist.github.com/4120476
>>
>> Changes are:
>>
>> Version numbers:  a couple of people asked privately about adding
>> version numbers to the messages. In general, Protocol Buffers don't
>> need version numbers if later versions add only optional fields.
>>
>> And best-practice is to know what version of something you're
>> expecting BEFORE you start parsing that something.
>>
>> So, if a bitcoin client is getting Invoice messages via email or from
>> a web server, the version will be specified as part of the MIME type;
>> for example:
>>    Content-Type: application/x-bitcoin-invoice; version=1
>> The version= syntax is part of the MIME standard.
>>
>> Following that best-practice of knowing what you're parsing before you
>> parse it, I added an invoice_version field to the SignedInvoice
>> message. It is now:
>>
>> message SignedInvoice {
>>     required bytes pki_data = 1;
>>     required string pki_type = 2 [default = "x509"];
>>     required bytes serialized_invoice = 3;
>>     required uint32 invoice_version = 4 [default = 1];
>>     required bytes signature = 5;
>> }
>>
>>
>> Handling of receiptURI errors:
>>
>> Following discussion here, I changed the spec to say:
>>
>> "Clients may handle errors communicating with the receiptURI server
>> however they like, but should assume that if they cannot communicate
>> at all with the server then the Payment should either be retried later
>> or immediately rejected."
>>
>> and under Receipt added:
>>
>> "The Bitcoin client must be prepared to handle the case of an evil
>> merchant that returns accepted=false but broadcasts the transactions
>> anyway."
>>
>>
>> I also added a TODO "Test Vectors" section with base64-encoded
>> examples of everything.
>>
>> --
>> --
>> Gavin Andresen
>>
>>
>> ------------------------------------------------------------------------------
>> Keep yourself connected to Go Parallel:
>> INSIGHTS What's next for parallel hardware, programming and related areas?
>> Interviews and blogs by thought leaders keep you ahead of the curve.
>> http://goparallel.sourceforge.net
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists•sourceforge•net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>
>
> --
> Mike Koss
> CTO, CoinLab
> (425) 246-7701 (m)
>
> A Bitcoin Primer <http://coinlab.com/a-bitcoin-primer.pdf> - What you
> need to know about Bitcoins.
>
>
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> BUILD Helping you discover the best ways to construct your parallel
> projects.
> http://goparallel.sourceforge.net
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>

On Sun, Dec 16, 2012 at 4:15 PM, Melvin Carvalho <melvincarvalho@gmail•com> wrote:

> On 3 December 2012 20:35, Mike Koss <mike@coinlab•com> wrote:
>> It would also be really nice to migrate to textual representations of data
>> structures as opposed to binary ones.  The most successful internet
>> standards are based on text, making them that much more accessible for
>> developers to deal with them.   JSON would be my preferred candidate.
>>
>> Why don't we sign the text representation of a (utf8) JSON, rather than
>> some complex encoding standard of JSON?  That way the signatures are simple
>> - and you need only retain the original textual representation of a message
>> to validate the signature (as well as the decoded version, if you don't want
>> to alway re-parse the message when writing programs that use it).

> Binary formats can be challenging to deal with and convert to other formats.
> The experiences in the PKI world of ASN.1 have not been great, in terms of
> interop.  It tends to create islands and silos.  This is probably one of the
> reasons why X.509 and GPG are fragmented and why we dont really have a
> widely deployed web of trust on the net.  Another reason is simply lack of
> developer resources to make tools.  In that respect I think JSON offers
> significant advantages, though I am interested in the security issues
> raised.

I thought this had already been covered up-thread?

When creating something that must be hashed and/or compared, the data structure must be created and reproduced precisely, byte-for-byte. JSON offers significant -disadvantages- in this regard. With JSON, you would therefore require an additional middle layer, between JSON and application, ensuring that all fields are output in the same order, all whitespace is not only perfectly preserved -- but reliably generates identical whitespace output for identical inputs, given two separate JSON implementations.

On 17 December 2012 03:18, Jeff Garzik <jgarzik@exmulti•com> wrote:

> On Sun, Dec 16, 2012 at 4:15 PM, Melvin Carvalho
> <melvincarvalho@gmail•com> wrote:
> > On 3 December 2012 20:35, Mike Koss <mike@coinlab•com> wrote:
> >> It would also be really nice to migrate to textual representations of
> data
> >> structures as opposed to binary ones.  The most successful internet
> >> standards are based on text, making them that much more accessible for
> >> developers to deal with them.   JSON would be my preferred candidate.
> >>
> >> Why don't we sign the text representation of a (utf8) JSON, rather than
> >> some complex encoding standard of JSON?  That way the signatures are
> simple
> >> - and you need only retain the original textual representation of a
> message
> >> to validate the signature (as well as the decoded version, if you don't
> want
> >> to alway re-parse the message when writing programs that use it).
>
> > Binary formats can be challenging to deal with and convert to other
> formats.
> > The experiences in the PKI world of ASN.1 have not been great, in terms
> of
> > interop.  It tends to create islands and silos.  This is probably one of
> the
> > reasons why X.509 and GPG are fragmented and why we dont really have a
> > widely deployed web of trust on the net.  Another reason is simply lack
> of
> > developer resources to make tools.  In that respect I think JSON offers
> > significant advantages, though I am interested in the security issues
> > raised.
>
> I thought this had already been covered up-thread?
>
> When creating something that must be hashed and/or compared, the data
> structure must be created and reproduced precisely, byte-for-byte.
> JSON offers significant -disadvantages- in this regard.  With JSON,
> you would therefore require an additional middle layer, between JSON
> and application, ensuring that all fields are output in the same
> order, all whitespace is not only perfectly preserved -- but reliably
> generates identical whitespace output for identical inputs, given two
> separate JSON implementations.
>

Apologies if I am a bit late to the thread. I bumped into someone that suggested I take a look at it. Will try and catch up!

You raise a good point.

Is there no good canonicalization algorithm / library for JSON?

I think that provided that each JSON object has an identifier, canonicalization of JSON is not that hard.

Then when you hash or sign the canonical form they can be compared reliably.

>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti•com
>

Can we please drop the binary vs text issue? We have been around it millions of times already. There are no compelling arguments to use text here and several obvious problems with it. If you think you've found a good argument to use JSON, please research protocol buffers more thoroughly and see if it changes your mind.

I've been following this thread closely, and Mike is correct here - protocol buffers is definitely the way to go.

On 17 December 2012 09:19, Mike Hearn <mike@plan99•net> wrote:

> Can we please drop the binary vs text issue? We have been around it
> millions of times already. There are no compelling arguments to use
> text here and several obvious problems with it. If you think you've
> found a good argument to use JSON, please research protocol buffers
> more thoroughly and see if it changes your mind.
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

On 17 December 2012 10:19, Mike Hearn <mike@plan99•net> wrote:

> Can we please drop the binary vs text issue? We have been around it
> millions of times already. There are no compelling arguments to use
> text here and several obvious problems with it. If you think you've
> found a good argument to use JSON, please research protocol buffers
> more thoroughly and see if it changes your mind.
>

Hi Mike, thanks you for the pointer. I have read up on Protocol Buffers.

If the decision has already been made, then let's go with that, but if not perhaps I can offer some comments.

Looking at:

http://en.wikipedia.org/wiki/Comparison_of_data_serialization_formats

And -- "Canonically, Protocol Buffers are serialized into a binary wire format which is compact, forwards-compatible, backwards-compatible, but not self-describing"

I can see there are advantages in this approach in that you can send messages quickly and with low bandwidth. However the non self describing data means that it's significantly harder to convert from one format to another. Also references are important, and can be achieved in JSON.

Yet in my opinion there is great advantage to growing the bitcoin ecosystem to interoperate with the whole net, kind of creating a complete web economy. The way to do this is to foster interoperability. Having looked at and worked with standards for the past 5-10 years that is the great challenge. Every system works in an island, and few talk to any others. However, a market based economy grows exponentially more valuable with extra liquidity.

Inventing yet another format may lead to balkanization. If history is a judge, the chances are high. A self describing JSON format, however is much more likely to interop.

I can understand the hesitation with JOSE. However, if you get a moment, please look at :

http://payswarm.com/specs/source/web-keys/

This should provide some of the tools that you need.

As I said above, if the matter is closed, that's fine and thanks for taking the time to read.

Can I at least propose to make it mandatory for the binary format to have a translation script to a self describing JSON format and back again. I would love to see the bitcoin ecosystem become a major part of the infrastructure of the web itself (leading to even nice things like a proper web of trust), as well as an awesome P2P system in its own right.

On Mon, Dec 17, 2012 at 6:23 AM, Melvin Carvalho <melvincarvalho@gmail•com>wrote:

> If the decision has already been made, then let's go with that
>
>

The decision has already been made.

Here are my (mostly half baked) thoughts on the payments protocol proposal.

My first observation is that the proposal is too heavily oriented around a merchant/customer interaction. I think it's equally important to consider the person to person scenarios. It would be very cool if people could send/receive payments by copying and pasting stuff on facebook or email (you can kind of do it now, but it's not safe unless you go to extraordinary lengths using PGP signatures and the like).

Protobufs vs JSON: Protobufs are fine, although I will mention that the serialization/JOSE arguments are irrelevant...you only need that if you need a reliable way of signing an in memory object structure...in this case you would be signing a serialized form of the object...the recipient doesn't have to be able to reproduce the serialized form, they only need to verify the signature on the already serialized bytes...I see protobufs as a good serialization format for storage, while JSON being more practical for communications in a web oriented environment...with protobufs & a web wallet, you may find yourself in a situation needing to parse a protobuf message in a web browser...the protobuf parsing and serializing code is just going to add bloat to the web page...personally, I probably would have gone with JSON, but hey, I'm not writing the code.

X.509 - nasty, but maybe ok ...as long as you can add root CAs to your Bitcoin client or explicitly trust a certificate, I don't see that it poses any privacy issues...but there are some other things to think about here ...like what about the casual user that wants to create a payment request to send to their friend over email (wrapped in a clear text block similar to PGP...it could also be sent as a file attachment)? Are you now requiring them to go and setup a certificate? Btw, I really like the use of a payment request in this manner because you have a signed payment request that can be verified against an address book of known identities. This could be much safer than simply emailing an unsigned bitcoin address around.

Refund addresses...this is not going to be as useful as people might think...most refunds that bitpay needs to process happen days or even months after the initial purchase...in that span of time, people can change wallets, rendering such a refund address useless...so, as I think about the situation, we would still need to contact the buyer to confirm a refund address anyway. What we really need is to verify the identity of the person we're potentially sending the refund to...we need a way of determining that the person we're sending the refund to is the same person that paid the original invoice. Bitcoin addresses are identities, but they are too low level. HD wallets come to mind...the top level or intermediate levels of a deterministic hierarchy could be used for identity purposes...but it also seems like it might be conflating payments and identity (which for many reasons you might want to keep separate). What if bitcoin clients could manage one or more identities used for the purpose of communications? You could have a bitcoin identity file that could be used by multiple wallets. These identities would be used for signing messages and verifying the authenticity of communications...when sending a payment, instead of a refund address, you would include one of these identities which could later be used to confirm a refund address. In fact, the refund would be processed by the buyer generating another payment request message signed by the identity used in the original payment.

People would understand that their identities are important for communications and they would keep those even when changing to new wallets and such (identities could be stored in ~/.bitcoin/id or something (encrypted of course)).

There are some other interesting possibilities if messaging and identities are done right...for example, I could add "check" feature (analogous to paper checks). It would work like this...you create a transaction that spends to a newly generated address...you put that transaction, along with the private key into an encrypted container (sent to the identity of the person you want to pay). The recipient can open it and their wallet would go ahead and generate and broadcast a transaction moving the funds into their wallet (optionally including a fee). But, if the recipient never cashes the check, the sender could pull those funds back after a certain period of time. This also eliminates the possibility of accidentally sending the funds to the wrong address (or an old address) and the bitcoins being forever lost...the recipient can sweep the transaction into any wallet of their choice.

As I'm writing this, I'm beginning to wonder if the identity management problem is unavoidable. Maybe that needs to be dealt with first. It would enable so many other interesting possibilities.

I like the use of merchant_data...this means that you no longer will need a unique bitcoin address for every invoice.

In the signed invoice structure, why embed the serialized invoice? Why not make that a reference using a hash? Generally speaking, I'm not a fan of embedding things like that. You could have an over-arching structure called Message which is just "repeated bytes objects" (in protobuf lingo) ...references between objects would use a hash and the first object would be treated as the message. In a payment request message, the SignedPaymentRequest would be the first object, the PaymentRequest the second.

I think the Payment structure should refer to the SignedPaymentRequest (by its hash) instead of the merchant_data...you can of course access the merchant_data through the SignedPaymentRequest. I suppose you could always index payment requests based on the merchant_data, but it just seems cleaner to refer back the the signed payment request when sending a payment.

You might want to include an optional memo for each output...I could imagine including one output that says "Don't forget to tip your waiter"...any amount sent to that address could go directly to the waiter's wallet.

What about payments from multiple wallets? We see this a lot. I think this scheme would handle it ok, but just want to mention it. I can imagine someone paying first from one wallet, then the invoice webpage updates with a clickable link to a new PaymentRequest for the remaining amount.

The Receipt should be signed...it could be used as proof of payment by wallets.

Finally, I've seen seen suggestions to tack on a payment request URI to the current bitcoin: URI for backward compatibility...I say no. A bitcoin URI already has a lot of data (especially if it includes a memo)...this makes QR codes more dense and hence more difficult to scan...I say we stake a claim on the "pay" URI .... pay:https://somewhere.com/payment/94kd83 ...or for a clickable link, you could embed it right on the web page, eliminating the need for a second https request... pay:data:<PaymentRequestMessage> ...and finally, to further shorten the URI, https could be assumed if the protocol is omitted ... pay:somewhere.com/payment/94kd83

We can deal with backward compatibility by including a link on invoices to display an old style bitcoin payment address.

Thanks for the thoughts. For those who don't know, Stephen works for BitPay.

> My first observation is that the proposal is too heavily oriented around a
> merchant/customer interaction.

The term "merchant" is just being used to mean the entity requesting the payment. I'm hopeful that in future mobile wallets will be creating these messages where today they'd make URLs and QRcodes. So it's definitely intended for person to person payments too.

> you may find yourself in a situation needing to parse a protobuf
> message in a web browser

Nothing stops you converting them into whatever form you want on the server side. If you don't care about the signature checking then it's no problem to use a server. If you do then you'd need to ship all the code for verifying signatures that to the client anyway, at which point a small protobuf parser is hardly a deal killer.

> ...like what about the casual user that wants to create a payment request to
> send to their friend over email

They can send an unsigned payment request. Note that if you mail it as an attachment from a competent, up to date email provider then the attachment isn't really unsigned. The whole thing is covered by the emails DKIM signature which is applied transparently by the ESP. If the signature fails to verify then the mail client can show that or treat the mail differently (as Gmail does). This is easy to use for the end user - they don't have to think about cryptography or PKI. As long as their email account is secure then they can send signed mails asserting to their identity.

> Refund addresses...this is not going to be as useful as people might
> think...most refunds that bitpay needs to process happen days or even months
> after the initial purchase

Useful feedback, thanks. Still, there may be other types of merchants for whom it's useful, and many users won't change their wallet. It certainly simplifies things if you can present the refund address and give a one-click option to use it. If the user wants to use a different address, then they can go onto the slow/complicated path.

This current spec deliberately punts on the topic of identifying end users. It's a difficult problem. Whereas many merchants have SSL certs, most end users don't have published keys in any useful form. By far the easiest way for 99% of people to generate a signed message is to send email that's signed by DKIM (from gmail, hotmail, yahoo, other providers etc). Then it's all transparent and behind the scenes. Their identity is their email address.

So for BitPays scenario, you could require an email to be sent by the end user containing new instructions. Your MTA can show you whether the mail is correctly DKIM signed or not when deciding whether to follow the instructions.

> But, if the recipient never cashes the check, the sender could pull those
> funds back after a certain period of time.

Yeah, I like the term "check"/"cheque" for that concept of a reversible payment.

> I like the use of merchant_data...this means that you no longer will need a
> unique bitcoin address for every invoice.

It's still a good idea to use one for privacy reasons. The merchant data is there so you can stuff whatever state you want into it. So it's like cookies. You don't have to keep state on the server side. Just encrypt/sign it, put it in the invoice, and when you get a payment message back there's no need to do database lookups or anything, you can just do some crypto and know who is submitting it.

> Generally speaking, I'm not a fan of embedding things like that

What's wrong with it? Isn't your proposal more complex? I don't see why it's better than just embedding it.

> The Receipt should be signed...it could be used as proof of payment by
> wallets.

There's no Receipt message, a SignedPaymentRequest + transactions that pay to the requested outputs are together the proof of payment.

Re a new URI scheme. Interesting idea, thanks for the suggestion. It seemed like it'd be easier for merchants to integrate if a single linked worked for all wallet implementations/versions. But I guess we could do both schemes, even.

On Thu, Dec 20, 2012 at 12:43 PM, Mike Hearn <mike@plan99•net> wrote:

> > you may find yourself in a situation needing to parse a protobuf
> > message in a web browser
> Nothing stops you converting them into whatever form you want on the
> server side. If you don't care about the signature checking then it's
> no problem to use a server. If you do then you'd need to ship all the
> code for verifying signatures that to the client anyway, at which
> point a small protobuf parser is hardly a deal killer.

No, it's not a killer...just a hassle. JSON is convenient and ubiquitous and there is something to be said for that (and I wanted to point out that the JOSE objection was invalid). Protobufs are nice and efficient, but who cares. You're talking about direct communications rather than something that will be bounced around every node in the mesh network. I don't really care much either way, it's not worth debating. I'm just thankful no one is arguing for XML or IIOP. :)

> ...like what about the casual user that wants to create a payment request
> to
> > send to their friend over email
>
> They can send an unsigned payment request. Note that if you mail it as
> an attachment from a competent, up to date email provider then the
> attachment isn't really unsigned. The whole thing is covered by the
> emails DKIM signature which is applied transparently by the ESP. If
> the signature fails to verify then the mail client can show that or
> treat the mail differently (as Gmail does). This is easy to use for
> the end user - they don't have to think about cryptography or PKI. As
> long as their email account is secure then they can send signed mails
> asserting to their identity.
>

This leaves too much to chance for my taste. Forget email, what about jabber, ICQ, skype, IRC? Email is just one communications medium, there are many others for which there would be no assurance that the payment request hasn't been tampered with. You could at a minimum allow a person to create a normal ECC key, but have it used as an identity in communications rather than a payment address. You store it in a separate file in ~/.bitcoin/id ...you don't have to solve the whole set of PKI problems, people could exchange identities using any secure channel they are comfortable with (email + phone verification of a short hash id would be sufficient). In another scenario, an id could be made available over https, using the normal SSL certificate and CA infrastructure to verify authenticity. This way all messages could be signed and/or encrypted without the user having to go out of their way to use external tools or infrastructure that is often not very user friendly. You also need encryption for the "cheque" feature...asking people to use GPG would be too much of a burden (and email DKIM doesn't offer encryption).

>>> wandering off topic >>>

Indeed, "cheques" could become the dominant method of person to person payments...first, you would obtain someone's id, which you might already have on file (rather than obtaining a bitcoin address), then you would generate a "cheque" for the amount desired and send it to them...the recipient then has full control over what address they want to sweep the funds to as well as whether they'd like to include a miner fee to speed the confirmation along. Despite the fact that you may send many payments to the same identity, the only thing showing up on the p2p network and the block chain is the one time use address for the cheque and the recipient's wallet address. This means the recipient has much more control over the address policy used (compared with simply giving out a bitcoin address that may be reused). <<<

> Refund addresses...this is not going to be as useful as people might
> > think...most refunds that bitpay needs to process happen days or even
> months
> > after the initial purchase
>
> Useful feedback, thanks. Still, there may be other types of merchants
> for whom it's useful, and many users won't change their wallet. It
> certainly simplifies things if you can present the refund address and
> give a one-click option to use it. If the user wants to use a
> different address, then they can go onto the slow/complicated path.
>
> This current spec deliberately punts on the topic of identifying end
> users. It's a difficult problem.
>

I know, but as I was responding, I began to realize this is a mistake. It's worthwhile to tackle that problem first...if done right, it would pay huge dividends. Also, identity is one thing, an elaborate trust based identity verification system (like CA's) is a whole other thing. I think the former is pretty simple actually...and it's all that's really needed for the time being (as I alluded, a bitcoin identity could be communicated or verified using the existing X.509/CA infrastructure if desired...you could also use the PGP infrastructure).

> > I like the use of merchant_data...this means that you no longer will
> need a
> > unique bitcoin address for every invoice.
>
> It's still a good idea to use one for privacy reasons.

Actually, I was speaking more in terms of relying on the address to match up a transaction to an invoice. The merchant_data field frees you from having to do that.

> The merchant
> data is there so you can stuff whatever state you want into it. So
> it's like cookies. You don't have to keep state on the server side.
> Just encrypt/sign it, put it in the invoice, and when you get a
> payment message back there's no need to do database lookups or
> anything, you can just do some crypto and know who is submitting it.
>

Yeah, that's neat...I hadn't thought of that possibility.

>  > Generally speaking, I'm not a fan of embedding things like that
>
> What's wrong with it? Isn't your proposal more complex? I don't see
> why it's better than just embedding it.
>

It's not a big deal, I just think a referential model is more general than embedding objects within each other.

>  > The Receipt should be signed...it could be used as proof of payment by
> > wallets.
>
> There's no Receipt message, a SignedPaymentRequest + transactions that
> pay to the requested outputs are together the proof of payment.
>

Ah, I see it was renamed PaymentACK...the point of signing a PaymentACK is that while you could prove that you paid according to a PaymentRequest, a signed PaymentACK is proof that the recipient acknowledged you have made that payment.

The more I think about this topic, the more I think the first task at hand is to implement secure, private messaging...the nature of any messages (payment requests or otherwise) sent between wallets is such that it needs to be secured. And the great thing is that it's easy to do and you don't need to solve the PKI problem. Have the wallet maintain one or more ECC key pairs for the purposes of signing and encrypting messages. Allow these to be shared between wallets, or exported/imported, etc. You can punt on the whole topic of verifying the others' public keys using PKI (I mean, people use bitcoin addresses today without the use of any formal or explicit PKI to verify them...people will make do without it for communications keys just fine...and they can always use PGP or other PKI if they feel the need...most people would just pick up the phone to verify a friend's public key)...this also doesn't preclude the use of X.509 for the merchant/customer scenario...

For a payment protocol, you could do something like this: use https & ssl certificates/CAs as one method of obtaining an ECC public key...pki_type could be "https" and pki_bytes could be a url for the https location to download the ECC public key. The software would reject (or warn) if the SSL certificate isn't considered valid by the normal CA validation process. The wallet would not necessarily need to permanently store ECC public keys obtained in this manner. This approach doesn't require people to obtain new certificates just for bitcoin.

In fact, there would be very little difference to the proposed payments protocol if this approach were taken...instead of using X.509 directly for signing and encrypting messages, you are using it for signing and encrypting the ECC public key exchange. And this allows people that don't have web servers or SSL certificates to exchange their ECC public keys by other means and be able to use this payment protocol as well as any others that one could imagine. So, I actually think this is a better way of keeping PKI out of the scope of the proposal.

Payment requests are just one kind of messaging between wallets. I've also mentioned the "cheques" feature. I'm sure there are many more possibilities. Having a uniform method of securing messages sent between wallets (that doesn't depend on external tools) would be a great step forward IMO.

I hope that this input does not come too late; I haven't had time to review the proposal until now.

For alt-chains that have time-varying value (Freicoin[1], currently), it is necessary in some applications to include a "reference height" in the invoice. Since the bitcoin protocol does not assume a universally agreed-upon time source, Freicoin (and presumably other yet-to-be-implemented time-varying chains) uses blocktime as the clock for time-value calculations: outputs lose 2**-20 of their value with each passing block. The reference height for an invoice is the blocktime at which amount values are specified and the reference point for time-varying calculations. As a concrete example, an invoice for payment of 50 frc today could be satisfied by 49.99313402 frc tomorrow.

To implement this, we would require an optional "uint64 refheight" field in the invoice structure. "refheight" or "nRefHeight" is what we call this value internally, but "blocktime" or "blockheight" would work as well.

Github is currently down, so I apologize if a suitable field has already been added.

Cheers, Mark Friedenbach

[1] http://freico.in/ "Freicoin: a P2P digital currency delivering freedom from usury."

On Mon, Nov 26, 2012 at 2:37 PM, Gavin Andresen <gavinandresen@gmail•com>wrote:

> This is the next big "lets all agree to do things the same way" thing
> I think we should tackle. I'm particularly looking for feedback from
> other bitcoin client developers, even if it is just a quick "looks
> reasonable, if everybody else is going to do it then I will
> (eventually) too..."
>
> Thanks to Pieter Wuille and Mike Hearn for lots of feedback and
> suggestions and brainstorming.
>
> This document is online at https://gist.github.com/4120476
>
> If you respond to this message, please be considerate of people who
> subscribe to the digest version of this mailing list and trim your
> response.
>
>
> Invoices, Payments and Receipts for Bitcoin Transactions
> ========================================================
>
> This document proposes protocol buffer-based formats for signed,
> authenticated "invoices" and "receipts" -- requests for payment, and
> proof-of-payment.
>
> Separate documents propose an extension to the Bitcoin URI syntax and
> new MIME types to support them.
>
> Motivation
> ==========
>
> The idea of a "payment protocol" to improve on Bitcoin addresses has
> been around for over a year. Users have been asking for some features
> in this proposal (like the ability to provide a refund address so
> overpayments or refunds can be returned to customers without the need
> to ask them for their address) for two or three years, and have
> started to work around shortcomings in the Bitcoin payment process
> with creative (but inefficient) uses of transactions.
>
> The key features of this proposal are:
>
> + Requests for payment (Invoices) are tied to authenticated identities
> using the only widely-deployed identity authentication system we have
> right now (X.509 certificates signed by root certificate authorities)
> + Invoices include a user-friendly description of what the payment is for
> + Payments include where refunds should be sent
> + At the end of the payment process, the customer holds a
> cryptographically signed Receipt that can be used as proof-of-payment
> if there is any dispute with the merchant.
>
>
> Specification
> =============
>
> Invoice/SignedInvoice
> ---------------------
>
> An Invoice is a request for payment from a merchant to a customer:
>
> ::
>
>     message Output {
>         optional uint64 amount = 1;
>         required bytes script = 2;
>     }
>
> amount: Number of satoshis (0.00000001 BTC) to be paid. If not given
> or zero, then the customer will be asked how much to pay.
>
> script: a "TxOut" script to which the customer should direct payment.
> This will normally be one of the standard Bitcoin transaction script
> (e.g. pubkey OP_CHECKSIG).
>
> ::
>
>     message Invoice {
>         repeated bytes x509chain = 1;
>         repeated Output outputs = 2;
>         required uint64 time = 3;
>         optional uint64 expires = 4;
>         optional bool single_use = 5 [default = true];
>         optional string memo = 6;
>         optional string receiptURI = 7;
>         optional bytes merchant_data = 8;
>     }
>
> outputs: one or more outputs where Bitcoins are to be sent.
>
> x509chain: one or more DER-encoded X.509 certificates that identifies
> the merchant. See the "Certificates" section below for details.
>
> time: Unix timestamp (seconds since 1-Jan-1970) when the Invoice was
> created.
>
> expires: Unix timestamp after which the Invoice should be considered
> invalid. If not given, the Invoice may be re-used until the earliest
> certificate expiration date in the X509chain.
>
> single_use: If true, this Invoice should be used for only one payment.
> If false, it may be added to the user's address book and used
> repeatedly until it expires (e.g. for donations or a recurring
> payment).
>
> memo: UTF-8 encoded, plain-text (no formatting) note that should be
> displayed to the customer, explaining what this Invoice is for.
>
> receiptURI: Secure (https) URI where a Payment message (see below) may
> be sent to obtain a SignedReceipt as proof-of-payment.
>
> merchant_data : Arbitrary data ignored by the client that may be used
> by the merchant to identify the Invoice.
>
> ::
>
>     message SignedInvoice {
>         required Invoice invoice = 1;
>         required bytes signature = 2;
>     }
>
> A SignedInvoice is an Invoice signed using the private key
> corresponding to the public key in the first certificate in the
> x509chain and the HMAC SHA-256 algorithm.
>
> When a Bitcoin client receives a SignedInvoice, it must authorize
> payment by doing the following:
>
> 1. Validate the x509chain certificate chain up to it's list of root
> certificate authorities
> 2. Validate that the time on the customer's system is before
> Invoice.expires
> 3. Display the "Common Name" (CN) string from the first x509chain
> certificate and ask the customer if they would like to submit payment
>
> Payment
> -------
>
> ::
>
>     message Payment {
>         required Invoice invoice = 1;
>         repeated bytes transactions = 2;
>         repeated Output refund_to = 3;
>         optional string memo = 4;
>     }
>
> invoice : the invoice received from the merchant. A merchant must
> validate the Invoice and may reject the Payment if the Invoice was
> altered by the customer.
>
> transactions : One or more valid, signed Bitcoin transactions that
> fully pay the Invoice
>
> refund_to : One or more outputs where the merchant may return funds,
> if necessary.
>
> memo : UTF-8 encoded, plain-text note from the customer to the merchant.
>
> If the customer authorizes payment, then the Bitcoin client:
>
> 1. Creates and signs a transaction with one output sending the
> Invoice.script
> 2. If there is no Invoice.receiptURI, then the transaction is
> broadcast on the Bitcoin p2p network.
> 3. Else POST a Payment message to Invoice.receiptURI and expect a
> SignedReceipt in response.
>
> Invoice.receiptURI must be secure against man-in-the-middle attacks
> that might alter Payment.refund_to.
>
> *Note: an alternative would be a SignedPayment message that ties the
> signatures in Payment.transactions to a signature for the entire
> Payment message. Spending multisig inputs that may be controlled by
> more than one person or spending arbitrary non-standard transactions
> makes that non-trivial.*
>
> Receipt/SignedReceipt
> ---------------------
>
> ::
>
>     message Receipt {
>         required Payment payment = 1;
>         required bool accepted = 2;
>         optional string memo = 3;
>     }
>
> accepted : true if the Payment is accepted and will be broadcast on
> the Bitcoin p2p network.
>
> memo : UTF-8 encoded note that should be displayed to the customer
> indicating that the transaction is complete.
>
> ::
>
>     message SignedReceipt {
>         required Receipt receipt = 1;
>         required bytes signature = 3;
>     }
>
> A SignedReceipt is a Receipt signed using the private key
> corresponding to the public key in the first certificate in the
> Receipt->Payment->Invoice.x509chain and the HMAC SHA-256 algorithm.
>
> Upon receiving a SignedReceipt, a Bitcoin client should validate the
> signature and, if valid, display the Receipt.memo and store the
> SignedReceipt as proof-of-payment.
>
> If a SignedReceipt is not received for any reason (timeout, error) and
> Payment.transactions has not been broadcast by the merchant on the
> Bitcoin p2p network, then the Bitcoin client should assume that the
> payment failed, inform the customer that the payment failed, and
> return coins involved in the transaction to the customer's wallet.
>
>
> Certificates
> ============
>
> The Invoice.x509chain (X.509 Certificate Chain) field contains the
> X.509 public key certificate or certificate chain [RFC5280]
> corresponding to the key used to digitally sign the Invoice and
> Receipt. The certificate or certificate chain is represented as an
> array of DER [ITU.X690.1994] PKIX certificate value. The certificate
> containing the public key of the entity that digitally signed the
> Invoice MUST be the first certificate. This MAY be followed by
> additional certificates, with each subsequent certificate being the
> one used to certify the previous one. The recipient MUST verify the
> certificate chain according to [RFC5280] and reject the payment
> request if any validation failure occurs.
>
> *What should we say about root certificates and certificate management
> in general? Any requirements, or leave it up to each Bitcoin client to
> determine which root CA's are trustworthy, as happens with web
> browsers? Gavin suggests trusting only (say) ten of the Extended
> Validation authorities:
>
> http://en.wikipedia.org/wiki/Extended_Validation_Certificate#Extended_Validation_certificate_identification
> *
>
> *X.509 is widely criticised for doing too much. However, it is the
> Public Key Infrastructure (PKI) system we're stuck with. Do web
> browsers / certificate authorities support the full X.509 spec, or
> only a subset? Should Bitcoin clients only support some well-defined
> subset of X.509 ? More research needed here... *
>
> Use Cases
> =========
>
> Merchant Payment Service
> ------------------------
>
> A merchant payment service (like Paysius or bit-pay.com) would use
> Invoices and Receipts as follows:
>
> 1. Merchant pays for a certificate from a certificate authority, and
> then gives the payment service the certificate and their private key.
> This could be the same certificate and private key as is used for the
> merchant's web site, but best security practice would be to purchase a
> separate certificate for authenticating Invoices. Very successful
> merchant payment services might act as intermediate certificate
> authorities, issuing certificates for their merchants.
> 2. Customer goes through the checkout process on either the merchant's
> or payment service's web site.
> 3. At the end of the checkout process, a SignedInvoice is generated
> and sent to the customer's Bitcoin client.
> 4. Customer's Bitcoin client displays the Invoice, showing that the
> payment is for the merchant.
> 5. On customer approval, a Payment is sent to the payment service's
> paymentURI. The merchant is notified of the payment, and the customer
> receives a SignedReceipt as proof-of-payment.
>
> SatoshiDice
> -----------
>
> SatoshiDice (www.satoshidice.com) is an extremely popular game that
> uses tiny transactions for some customer/service communications. In
> particular, customers can add an extra output to their transactions to
> indicate where winnings should be sent. And SatoshiDice creates tiny
> transactions to let their customers know that a bet was received, but
> lost.
>
> Assuming Bitcoin clients upgrade to support this proposal, a bet on
> SatoshiDice would proceed as follows:
>
> 1. Customer clicks on a link on SatoshiDice.com and their Bitcoin
> client receives a SignedInvoice.
> 2. Customer authorizes payment, and their Bitcoin client creates a
> Payment message and submits it directly to
> https://satoshidice.com/something
> 3. The SatoshiDice web server checks to make sure the transaction is
> valid, broadcasts it, and determines whether the customer wins or
> loses. It returns a SignedReceipt with either a "You win" or "You
> lost" memo.
> 4. If the customer won, it broadcasts a transaction to pay them using
> Payment.refund_to
> 5. Customer's Bitcoin client displays the win/lose memo, and if they
> won the winnings appear in their wallet when received over the p2p
> network.
>
> Multiperson Wallet
> ------------------
>
> This use case starts with a multi-signature Bitcoin address or wallet,
> with keys held by two different people (Alice and Bob). Payments from
> that address/wallet must be authorized by both Alice and Bob, and both
> are running multi-signature-capable Bitcoin clients.
>
> Alice begins the payment process by getting a SignedInvoice from a
> merchant that needs to be paid. She authorizes payment and her Bitcoin
> client creates a Payment message with a partially-signed transaction,
> which is then sent to Bob any way that is convenient (email
> attachment, smoke signals...).
>
> Bob's Bitcoin client validates the SignedInvoice and asks Bob to
> authorize the transaction. He says OK, his Bitcoin client completes
> the transaction by providing his signature, submits the payment to the
> merchant, and then sends a message to Alice with the SignedReceipt he
> received from the merchant, completing the payment process.
>
>
> Design Notes
> ============
>
> Why X.509 Certificates?
> -----------------------
>
> This proposal uses X.509 certificates as the identity system for
> merchants because most of them will have already purchased a
> certificate to secure their website and will be familiar with the
> process of proving their identity to a certificate issuing authority.
>
> Implementing a better global PKI is outside the scope of this
> proposal. If a better PKI is adopted, the only change to this proposal
> would be to replace the Invoice.x509chain with whatever that better
> infrastructure uses to identify entities.
>
>
> Why not JSON?
> -------------
>
> Invoice, Payment and Receipt messages could all be JSON-encoded. And
> the Javascript Object Signing and Encryption (JOSE) working group at
> the IETF has a draft specification for signing JSON data.
>
> But the spec is non-trivial. Signing JSON data is troublesome because
> JSON can encode the same data in multiple ways (whitespace is
> insignificant, characters in strings can be represented escaped or
> un-escaped, etc.), and the standards committee identified at least one
> security-related issue that will require special JSON parsers for
> handling JSON-Web-Signed (JWS) data (duplicate keys must be rejected
> by the parser, which is more strict than the JSON spec requires).
>
> A binary message format has none of those complicating issues. Which
> encoding format to pick is largely a matter of taste, but Protocol
> Buffers is a simple, robust, multi-programming-language,
> well-documented, easy-to-work-with, extensible format.
>
> What about a merchant-pays-fee feature?
> ---------------------------------------
>
> It is desireable to allow a merchant to pay the cost of any Bitcoin
> network transaction processing fees, so if a customer is paying for a
> 1 BTC item they pay exactly 1 BTC.
>
> One way of accomplishing that is to add a 'maxfee' field to the
> Invoice, and have the Bitcoin client construct a transaction that pays
> the merchant (amount-maxfee).
>
> Another way of accomplishing that is to change the transaction
> selection code used by Bitcoin miners, so that dependent transactions
> are considered as a group. Then a merchant with several unconfirmed
> zero-fee transaction from customers can create a pay-to-self
> transaction with a large enough fee to pay for the set of transactions
> to be confirmed.
>
> A third way of accomplishing that is for the Bitcoin client to sign
> Payment.transactions[0] using the SIGHASH_ANYONECANPAY flag, and for
> the merchant to add an additional, small-BTC-value input to the
> transaction before broadcasting it. That additional input would go
> directly to miners as a fee. *Note: Gavin is not sure if he loves or
> hates this idea.*
>
> Checking for revoked certificates
> ---------------------------------
>
> The Online Certificate Checking Protocol (OCSP) is supposed to be a
> quick and easy way for applications to check for revoked certificates.
>
> In practice, it doesn't work very well. Certificate Authorities have
> no financial incentive to support a robust infrastructure that can
> handle millions of OCSP validation requests quickly.
>
> Ideally, Bitcoin clients would use OCSP to check certificate statuses
> every time they received or re-used an Invoice. But if that results in
> long pauses or lots of false-positive rejections (because an OCSP
> endpoint is offline or overwhelmed, perhaps) then merchants and
> customers might revert to just using "never fails" Bitcoin addresses.
>
>
>
> References
> ==========
>
> Public-Key Infrastructure (X.509) working group :
> http://datatracker.ietf.org/wg/pkix/charter/
>
> RFC 2560, X.509 Internet Public Key Infrastructure Online Certificate
> Status Protocol - OCSP : http://tools.ietf.org/html/rfc2560
>
> Protocol Buffers : https://developers.google.com/protocol-buffers/
>
> See Also
> ========
>
> Javascript Object Signing and Encryption working group :
> http://datatracker.ietf.org/wg/jose/
>
> sipa's payment protocol proposal: https://gist.github.com/1237788
>
> ThomasV's "Signed Aliases" proposal : http://ecdsa.org/bitcoin_URIs.html
>
>
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

On 2012-12-21 17:05:21 +0000, Stephen Pair said:

> Also, identity is one thing, an elaborate trust based identity
> verification system (like CA's) is a whole other thing.

Your distinction between "identity" and "trust-based identity" is one of the most important insights to emerge from this thread. Thank you for pointing this out.

> the first task at hand is to implement secure, private messaging...
> it's easy to do and you don't need to solve the PKI problem.

Alan Reiner <etotheipi <at> gmail.com> writes:

>
>
>
> On Thu, Dec 6, 2012 at 11:56 AM, Gavin Andresen <gavinandresen <at>

gmail.com> wrote:When I say "pass around" I'm not thinking of users copying and pasting, that would be a terrible user experience; all of that communication needs to happen automatically behind the scenes. Lets tackle that after we've got the simpler customer-pays-merchant flow working nicely (funded-escrow-pays-merchant is a subset of that, anyway).

>
>
>
>
> I think that the "pass around" method needs to happen in addition to the

methods of transparent protocols that occur behind the scenes.  For one, there's a lot of CONOPs that need to be worked out by getting knowledgeable people using it, and providing feedback about how it could/should/will be used and how it could be improved.  The pass-around method is simpler to implement and still usable by the types of users that will be using it in the beginning -- experts.  Also, I see that for very large, important multi-sig tx/contracts/escrow, the "manual" method might be preferred -- much the same way many people prefer manual-transmission cars even though automatics are "easier" -- some people/organizations will want the control.

>
> I'm all for protocols that enable higher-level access to this

functionality, I'm just saying there should be lower-level access, too.

>
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
>
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development <at> lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Another online space wagering stage has been propelled and targets Bitcoin, Litecoin and Dogecoin fans from everywhere throughout the world.

Presented in August, Crypto-Games.net is a straightforward space machine diversion that might be immediately played in any web program with the utilization of cryptocurrencies, making it one of the first few wagering destinations that are intended for Slot machine betting.

As indicated by its site, it emphasizes a four-reel space machine with one pay line that acknowledges wagers measuring from 50 Credits to a greatest of 9,999 Credits, where playing with most extreme wager yields a gigantic big stake of 4,999,500 Credits.

Then, the store exchange routines for this opening machine amusement have been created to be productive with the stores taking online 2 to 10 minutes to be transformed and the withdrawals just 6 to 60 minutes without any transaction charges. Moreover, a QR code has as of now been added to suit versatile bettors utilizing their cell phones and tablets.

Also, the administrators have taken the additional mile to guarantee reasonableness by depending on the random.org API to deliver genuine arbitrariness that supplements the diversion's reasonable calculation.

The https://www.crypto-games.net opening diversion, on the other hand, cautions players to observe their remarkable ID number to abstain from losing access to their account. This is essential on the grounds that Crypto-Games does not oblige any type of record enlistment; rather, it composes into cookies, which are erased following 14 days.

Besides, it has taken off new extra arrangements for Crypto bettors aside from the 25% house edge that are up for gets through the referral system.