BIP 32.5

I am wondering if we shouldn't have a BIP32 addendum which makes the following signing related recommendations:

(1) Recommend a specific deterministic DSA derandomization procedure (a deterministic way to generate the DSA nonce), presumably one based on HMAC-SHA512 (since BIP32 uses that construct) or SHA256 in the style of RFC 6979.

DSA systems being compromised due to poor randomness at runtime is not new. It effected other systems before it effected Bitcoin systems, it's not a new problem and it's not going away. It's difficult to tell if an implementation is correct or not.

Use of a fully deterministic signature would allow for complete test vectors in signing and complete confidence that there is no random number related weakness in a signing implementation.

In particular, with relevance to our ecosystem a maliciously modified difficult to audit hardware wallet could be leaking its keys material via its signatures. Even without producing insecure K values it could use the choice of K to leak a couple bits of an encrypted root key with every signature, and allow the malicious party to recover the keys by simply observing the network. Making the signatures deterministic would make this kind of misbehavior practically discoverable.

We wouldn't be alone in making this change, in general industry is moving in this direction because it has become clear that DSA is a hazard otherwise.

The primary arguments in most spaces against derandomizing DSA are FIPS conformance (irrelevant for us) and reasonable concerns about the risks of using a (less) reviewed cryptographic construct. With widespread motion towards derandomized DSA this latter concern is less of an issue.

Libcrypt has also implemented derandomized DSA in git. The ed25519 signature system of DJB, et. al. also uses a similar derandomization.

An alternative is implementing a still random construct where K is
some H(message||key||random) which should remain secure even where the
randomness is poor, but this loses the advantage of being able to
externally verify that an implementation is not leaking information.
OpenSSL development has implemented a form of this recently.

See also: http://tools.ietf.org/rfc/rfc6979.txt

(2) Recommends a procedure for using only even S values in signatures, eliminating this source of mutability in transactions.

This can be accomplished via post-processing of existing signatures, but since it requires bignum math it is usually preferable to implement it along with signing. I believe someday this will become a network requirement for Bitcoin, but regardless it makes sense to implement it as a best practice sooner rather than later.

Thoughts?

I filed a bug in the bitcoinj tracker for this a few days ago referencing
rfc 6967, but that RFC is very complicated and I'm not sure it's really
necessary to go that far. H(sighash||key) is easy to implement and I feel I
understand it better.

In our case it wouldn't have helped anyway - if anything it would just delayed discovery of the underlying weakness. The same RNG is typically used to generate both keys and signatures today. However in future it may be the case that people put more effort into generating a really random key because they only have to do it once, and then the signing RNG would be different.

Your concern about hardware devices leaking private key bits via a side channel is also well made, although I think you have to find some way to establish trust in these devices anyway as sniffing all their IO traffic and analysing it is really hard (plus it inverts the threat model - if you trust your computer and not your hardware wallet, why do you have a hardware wallet?)

The other advantage is that deterministic keys and signatures together mean two instances of the same wallet generate identical transactions given an identical sequence of commands. This could help keep wallets in sync. For example we had a few users who got confused because they had cloned their Android wallets across devices (NOT SUPPORTED!) and then one device updated first, did key rotation, and then the other device showed a transaction that sent all their money to a new address it knew nothing about. If they didn't realise the other device had updated this looked identical to theft!

I don't think fractional BIP numbers are the way to go :) but a new BIP that standardised a way to select K would, if reviewed, be something I'd implement.

On Fri, Aug 16, 2013 at 4:26 AM, Gregory Maxwell <gmaxwell@gmail•com> wrote:

> I am wondering if we shouldn't have a BIP32 addendum which makes the
> following signing related recommendations:
>
> (1) Recommend a specific deterministic DSA derandomization procedure
> (a deterministic way to generate the DSA nonce), presumably one based
> on HMAC-SHA512 (since BIP32 uses that construct) or SHA256 in the
> style of RFC 6979.
>
> DSA systems being compromised due to poor randomness at runtime is not
> new. It effected other systems before it effected Bitcoin systems,
> it's not a new problem and it's not going away.  It's difficult to
> tell if an implementation is correct or not.
>
> Use of a fully deterministic signature  would allow for complete test
> vectors in signing and complete confidence that there is no random
> number related weakness in a signing implementation.
>
> In particular, with relevance to our ecosystem a maliciously modified
> difficult to audit hardware wallet could be leaking its keys material
> via its signatures. Even without producing insecure K values it could
> use the choice of K to leak a couple bits of an encrypted root key
> with every signature, and allow the malicious party to recover the
> keys by simply observing the network. Making the signatures
> deterministic would make this kind of misbehavior practically
> discoverable.
>
> We wouldn't be alone in making this change, in general industry is
> moving in this direction because it has become clear that DSA is a
> hazard otherwise.
>
> The primary arguments in most spaces against derandomizing DSA are
> FIPS conformance (irrelevant for us) and reasonable concerns about the
> risks of using a (less) reviewed cryptographic construct. With
> widespread motion towards derandomized DSA this latter concern is less
> of an issue.
>
> Libcrypt has also implemented derandomized DSA in git. The ed25519
> signature system of DJB, et. al. also uses a similar derandomization.
>
> An alternative is implementing a still random construct where K is
> some H(message||key||random) which should remain secure even where the
> randomness is poor, but this loses the advantage of being able to
> externally verify that an implementation is not leaking information.
> OpenSSL development has implemented a form of this recently.
>
> See also: http://tools.ietf.org/rfc/rfc6979.txt
>
> (2) Recommends a procedure for using only even S values in signatures,
> eliminating this source of mutability in transactions.
>
> This can be accomplished via post-processing of existing signatures,
> but since it requires bignum math it is usually preferable to
> implement it along with signing.  I believe someday this will become a
> network requirement for Bitcoin, but regardless it makes sense to
> implement it as a best practice sooner rather than later.
>
> Thoughts?
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

On Fri, Aug 16, 2013 at 01:32:39PM +0200, Mike Hearn wrote:

> and analysing it is really hard (plus it inverts the threat model - if you
> trust your computer and not your hardware wallet, why do you have a
> hardware wallet?)

Myself I would trust neither the hardware wallet nor my computer...

So looks like the first version of the TREZOR won't support multisig - how far away are we from support? What about other manufacturers?

P2SH support is probably going to be a major sticking point. The payment protocol is all well and good, but it doesn't (yet) help getting money to the individual. bitcoinj P2SH support for sending would be a major help here - lots of person-to-person trades happen via Android wallets.

I personally like the full-measure of eliminating the "CS-PRNG" entirely from signing. If the "random" component is assumed to be untrusted, keeping it in there adds no value, while eschewing the main benefit of deterministic signing (ease of testing, auditing)

This just leaves the CS-PRNG at the heart of the security system -- when generating the root master key of an HD wallet. Adding to what Mike said, a single invocation of a CS-PRNG driving all subsequent keys increases the attack value if that one invocation turns out to be weak. By comparison, at least compromised DSA signatures were one-off events which didn't allow theft of funds beyond the one compromised address.

Cumulative / rolling entropy collection over time through multiple CS-PRNG invocations, or multiple entropy sources, could serve to recover from an "occasionally weak" CS-PRNG. I've read claims that this is bad practice because a single low entropy source can take entropy out of the result, but this seems like FUD. If you're using SHA512-HMAC to hash chain a few entropy sources, even "return 4; // chosen by random dice roll" is not going to help, but it's not going to hurt.

The DSA 'repeated-k' basically advertises itself on the block-chain and people were actively scanning for this weakness, whereas a weak key in the BIP32 root might not be as apparent, so exploitation may be more difficult, but also more insidious. Of course this depends on the exact failure mode of the CS-PRNG being used -- I wonder if anyone is searching for BIP32 keys based off of one of the 32k Debian random numbers being used as a master key?

Smartphones in particular have lots of sensors which could provide entropy. For example, if you pulled 64 bytes from "secure random", you could at least HMAC that with the SHA512 of a picture or a short video sample taken by the user. I'm guessing some people would cringe at this, but it seems to me like it provides some measure of protection to justify the increased code complexity.

TL;DR - Really like the idea of minimizing CS-PRNG use whenever possible (deterministic signing) and also would love to learn more best practices for placing less trust in the so called "CS-PRNG" when we do have to use them.

Thanks, Jeremy

On Thu, Aug 15, 2013 at 7:26 PM, Gregory Maxwell <gmaxwell@gmail•com> wrote:

> I am wondering if we shouldn't have a BIP32 addendum which makes the
> following signing related recommendations:

Looks like we're in the midst of another DSA duplicated K disaster. (Now, blockchain.info mywallet)

I talked to Pieter about this some earlier today and he sounded pretty positive. I'll go ahead and start on an actual BIP document for it.