soft-fork block size increase (extension blocks)

Hi Gavin

Sorry for slow response & broken threading - mailbox filled up & only saw your response on archive.

I do earnestly think opt-in block-size increases are politically cleaner (gives different people different sized blocks by their own volition without forcing a compromise) and less risky than hard forks. Particularly if a hard-fork were really provoked without clear and wide consensus - dragons lay there.

> Then ask the various wallet developer how long it would take them to update
> their software to support something like this,

I don't think thats any particular concern, extension block payments are forwards and backwards compatible. Businesses who are keen to have more transactions, would make it their problem to implement in their wallet, or ask the wallet vendor/maintainer they're working with to do it. Nothing breaks if they dont use it. The people that have the need for it will work on it. Market at work. If it turns out they dont really have a need for it, just projected huge numbers for their business plan that say dont materialise, well no foul.

> and do some UI mockups of what the experience would look like for users.

I am not a UX guy, but for example it might be appropriate for tipping services or other micropayments to use an extension block. Or small retail payments. They can choose what address they use. Merchants, integrators etc can do likewise. It gives plenty enough scope that people can work with useful trade-offs while others work on lightning.

> If there are two engineering solutions to a problem, one really simple, and
> one complex, why would you pick the complex one?

Because the more complex one is safer, more flexible, more future proof and better for decentralisation (and so as a bonus and might actually get done without more months of argument as its less contentious because it gives users choice to opt-in). Bitcoin itself is complex, a central ledger is simpler but as we know uninteresting which is to say this is a security tradeoff.

Obviously I do appreciate KISS as a design principle, and utility of incremental improvements, but this is a security trade-off we're discussing here. I am proposing a way to not weaken security, while getting what you think is important - access to more TPS with a higher centralisation tradeoff (for those who opt-in to it, rather than for everyone whether that tradeoff is strongly against their interests or not).

The decentralisation metrics are getting worse, not better, see Greg Maxwell's comments http://www.reddit.com/r/Bitcoin/comments/37vg8y/is_the_blockstream_company_the_reason_why_4_core/crqg381

This would not by those metrics be a good moment in history to make the situation worse.

> Especially if the complex solution has all of the problems of the simple
> one (20MB extension blocks are just as "dangerous" as 20MB main blocks,
> yes? If not, why not?)

Not at all, thats the point. Bitcoin has a one-size fits all blocksize. People can pool mine the 8MB extension block, while solo or GBT mining the 1MB block. Thats more centralising than staying at 1MB (because to get the fees from the extension block some people without that kind of bandwidth are pool mining 8/9th of the lower security/decentralisation transactions. But its less centralising than a fixed blocksize of 9MB (1+8 for apples/apples) because realistically if those transactions are not spam, they would've happened offchain, and offchain until we get lightning like systems up, means central systems which are worse than the slight centralisation of 8MB blocks being single servers and prone to custody & security failure. I think you made that point yourself in a recent post also.

Sound good? ;) Seriously I think its the least bad idea I've heard on this topic.

As an aside, a risk with using companies as a sounding board, is that you can get a misleading sense of consensus. Did they understand the tradeoff between security (decentralisation) and blocksize. Did they care? Do they represent users interests? Would they have "voted" instead for extension blocks if it was presented in similar terms? (I have to imagine they might have preferred extension blocks given the better story if you gloss over complexity and tradeoffs).

Adam

Hi Adam,

I have more experience than Gavin of building consumer wallets, so I'll make an attempt to answer your questions.

> Then ask the various wallet developer how long it would take them to
> update
> > their software to support something like this,
>
> I don't think thats any particular concern

I am a wallet developer and I am telling you that it is.

> Businesses who are keen to
> have more transactions, would make it their problem to implement in
> their wallet, or ask the wallet vendor/maintainer they're working with
> to do it.  Nothing breaks if they dont use it.

I don't see how this is the case. If an exchange supports extension blocks and I withdraw from that to a wallet that doesn't, the money will never arrive from my perspective. Yet the exchange will claim they sent it and they will wash their hands of the matter. Disaster.

I am not a UX guy

>

But I am. I've designed both consumer and engineering UI's at Google, and also more recently for Lighthouse.

Attempting to explain to a user why they sent money that didn't show up on the other end is a non starter. It's bad enough when things take a long time to confirm or bugs cause propagation failures. Doing it deliberately is not going to work. Payments *must* be reliable and wallets *must* be compatible with each other.

This is one reason why a Lightning style approach also isn't going to work any time soon. For example, it would require people to abandon Bitcoin addresses. I pushed for that before, around the P2SH time, and Gavin correctly intuited that the community wasn't ready for it yet. I'm not sure much has changed.

> Because the more complex one is safer, more flexible, more future
> proof and better for decentralisation

I disagree with all of those points. I find Lightning/Stroem etc to be more dangerous, less flexible, and worse for decentralisation. I explain why here:

https://medium.com/@octskyward/the-capacity-cliff-586d1bf7715e

You mentioned decentralisation metrics. Gregory's post is ignoring one of the most important decentralisation metrics, which is number of wallets made by independent developers. That has got dramatically better over time. It would get worse if wallets became more complex very suddenly.

> As an aside, a risk with using companies as a sounding board, is that
> you can get a misleading sense of consensus.

From what I can tell Blockstream employees are just ignoring those
companies entirely, which will give you a radically more distorted view of
the consensus. As companies providing services to our community have
serious economic weight, it stands to reason that their opinions would
matter a great deal. Yet on this mailing list I see zero effort to even
recognise their concerns, let alone care about them.
  <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>
Anyway, let me repeat again to make it clear - as someone who has spent
five years writing SPV wallets, I am not on board with extension blocks or
any other Rube Goldberg contraption that exists purely to work around
theoretical objections by Blockstream employees+Peter Todd, which is what
this feels like to me.

Mike wrote:

>> Businesses who are keen to
>> have more transactions, would make it their problem to implement in
>> their wallet, or ask the wallet vendor/maintainer they're working with
>> to do it.  Nothing breaks if they dont use it.
>
>
> I don't see how this is the case. If an exchange supports extension blocks
> and I withdraw from that to a wallet that doesn't, the money will never
> arrive from my perspective. Yet the exchange will claim they sent it and
> they will wash their hands of the matter. Disaster.

To be clear in case you are missing part of the mechanism.: it is forward and backwards compatible meaning a 1MB address can receive payments from an 8MB address (at reduced security if it has software that doesnt understand it) and a 1MB address can pay an 8MB address by paying to an OP_TRUE that has meaning to the extension block nodes.

A 1MB client wont even understand the difference between a 1MB and 8MB out payment. An 8MB client will understand and pay 1MB addresses in a different way (moving the coin back to the 1MB chain).

So its opt-in and incrementally deployable. Exchanges could encourage their users to use wallets that support 8MB blocks, eg by charging a fee for 1MB transactions. If 1MB blocks experience significant fee pressure, this will be persuasive. Or they could chose not to and eat the cost. This is all normal market adoption of a new cheaper technical option (in this case with a tradeoff of reduced security/more centralisation for those opting in to it).

>> Because the more complex one is safer, more flexible, more future
>> proof and better for decentralisation
>
> I disagree with all of those points. I find Lightning/Stroem etc to be more
> dangerous, less flexible, and worse for decentralisation. I explain why
> here:

Extension blocks & lightning are unrelated things.

While I understand the need for being practical, there is IMO, amongst engineering maxims something as far as being too pragmatic, dangerously pragmatic even. We cant do stuff in bitcoin that has bad carry costs, nor throw out the baby with the bathwater.

The situation is just that we are facing a security vs volume tradeoff and different people will have different requirements and comfort zones. If I am not misremembering, I think you've sided typically with the huge block, big data center only end of the spectrum. What I am proposing empowers you to do experiments in that direction without getting into a requirements conflict with people who value more strongly the bitcoin properties arising from it being robustly decentralised.

I am not sure personally where the blocksize discussion comes out - if it stays as is for a year, in a wait and see, reduce spam, see fee-pressure take effect as it has before, work on improving improve decentralisation metrics, relay latency, and do a blocksize increment to kick the can if-and-when it becomes necessary and in the mean-time try to do something more long-term ambitious about scale rather than volume. Bitcoin without scale improvements probably wont get the volume people would like. So scale is more important than volume; and security (decentralisation) is important too. To the extreme analogy we could fix scale tomorrow by throwing up a single high perf database, but then we'd break the security properties arising from decentralisation. We should improve both within an approximately safe envelope IMO.

Adam

>
> (at reduced security if it has software that doesnt understand it)

Well, yes. Isn't that rather key to the issue? Whereas by simply increasing the block size, SPV wallets don't care (same security and protocol as before) and fully validating wallets can be updated with a very small code change.

> A 1MB client wont even understand the difference between a 1MB and 8MB
> out payment.

Let's say an old client makes a payment that only gets confirmed in an extension block. The wallet will think the payment is unconfirmed and show that to the user forever, no?

Can you walk through the UX for each case?

> If I am not misremembering, I think you've sided typically
> with the huge block, big data center only end of the spectrum.

It would be Satoshi, that argued that.

I think there must be a communication issue here somewhere. I'm not sure how this meme has taken hold amongst you guys, as I am the guy who wrote the scalability page back in 2011:

https://en.bitcoin.it/wiki/Scalability

It says:

*The core Bitcoin network can scale to much higher transaction rates than are seen today, assuming that nodes in the network are primarily running on high end servers rather than desktops. *

By "much higher rates" I meant VISA scale and by "high end server" I meant high end by today's standards not tomorrows. There's a big difference between a datacenter and a single server! By definition a single server is not a datacenter, although it would be conventional to place it in one. But even with the most wildly optimistic growth imaginable, I couldn't foresee a time when you needed more than a single machine to keep up with the transaction stream.

And we're not going to get to VISA scale any time soon: I don't think I've ever argued we will. If it does happen it would presumably be decades away. Again, short of some currently unimagined killer app.

So I don't believe I've ever argued this, and honestly I kinda feel people are putting words in my mouth.

> And we're not going to get to VISA scale any time soon

No, not at these block size limits. The closer we get to the maximum block size, the slower we grow the average block size toward it. Number of transactions per day is of course highly correlated with average block size. Based on these graphs we can expect that hitting 1 million transactions per day will be impossible without raising the maximum block size.

https://blockchain.info/charts/avg-block-size?showDataPoints=false&show_header=true&daysAverageString=7&timespan=all&scale=1&address=

https://blockchain.info/charts/n-transactions?showDataPoints=false&timespan=all&show_header=true&daysAverageString=7&scale=1&address=

From: Mike Hearn Sent: Monday, June 01, 2015 11:01 AM To: Adam Back Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] soft-fork block size increase (extensionblocks)

  (at reduced security if it has software that doesnt understand it)

Well, yes. Isn't that rather key to the issue? Whereas by simply increasing the block size, SPV wallets don't care (same security and protocol as before) and fully validating wallets can be updated with a very small code change.

  A 1MB client wont even understand the difference between a 1MB and 8MB
  out payment.

Let's say an old client makes a payment that only gets confirmed in an extension block. The wallet will think the payment is unconfirmed and show that to the user forever, no?

Can you walk through the UX for each case?

  If I am not misremembering, I think you've sided typically
  with the huge block, big data center only end of the spectrum.

It would be Satoshi, that argued that.

I think there must be a communication issue here somewhere. I'm not sure how this meme has taken hold amongst you guys, as I am the guy who wrote the scalability page back in 2011:

https://en.bitcoin.it/wiki/Scalability

It says:

  The core Bitcoin network can scale to much higher transaction rates than are seen today, assuming that nodes in the network are primarily running on high end servers rather than desktops.

By "much higher rates" I meant VISA scale and by "high end server" I meant high end by today's standards not tomorrows. There's a big difference between a datacenter and a single server! By definition a single server is not a datacenter, although it would be conventional to place it in one. But even with the most wildly optimistic growth imaginable, I couldn't foresee a time when you needed more than a single machine to keep up with the transaction stream.

And we're not going to get to VISA scale any time soon: I don't think I've ever argued we will. If it does happen it would presumably be decades away. Again, short of some currently unimagined killer app.

So I don't believe I've ever argued this, and honestly I kinda feel people are putting words in my mouth.

--------------------------------------------------------------------------------

On 6/1/2015 10:21 AM, Adam Back wrote:

> if it stays as is for a year, in a wait and see, reduce spam, see
> fee-pressure take effect as it has before, work on improving improve
> decentralisation metrics, relay latency, and do a blocksize increment
> to kick the can if-and-when it becomes necessary and in the mean-time
> try to do something more long-term ambitious about scale rather than
> volume.

What's your estimate of the lead time required to kick the can, if-and-when it becomes necessary?

The other time-series I've seen all plot an average block size. That's misleading, because there's a distribution of block sizes. If you bin by retarget interval and plot every single block, you get this

http://i.imgur.com/5Gfh9CW.png

The max block size has clearly been in play for 8 months already.

Awesome graph!

It would be interesting if you could use your Excel voodoo (or whatever graph app you use) to add density as a heat map, making areas with more blocks red and regions with less blocks blue…

> On Jun 1, 2015, at 8:42 PM, Tom Harding <tomh@thinlink•com> wrote:
>
> On 6/1/2015 10:21 AM, Adam Back wrote:
>> if it stays as is for a year, in a wait and see, reduce spam, see
>> fee-pressure take effect as it has before, work on improving improve
>> decentralisation metrics, relay latency, and do a blocksize increment
>> to kick the can if-and-when it becomes necessary and in the mean-time
>> try to do something more long-term ambitious about scale rather than
>> volume.
>
> What's your estimate of the lead time required to kick the can,
> if-and-when it becomes necessary?
>
> The other time-series I've seen all plot an average block size.  That's
> misleading, because there's a distribution of block sizes.  If you bin
> by retarget interval and plot every single block, you get this
>
> http://i.imgur.com/5Gfh9CW.png
>
> The max block size has clearly been in play for 8 months already.
>
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸¸.·´¯`·.¸><(((º>

Richard Moore ~ Founder Genetic Mistakes Software inc. phone: (778) 882-6125 email: ricmoo@geneticmistakes•com <mailto:ricmoo@geneticmistakes.com> www: http://GeneticMistakes.com <http://geneticmistakes.com/>