Weak block thoughts...

> First:  the idea of 'weak blocks' (hat tip to Rusty for the term)
> I've been thinking about 'weak blocks' and SPV mining, and it seems to me
> weak blocks will make things better, not worse, if we improve the mining
> code a little bit.
>
> First:  the idea of 'weak blocks' (hat tip to Rusty for the term) is for
> miners to pre-announce blocks that they're working on, before they've
> solved the proof-of-work puzzle. To prevent DoS attacks, assume that some
> amount of proof-of-work is done (hence the term 'weak block') to rate-limit
> how many 'weak block' messages are relayed across the network.
>
>
> Today, miners are incentivized to start mining an empty block as soon as
> they see a block with valid proof-of-work, because they want to spend as
> little time as possible mining a not-best chain.
>
> Imagine miners always pre-announce the blocks they're working on to their
> peers, and peers validate those 'weak blocks' as quickly as they are able.
>
> Because weak blocks are pre-validated, when a full-difficulty block based
> on a previously announced weak block is found, block propagation should be
> insanely fast-- basically, as fast as a single packet can be relayed across
> the network the whole network could be mining on the new block.
>
> I don't see any barrier to making accepting the full-difficulty block and
> CreateNewBlock() insanely fast, and if those operations take just a
> microsecond or three, miners will have an incentive to create blocks with
> fee-paying transactions that weren't in the last block, rather than mining
> empty blocks.
>
> .................
>
> A miner could try to avoid validation work by just taking a weak block
> announced by somebody else, replacing the coinbase and re-computing the
> merkle root, and then mining. They will be at a slight disadvantage to
> fully validating miners, though, because they WOULD have to mine empty
> blocks between the time a full block is found and a fully-validating miner
> announced their next weak block.
>
> .................
>
> Weak block announcements are great for the network; they give transaction
> creators a pretty good idea of whether or not their transactions are likely
> to be confirmed in the next block. And if we're smart about implementing
> them, they shouldn't increase bandwidth or CPU usage significantly, because
> all the weak blocks at a given point in time are likely to contain the same
> transactions.
>
> --
> --
> Gavin Andresen
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> I've been thinking about 'weak blocks' and SPV mining, and it seems to me weak blocks will make things better, not worse, if we improve the mining code a little bit.
>
> First:  the idea of 'weak blocks' (hat tip to Rusty for the term) is for miners to pre-announce blocks that they're working on, before they've solved the proof-of-work puzzle. To prevent DoS attacks, assume that some amount of proof-of-work is done (hence the term 'weak block') to rate-limit how many 'weak block' messages are relayed across the network.
>
>
> Today, miners are incentivized to start mining an empty block as soon as they see a block with valid proof-of-work, because they want to spend as little time as possible mining a not-best chain.
>
> Imagine miners always pre-announce the blocks they're working on to their peers, and peers validate those 'weak blocks' as quickly as they are able.
>
> Because weak blocks are pre-validated, when a full-difficulty block based on a previously announced weak block is found, block propagation should be insanely fast-- basically, as fast as a single packet can be relayed across the network the whole network could be mining on the new block.
>
> I don't see any barrier to making accepting the full-difficulty block and CreateNewBlock() insanely fast, and if those operations take just a microsecond or three, miners will have an incentive to create blocks with fee-paying transactions that weren't in the last block, rather than mining empty blocks.
>
> .................
>
> A miner could try to avoid validation work by just taking a weak block announced by somebody else, replacing the coinbase and re-computing the merkle root, and then mining. They will be at a slight disadvantage to fully validating miners, though, because they WOULD have to mine empty blocks between the time a full block is found and a fully-validating miner announced their next weak block.
>
> .................
>
> Weak block announcements are great for the network; they give transaction creators a pretty good idea of whether or not their transactions are likely to be confirmed in the next block. And if we're smart about implementing them, they shouldn't increase bandwidth or CPU usage significantly, because all the weak blocks at a given point in time are likely to contain the same transactions.
>
> --
> --
> Gavin Andresen
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> On Sep 23, 2015, at 12:28 PM, Peter R <peter_r@gmx•com> wrote:
>
> Hi Gavin,
>
> One thing that's not clear to me is whether it is even necessary--from the perspective of the block size limit--to consider block propagation.
> Imagine miners always pre-announce the blocks they're working on to their
> peers, and peers validate those 'weak blocks' as quickly as they are able.
>
> Because weak blocks are pre-validated, when a full-difficulty block based
> on a previously announced weak block is found, block propagation should be
> insanely fast-- basically, as fast as a single packet can be relayed across
> the network the whole network could be mining on the new block.
>
> I don't see any barrier to making accepting the full-difficulty block and
> CreateNewBlock() insanely fast, and if those operations take just a
> microsecond or three, miners will have an incentive to create blocks with
> fee-paying transactions that weren't in the last block, rather than mining
> empty blocks.
>
- receive weak block
- validate
- create child block
> A miner could try to avoid validation work by just taking a weak block
> announced by somebody else, replacing the coinbase and re-computing the
> merkle root, and then mining. They will be at a slight disadvantage to
> fully validating miners, though, because they WOULD have to mine empty
> blocks between the time a full block is found and a fully-validating miner
> announced their next weak block.
>
> Weak block announcements are great for the network; they give transaction
> creators a pretty good idea of whether or not their transactions are likely
> to be confirmed in the next block.
>
> And if we're smart about implementing them, they shouldn't increase
> bandwidth or CPU usage significantly, because all the weak blocks at a
> given point in time are likely to contain the same transactions.
>
>
>
> --
> --
> Gavin Andresen
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> more recently:
> http://gnusha.org/bitcoin-wizards/2015-09-20.log
> http://diyhpl.us/wiki/transcripts/scalingbitcoin/roundgroup-roundup-1/
> http://diyhpl.us/wiki/transcripts/scalingbitcoin/bitcoin-block-propagation-iblt-rusty-russell/
> Imagine miners always pre-announce the blocks they're working on to their
> peers, and peers validate those 'weak blocks' as quickly as they are able.
>
> Because weak blocks are pre-validated, when a full-difficulty block based on
> a previously announced weak block is found, block propagation should be
> insanely fast--
> A miner could try to avoid validation work by just taking a weak block
> announced by somebody else, replacing the coinbase and re-computing the
> merkle root, and then mining. They will be at a slight disadvantage to fully
> Shouldn't mining pools and miners be paying you guys for coding solutions
> that improve their profitability?
> On Wed, Sep 23, 2015 at 3:43 PM, Gavin Andresen via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation•org> wrote:
> [...]
> > A miner could try to avoid validation work by just taking a weak block
> > announced by somebody else, replacing the coinbase and re-computing the
> > merkle root, and then mining. They will be at a slight disadvantage to
> fully
>
> Take care, here-- if a scheme is used where e.g. the full solution had
> to be exactly identical to a prior weak block then the result would be
> making mining not progress free because bigger miners would have
> disproportionately more access to the weak/strong one/two punch. I
> think what you're thinking here is okay, but it wasn't clear to me if
> you'd caught that particular potential issue.
>
> Avoiding this is why I've always previously described this idea as
> merged mined block DAG (with blocks of arbitrary strength) which are
> always efficiently deferentially coded against prior state. A new
> solution (regardless of who creates it) can still be efficiently
> transmitted even if it differs in arbitrary ways (though the
> efficiency is less the more different it is).
>
> I'm unsure of what approach to take for incentive compatibility
> analysis. In the worst case this approach class has no better delays
> (and higher bandwidth); but it doesn't seem to me to give rise to any
> immediate incrementally strategic behavior (or at least none worse
> than you'd get from just privately using the same scheme).
>
> Take care, here-- if a scheme is used where e.g. the full solution had
> to be exactly identical to a prior weak block then the result would be
> making mining not progress free because bigger miners would have
> disproportionately more access to the weak/strong one/two punch. I
> think what you're thinking here is okay, but it wasn't clear to me if
> you'd caught that particular potential issue.
>
> I'm assuming the optimized protocol would be forward-error-coded (e.g. using IBLTs)  and NOT require the full solution (or follow-on weak blocks) to be exactly the same.
>
> I don't see any incentive problems, either. Worst case is more miners
> decide to skip validation and just mine a variation of the
> highest-fee-paying weak block they've seen, but that's not a disaster--
> invalid blocks will still get rejected by all the non-miners running full
> nodes.
> If we did see that behavior, I bet it would be a good strategy for a big
> hashrate miner to dedicate some of their hashrate to announcing invalid
> weak blocks; if you can get your lazy competitors to mine it, then you
> win....
> I've been thinking about 'weak blocks' and SPV mining, and it seems to me
> weak blocks will make things better, not worse, if we improve the mining
> code a little bit.
>
> First:  the idea of 'weak blocks' (hat tip to Rusty for the term) is for
> miners to pre-announce blocks that they're working on, before they've
> solved the proof-of-work puzzle. To prevent DoS attacks, assume that some
> amount of proof-of-work is done (hence the term 'weak block') to rate-limit
> how many 'weak block' messages are relayed across the network.
>
>
> Today, miners are incentivized to start mining an empty block as soon as
> they see a block with valid proof-of-work, because they want to spend as
> little time as possible mining a not-best chain.
>
> Imagine miners always pre-announce the blocks they're working on to their
> peers, and peers validate those 'weak blocks' as quickly as they are able.
>
> Because weak blocks are pre-validated, when a full-difficulty block based
> on a previously announced weak block is found, block propagation should be
> insanely fast-- basically, as fast as a single packet can be relayed across
> the network the whole network could be mining on the new block.
Chers,
Rusty.
PS.  Original idea came to me from Greg Maxwell; Peter Todd called it
     "near blocks" and extolled their virtues 2 years ago...
>> Avoiding this is why I've always previously described this idea as
>> merged mined block DAG (with blocks of arbitrary strength) which are
>> always efficiently deferentially coded against prior state. A new
>> solution (regardless of who creates it) can still be efficiently
>> transmitted even if it differs in arbitrary ways (though the
>> efficiency is less the more different it is).
>
> Yup, although I don't get the 'merge mined' bit; the weak blocks are
> ephemeral, probably purged out of memory as soon as a few full blocks are
> found...
> Unless the weak block transaction list can be a superset of the block
> transaction list size proportional propagation costs are not totally
> eliminated.
>
> As even if the weak block criteria is MUCH lower than the block
> criteria (which would become problematic in its own right at some
> point) the network will sometimes find blocks when there hasn't been
> any weak block priming at all (e.g. all prior priming has made it into
> blocks already).
>
>
>
> On Sun, Sep 27, 2015 at 2:39 AM, Gregory Maxwell via bitcoin-dev <
> bitcoin-dev@lists•linuxfoundation•org> wrote:
>
>> Unless the weak block transaction list can be a superset of the block
>> transaction list size proportional propagation costs are not totally
>> eliminated.
>>
>
> The POW threshold could be dynamic.  The first weak-block that builds on a
> new block could be forwarded with a smaller target.
>
> This reduces  the window size until at least one weak block is
> propagated.
>
> The change in threshold could be time based (for the first 30 seconds or
> so).  This would cause a surge of traffic when a new block once a new block
> has propagated, so perhaps not so good an idea.
>
>
>> As even if the weak block criteria is MUCH lower than the block
>> criteria (which would become problematic in its own right at some
>> point) the network will sometimes find blocks when there hasn't been
>> any weak block priming at all (e.g. all prior priming has made it into
>> blocks already).
>>
>
> If there is a transaction backlog, then miners could forward merkle
> branches with transactions in the memory pool with a commitment in the
> coinbase.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation•org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
> I was mansplaining weak blocks to my wife. She asked a simple question:
>
> Why would I, as a miner, publish a weak block if I find one?
>
> I don't know.
> Sure, I will get faster propagation for my solved block, should I find one.
> On the other hand everybody else mining a similar block will enjoy the same
> benefit. Assuming that I'm not a huge miner, it's unlikely that I will
> actually solve the block, so I'm probably just giving away fast propagation
> times to someone else.
> So how does publishing a weak block benefit the producer of it more than the
> other miners? Please help me understand this.
> On Sun, Sep 27, 2015 at 3:10 PM, Kalle Rosenbaum via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation•org> wrote:
> > I was mansplaining weak blocks to my wife. She asked a simple question:
> >
> > Why would I, as a miner, publish a weak block if I find one?
> >
> > I don't know.
> > Sure, I will get faster propagation for my solved block, should I find
> one.
> > On the other hand everybody else mining a similar block will enjoy the
> same
> > benefit. Assuming that I'm not a huge miner, it's unlikely that I will
> > actually solve the block, so I'm probably just giving away fast
> propagation
> > times to someone else.
> > So how does publishing a weak block benefit the producer of it more than
> the
> > other miners? Please help me understand this.
>
> Keep in mind, because of efficient differential transmission the cost
> to you is effectively nothing if your transaction acceptance policy is
> predictable, it's a hand-full of bytes sent. And by failing to send
> yours you do little to nothing to deny others the improvement.
>
>
> Lets imagine an alternative weak-blockless weak block implementation:
>
> Every N seconds, every miner send to every other miner what they're
> working on.  This isn't totally crazy-- efficient differential
> transmission will keep the amount transmitted small.
>
> Any block found can be referenced to any of these earlier worklists.
>
> What the effect be of not transmitting yours?
>
> If your block is unlike everyone elses, you would suffer great delays
> in the event you found a block.
> If your block is mostly like everyone elses, you wouldn't suffer as
> much delay-- but the transmission costs would be negligible in that
> case. ... the size sent is proportional to the improvement you get
> when finding a block.
>
>
> In either case, no one else is harmed by you not sending yours... they
> still send their lists.
>
> A problem with that scheme is that unless you've layered an identity
> based access control system on it anyone can DOS attack it, because
> anyone can send as much as they want, they don't even have to be
> actual miners.
>
> What weak blocks adds to that is using hashcash as a rate limiting
> mechanism-- a coordination free lottery weighed by hash-power decides
> who can transmit.
>
> What if you don't participate in the lottery and share your solutions?
>  No major harm for the other users... the other users will just choose
> a somewhat lower weak-block threshold to get the updates at the
> desired rate than they would otherwise. To the extent that what you
> were working on was different from anyone else, you'll suffer because
> you failed to make use of your chance to influence what could be
> efficiently transmitted to include your own blocks.
>
> You could also ask a question of why would you transitively relay
> someone elses announcement-- well if it helped their blocks too  (by
> reflecting things they also want to mine) the answer is obvious. But
> what if it was disjoint from the things they wanted to mine and didn't
> help compared to the weak blocks they already relayed?  In that case
> it's still in likely in their interest to relay it because if a block
> similar to it is produced and they extend that block they may end up
> orphaned because of propagation delays their parent block suffered.
> What if they receive an announcement which is so "ugly" that they
> wouldn't extend the chain with the strong block version of it (they'd
> intentionally try to fork it off?)-- in that case they wouldn't want
> to relay it.  So much the same logic as why you relay other parties
> blocks applies, including-- relaying helps the network, but if you
> don't it'll still get along fine without you.
>
> Suppose that you've solved a block Z (weak or not) and you want to propagate it using a previous weak block Y. With "efficient differential transmission", I assume that you refer to the transmission of the differences between Y and Z to a peer? What encodings are discussed? I guess IBLTs are a hot candidate, but are there other schemes in the making? I suppose that sending something like "weak block Y plus transactions A, B, C minus transaction ids h(D), h(E)" is not considered an efficient differential transmission. Then that's part of the answer to my question.
>