LearnBitcoin

Bitcoin Dev Archive 2012

Punishing empty blocks?

31 messages, 24 to 29 May 2012. Started by Jeff Garzik.

Luke Dashjr (7), Jeff Garzik (6), Peter Vessenes (6), Greg Maxwell (2), Arthur Britto, Joel Joonatan Kaartinen, Robert McKay, Christian Decker, +6 more

Permalink Original thread on gnusha.org

Jeff Garzik permalink original

There appears to be some non-trivial mining power devoted to mining empty blocks. Even with satoshi's key observation -- hash a fixed 80-byte header, not the entire block -- some miners still find it easier to mine empty blocks, rather than watch the network for new transactions.

Therefore I was wondering what people thought about a client implementation change:

     - Do not store or relay empty blocks, if time since last block < X
       (where X = 60 minutes, perhaps)

or even stronger,

     - Ensure latest block includes at least X percent of mempool
unconfirmed TXs

The former is easier to implement, though there is the danger that no-TX miners simply include a statically generated transaction or two.

The latter might be considered problematic, as it might refuse to relay quickly found blocks.

Comments? It wouldn't be a problem if these no-TX blocks were not already getting frequent (1 in 20).

Arthur Britto in reply to Jeff Garzik permalink original

I think you need the stronger change. Otherwise, the mystery miner could just put in a few transactions to himself to mask his block. His block would appear to be of some use while not being helpful.

-Arthur

On Thu, May 24, 2012 at 9:33 AM, Jeff Garzik <jgarzik@exmulti•com> wrote:

Quoted text (43 lines)
> There appears to be some non-trivial mining power devoted to mining
> empty blocks.  Even with satoshi's key observation -- hash a fixed
> 80-byte header, not the entire block -- some miners still find it
> easier to mine empty blocks, rather than watch the network for new
> transactions.
>
> Therefore I was wondering what people thought about a client
> implementation change:
>
>     - Do not store or relay empty blocks, if time since last block < X
>       (where X = 60 minutes, perhaps)
>
> or even stronger,
>
>     - Ensure latest block includes at least X percent of mempool
> unconfirmed TXs
>
> The former is easier to implement, though there is the danger that
> no-TX miners simply include a statically generated transaction or two.
>
> The latter might be considered problematic, as it might refuse to
> relay quickly found blocks.
>
> Comments?  It wouldn't be a problem if these no-TX blocks were not
> already getting frequent (1 in 20).
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti•com
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
Joel Joonatan Kaartinen in reply to Jeff Garzik permalink original

I think the strong verification would go well if you add it along with an optimization that avoids rechecking transactions that have already been verified as valid. Any transactions it doesn't have to verify are from the pool, of course :)

On Thu, May 24, 2012 at 7:33 PM, Jeff Garzik <jgarzik@exmulti•com> wrote:

Quoted text (43 lines)
> There appears to be some non-trivial mining power devoted to mining
> empty blocks.  Even with satoshi's key observation -- hash a fixed
> 80-byte header, not the entire block -- some miners still find it
> easier to mine empty blocks, rather than watch the network for new
> transactions.
>
> Therefore I was wondering what people thought about a client
> implementation change:
>
>     - Do not store or relay empty blocks, if time since last block < X
>       (where X = 60 minutes, perhaps)
>
> or even stronger,
>
>     - Ensure latest block includes at least X percent of mempool
> unconfirmed TXs
>
> The former is easier to implement, though there is the danger that
> no-TX miners simply include a statically generated transaction or two.
>
> The latter might be considered problematic, as it might refuse to
> relay quickly found blocks.
>
> Comments?  It wouldn't be a problem if these no-TX blocks were not
> already getting frequent (1 in 20).
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti•com
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
Jeff Garzik in reply to Joel Joonatan Kaartinen permalink original

On Thu, May 24, 2012 at 1:13 PM, Joel Joonatan Kaartinen <joel.kaartinen@gmail•com> wrote:

Quoted text (3 lines)
> optimization that avoids rechecking transactions that have already been
> verified as valid. Any transactions it doesn't have to verify are from the
> pool, of course :)

Work in this area is already progressing, though it is outside the scope of this proposal regarding lazy miners and empty blocks.

Robert McKay in reply to Jeff Garzik permalink original

On Thu, 24 May 2012 12:33:12 -0400, Jeff Garzik wrote:

Quoted text (17 lines)
> There appears to be some non-trivial mining power devoted to mining
> empty blocks.  Even with satoshi's key observation -- hash a fixed
> 80-byte header, not the entire block -- some miners still find it
> easier to mine empty blocks, rather than watch the network for new
> transactions.
>
> Therefore I was wondering what people thought about a client
> implementation change:
>
>      - Do not store or relay empty blocks, if time since last block <
> X
>        (where X = 60 minutes, perhaps)
>
> or even stronger,
>
>      - Ensure latest block includes at least X percent of mempool
> unconfirmed TXs

If miners wanted to continue mining empty blocks without bothering to monitor the Tx pool they would just switch to stuffing the empty blocks with a dummy transaction of their own to get round your new rules. They could also spam them to the p2p network so that they end up in the mempool if that were necessary. This would probably still be slightly easier than 'doing it properly'.

Once the block reward halves in a few months time then receiving transaction fees will probably become more important to the miner's profit and loss calculations and they'll spend the extra time to implement proper transaction processing. I suspect if we do nothing this particular issue will go away. Perhaps it could be helped along by publishing some example code to make it easier for them.

The ability to refuse transactions seems like an important part of the game theory of transaction pricing. Miners are supposed to be able to jack up transaction costs by declining to process no fee or too low fee (in their opinion) transactions.. the counter balance is that they are losing money by doing that and leaving more on the table for the next miner to score a block.

I expect that in the future there will be other instances when people complain that the miners are being 'unfair' and that the rules should be changed in some way to lower transaction fees (ie: increase block size).

If block sizes are increased ever larger and miners aren't allowed to refuse to process transactions it will get rid of the financial motivation for mining and less mining will happen. We should be very careful when making these kinds of changes.

Setting percentage quotas of stuff in the mempool sounds dangerous.. miners that hear about a block from a rival miner soon enough could possibly DOS the mempool on the rest of the network to get the block rejected. I think it should be legitimate not to publish a transaction to the p2p network at all.. in the future there will probably be lots of networks other than the p2p network.. right now we have the IPv6 network and the IPv4 network.. in the future there could be many other protocols and perhaps not all transactions will make it back to the old legacy ipv4 p2p network or into the mempool of bitcoin nodes on that network.. but they should still be able to get into the block chain.

Rob

Jeff Garzik in reply to Robert McKay permalink original

On Thu, May 24, 2012 at 1:27 PM, Robert McKay <robert@mckay•com> wrote:

Quoted text (3 lines)
> If miners wanted to continue mining empty blocks without bothering to
> monitor the Tx pool they would just switch to stuffing the empty blocks
> with a dummy transaction of their own to get round your new rules.

Yes. This was stated in the original email.

Quoted text (6 lines)
> Once the block reward halves in a few months time then receiving
> transaction fees will probably become more important to the miner's
> profit and loss calculations and they'll spend the extra time to
> implement proper transaction processing. I suspect if we do nothing this
> particular issue will go away. Perhaps it could be helped along by
> publishing some example code to make it easier for them.

At current rates it is potentially years before that point is reached -- years of degraded service for existing users.

Quoted text (10 lines)
> The ability to refuse transactions seems like an important part of the
> game theory of transaction pricing. Miners are supposed to be able to
> jack up transaction costs by declining to process no fee or too low fee
> (in their opinion) transactions.. the counter balance is that they are
> losing money by doing that and leaving more on the table for the next
> miner to score a block.
>
> I expect that in the future there will be other instances when people
> complain that the miners are being 'unfair' and that the rules should be
> changed in some way to lower transaction fees (ie: increase block size).

If you see a rule change, you have misunderstood the proposal.

This is an -implementation- change, which users and miners are free to accept or reject as part of their choice of software to use in the bitcoin ecosystem.

As such, miners continue to be free to build upon empty blocks, and let those blocks become part of a useful chain. You would not simply /ban/ empty blocks completely, but avoid relaying top-of-chain empty blocks.

Mining power and network collaborate to choose the best chain at that point -- perhaps even including those empty blocks. Clients will continue to follow the longest, strongest chain, even after this implementation change.

An implementation change is a soft vote of choice by the user, not a hard requirement on all users.

Quoted text (7 lines)
> I think it should be legitimate not to publish a transaction
> to the p2p network at all.. in the future there will probably be lots of
> networks other than the p2p network.. right now we have the IPv6 network
> and the IPv4 network.. in the future there could be many other protocols
> and perhaps not all transactions will make it back to the old legacy
> ipv4 p2p network or into the mempool of bitcoin nodes on that network..
> but they should still be able to get into the block chain.

See above -- such behavior is perfectly fine.

It should be noted that out of band (OOB) TXs, transited through third party means outside P2P network, would not cause _empty_ blocks, as the block chain will continue to have traffic for the foreseeable future.

OOB TXs are a great idea, too. In a hyperscaled bitcoin future, OOB TXs might even be the norm.

Luke Dashjr in reply to Jeff Garzik permalink original

On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote:

Quoted text (16 lines)
> There appears to be some non-trivial mining power devoted to mining
> empty blocks.  Even with satoshi's key observation -- hash a fixed
> 80-byte header, not the entire block -- some miners still find it
> easier to mine empty blocks, rather than watch the network for new
> transactions.
>
> Therefore I was wondering what people thought about a client
> implementation change:
>
>      - Do not store or relay empty blocks, if time since last block < X
>        (where X = 60 minutes, perhaps)
>
> or even stronger,
>
>      - Ensure latest block includes at least X percent of mempool
> unconfirmed TXs

These are problematic for legitimate miners: 1) The freedom to reject transactions based on fees or spam filters, is severely restricted. As mentioned in other replies, this is an important point of Bitcoin's design. 1b) This punishes miners with superior transaction spam filtering. As with all spam filtering, it is often an "arms race" and therefore the filter rules must be kept private by the miners, and therefore cannot be disclosed for the validating clients to take into consideration. 2) For a few seconds after a new block is received, the new transaction merkle root(s) are not finished calculating. During this time, most miners are working on "blank" blocks with the new previousblockhash but no transactions. If those blocks are ignored, miners are forced to shutdown mining during this time. 3) As you mentioned, illegitimate miners can easily workaround these restrictions (even the second one, by flooding the network with their own transactions). This puts the legitimate miners at a disadvantage in their own search for valid blocks, unless they also come up with counter-measures themselves.

The argument that these are not rule changes is flawed: 1) As of right now, 99% of the network runs a single client. Anything this client rejects does de facto become a rule change. 2) Even if there were a diverse ecosystem of clients in place, discouragement rules that potentially affect legitimate miners significantly mess with the odds of finding a block. 3) If legitimate miners do not adopt counter-rules to bypass these new restrictions, the illegitimate miners are left with an even larger percentage of blocks found.

To summarize, I believe such a change as proposed would be very harmful to Bitcoin.

Luke

Jeff Garzik in reply to Luke Dashjr permalink original

On Thu, May 24, 2012 at 4:31 PM, Luke-Jr <luke@dashjr•org> wrote:

Quoted text (8 lines)
> These are problematic for legitimate miners:
> 1) The freedom to reject transactions based on fees or spam filters, is
> severely restricted. As mentioned in other replies, this is an important point
> of Bitcoin's design.
> 1b) This punishes miners with superior transaction spam filtering. As with all
> spam filtering, it is often an "arms race" and therefore the filter rules must
> be kept private by the miners, and therefore cannot be disclosed for the
> validating clients to take into consideration.

This is simply not true given current available data, i.e. the current blockchain and ongoing not-spam transaction rate/pool.

Quoted text (3 lines)
> The argument that these are not rule changes is flawed:
> 1) As of right now, 99% of the network runs a single client. Anything this
> client rejects does de facto become a rule change.

According to your own numbers even, this is not true. 99% of the network runs a wide variety of rules and versions. Even with a "critical" security announcement, the percentage of those running the latest version is not large.

Quoted text (6 lines)
> 2) Even if there were a diverse ecosystem of clients in place, discouragement
> rules that potentially affect legitimate miners significantly mess with the
> odds of finding a block.
> 3) If legitimate miners do not adopt counter-rules to bypass these new
> restrictions, the illegitimate miners are left with an even larger percentage
> of blocks found.

Miners are not the -only- ones that get a say in what is spam, and what is not. If miners are generating garbage, network users have the right to veto that garbage.

Luke Dashjr in reply to Jeff Garzik permalink original

On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote:

Quoted text (2 lines)
> Comments?  It wouldn't be a problem if these no-TX blocks were not
> already getting frequent (1 in 20).

FWIW, based on statistics for Eligius's past 100 blocks, it seems 10% (1 in 10) of 1-txn blocks is not actually unreasonable. This also means these 1-txn mined blocks are not necessarily harming Bitcoin intentionally. Anyone care to figure out the math for how fast miners need to finish processing transactions to reduce the number of 1txn blocks?

Jeff Garzik in reply to Luke Dashjr permalink original

On Thu, May 24, 2012 at 8:45 PM, Luke-Jr <luke@dashjr•org> wrote:

Quoted text (9 lines)
> On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote:
>> Comments?  It wouldn't be a problem if these no-TX blocks were not
>> already getting frequent (1 in 20).
>
> FWIW, based on statistics for Eligius's past 100 blocks, it seems 10% (1 in
> 10) of 1-txn blocks is not actually unreasonable. This also means these 1-txn
> mined blocks are not necessarily harming Bitcoin intentionally. Anyone care to
> figure out the math for how fast miners need to finish processing transactions
> to reduce the number of 1txn blocks?

Look at the time since last block, and correlate with the number of non-spam TX's in the memory pool at the time. It is obvious which ones are quick blocks (<60 seconds since last block, no big deal) and which ones are the lazy miners (> 120 seconds since last block).

Luke Dashjr in reply to Jeff Garzik permalink original

On Friday, May 25, 2012 12:51:09 AM Jeff Garzik wrote:

Quoted text (16 lines)
> On Thu, May 24, 2012 at 8:45 PM, Luke-Jr <luke@dashjr•org> wrote:
> > On Thursday, May 24, 2012 4:33:12 PM Jeff Garzik wrote:
> >> Comments?  It wouldn't be a problem if these no-TX blocks were not
> >> already getting frequent (1 in 20).
> >
> > FWIW, based on statistics for Eligius's past 100 blocks, it seems 10% (1
> > in 10) of 1-txn blocks is not actually unreasonable. This also means
> > these 1-txn mined blocks are not necessarily harming Bitcoin
> > intentionally. Anyone care to figure out the math for how fast miners
> > need to finish processing transactions to reduce the number of 1txn
> > blocks?
>
> Look at the time since last block, and correlate with the number of
> non-spam TX's in the memory pool at the time.  It is obvious which
> ones are quick blocks (<60 seconds since last block, no big deal) and
> which ones are the lazy miners (> 120 seconds since last block).

Block times are not accurate enough for that.

Greg Maxwell in reply to Jeff Garzik permalink original

On Thu, May 24, 2012 at 8:51 PM, Jeff Garzik <jgarzik@exmulti•com> wrote:

Quoted text (1 line)
> which ones are the lazy miners (> 120 seconds since last block).

It's important to understand the motivations before acting— otherwise you'll fail to do anything useful.

E.g. if they're empty because some miners want to drive up fees or fight against the rapidly increasing blockchain size there isn't much you can do there.

If they're empty because they're mined by botnets which don't have a local copy of the chain in order to load their victims less (and avoid central pooling) then you want something like https://bitcointalk.org/index.php?topic=68396.0

If they're produced by people who think they gain a mining speed advantage by not including them then then we need education— dropping their blocks won't help much: we've seen miners go a month with 100% of their blocks being orphaned.

Jeff Garzik in reply to Luke Dashjr permalink original

On Thu, May 24, 2012 at 8:57 PM, Luke-Jr <luke@dashjr•org> wrote:

Quoted text (1 line)
> Block times are not accurate enough for that.

The times in your log are very accurate, assuming your system clock is remotely accurate.

Christian Decker in reply to Jeff Garzik permalink original

How about a simple proof of work test? This one though does not ask for CPU work but asks the miner for a random old transaction. If the miner really stores the entire blockchain he will not have any problem answering to that getdata request, whereas a botnet would have to ask someone else for it, which could be detected if the response time deviates too much from what has been previously measured (compare it against getdata for the block they advertise). It's not perfect but it allows an estimate of whether it is a chainless miner.

Regards, Chris

Alan Reiner in reply to Christian Decker permalink original

I like the concept except that it only works if every node connected to the miner enforces the rule (if it works). Once any one of the nodes forwards the block, other nodes see it coming from a node that can pass the challenge.

I don't think any solution based on node queries will succeed, especially if it requires spontaneous super-majority-of-nodes acceptance. I think it's gotta be based on the block itself and each nodes' own info.

If you could spontaneously get all miners to agree not to build off of anti-social blocks (however that is defined) , it would have a chance of making a difference, but individual miners would have an advantage building off the antisocial block because they only need to produce one to create the longest chain (and collect reward) while the miners following the rules need two blocks.

--Sent from my overpriced smartphone On May 25, 2012 3:48 AM, "Christian Decker" <decker.christian@gmail•com> wrote:

Quoted text (56 lines)
> How about a simple proof of work test? This one though does not ask for
> CPU work but asks the miner for a random old transaction. If the miner
> really stores the entire blockchain he will not have any problem answering
> to that getdata request, whereas a botnet would have to ask someone else
> for it, which could be detected if the response time deviates too much from
> what has been previously measured (compare it against getdata for the block
> they advertise). It's not perfect but it allows an estimate of whether it
> is a chainless miner.
>
> Regards,
> Chris
> --
> Christian Decker
>
>
>
> On Fri, May 25, 2012 at 3:17 AM, Jeff Garzik <jgarzik@exmulti•com> wrote:
>
>> On Thu, May 24, 2012 at 8:57 PM, Luke-Jr <luke@dashjr•org> wrote:
>> > Block times are not accurate enough for that.
>>
>> The times in your log are very accurate, assuming your system clock is
>> remotely accurate.
>>
>> --
>> Jeff Garzik
>> exMULTI, Inc.
>> jgarzik@exmulti•com
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists•sourceforge•net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
Peter Vessenes in reply to Alan Reiner permalink original

We just implemented our own mining tool, soup-to-nuts, and I would say that the likely motivation for what I presume are botnet owners is just economic.

It's a lot more work to make sure your merkleing and keeping up-to-date are happening than it is to just get an 80 byte header from a central server, and re-calc a single transaction merkle client-side.

Not to mention the extra work to keep track of what version of getmemorypool output you're receiving work for in a broadly distributed pool.

For what it's worth, we did this extra engineering work since we care about Bitcoin, but if I just wanted to pull value out of the ecosystem, we would have skipped it.

The only solutions to this are economic solutions -- making such 'cheater' blocks less valuable, or increasing the value of the transactions.

Also note that botnet operators likely care, in the end, about fiat currency, so going to 25 btc per block in what I think of as transaction fee subsidies won't necessarily impact this -- it's a matter of what happens to exchange rates vs generation rates that will matter.

I think we also have to moderate this consideration against the rights (and arguable benefits) of someone wanting to build an express-delivery mining service, one that will provide provably faster certification for those adding a transaction fee of, say, 1 btc.

My own experience now in the MMO world is that we have to carefully understand how we deal with griefers who control massive resources (compute or gold-farmers). It may not be a winning battle to choose a solution which harms the rest of the network in exchange for harming the griefers.

This is definitely out of the box, but one solution might be to change the difficulty calculations to just ignore 1tx blocks; that would minimize impact on others to a great extent, and would let someone set up an express block service if they chose. I guess we'd have to settle on whether or not such blocks counted towards the issuance countdown as well. Or, we could allow only 1/10 generation fees on such blocks.

Peter

On Fri, May 25, 2012 at 9:44 AM, Alan Reiner <etotheipi@gmail•com> wrote:

Quoted text (89 lines)
> I like the concept except that it only works if every node connected to
> the miner enforces the rule (if it works).  Once any one of the nodes
> forwards the block,  other nodes see it coming from a node that can pass
> the challenge.
>
> I don't think any solution based on node queries will succeed,  especially
> if it requires spontaneous super-majority-of-nodes acceptance.  I think
> it's gotta be based on the block itself and each nodes' own info.
>
> If you could spontaneously get all miners to agree not to build off of
> anti-social blocks (however that is defined) ,  it would have a chance of
> making a difference,  but individual miners would have an advantage
> building off the antisocial block because they only need to produce one to
> create the longest chain (and collect reward) while the miners following
> the rules need two blocks.
>
> --Sent from my overpriced smartphone
> On May 25, 2012 3:48 AM, "Christian Decker" <decker.christian@gmail•com>
> wrote:
>
>> How about a simple proof of work test? This one though does not ask for
>> CPU work but asks the miner for a random old transaction. If the miner
>> really stores the entire blockchain he will not have any problem answering
>> to that getdata request, whereas a botnet would have to ask someone else
>> for it, which could be detected if the response time deviates too much from
>> what has been previously measured (compare it against getdata for the block
>> they advertise). It's not perfect but it allows an estimate of whether it
>> is a chainless miner.
>>
>> Regards,
>> Chris
>> --
>> Christian Decker
>>
>>
>>
>> On Fri, May 25, 2012 at 3:17 AM, Jeff Garzik <jgarzik@exmulti•com> wrote:
>>
>>> On Thu, May 24, 2012 at 8:57 PM, Luke-Jr <luke@dashjr•org> wrote:
>>> > Block times are not accurate enough for that.
>>>
>>> The times in your log are very accurate, assuming your system clock is
>>> remotely accurate.
>>>
>>> --
>>> Jeff Garzik
>>> exMULTI, Inc.
>>> jgarzik@exmulti•com
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists•sourceforge•net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists•sourceforge•net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
Zooko Wilcox-O'Hearn in reply to Jeff Garzik permalink original

For what it is worth, I question whether this is a problem. Or, I guess I question whether the best solution to it isn't for people to start including more transaction fees. In fact, I'm not entirely sure that this problem doesn't actually *encourage* people to that solution, which would be very good if true.

I would be more comfortable if the reward for mining were more commensurate with the value it provides. Ultimately, of course, that means that each transaction fee would have to be more of a proportion of the value *to the spender* of that transaction being included in the blockchain.

(Aside: in order to convey to outsiders that miners are providing a useful service rather than gaining undeserved reward for wasting electricity, I refer to them as "distributed transaction verification servers" rather than "miners" whenever possible.)

I'm pretty sure that — assuming there isn't some Bitcoin-killing disaster — transaction fees will eventually rise, but sooner might be better, especially with the first coinbase-halving looming.

Perhaps people will be motivated to include transaction fees if they know that some miners don't bother to validate their transactions and others do. They may feel motivated to reward the miners that are serving them and punish the ones that are not. (Note: this wouldn't be a valid strategy on their part from a strictly game-theoretic perspective, but if they act on those motivations, then I don't care if it was rational or not.)

Also, they may decide that they want to counteract the added delay which those no-transactions miners are adding to *all* transactions (with or without fee), by putting a fee on their transactions in order to make them take less long when they are processed by a miner which does process (some) transactions.

Already this visualization, which I typically glance at a few times a day, usually shows a good separation with fee-included transactions sometimes doing much better than (some) free transactions:

http://bitcoinstats.org/

However, this graph shows that the aggregate reward to the miners for processing transaction is minimal:

http://blockchain.info/charts/transaction-fees?timespan=60days&showDataPoints=false&daysAverageString=1&show_header=true&scale=0&address=

You can see from the first visualization (assuming it is showing the typical pattern that I've seen) how you risk greater delay by sending your transaction without fees. The no-transactions miners push *all* transactions, fee or no-fee to the right. This may incentivize more people to change their transactions from red diamonds into blue circles, in order to move their transactions further to the left, even though the no-transactions miners are not currently discriminating among the two types.

Therefore, the presence of those miners may help push the aggregate fees in that latter graph up, which is something I would very much like to see.

Regards,

Zooko

Stefan Thomas in reply to Zooko Wilcox-O'Hearn permalink original

Zooko is spot on - slower confirmations will give people a reason to set higher fees. As soon as fees reach a level where they matter, even botnet operators will be looking into ways of including transactions for some extra profit.

In the meantime slightly slower confirmations aren't a problem. Consider that even if it takes four blocks to get your transaction included instead of one, once it is included, you still benefit from every new block in terms of security. So if you're looking for six confirmations for example, even a three block delay will only be a 50% delay for you. And of course there are techniques for instant transactions which continue to be refined and improved.

As for the proposed solutions: Punishing 1-tx blocks is complete and utter nonsense. It's trivial to include a bogus second transaction.

Any additional challenges towards miners like hashes of the previous block are at best useless. If I was running a botnet, I'd just grab that hash from a website (pretty good chance Blockchain.info will have it :P) or mining pool or wherever and keep going undeterred. At worst they may affect scalability one day. You might imagine a peer-to-peer network of miners who for cost reasons don't download all blocks anymore, but verify only a percentage of them at random. They might then exchange messages about invalid blocks including a proof (invalid tx, merkle branch) why the block is invalid. This is just one idea, the point is that assumptions about what a legitimate miner looks like may not always hold in the future.

Finally, there is an ethical aspect as well. If a miner wishes not to include my transaction that is his choice. He has no more an obligation to sell his service to me than I have to buy it from him. If I really, really want him to include my transaction I will have to offer to pay more.

If we as developers think that confirmations are too slow or that more blocks should include transactions, then the right measures would be:

- Educating users about the relationship between confirmation speed and fees
- Raising the default transaction fee

Every market has a supply curve, so it is economically to be expected that there will be some miners who don't include transactions, simply because they are at that end of the supply curve where it is not worth it for them to sell their service. All markets must have a certain tension - there must be miners who don't include transactions for there to be users who want their transactions included more quickly. In other words there must be somebody not confirming if confirmations are to have value. If you interfere with that all you'll accomplish is keep transaction fees below market level, which will make the transition from inflation-financed hashing to transaction-financed hashing more painful and disruptive.

Cheers,

Stefan

Peter Vessenes in reply to Stefan Thomas permalink original

One of the issues here though is that it would be nice if miners published their own tx rules -- it might be hard to impute them from data.

I had started a thread about this on bitcoin.org some time ago, and I don't recall what the general outcome was.

I had imagined an open service whereby a miner could publish a short string in their conbase tying to the service and the service would have different metadata, including the miner's transaction guarantees.

We offered to host this before, and would still be willing to host such a service.

Peter

On Sat, May 26, 2012 at 7:52 AM, Stefan Thomas <moon@justmoon•de> wrote:

Quoted text (68 lines)
> Zooko is spot on - slower confirmations will give people a reason to set
> higher fees. As soon as fees reach a level where they matter, even
> botnet operators will be looking into ways of including transactions for
> some extra profit.
>
> In the meantime slightly slower confirmations aren't a problem. Consider
> that even if it takes four blocks to get your transaction included
> instead of one, once it is included, you still benefit from every new
> block in terms of security. So if you're looking for six confirmations
> for example, even a three block delay will only be a 50% delay for you.
> And of course there are techniques for instant transactions which
> continue to be refined and improved.
>
> As for the proposed solutions: Punishing 1-tx blocks is complete and
> utter nonsense. It's trivial to include a bogus second transaction.
>
> Any additional challenges towards miners like hashes of the previous
> block are at best useless. If I was running a botnet, I'd just grab that
> hash from a website (pretty good chance Blockchain.info will have it :P)
> or mining pool or wherever and keep going undeterred. At worst they may
> affect scalability one day. You might imagine a peer-to-peer network of
> miners who for cost reasons don't download all blocks anymore, but
> verify only a percentage of them at random. They might then exchange
> messages about invalid blocks including a proof (invalid tx, merkle
> branch) why the block is invalid. This is just one idea, the point is
> that assumptions about what a legitimate miner looks like may not always
> hold in the future.
>
> Finally, there is an ethical aspect as well. If a miner wishes not to
> include my transaction that is his choice. He has no more an obligation
> to sell his service to me than I have to buy it from him. If I really,
> really want him to include my transaction I will have to offer to pay more.
>
> If we as developers think that confirmations are too slow or that more
> blocks should include transactions, then the right measures would be:
>
> - Educating users about the relationship between confirmation speed and
> fees
> - Raising the default transaction fee
>
> Every market has a supply curve, so it is economically to be expected
> that there will be some miners who don't include transactions, simply
> because they are at that end of the supply curve where it is not worth
> it for them to sell their service. All markets must have a certain
> tension - there must be miners who don't include transactions for there
> to be users who want their transactions included more quickly. In other
> words there must be somebody not confirming if confirmations are to have
> value. If you interfere with that all you'll accomplish is keep
> transaction fees below market level, which will make the transition from
> inflation-financed hashing to transaction-financed hashing more painful
> and disruptive.
>
> Cheers,
>
> Stefan
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
Amir Taaki in reply to Peter Vessenes permalink original

That's a cool idea. Very meta.

________________________________ From: Peter Vessenes <peter@coinlab•com> To: Stefan Thomas <moon@justmoon•de> Cc: bitcoin-development@lists•sourceforge•net Sent: Monday, May 28, 2012 4:54 PM Subject: Re: [Bitcoin-development] Punishing empty blocks?

One of the issues here though is that it would be nice if miners published their own tx rules -- it might be hard to impute them from data.

I had started a thread about this on bitcoin.org some time ago, and I don't recall what the general outcome was.

I had imagined an open service whereby a miner could publish a short string in their conbase tying to the service and the service would have different metadata, including the miner's transaction guarantees.

We offered to host this before, and would still be willing to host such a service.

Peter

On Sat, May 26, 2012 at 7:52 AM, Stefan Thomas <moon@justmoon•de> wrote:

Zooko is spot on - slower confirmations will give people a reason to set

Quoted text (66 lines)
>higher fees. As soon as fees reach a level where they matter, even
>botnet operators will be looking into ways of including transactions for
>some extra profit.
>
>In the meantime slightly slower confirmations aren't a problem. Consider
>that even if it takes four blocks to get your transaction included
>instead of one, once it is included, you still benefit from every new
>block in terms of security. So if you're looking for six confirmations
>for example, even a three block delay will only be a 50% delay for you.
>And of course there are techniques for instant transactions which
>continue to be refined and improved.
>
>As for the proposed solutions: Punishing 1-tx blocks is complete and
>utter nonsense. It's trivial to include a bogus second transaction.
>
>Any additional challenges towards miners like hashes of the previous
>block are at best useless. If I was running a botnet, I'd just grab that
>hash from a website (pretty good chance Blockchain.info will have it :P)
>or mining pool or wherever and keep going undeterred. At worst they may
>affect scalability one day. You might imagine a peer-to-peer network of
>miners who for cost reasons don't download all blocks anymore, but
>verify only a percentage of them at random. They might then exchange
>messages about invalid blocks including a proof (invalid tx, merkle
>branch) why the block is invalid. This is just one idea, the point is
>that assumptions about what a legitimate miner looks like may not always
>hold in the future.
>
>Finally, there is an ethical aspect as well. If a miner wishes not to
>include my transaction that is his choice. He has no more an obligation
>to sell his service to me than I have to buy it from him. If I really,
>really want him to include my transaction I will have to offer to pay more.
>
>If we as developers think that confirmations are too slow or that more
>blocks should include transactions, then the right measures would be:
>
>- Educating users about the relationship between confirmation speed and fees
>- Raising the default transaction fee
>
>Every market has a supply curve, so it is economically to be expected
>that there will be some miners who don't include transactions, simply
>because they are at that end of the supply curve where it is not worth
>it for them to sell their service. All markets must have a certain
>tension - there must be miners who don't include transactions for there
>to be users who want their transactions included more quickly. In other
>words there must be somebody not confirming if confirmations are to have
>value. If you interfere with that all you'll accomplish is keep
>transaction fees below market level, which will make the transition from
>inflation-financed hashing to transaction-financed hashing more painful
>and disruptive.
>
>Cheers,
>
>Stefan
>
>
>------------------------------------------------------------------------------
>Live Security Virtual Conference
>Exclusive live event will cover all the ways today's security and
>threat landscape has changed and how IT managers can respond. Discussions
>will include endpoint security, mobile security and the latest in malware
>threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>_______________________________________________
>Bitcoin-development mailing list
>Bitcoin-development@lists•sourceforge•net
>https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
Michael Gronager in reply to Peter Vessenes permalink original

Peter, I like the idea of being able to know what fees to expect from different miners (it is like a service description / SLA for their service), but I would prefer a more distributed discovery mechanism for the information on the fees (Spent 10 years on Grid Computing...).

Miners could e.g. include a pointer to a webpage (or even their min fee) in the coinbase (encoded properly, like the "/P2SH/" string for BIP0016). That way clients could look it up them selves or you could create sites accumulating this information from the chain it self.

So something like :
        const char* service_sla = "|https://my_ubercool_asic_mining_pool/sla.php|";
        COINBASE_FLAGS << std::vector<unsigned char>(service_sla, service_sla+strlen(service_sla));
The format of the sla.php page should then be specified too - but it could be a json-rpc call returning a json object like (as result):
{
    sla_version: "0.1",
    accept_no_fee_tx: false,
    min_fee: 50000,
    big_tx_fee: 10000, // extra fee pr kb
}
I guess miners could work out a more suitable set of fees...

Seems like this calls for a BIP ?

/M

On 28/05/2012, at 16:54, Peter Vessenes wrote:

Quoted text (96 lines)
> One of the issues here though is that it would be nice if miners published their own tx rules -- it might be hard to impute them from data.
>
> I had started a thread about this on bitcoin.org some time ago, and I don't recall what the general outcome was.
>
> I had imagined an open service whereby a miner could publish a short string in their conbase tying to the service and the service would have different metadata, including the miner's transaction guarantees.
>
> We offered to host this before, and would still be willing to host such a service.
>
> Peter
>
> On Sat, May 26, 2012 at 7:52 AM, Stefan Thomas <moon@justmoon•de> wrote:
> Zooko is spot on - slower confirmations will give people a reason to set
> higher fees. As soon as fees reach a level where they matter, even
> botnet operators will be looking into ways of including transactions for
> some extra profit.
>
> In the meantime slightly slower confirmations aren't a problem. Consider
> that even if it takes four blocks to get your transaction included
> instead of one, once it is included, you still benefit from every new
> block in terms of security. So if you're looking for six confirmations
> for example, even a three block delay will only be a 50% delay for you.
> And of course there are techniques for instant transactions which
> continue to be refined and improved.
>
> As for the proposed solutions: Punishing 1-tx blocks is complete and
> utter nonsense. It's trivial to include a bogus second transaction.
>
> Any additional challenges towards miners like hashes of the previous
> block are at best useless. If I was running a botnet, I'd just grab that
> hash from a website (pretty good chance Blockchain.info will have it :P)
> or mining pool or wherever and keep going undeterred. At worst they may
> affect scalability one day. You might imagine a peer-to-peer network of
> miners who for cost reasons don't download all blocks anymore, but
> verify only a percentage of them at random. They might then exchange
> messages about invalid blocks including a proof (invalid tx, merkle
> branch) why the block is invalid. This is just one idea, the point is
> that assumptions about what a legitimate miner looks like may not always
> hold in the future.
>
> Finally, there is an ethical aspect as well. If a miner wishes not to
> include my transaction that is his choice. He has no more an obligation
> to sell his service to me than I have to buy it from him. If I really,
> really want him to include my transaction I will have to offer to pay more.
>
> If we as developers think that confirmations are too slow or that more
> blocks should include transactions, then the right measures would be:
>
> - Educating users about the relationship between confirmation speed and fees
> - Raising the default transaction fee
>
> Every market has a supply curve, so it is economically to be expected
> that there will be some miners who don't include transactions, simply
> because they are at that end of the supply curve where it is not worth
> it for them to sell their service. All markets must have a certain
> tension - there must be miners who don't include transactions for there
> to be users who want their transactions included more quickly. In other
> words there must be somebody not confirming if confirmations are to have
> value. If you interfere with that all you'll accomplish is keep
> transaction fees below market level, which will make the transition from
> inflation-financed hashing to transaction-financed hashing more painful
> and disruptive.
>
> Cheers,
>
> Stefan
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
> --
>
> Peter J. Vessenes
> CEO, CoinLab
> M: 206.595.9839
> Skype: vessenes
> Google+
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge•net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development

Michael Gronager, PhD Director, Ceptacle Jens Juels Gade 33 2100 Copenhagen E Mobile: +45 31 45 14 01 E-mail: gronager@ceptacle•com Web: http://www.ceptacle.com/

Luke Dashjr in reply to Michael Gronager permalink original

On Tuesday, May 29, 2012 8:52:49 AM Michael Grønager wrote:

Quoted text (8 lines)
> The format of the sla.php page should then be specified too - but it could
> be a json-rpc call returning a json object like (as result): {
>     sla_version: "0.1",
>     accept_no_fee_tx: false,
>     min_fee: 50000,
>     big_tx_fee: 10000, // extra fee pr kb
> }
> I guess miners could work out a more suitable set of fees...

Please not JSON, and not hard-coded logic. Bitcoin already has a secure scripting system - perhaps we can decide on an initial stack format and run a script retrieved from the URI?

Peter Vessenes in reply to Luke Dashjr permalink original

OK, I have a few thoughts on this:

1) Germane to the original conversation, anything hard to implement will not get implemented by miners. 2) Coinbase is hard-limited to 100 bytes; this has to include space for voting as well as extra nonce, etc. So, I'm not sure that a full URL is a good plan. 3) I'm a little fuzzy on the details of BIP governance; but I'm happy to write one up and get my thoughts down, or someone who's more familiar could do it, I suppose.

I propose the following spec:

periodically a miner may choose to publish a url through their coinbase as follows:

1) They shall prepend \mi: to the url to designate it as a url for miner info, and append a trailing \ to the url 2) The url given in the coinbase shall have http:// prepended to it before processing. 3) The destination may be a redirect (to allow short URLs), or may deliver content 4) The content-type returned by the final site post-redirect shall be either (preferred text/json) or text/plain or text/html 4) The text of the document delivered shall be a JSON format dictionary, and shall include at minimum the following fields: 'min_fee', 'pool_name', and 'last_modified' Optional fields can be determined over time as necessary by the mining community 5) The Service Level Agreement isn't binding, but miners who implement it are expected to make a best efforts attempt to follow it.

So a valid coinbase could be: /P2SH/\mi:goo.gl/mr2D\extra_nonce:2110

Generally a miner would occasionally publish the \mi:\ when they had updated their SLA, or just every so often, but the canonical location would be the final destination URL from the redirects.

Inre: Luke's complaint about JSON, it is the language of the web. There is no easier format for both computers and humans to read, and in this case, it includes extensibility, which is nice, since we have no idea how miners will wish to divvy up their services; I think one would need to make a strong case against JSON for a specific reason to not choose it by default.

Thoughts welcome!

Best,

Peter

On Tue, May 29, 2012 at 10:47 AM, Luke-Jr <luke@dashjr•org> wrote:

Quoted text (17 lines)
> On Tuesday, May 29, 2012 8:52:49 AM Michael Grønager wrote:
> > The format of the sla.php page should then be specified too - but it
> could
> > be a json-rpc call returning a json object like (as result): {
> >     sla_version: "0.1",
> >     accept_no_fee_tx: false,
> >     min_fee: 50000,
> >     big_tx_fee: 10000, // extra fee pr kb
> > }
> > I guess miners could work out a more suitable set of fees...
>
> Please not JSON, and not hard-coded logic. Bitcoin already has a secure
> scripting system - perhaps we can decide on an initial stack format and
> run a
> script retrieved from the URI?
>
>
Luke Dashjr in reply to Peter Vessenes permalink original

On Tuesday, May 29, 2012 3:05:18 PM Peter Vessenes wrote:

Quoted text (2 lines)
> 1) Germane to the original conversation, anything hard to implement will
> not get implemented by miners.

Without my got-tired-of-waiting-for-someone-to-merge-it coinbaser branch, anything modifying the coinbase is hard to implement.

Quoted text (3 lines)
> 2) Coinbase is hard-limited to 100 bytes; this has to include space for
> voting as well as extra nonce, etc. So, I'm not sure that a full URL is a
> good plan.

Rather, I would suggest a 20 byte keyhash, which allows the owner to broadcast a full URI out-of-band.

Quoted text (2 lines)
> 1) They shall prepend \mi: to the url to designate it as a url for miner
> info, and append a trailing \ to the url

How about a simple prefix to the fixed-size keyhash? Perhaps "MFR=" (Mining Fee Rules)

Quoted text (2 lines)
> 2) The url given in the coinbase shall have http:// prepended to it before
> processing.

I would recommend miners use https, with a specified SSL keyhash in the URI (so we don't need to pay for a "proper" SSL cert).

Quoted text (2 lines)
> 3) The destination may be a redirect (to allow short URLs), or may deliver
> content

Clients should simply be required to follow the relevant HTTP specification.

Quoted text (2 lines)
> 4) The content-type returned by the final site post-redirect shall be
> either (preferred text/json) or text/plain or text/html

text/plain and text/html are just wrong and don't make any sense here.

Quoted text (5 lines)
> Inre: Luke's complaint about JSON, it is the language of the web. There is
> no easier format for both computers and humans to read, and in this case,
> it includes extensibility, which is nice, since we have no idea how miners
> will wish to divvy up their services; I think one would need to make a
> strong case against JSON for a specific reason to not choose it by default.

Bitcoin isn't "the web", it's a complicated script-based cryptocurrency. Everything in the Bitcoin protocol requires a computer's interpretation for humans, and there's no reason to stray from this default. Also, JSON is not extensible in any of the ways needed for this specific purpose.

Quoted text (4 lines)
> 4) The text of the document delivered shall be a JSON format dictionary,
> and shall include at minimum the following fields: 'min_fee', 'pool_name',
> and 'last_modified' Optional fields can be determined over time as
> necessary by the mining community

Last Modified and other caching rules are dealt with in the relevant HTTP specification...

Quoted text (2 lines)
> 5) The Service Level Agreement isn't binding, but miners who implement it
> are expected to make a best efforts attempt to follow it.

While it doesn't make sense to give it the full legal force of a contract, I think it should be expressed as a "MUST" in the BIP.

Quoted text (3 lines)
> Generally a miner would occasionally publish the \mi:\ when they had
> updated their SLA, or just every so often, but the canonical location would
> be the final destination URL from the redirects.

The coinbase advertisement MUST be part of every coinbase mined by the miner, or there's no reliable way to prove which blocks are theirs.

Peter Vessenes in reply to Luke Dashjr permalink original

I disagree with a bunch of your points, but I'll wait on others to comment, except I will say that I don't understand what the 20 byte keyhash is. Can you elucidate?

I am assuming major mining folks have written their own coinbasing facilities, but perhaps this is not the case -- if so, I agree that some work is necessary for such miners.

Finally I will just comment that I am guided by the general perspective that many things about bitcoins are opt-in; therefore it makes sense to me put difficult work onto those who are motivated to do it, and keep things as easy as possible for the 'maybes' to participate -- hence small courtesies like allowing text/plain or text/html.

Peter

On Tue, May 29, 2012 at 11:18 AM, Luke-Jr <luke@dashjr•org> wrote:

Quoted text (79 lines)
> On Tuesday, May 29, 2012 3:05:18 PM Peter Vessenes wrote:
> > 1) Germane to the original conversation, anything hard to implement will
> > not get implemented by miners.
>
> Without my got-tired-of-waiting-for-someone-to-merge-it coinbaser branch,
> anything modifying the coinbase is hard to implement.
>
> > 2) Coinbase is hard-limited to 100 bytes; this has to include space for
> > voting as well as extra nonce, etc. So, I'm not sure that a full URL is a
> > good plan.
>
> Rather, I would suggest a 20 byte keyhash, which allows the owner to
> broadcast
> a full URI out-of-band.
>
> > 1) They shall prepend \mi: to the url to designate it as a url for miner
> > info, and append a trailing \ to the url
>
> How about a simple prefix to the fixed-size keyhash?
> Perhaps "MFR=" (Mining Fee Rules)
>
> > 2) The url given in the coinbase shall have http:// prepended to it
> before
> > processing.
>
> I would recommend miners use https, with a specified SSL keyhash in the URI
> (so we don't need to pay for a "proper" SSL cert).
>
> > 3) The destination may be a redirect (to allow short URLs), or may
> deliver
> > content
>
> Clients should simply be required to follow the relevant HTTP
> specification.
>
> > 4) The content-type returned by the final site post-redirect shall be
> > either (preferred text/json) or text/plain or text/html
>
> text/plain and text/html are just wrong and don't make any sense here.
>
> > Inre: Luke's complaint about JSON, it is the language of the web. There
> is
> > no easier format for both computers and humans to read, and in this case,
> > it includes extensibility, which is nice, since we have no idea how
> miners
> > will wish to divvy up their services; I think one would need to make a
> > strong case against JSON for a specific reason to not choose it by
> default.
>
> Bitcoin isn't "the web", it's a complicated script-based cryptocurrency.
> Everything in the Bitcoin protocol requires a computer's interpretation for
> humans, and there's no reason to stray from this default. Also, JSON is not
> extensible in any of the ways needed for this specific purpose.
>
> > 4) The text of the document delivered shall be a JSON format dictionary,
> > and shall include at minimum the following fields: 'min_fee',
> 'pool_name',
> > and 'last_modified' Optional fields can be determined over time as
> > necessary by the mining community
>
> Last Modified and other caching rules are dealt with in the relevant HTTP
> specification...
>
> > 5) The Service Level Agreement isn't binding, but miners who implement it
> > are expected to make a best efforts attempt to follow it.
>
> While it doesn't make sense to give it the full legal force of a contract,
> I
> think it should be expressed as a "MUST" in the BIP.
>
> > Generally a miner would occasionally publish the \mi:\ when they had
> > updated their SLA, or just every so often, but the canonical location
> would
> > be the final destination URL from the redirects.
>
> The coinbase advertisement MUST be part of every coinbase mined by the
> miner,
> or there's no reliable way to prove which blocks are theirs.
>
Greg Maxwell in reply to Jeff Garzik permalink original

On Thu, May 24, 2012 at 12:33 PM, Jeff Garzik <jgarzik@exmulti•com> wrote:

Quoted text (2 lines)
> There appears to be some non-trivial mining power devoted to mining
> empty blocks.

In the last 2016 blocks, as I write this, there are only 35 1 txn blocks.

This is about 1.73%, which wouldn't be surprising just from timing alone. Moreover, a fair amount (I didn't measure the percentage) appear to be mined by Eligius— Luke does some clever pre-computation of the hash tree for faster distribution right after new blocks.

Resources expended on fancy (and potentially risky) techno-economic hacks to discourage empty blocks would probably be better spent writing very fast transaction tree generating code.

Can we kill this thread now?

Luke Dashjr in reply to Peter Vessenes permalink original

On Tuesday, May 29, 2012 3:28:56 PM Peter Vessenes wrote:

Quoted text (1 line)
> I don't understand what the 20 byte keyhash is. Can you elucidate?

20 byte keyhashes are a fundamental building block of the Bitcoin protocol.

ripemd160(sha256(ecdsaPubKey))

Peter Vessenes in reply to Luke Dashjr permalink original

I suppose I mean that I don't understand how to reverse that into a URL when one is presented only with a block, or perhaps a coinbase in a transaction.

Best,

Peter

On Tue, May 29, 2012 at 11:34 AM, Luke-Jr <luke@dashjr•org> wrote:

Quoted text (7 lines)
> On Tuesday, May 29, 2012 3:28:56 PM Peter Vessenes wrote:
> > I don't understand what the 20 byte keyhash is. Can you elucidate?
>
> 20 byte keyhashes are a fundamental building block of the Bitcoin protocol.
>
> ripemd160(sha256(ecdsaPubKey))
>
Luke Dashjr in reply to Peter Vessenes permalink original

On Tuesday, May 29, 2012 3:36:34 PM Peter Vessenes wrote:

Quoted text (3 lines)
> I suppose I mean that I don't understand how to reverse that into a URL
> when one is presented only with a block, or perhaps a coinbase in a
> transaction.

A new message can be added to the p2p relay network, similar to tx and alert broadcasts, that allow miners to publish/update their policy URI signed by the key in question. Counter-DDoS rules could decline to relay or store URIs for keys that haven't been published in - or achieved statistical significance in - the last N blocks.

Peter Vessenes in reply to Luke Dashjr permalink original

I see. That is undeniably more secure and "bitcoin-y" than my suggestion.

It's also really a lot more work, especially in that it requires extra linkages between codebases that in my mind are largely separate.

I'm just one voice, but I persist in believing that the 'lighter' solution, especially for something that may not be a particularly big problem in the bitcoin world is good -- it carries much less technical implementation debt going forward, and has a lower risk of sort of seizing up development with additional necessary code to worry about for those implementing to-spec clients.

If that lighter solution turns out to be gameable, or has problems that require the full force of the bitcoin network and concepts, that would be the time to implement the improved version. That's just my approach, however. I worry that building in any additional requirements to the protocol or codebase adds significant cost to the network as a whole over the next 10 years.

Peter

On Tue, May 29, 2012 at 11:39 AM, Luke-Jr <luke@dashjr•org> wrote:

Quoted text (15 lines)
> On Tuesday, May 29, 2012 3:36:34 PM Peter Vessenes wrote:
> > I suppose I mean that I don't understand how to reverse that into a URL
> > when one is presented only with a block, or perhaps a coinbase in a
> > transaction.
>
> A new message can be added to the p2p relay network, similar to tx and
> alert
> broadcasts, that allow miners to publish/update their policy URI signed by
> the
> key in question. Counter-DDoS rules could decline to relay or store URIs
> for
> keys that haven't been published in - or achieved statistical significance
> in
> - the last N blocks.
>
Rebroad (sourceforge) in reply to Stefan Thomas permalink original

I'd like to garner consensus on whether anyone else thinks it desirable to have a flag option for bitcoin to punish blocks for not including transactions. I notice there are already pro-miner options, such as the restricting the relaying of free transactions, and so including an option to restrict relaying of blocks from "stingy" miners to balance against the current bias, so that the default bitcoin client can be run as much pro-miner as pro-non-miner.

On Monday, May 28, 2012, rebroad@gmail•com wrote:

Quoted text (10 lines)
> What i think this thread reveals is whether a bitcoin client is pro-miner
> or pro-non-miner. What i think is needed is a fork where one benefits
> miners (e.g. Limits relaying of free transactions, as has been added to the
> current default client), and one that benefits non-miners (e.g. Limits
> relaying of blocks not including free transactions). Then people can vote
> based on the client they use.
>
> It seems to me that the current main client is a pro-miner one, and
> possibly not what most people would vote for if they were given an easier
> choice.