Blocking uneconomical UTXO creation
23 messages, 10 to 13 Mar 2013. Started by Peter Todd.
Jorge Timón (6), Mike Hearn (3), Tadas Varanavičius (3), Peter Todd (2), Michael Gronager (2), Gavin Andresen, Jeff Garzik, Benjamin Lindner, +4 more
As discussed endlessly data in the UTXO set is more costly, especially in the long run, than transaction data itself. The fee system is per KB in a block, and thus doesn't properly capture the long-term costs of UTXO creation.
It's also been suggested multiple times to make transaction outputs with a value less than the transaction fee non-standard, either with a fixed constant or by some sort of measurement.
https://github.com/petertodd/bitcoin/tree/block-uneconomic-utxo-creation
The above patch that implements the latter approach, and thus will not accept into the mempool any txout whose value is <= the fee per KB paid by the transaction. That fee is then bounded between MIN_TX_FEE and COIN_DUST, 0.0005BTC and 0.01BTC respectively. The former due to the fact that the fee can be zero, and the latter so that delibrate high-fee creation is still allowed. (provably unspendable txouts can of course be handled specially later)
By basing the calculation on the fee per KB the transaction itself pays the limit automatically adjustes as the market for blockchain space changes, and the value of Bitcoins change.
Since scriptSigs greater than 500 bytes are non-standard the marginal bytes required to spend a txout is always less than 540 bytes. For standard transactions the marginal cost is usually just a 80+1 byte signature, and a 33+1 byte pubkey, or 155 bytes. Thus the choice of the fee for 1000bytes allows a margin to ensure a net positive return, even if tx fees become more expensive. In particular I think a reasonable margin is important to deter users from simply deleting wallets filled with dust-spam, something which gets reported as happening frequently. It also protects users who do not understand how Bitcoin works from thinking that repeated small amounts of coins collected from sites giving away small amounts will add up to an amount that they can usefully use, and equally protects the long-term health of the network from those services.
By basing the threshold for what is considered a too-low output value on ensuring that spending outputs has a net positive return, rather than trying to come up with some sort of model of UTXO cost, we make the logic significantly easier to reason about. In particular, it means that Bitcoin clients can use an unchanging rule based on fees paid, rather than some constant subject to change as the economics of UTXO costs change. Note how the total cost of maintaining the UTXO set is determined by the number of validating nodes, and what that number will be in the future heavily debated with a possible range spanning many orders of magnitude.
Short-term ----------
SatoshiDice will have to change their betting system to have their "failed bet" messages return enough coins to be economically spendable. It's notable that Satoshidice seems to have already changed their system to return what appear to be randomly chosen amounts, likely to get around the users who have applied custom patches to consider 1 satoshi output values non-standard. Because this patch does not block SatoshiDice, nor do I expect it to result in less SatoshiDice traffic, I expect pool operators to be open to applying it.
Other services such as CoinAd will also have to make changes to either collect multiple payments together, or use off-chain transactions. I've spoken with a person who runs one of these sites, CoinAd IIRC, and he was open to opening an instawallet or easywallet account and using it to do direct off-chain transactions for users who wanted to be paid immediately.
Part of the patch includes code that sends change to fees if creating a change output would produce an uneconomic txout. This will likely occasionally generate confusion from users, especially as it will only happen if they try to send almost all of their wallet.
Security --------
For a non-upgraded client, accepting zero-confirmation transactions becomes more risky as the change represents yet another way of creating a transaction that won't be mined. Fortnately the nLockTime problem has served to warn people yet again about those dangers.
Long-term ---------
If fees required on transactions go up in numerical value, the patch adapts the minimum output size as required.
If fees go down numerically, the minimum output size is also adjusted as required. If they go down sufficiently that MIN_TX_FEE requires changing, only one constant needs to change. In particular, the MIN_RELAY_TX_FEE blocks relaying small output values with small fees anyway, and it's set to one fifth of MIN_TX_FEE. Additionally most miners follow the MIN_TX_FEE default, so using that value ensures that the logic holds true for the more likely case that fees numerically stay stable or rise. In any case, Bitcoin will never be a good microtransaction system.
Ouputs representing other assets; "colored coins" and "smartcoins" ------------------------------------------------------------------
The colored coin ideas being passed around on the forums often used fixed representations of assets, that is 1 bond unit == 1 satoshi or similar. Since proving the state of a colored coin to an SPV client requires providing the full transaction history changing these protocols to allow a floating numerical ratio Bitcoins to assets is always possible by allowing for ordinary, non-marked, transaction inputs and outputs to add or remove coins as required. This has the additional advantage of making divisibility easy.
If the asset itself is worth less than a tx fee, moving it will still incur a net loss. Equally if the asset is worth more than a tx fee, the quickly the burden of requiring an additional tx fee to be locked up by the asset becomes minor.
Testing Required ----------------
To be written after more consensus. Essentially a UI testing script, and unittests in wallet_tests.cpp need to be written.
On 3/10/13, Peter Todd <pete@petertodd•org> wrote:
Quoted text (3 lines)
> It's also been suggested multiple times to make transaction outputs with > a value less than the transaction fee non-standard, either with a fixed > constant or by some sort of measurement.
As said on the bitcointalk thread, I think this is the wrong approach. This way you effectively disable legitimate use cases for payments that "are worth" less than the fees like smart property/colored coins. While the transactions pay fees, they should not be considered spam regardless of how little the quantities being moved are.
Then your only concern are unspent outputs and comparing fees with values doesn't help in any way. Just activate a non-proportional demurrage (well, I won't complain if you just turn bitcoin into freicoin, just think that non-proportional would be more acceptable by most bitcoiners) that incentives old transactions to be moved and destroys unspent transactions with small amounts that don't move to another address periodically. This has been proposed many times before too, and I think it makes a lot more sense.
Quoted text (1 line)
> Just activate a non-proportional demurrage
demurrage of any kind will never, ever happen, just give up on that idea.
The negative publicity of "the bitcoin developers are destroying YOUR coins!" would be devastating.
"The Bitcoin network will destroy your coins IF you don't move your coins" Is pretty different. By the way, doesn't have to destroy them, can just give them to miners.
In any case, what's wrong with my reasoning? Smart property/colored coins are not spam transactions because they pay fees.
The problem for the network are not transactions that move less coins than they pay fees, but old UNSPENT OUTPUTS. So why don't you focus on that instead of a formula to check what transactions make "economic sense"?
I even prefer the sudden "destruction" (or re-generation by miners) of the account after the X period (killerstorm's proposal) instead of just rejecting great potential use cases for the chain.
I mean, I still prefer a small fixed demurrage fee after those X blocks without moving them, but since this community is demurrage allergic and that possibility cannot even be considered (doesn't matter what reflects better the costs for miners/the network I guess), I'll go with the second best option IMO.
This would be just a fee for a resource that users are enjoying and has real costs for the network. Why would constant demurrage fees after a free storage period would be perceived so different from transaction fees?
I haven't heard anyone complaining about "the bitcoin developers are destroying part of YOUR coins every time you move them!!"
On 3/11/13, Gavin Andresen <gavinandresen@gmail•com> wrote:
Quoted text (11 lines)
>> Just activate a non-proportional demurrage > > demurrage of any kind will never, ever happen, just give up on that idea. > > The negative publicity of "the bitcoin developers are destroying YOUR > coins!" would be devastating. > > -- > -- > Gavin Andresen >
Unless of course everlasting physical "bitcoins" are much more important than smart property and colored coins...
On 3/11/13, Jorge Timón <jtimonmv@gmail•com> wrote:
Quoted text (52 lines)
> "The Bitcoin network will destroy your coins IF you don't move your coins" > Is pretty different. By the way, doesn't have to destroy them, can > just give them to miners. > > In any case, what's wrong with my reasoning? > Smart property/colored coins are not spam transactions because they pay > fees. > > The problem for the network are not transactions that move less coins > than they pay fees, but old UNSPENT OUTPUTS. So why don't you focus on > that instead of a formula to check what transactions make "economic > sense"? > > I even prefer the sudden "destruction" (or re-generation by miners) of > the account after the X period (killerstorm's proposal) instead of > just rejecting great potential use cases for the chain. > > I mean, I still prefer a small fixed demurrage fee after those X > blocks without moving them, but since this community is demurrage > allergic and that possibility cannot even be considered (doesn't > matter what reflects better the costs for miners/the network I guess), > I'll go with the second best option IMO. > > This would be just a fee for a resource that users are enjoying and > has real costs for the network. Why would constant demurrage fees > after a free storage period would be perceived so different from > transaction fees? > > I haven't heard anyone complaining about "the bitcoin developers are > destroying part of YOUR coins every time you move them!!" > > > On 3/11/13, Gavin Andresen <gavinandresen@gmail•com> wrote: >>> Just activate a non-proportional demurrage >> >> demurrage of any kind will never, ever happen, just give up on that idea. >> >> The negative publicity of "the bitcoin developers are destroying YOUR >> coins!" would be devastating. >> >> -- >> -- >> Gavin Andresen >> > > > -- > Jorge Timón > > http://freico.in/ > http://archive.ripple-project.org/ >
Why does demurrage even still come up? The base rules of Bitcoin will not be changing in such a fundamental way.
With regards to trying to minimize the size of the UTXO set, this again feels like a solution in search of a problem. Even with SD abusing micropayments as messages, it's only a few hundred megabytes today. That fits in RAM, let alone disk. If one day people do get concerned about the working set size, miners can independently set their own policies for what they confirm, for instance maybe they just bump the priority of any transaction that has fewer outputs than inputs. An IsStandard() rule now that tries to ban micropayments will just risk hurting interesting applications for no real benefit. It's like trying to anticipate and fix problems we might face in 2020.
There are lots of less invasive changes for improving scalability, like making transaction validation multi-threaded in every case, transmitting merkle blocks instead of full blocks, moving blocking disk IO off the main loop so nodes don't go unresponsive when somebody downloads the chain from them, and finishing the payment protocol work so there's less incentive to replicate the SD "transactions as messages" design.
Well, my initial idea was that nothing was really needed too. But if something must be done, I dislike very much the "ban micropayments" approach. I was just offering other solutions that I consider much better, but if nothing is done I won't be pushing for those alternative solutions (to a problem that we may not even have).
On 3/11/13, Mike Hearn <mike@plan99•net> wrote:
Quoted text (22 lines)
> Why does demurrage even still come up? The base rules of Bitcoin will > not be changing in such a fundamental way. > > With regards to trying to minimize the size of the UTXO set, this > again feels like a solution in search of a problem. Even with SD > abusing micropayments as messages, it's only a few hundred megabytes > today. That fits in RAM, let alone disk. If one day people do get > concerned about the working set size, miners can independently set > their own policies for what they confirm, for instance maybe they just > bump the priority of any transaction that has fewer outputs than > inputs. An IsStandard() rule now that tries to ban micropayments will > just risk hurting interesting applications for no real benefit. It's > like trying to anticipate and fix problems we might face in 2020. > > There are lots of less invasive changes for improving scalability, > like making transaction validation multi-threaded in every case, > transmitting merkle blocks instead of full blocks, moving blocking > disk IO off the main loop so nodes don't go unresponsive when somebody > downloads the chain from them, and finishing the payment protocol work > so there's less incentive to replicate the SD "transactions as > messages" design. >
On Mon, Mar 11, 2013 at 11:36 AM, Gavin Andresen <gavinandresen@gmail•com> wrote:
Quoted text (6 lines)
>> Just activate a non-proportional demurrage > > demurrage of any kind will never, ever happen, just give up on that idea. > > The negative publicity of "the bitcoin developers are destroying YOUR > coins!" would be devastating.
While 100% agreed, I do think there is space in the alt-currency world for a well-done coin THAT IS NOT BITCOIN (i.e. merge-mined or whatever) with a finite lifespan. Call it "tempcoin" For example: any coin older than (144 * 365 * 4) blocks may be reclaimed by a miner.
Even though, sadly, many of the alt-coins have been pre-mined scams, the alt-coin concept in general is great for experimenting. If the idea can be proven in the field without modifying mainnet bitcoind, then perhaps it should go onto the mainnet hard fork wishlist, many years in the future.
That's the great thing about open source. People can experiment with these ideas, and bitcoin.git need not change at all :)
On Mar 11, 2013, at 12:54 PM, Mike Hearn <mike@plan99•net> wrote:
Quoted text (4 lines)
> With regards to trying to minimize the size of the UTXO set, this > again feels like a solution in search of a problem. Even with SD > abusing micropayments as messages, it's only a few hundred megabytes > today. That fits in RAM, let alone disk. If one day people do get
The problem of UTXO in principal scales with the block size limit. Thus it should be fixed BEFORE you consider increasing the block size limit. Otherwise you just kick the can down the road, making it bigger.
Quoted text (2 lines)
> concerned about the working set size, miners can independently set > their own policies for what they confirm, for instance maybe they just
Problem is the skewed incentive structure. Rational miners will always include dust output with fees, because the eternal cost of UTXO is payed by the network and future miners, not the current/individual miner.
On Mar 11, 2013, at 7:01 AM, Jorge Timón <jtimonmv@gmail•com> wrote:
Quoted text (8 lines)
> On 3/10/13, Peter Todd <pete@petertodd•org> wrote: >> It's also been suggested multiple times to make transaction outputs with >> a value less than the transaction fee non-standard, either with a fixed >> constant or by some sort of measurement. > > As said on the bitcointalk thread, I think this is the wrong approach. > This way you effectively disable legitimate use cases for payments > that "are worth" less than the fees like smart property/colored coins.
this.
Quoted text (4 lines)
> Just activate a non-proportional > demurrage (well, I won't complain if you just turn bitcoin into > freicoin, just think that non-proportional would be more acceptable by > most bitcoiners) that incentives old transactions to be moved
You could delegate the decision to the user with a rule like:
if (output<fee): limit lifetime of the UTXO to 10 years. if (output>fee): unlimited lifetime
Then, when a user creates a transaction, he can decide whether he wants to have limited or unlimited lifetime. The rationale for limiting the lifetime for (output<fee) transactions is that they may have no inherent economic incentive to be spend.
On Mon, Mar 11, 2013 at 11:17 AM, Benjamin Lindner <ben@benlabs•net> wrote:
Quoted text (17 lines)
> > Just activate a non-proportional > > demurrage (well, I won't complain if you just turn bitcoin into > > freicoin, just think that non-proportional would be more acceptable by > > most bitcoiners) that incentives old transactions to be moved > > You could delegate the decision to the user with a rule like: > > if (output<fee): > limit lifetime of the UTXO to 10 years. > if (output>fee): > unlimited lifetime > > Then, when a user creates a transaction, he can decide whether he wants to > have limited or unlimited lifetime. The rationale for limiting the lifetime > for (output<fee) transactions is that they may have no inherent economic > incentive to be spend. >
If you think demurrage has a bad rep, wait until you see the response to escheatment (which is what's really being proposed here).
UTXO growth over time is worst-case linear, while computational resources increase exponentially. Mike nailed it on the head: all of this is a solution in search of a problem.
Mark
That solution seems good enough to me. Smartcoin users would just need to move their assets before 10 years, totally acceptable. And regular users don't need to think about it since they're probably always sending more than they pay in fees.
On 3/11/13, Benjamin Lindner <ben@benlabs•net> wrote:
Quoted text (60 lines)
> > On Mar 11, 2013, at 12:54 PM, Mike Hearn <mike@plan99•net> wrote: >> With regards to trying to minimize the size of the UTXO set, this >> again feels like a solution in search of a problem. Even with SD >> abusing micropayments as messages, it's only a few hundred megabytes >> today. That fits in RAM, let alone disk. If one day people do get > > The problem of UTXO in principal scales with the block size limit. Thus it > should be fixed BEFORE you consider increasing the block size limit. > Otherwise you just kick the can down the road, making it bigger. > >> concerned about the working set size, miners can independently set >> their own policies for what they confirm, for instance maybe they just > > Problem is the skewed incentive structure. Rational miners will always > include dust output with fees, because the eternal cost of UTXO is payed by > the network and future miners, not the current/individual miner. > > On Mar 11, 2013, at 7:01 AM, Jorge Timón <jtimonmv@gmail•com> wrote: > >> On 3/10/13, Peter Todd <pete@petertodd•org> wrote: >>> It's also been suggested multiple times to make transaction outputs with >>> a value less than the transaction fee non-standard, either with a fixed >>> constant or by some sort of measurement. >> >> As said on the bitcointalk thread, I think this is the wrong approach. >> This way you effectively disable legitimate use cases for payments >> that "are worth" less than the fees like smart property/colored coins. > > this. > >> Just activate a non-proportional >> demurrage (well, I won't complain if you just turn bitcoin into >> freicoin, just think that non-proportional would be more acceptable by >> most bitcoiners) that incentives old transactions to be moved > > You could delegate the decision to the user with a rule like: > > if (output<fee): > limit lifetime of the UTXO to 10 years. > if (output>fee): > unlimited lifetime > > Then, when a user creates a transaction, he can decide whether he wants to > have limited or unlimited lifetime. The rationale for limiting the lifetime > for (output<fee) transactions is that they may have no inherent economic > incentive to be spend. > > > ------------------------------------------------------------------------------ > Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester > Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the > endpoint security space. For insight on selecting the right partner to > tackle endpoint security challenges, access the full report. > http://p.sf.net/sfu/symantec-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
On 03/11/2013 08:17 PM, Benjamin Lindner wrote:
Quoted text (1 line)
> The problem of UTXO in principal scales with the block size limit. Thus it should be fixed BEFORE you consider increasing the block size limit. Otherwise you just kick the can down the road, making it bigger.
Let's assume bitcoin has scaled up to 2000 tx/s. We all want this, right? https://en.bitcoin.it/wiki/Scalability. Block size is 500 MB. CPU, network and archival blockchain storage seem to solvable.
Let's say SatoshiDice-like systems are doing informational transactions that produce unspendable outputs, because they can and users are paying for it anyway (proved in real life). 400 unspendable outputs per second would be realistic.
This would be bloating UTXO data at a speed of 52 GB/year. That's a very big memory leak. And this is just the unspendable outputs.
Bitcoin cannot scale up until such dust output spamming is discouraged at the protocol level.
On Mon, Mar 11, 2013 at 12:01 PM, Jorge Timón <jtimonmv@gmail•com> wrote:
Quoted text (21 lines)
> On 3/10/13, Peter Todd <pete@petertodd•org> wrote: > > It's also been suggested multiple times to make transaction outputs with > > a value less than the transaction fee non-standard, either with a fixed > > constant or by some sort of measurement. > > As said on the bitcointalk thread, I think this is the wrong approach. > This way you effectively disable legitimate use cases for payments > that "are worth" less than the fees like smart property/colored coins. > While the transactions pay fees, they should not be considered spam > regardless of how little the quantities being moved are. > > Then your only concern are unspent outputs and comparing fees with > values doesn't help in any way. > Just activate a non-proportional > demurrage (well, I won't complain if you just turn bitcoin into > freicoin, just think that non-proportional would be more acceptable by > most bitcoiners) that incentives old transactions to be moved and > destroys unspent transactions with small amounts that don't move to > another address periodically. This has been proposed many times before > too, and I think it makes a lot more sense. >
From an economic point of view this *does* make sense, in my opinion. Storing an unspent transaction in the block chain costs money because we can't prune it. However, it would completely destroy confidence in Bitcoin, as far as I can see. It makes sense economically, but it isn't feasible if we want to maintain people's confidence in Bitcoin.
I like Jeff's proposal of letting an alt-coin implement this. If it gets to the point where Bitcoin can't function without this functionality, it'll be a lot easier to make the transition, instead of now, when it's not really needed, and the trust in Bitcoin really isn't that great.
/Rune
Quoted text (13 lines)
> > > ------------------------------------------------------------------------------ > Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester > Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the > endpoint security space. For insight on selecting the right partner to > tackle endpoint security challenges, access the full report. > http://p.sf.net/sfu/symantec-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development >
The point with UTXO is in the long run to be able to switch from a p2p network where everyone stores, validates and verifies everything to a DHT where the load of storing, validating and verifying can be shared.
If we succeed with that then I don't see a problem in a growing set of UTXO, may that be due to abuse/misuse or just massive use. A properly designed DHT should be able to scale to this.
However, that being said, if you worry about the size of the UTXO set you should change the current coin choosing algorithm to simply get rid of dust.
The current algorithm (ApproximateBestSubset) tend to accumulate dust as dust tend to be on an other scale than a real transactions and hence it is never included.
Regarding the demurrage/escheatment road, I agree that this is for another project. However, if users/developers like this idea, they can just implement a coin choosing algorithm donating dust as miner fee and use it on their satoshi-dice polluted wallet ;)
/M
On 11/03/2013, at 21:08, Rune Kjær Svendsen <runesvend@gmail•com> wrote:
Quoted text (52 lines)
> On Mon, Mar 11, 2013 at 12:01 PM, Jorge Timón <jtimonmv@gmail•com> wrote: > On 3/10/13, Peter Todd <pete@petertodd•org> wrote: > > It's also been suggested multiple times to make transaction outputs with > > a value less than the transaction fee non-standard, either with a fixed > > constant or by some sort of measurement. > > As said on the bitcointalk thread, I think this is the wrong approach. > This way you effectively disable legitimate use cases for payments > that "are worth" less than the fees like smart property/colored coins. > While the transactions pay fees, they should not be considered spam > regardless of how little the quantities being moved are. > > Then your only concern are unspent outputs and comparing fees with > values doesn't help in any way. > > > Just activate a non-proportional > demurrage (well, I won't complain if you just turn bitcoin into > freicoin, just think that non-proportional would be more acceptable by > most bitcoiners) that incentives old transactions to be moved and > destroys unspent transactions with small amounts that don't move to > another address periodically. This has been proposed many times before > too, and I think it makes a lot more sense. > > From an economic point of view this *does* make sense, in my opinion. Storing an unspent transaction in the block chain costs money because we can't prune it. However, it would completely destroy confidence in Bitcoin, as far as I can see. It makes sense economically, but it isn't feasible if we want to maintain people's confidence in Bitcoin. > > I like Jeff's proposal of letting an alt-coin implement this. If it gets to the point where Bitcoin can't function without this functionality, it'll be a lot easier to make the transition, instead of now, when it's not really needed, and the trust in Bitcoin really isn't that great. > > /Rune > > > > ------------------------------------------------------------------------------ > Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester > Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the > endpoint security space. For insight on selecting the right partner to > tackle endpoint security challenges, access the full report. > http://p.sf.net/sfu/symantec-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > ------------------------------------------------------------------------------ > Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester > Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the > endpoint security space. For insight on selecting the right partner to > tackle endpoint security challenges, access the full report. > http://p.sf.net/sfu/symantec-dev2dev_______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
On Mon, Mar 11, 2013 at 1:36 PM, Michael Gronager <gronager@ceptacle•com> wrote:
Quoted text (1 line)
> The point with UTXO is in the long run to be able to switch from a p2p network where everyone stores, validates and verifies everything to a DHT where the load of storing, validating and verifying can be shared.
I believe you are confusing disjoint things.
Quoted text (3 lines)
>> The point with UTXO is in the long run to be able to switch from a p2p network where everyone stores, validates and verifies everything to a DHT where the load of storing, validating and verifying can be shared. > > I believe you are confusing disjoint things.
Nope, ahh well, I agree that the use of UTXOs in the Satoshi client today by no means a directed towards a DHT, though it does help speeding up validation (db lookup to check if an output is indeed unspent).
However, an alternative way to bootstrap and validate transactions exist, needing only the UTXOs and not the rest of the blockchain history: An authenticated data structure storing the UTXOs in a DHT.
/M
Quoted text (11 lines)
> > ------------------------------------------------------------------------------ > Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester > Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the > endpoint security space. For insight on selecting the right partner to > tackle endpoint security challenges, access the full report. > http://p.sf.net/sfu/symantec-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Quoted text (2 lines)
> This would be bloating UTXO data at a speed of 52 GB/year. That's a very > big memory leak. And this is just the unspendable outputs.
Firstly, the UTXO set is a LevelDB, it's not stored in memory. Outputs that never get spent are not in the working set by definition, after a while they just end up in the bottom levels and hardly ever get accessed. If need be we can always help LevelDB out a bit by moving outputs that we suspect are unlikely to get spent into a separate database, but I doubt it's needed.
Secondly, if an output can be proven unspendable it can be pruned immediately. We already reached consensus on adding some standard template using OP_RETURN that results in insta-pruning. So people who want to create unspendable outputs can do so with the only side-effect being long term chain storage. It would be effectively "free" to pruning nodes.
So the issue is not really with unspendable outputs but with low-value spendable outputs. Wallets with lots of tiny outputs end up generating large transactions that take a long time to verify, in situations where the network redlines those transactions would end up at the bottom of the priority queue and might take longer to confirm. So wallet apps already have incentives to try and find a good balance in output sizes and defragment themselves if their average output gets too low in value, eg, by send-to-self transactions at night.
On 03/12/2013 12:19 AM, Mike Hearn wrote:
Quoted text (6 lines)
> Firstly, the UTXO set is a LevelDB, it's not stored in memory. Outputs > that never get spent are not in the working set by definition, after a > while they just end up in the bottom levels and hardly ever get > accessed. If need be we can always help LevelDB out a bit by moving > outputs that we suspect are unlikely to get spent into a separate > database, but I doubt it's needed.
Isn't there danger of an attack if UTXO is not stored in fast storage?
Quoted text (1 line)
> Isn't there danger of an attack if UTXO is not stored in fast storage?
RAM is used as a database cache.
But regardless, what kind of attack are you thinking of? Using up all available disk seeks by sending a node a lot of fake transactions that connect to unspent outputs, but have invalid transactions? You'll get yourself disconnected and the IP banned even with todays code.
It's much easier to hose a node by just asking it to send you the block chain. Watch your own node when something is syncing the chain from it. Ping times go through the roof because there's only one network thread. If you're worried about DoS attacks on Bitcoin, it'd be better to fix that first.
On 03/12/2013 12:39 AM, Mike Hearn wrote:
Quoted text (6 lines)
> RAM is used as a database cache. > > But regardless, what kind of attack are you thinking of? Using up all > available disk seeks by sending a node a lot of fake transactions that > connect to unspent outputs, but have invalid transactions? You'll get > yourself disconnected and the IP banned even with todays code.
I'm thinking that (assuming 2000 tx/s and UTXO growing 50 GB/year) a malicious miner could create 1 GB of unspent outputs and then spam nodes with valid transactions. This would not be dangerous, if UTXO were smaller and fit on RAM.
Thank you for reading and correcting me :)
On Sat, Mar 09, 2013 at 11:31:55PM -0500, Peter Todd wrote:
Quoted text (4 lines)
> As discussed endlessly data in the UTXO set is more costly, especially > in the long run, than transaction data itself. The fee system is per KB > in a block, and thus doesn't properly capture the long-term costs of > UTXO creation.
There's been a lot of discussion about this issue, and many people have asked that Bitcoin not arbitrarily block interesting potential uses of provably unspendable txouts for data applications, and similarly spendable txouts representing assets. I've changed my hardline position and now think we should support all that stuff. However, there is one remaining class of txout not yet talked about, unspendable but not provably so txouts. For instance we could make the following a standard transaction type:
scriptPubKey: OP_HASH160 <20 byte digest> OP_EQUALVERIFY <data> scriptSig: <data>
Of course, usually the 20 byte digest would be picked randomly, but it might not be, and thus all validating nodes will always have a copy of the data. With the 10KB limit on script sizes you can fit 9974 bytes of data per transaction output with very little waste.
A good application is timestamping, with the advantage over coinbase/merkle tree systems in that you don't have to wait until your timestamp confirms, or even store the timestamp at all. Another application, quite possible with large block sizes and hence cheap or free transactions, is secure data backups. In particular such a service, perhaps called Google Chain Storage, can offer the unique guarantee that you can know you're data is secure by simply performing a successful Bitcoin transaction.
Instead of thinking in terms of blocking uneconomical transactions (how would a node even determine what's economical?), what about thinking in terms of paying for a feed of economical (i.e. profitable) transactions? There is a market for fee bearing, profitable transactions...if there is no one willing to pay to receive a transaction, then no one will bother propagating it. Such a system would make it possible to determine the probability of confirmation in a given timeframe for a given fee.
On Tue, Mar 12, 2013 at 3:49 AM, Peter Todd <pete@petertodd•org> wrote:
Quoted text (48 lines)
> On Sat, Mar 09, 2013 at 11:31:55PM -0500, Peter Todd wrote: > > As discussed endlessly data in the UTXO set is more costly, especially > > in the long run, than transaction data itself. The fee system is per KB > > in a block, and thus doesn't properly capture the long-term costs of > > UTXO creation. > > There's been a lot of discussion about this issue, and many people have > asked that Bitcoin not arbitrarily block interesting potential uses of > provably unspendable txouts for data applications, and similarly > spendable txouts representing assets. I've changed my hardline position > and now think we should support all that stuff. However, there is one > remaining class of txout not yet talked about, unspendable but not > provably so txouts. For instance we could make the following a standard > transaction type: > > scriptPubKey: OP_HASH160 <20 byte digest> OP_EQUALVERIFY <data> > scriptSig: <data> > > Of course, usually the 20 byte digest would be picked randomly, but it > might not be, and thus all validating nodes will always have a copy of > the data. With the 10KB limit on script sizes you can fit 9974 bytes of > data per transaction output with very little waste. > > A good application is timestamping, with the advantage over > coinbase/merkle tree systems in that you don't have to wait until your > timestamp confirms, or even store the timestamp at all. Another > application, quite possible with large block sizes and hence cheap or > free transactions, is secure data backups. In particular such a service, > perhaps called Google Chain Storage, can offer the unique guarantee that > you can know you're data is secure by simply performing a successful > Bitcoin transaction. > > -- > 'peter'[:-1]@petertodd.org > > > ------------------------------------------------------------------------------ > Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester > Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the > endpoint security space. For insight on selecting the right partner to > tackle endpoint security challenges, access the full report. > http://p.sf.net/sfu/symantec-dev2dev > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > >
I'm not sure I understand your proposal, but its sounds good. Can you elaborate with an example? Are you considering colored coins/smart property?
On 3/13/13, Stephen Pair <stephen@bitpay•com> wrote:
Quoted text (70 lines)
> Instead of thinking in terms of blocking uneconomical transactions (how > would a node even determine what's economical?), what about thinking in > terms of paying for a feed of economical (i.e. profitable) transactions? > There is a market for fee bearing, profitable transactions...if there is no > one willing to pay to receive a transaction, then no one will bother > propagating it. Such a system would make it possible to determine the > probability of confirmation in a given timeframe for a given fee. > > > On Tue, Mar 12, 2013 at 3:49 AM, Peter Todd <pete@petertodd•org> wrote: > >> On Sat, Mar 09, 2013 at 11:31:55PM -0500, Peter Todd wrote: >> > As discussed endlessly data in the UTXO set is more costly, especially >> > in the long run, than transaction data itself. The fee system is per KB >> > in a block, and thus doesn't properly capture the long-term costs of >> > UTXO creation. >> >> There's been a lot of discussion about this issue, and many people have >> asked that Bitcoin not arbitrarily block interesting potential uses of >> provably unspendable txouts for data applications, and similarly >> spendable txouts representing assets. I've changed my hardline position >> and now think we should support all that stuff. However, there is one >> remaining class of txout not yet talked about, unspendable but not >> provably so txouts. For instance we could make the following a standard >> transaction type: >> >> scriptPubKey: OP_HASH160 <20 byte digest> OP_EQUALVERIFY <data> >> scriptSig: <data> >> >> Of course, usually the 20 byte digest would be picked randomly, but it >> might not be, and thus all validating nodes will always have a copy of >> the data. With the 10KB limit on script sizes you can fit 9974 bytes of >> data per transaction output with very little waste. >> >> A good application is timestamping, with the advantage over >> coinbase/merkle tree systems in that you don't have to wait until your >> timestamp confirms, or even store the timestamp at all. Another >> application, quite possible with large block sizes and hence cheap or >> free transactions, is secure data backups. In particular such a service, >> perhaps called Google Chain Storage, can offer the unique guarantee that >> you can know you're data is secure by simply performing a successful >> Bitcoin transaction. >> >> -- >> 'peter'[:-1]@petertodd.org >> >> >> ------------------------------------------------------------------------------ >> Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester >> Wave(TM): Endpoint Security, Q1 2013 and "remains a good choice" in the >> endpoint security space. For insight on selecting the right partner to >> tackle endpoint security challenges, access the full report. >> http://p.sf.net/sfu/symantec-dev2dev >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists•sourceforge•net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> >> > > > -- > Stephen Pair, Co-Founder, CTO > > Does *your* website accept cash? bitpay.com > > [image: bitpay-small] > > ABC6 C11B BF75 9E2B FC6A B3E0 7B96 40B2 CAC0 C158 >