Composite priority: combining fees and bitcoin-days into one number

Assigning 5% of block space based on bitcoin-days destroyed (BDD) and the other 95% based on fees seems like a rather awkward approach to me. For one thing, it means two code paths in pretty much every procedure dealing with a constrained resource (e.g. mempool, CNB). This makes code harder two write, harder to maintain, and slower to execute. As a result, some people have proposed eliminating BDD priority altogether. I have another idea.

We can create and maintain a conversion rate between BDD and fees to create a composite priority metric. Then we just do compPrio = BDD * conversionRate + txFee.

How do we calculate conversionRate? We want the following equation to be true:

sum(fees) = sum(BDD) * conversionRate * BDDweight

So we sum up the mempool fees, and we sum up the mempool BDD. We get a policy statement from the command line for a relative weight of BDD vs fees (default 0.05), and then conversionRate = (summedFees / summedBDD) * BDDWeight.

As an optimization, rather than scanning over the whole mempool to calculate this, we can just store the sum and add or subtract from it each time a tx enters or leaves the mempool. In order to minimize drift (the BDD for a transaction changes over time), we recalculate the whole thing each time a new block is found.

On Wednesday, October 28, 2015 4:26:52 AM Jonathan Toomim via bitcoin-dev wrote:

> Assigning 5% of block space based on bitcoin-days destroyed (BDD) and the
> other 95% based on fees seems like a rather awkward approach to me. For
> one thing, it means two code paths in pretty much every procedure dealing
> with a constrained resource (e.g. mempool, CNB). This makes code harder
> two write, harder to maintain, and slower to execute.

This is all in the realm of node policy, which must be easy to modify/customise in a flexible manner. So simplifying other code in a way that makes the policy harder to configure is not a welcome change.

That is, by making the code simpler, if you make custom policies (such as the current default) harder, it is better to leave the main code less simple.

Luke

On Oct 28, 2015, at 12:13 AM, Luke Dashjr <luke@dashjr•org> wrote:

> On Wednesday, October 28, 2015 4:26:52 AM Jonathan Toomim via bitcoin-dev
> wrote:
>
> This is all in the realm of node policy, which must be easy to
> modify/customise in a flexible manner. So simplifying other code in a way that
> makes the policy harder to configure is not a welcome change.
>
> That is, by making the code simpler, if you make custom policies (such as the
> current default) harder, it is better to leave the main code less simple.

I think the only custom policy that this change would make harder to implement is the current default policy of 5% reserved space. Right now, in e.g. CreateNewBlock, you have two loops, each of which follows a completely different policy, plus additional code for corner cases like ensuring that a tx isn't added twice. If I were a miner and a mediocre programmer (which I actually am, on both accounts), and I wanted to change the mining policy, I would probably take a look at that code, groan, give up, and go sharpen my pickaxe instead.

This change could be written in an abstract way. We could define an API that is calibrated on the whole mempool, then has a method that takes transactions and returns priority scores.

If someone wanted to write a reserved-space algorithm in this priority API scheme, then they could just set it up so that most transactions would get a priority score between e.g. zero and 8999, and any transactions that were supposed to be prioritized would get a priority level over 9000. Easy enough?

On Wednesday, October 28, 2015 10:41:39 PM Jonathan Toomim wrote:

> On Oct 28, 2015, at 12:13 AM, Luke Dashjr <luke@dashjr•org> wrote:
> > On Wednesday, October 28, 2015 4:26:52 AM Jonathan Toomim via bitcoin-dev
> > wrote:
> >
> > This is all in the realm of node policy, which must be easy to
> > modify/customise in a flexible manner. So simplifying other code in a way
> > that makes the policy harder to configure is not a welcome change.
> >
> > That is, by making the code simpler, if you make custom policies (such as
> > the current default) harder, it is better to leave the main code less
> > simple.
>
> I think the only custom policy that this change would make harder to
> implement is the current default policy of 5% reserved space. Right now,
> in e.g. CreateNewBlock, you have two loops, each of which follows a
> completely different policy, plus additional code for corner cases like
> ensuring that a tx isn't added twice. If I were a miner and a mediocre
> programmer (which I actually am, on both accounts), and I wanted to change
> the mining policy, I would probably take a look at that code, groan, give
> up, and go sharpen my pickaxe instead.

Yes, I hope to improve the code significantly.

> This change could be written in an abstract way. We could define an API
> that is calibrated on the whole mempool, then has a method that takes
> transactions and returns priority scores.

Trying to communicate policies as simple numbers is significantly more complicated for the policy-writer than what we have now.

> If someone wanted to write a reserved-space algorithm in this priority API
> scheme, then they could just set it up so that most transactions would get
> a priority score between e.g. zero and 8999, and any transactions that
> were supposed to be prioritized would get a priority level over 9000. Easy
> enough?

No, because it gets exponentially harder when there are more than two factors involved.

Luke