New "feefilter" p2p message

Hi,

I'm proposing the addition of a new optional p2p message to help reduce unnecessary network traffic. The draft BIP is available here and pasted below: https://gist.github.com/morcos/9aab223c443c9258c979

The goal of this message is to take advantage of the fact that when a node has reached its mempool limit, there is a minimum fee below which no transactions are accepted to the mempool. Informing peers of this minimum would save them inv'ing your node for those transaction id's and save your node requesting them if they are not in your recentRejects filter.

This message is optional and may be ignored as a protocol rule. There is also an option to turn off sending the messages in the implementation.

Thanks to Suhas Daftuar, Greg Maxwell, and others for helping develop the idea.

-Alex

Draft BIP text:

<pre>
  BIP: <unassigned>
  Title: feefilter message
  Author: Alex Morcos <morcos@chaincode•com>
  Status: Draft
  Type: Standards Track
  Created: 2016-02-13
</pre>

==Abstract==

Add a new message, "feefilter", which serves to instruct peers not to send "inv"'s to the node for transactions with fees below the specified fee rate.

==Motivation==

The concept of a limited mempool was introduced in Bitcoin Core 0.12 to provide protection against attacks or spam transactions of low fees that are not being mined. A reject filter was also introduced to help prevent repeated requests for the same transaction that might have been recently rejected for insufficient fee. These methods help keep resource utilization on a node from getting out of control.

However, there are limitations to the effectiveness of these approaches. The reject filter is reset after every block which means transactions that are inv'ed over a longer time period will be rerequested and there is no method to prevent requesting the transaction the first time. Furthermore, inv data is sent at least once either to or from each peer for every transaction accepted to the mempool and there is no mechanism by which to know that an inv sent to a given peer would not result in a getdata request because it represents a transaction with too little fee.

After receiving a feefilter message, a node can know before sending an inv that a given transaction's fee rate is below the minimum currently required by a given peer, and therefore the node can skip relaying an inv for that transaction to that peer.

==Specification==

# The feefilter message is defined as a message containing an int64_t where pchCommand == "feefilter" # Upon receipt of a "feefilter" message, the node will be permitted, but not required, to filter transaction invs for transactions that fall below the feerate provided in the feefilter message interpreted as satoshis per kilobyte. # The fee filter is additive with a bloom filter for transactions so if an SPV client were to load a bloom filter and send a feefilter message, transactions would only be relayed if they passed both filters. # Inv's generated from a mempool message are also subject to a fee filter if it exists. # Feature discovery is enabled by checking protocol version >= 70013

==Considerations== The propagation efficiency of transactions across the network should not be adversely affected by this change. In general, transactions which are not accepted to your mempool are not relayed and the funcionality implemented with this message is meant only to filter those transactions. There could be a small number of edge cases where a node's mempool min fee is actually less than the filter value a peer is aware of and transactions with fee rates between these values will now be newly inhibited.

Feefilter messages are not sent to whitelisted peers if the "-whitelistforcerelay" option is set. In that case, transactions are intended to be relayed even if they are not accepted to the mempool.

There are privacy concerns with deanonymizing a node by the fact that it is broadcasting identifying information about its mempool min fee. To help ameliorate this concern, the implementaion quantizes the filter value broadcast with a small amount of randomness, in addition, the messages are broadcast to different peers at individually randomly distributed times.

If a node is using prioritisetransaction to accept transactions whose actual fee rates might fall below the node's mempool min fee, it may want to consider setting "-nofeefilter" to make sure it is exposed to all possible txid's.

==Backward compatibility==

Older clients remain fully compatible and interoperable after this change. The sending of feefilter messages can be disabled by unsetting the "-feefilter" option.

==Implementation==

https://github.com/bitcoin/bitcoin/pull/7542

On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev wrote:

> # The feefilter message is defined as a message containing an int64_t where
> pchCommand == "feefilter"

What happened to extensibility? And why waste 64 bits for what is almost certainly a small number?

> # The fee filter is additive with a bloom filter for transactions so if an
> SPV client were to load a bloom filter and send a feefilter message,
> transactions would only be relayed if they passed both filters.

This seems to make feefilter entirely useless for wallets?

Luke

On Tue, Feb 16, 2016 at 6:46 PM, Luke Dashjr <luke@dashjr•org> wrote:

> On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev wrote:
> > # The feefilter message is defined as a message containing an int64_t
> where
> > pchCommand == "feefilter"
>
> What happened to extensibility? And why waste 64 bits for what is almost
> certainly a small number?
>

I thought that extensibility was already sufficient with the command string system. If we come up with a better version of the feefilter later we can just give it a different command name. This seemed to encapsulate a fairly complete idea for now. As for the 8 bytes, it didn't seem necessary to me to over optimize with a custom encoding for what amounts to well under 20% of ping traffic. (pings are sent every 2 mins per peer, feefilters on average every 10 mins, but when the quantized value to be sent would be the same it is skipped)

> > # The fee filter is additive with a bloom filter for transactions so if
> an
> > SPV client were to load a bloom filter and send a feefilter message,
> > transactions would only be relayed if they passed both filters.
>
> This seems to make feefilter entirely useless for wallets?
>
>

I don't follow this comment? Transactions aren't synced with the wallet unless they are accepted into the mempool. Sending feefilter messages should not reduce the number of transactions that are accepted to the mempool.

> Luke
>

On Wednesday, February 17, 2016 2:28:31 AM Alex Morcos wrote:

> On Tue, Feb 16, 2016 at 6:46 PM, Luke Dashjr <luke@dashjr•org> wrote:
> > On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev

wrote:

> > > # The feefilter message is defined as a message containing an int64_t
> >
> > where
> >
> > > pchCommand == "feefilter"
> >
> > What happened to extensibility? And why waste 64 bits for what is almost
> > certainly a small number?
>
> I thought that extensibility was already sufficient with the command string
> system.  If we come up with a better version of the feefilter later we can
> just give it a different command name.

We shouldn't need a new protocol [extension] for every new policy. Obviously this can't be perfectly flexible, but supporting different feerate definition versions is trivial and obvious.

> > > # The fee filter is additive with a bloom filter for transactions so if
> > an
> > > SPV client were to load a bloom filter and send a feefilter message,
> > > transactions would only be relayed if they passed both filters.
> >
> > This seems to make feefilter entirely useless for wallets?
>
> I don't follow this comment?  Transactions aren't synced with the wallet
> unless they are accepted into the mempool.  Sending feefilter messages
> should not reduce the number of transactions that are accepted to the
> mempool.

In Core, they aren't (but Core never uses bloom filters anyway) - because otherwise it would leak privacy. But light clients (particularly overlapping with those that use bloom filters!) have no privacy in the first place, so they have no reason to use this rule.

Luke

On Wed, Feb 17, 2016 at 12:46 AM, Luke Dashjr via bitcoin-dev <bitcoin-dev@lists•linuxfoundation•org> wrote:

> On Tuesday, February 16, 2016 8:20:26 PM Alex Morcos via bitcoin-dev wrote:
>> # The feefilter message is defined as a message containing an int64_t where
>> pchCommand == "feefilter"
>
> What happened to extensibility?

I did think it might be interesting to do a priorityrate filter. but since it seems no one is even working on adding an index for ancestor priorityrate... or working on a space limited priority mempool... if that extension would be needed it could just be a new "priorityfilt" command.

> And why waste 64 bits for what is almost
> certainly a small number?

Technically fees per byte could be greater than 32 bits (e.g. a 9000 BTC fee is enough). Values are normally 64 bits already.

>> # The fee filter is additive with a bloom filter for transactions so if an
>> SPV client were to load a bloom filter and send a feefilter message,
>> transactions would only be relayed if they passed both filters.
>
> This seems to make feefilter entirely useless for wallets?

I think your reasoning is that you want to learn of your own transactions even if they don't meet the filter?

I'm not sure this reasoning plays out though-- regardless of what your own feefilter is, if a tx has too low a rate for your peers to relay it, they won't and you won't learn of it.

I might wave my hands at a use case for OR "I will relay very high fee txn ... or my own"; but considering that performance/privacy disaster that bloom filters are-- and that to relay third party txn at all you need to be able to validate them (or will get yourself banned). Also, if you really want the OR behavior you could make two connections... the same cannot be said for and.

Maybe one argument I could add is that if we added a priorityrate filter, that one would be an OR