Mailman incompatibility with DKIM ...
9 messages, 19 to 20 Jun 2015. Started by Warren Togami.
Warren Togami (3), Mike Hearn (3), Adam Weiss (2), Jeff Garzik
Both you and jgarzik experienced mail getting tossed into gmail's spam folder thanks to DKIM... I am concerned that DKIM is too fragile and not very compatible with mailing lists.
We already removed the footer because it was incompatible with DKIM signing. Keeping the "[Bitcoin-dev] " prepend tag in subject is compatible with DKIM header signing only if the poster manually prepends it in their subject header.
I am already concerned that the lack of the Mailman footer will make it hard to identify where exactly subscribers need to go to unsubscribe or look at archives. Removing the subject tag might make DKIM enforcement work a lot better, but I can easily see our obtuse subscribers as being extra confused by this.
Opinions?
Warren
On Thu, Jun 18, 2015 at 11:38 PM, Arthur <arthur@powaaa•com> wrote:
Quoted text (6 lines)
> warren | bad_duck: try manually adding "[Bitcoin-dev] " to the beginning > of the subject > > -- > Arthur >
Quoted text (6 lines)
> > We already removed the footer because it was incompatible with DKIM > signing. Keeping the "[Bitcoin-dev] " prepend tag in subject is compatible > with DKIM header signing only if the poster manually prepends it in their > subject header. >
I still see footers being added to this list by SourceForge?
Quoted text (2 lines)
> Opinions? >
I've asked Jeff to not use his @bitpay.com account for now.
The only real fix is to use a mailing list operator that is designed to operate correctly with DKIM/DMARC, either by not modifying messages in transit, or by re-sending (and ideally re-signing) under their own identity.
Though I'm sure this won't be an issue for the Linux Foundation, the latter approach is dangerous because it means the list operator takes full responsibility for any spamming that occurs from that domain. If the mail server is ever hacked or spammers start posting to the lists themselves, all that spam will be seen as originating from the listserv itself and the reputation will be degraded. It can end with everyone's mail going to the spam folder.
On Thu, Jun 18, 2015 at 11:56 PM, Mike Hearn <mike@plan99•net> wrote:
Quoted text (8 lines)
> We already removed the footer because it was incompatible with DKIM >> signing. Keeping the "[Bitcoin-dev] " prepend tag in subject is compatible >> with DKIM header signing only if the poster manually prepends it in their >> subject header. >> > > I still see footers being added to this list by SourceForge? >
The new list currently has footers removed during testing. I am not pleased with the need to remove the subject tag and footer to be more compatible with DKIM users.
Quoted text (8 lines)
> > >> Opinions? >> > > I've asked Jeff to not use his @bitpay.com account for now. > >
I'm guessing DKIM enforcement is not very common because of issues like this?
It seems that Sourceforge silently drops DKIM enforced mail like jgarzik's. LF seems to pass along their mail but mangles the header/body and makes DKIM verification fail, which causes gmail to toss it into the spam folder. I think this behavior is slightly worse than Sourceforge because it makes the poster think their message was successfully sent (it is in the archive), but many subscribers never see it due to the spam binning.
I don't see any good solution to this except an auto-reject for DKIM enforced domain postings. Yes this is rather terrible, but the instant rejection is vastly better than Sourceforge silently dropping the post or LF getting stuck in spam filters.
We should also auto-reject any other reason for mail getting stuck in the moderation queue like including non-subscribers. I considered auto-rejecting spam too, but that could go horribly wrong as a false From address could make the Mailman server into a spammer itself. We may have no choice but to silently drop spam for that reason.
Warren
Quoted text (5 lines)
> > The new list currently has footers removed during testing. I am not > pleased with the need to remove the subject tag and footer to be more > compatible with DKIM users. >
Lists can do what are effectively MITM attacks on people's messages in any way they like, if they resign for the messages themselves. That seems fair to me! :)
Quoted text (3 lines)
> I'm guessing DKIM enforcement is not very common because of issues like > this? >
DKIM is used by most mail on the internet. DMARC rules that publish in DNS statements like "All mail from bitpay.com is signed correctly so trash any that isn't" are used on some of the worlds most heavily phished domains like google.com, PayPal, eBay, and indeed BitPay.
These rules are understood and enforced by all major webmail providers including Gmail. It's actually only rusty geek infrastructure that has problems with this, I've never heard of DKIM/DMARC users having issues outside of dealing with mailman. The vast majority of email users who never post to technical mailing lists benefit from it significantly.
Really everyone should use them. Adding cryptographic integrity to email is hardly a crazy idea :)
Quoted text (2 lines)
> It seems that Sourceforge silently drops DKIM enforced mail like jgarzik's. >
It's not SourceForge, it's your spam filter. His mail gets through to me but it's all in the spam folder.
On Fri, Jun 19, 2015 at 12:24 AM, Mike Hearn <mike@plan99•net> wrote:
Quoted text (9 lines)
> The new list currently has footers removed during testing. I am not >> pleased with the need to remove the subject tag and footer to be more >> compatible with DKIM users. >> > > Lists can do what are effectively MITM attacks on people's messages in any > way they like, if they resign for the messages themselves. That seems fair > to me! :) >
Mailman isn't resigning it. Should it be? Does other mailing list software?
Quoted text (20 lines)
> > >> I'm guessing DKIM enforcement is not very common because of issues like >> this? >> > > DKIM is used by most mail on the internet. DMARC rules that publish in DNS > statements like "All mail from bitpay.com is signed correctly so trash > any that isn't" are used on some of the worlds most heavily phished domains > like google.com, PayPal, eBay, and indeed BitPay. > > These rules are understood and enforced by all major webmail providers > including Gmail. It's actually only rusty geek infrastructure that has > problems with this, I've never heard of DKIM/DMARC users having issues > outside of dealing with mailman. The vast majority of email users who never > post to technical mailing lists benefit from it significantly. > > Really everyone should use them. Adding cryptographic integrity to email > is hardly a crazy idea :) >
I understand the reason to protect the "heavily phished" domains. I heard that LKML does not modify the subject or add a footer, perhaps because it would make it incompatible with DKIM of the several big corporate domains who participate.
I suppose it is somewhat acceptable for us to remove subject tags and footers if we have no choice...
Warren
Quoted text (4 lines)
> > Mailman isn't resigning it. Should it be? Does other mailing list > software? >
Mailman must take responsibility for the mail itself. It doesn't have to actually sign with DKIM to do so: for backwards compatibility, spam filters fall back to other heuristics to try and figure out the 'owner' of the mail if it doesn't use DKIM. Those heuristics can go wrong of course. Ideally all mail would be DKIM signed. There's no reason not to do it, really.
Yes mailing lists that edit people's emails resign. For example, from a recent message to the bitcoinj list
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
*d=googlegroups.com <http://googlegroups.com>*; s=20120806;
h=to:from:subject:date:lines:message-id:references:mime-version
:content-type:user-agent:in-reply-to:x-original-sender
:x-original-authentication-results:reply-to:precedence:mailing-list
:list-id:list-post:list-help:list-archive:sender:list-subscribe
:list-unsubscribe;
Quoted text (3 lines)
> I suppose it is somewhat acceptable for us to remove subject tags and > footers if we have no choice... >
Good email clients can extract the same information from the headers anyway. I filter all my mail based on them, and the headers also contain unsubscribe instructions. Gmail is capable of using them programmatically.
Hi Warren,
If you set dmarc_moderation_action to "Munge from", the list will detect when someone posts from a domain that publishes a request for strict signature checking for all mails originating from it (in DNS) and rewrite the envelope-from to the list's address. Reply-to will be added and set to the original sender.
I think that this is probably a better way to workaround the issue (rather than playing with getting the list to not break the signature) until these things mature further.
Thoughts?
--adam
On Fri, Jun 19, 2015 at 6:38 AM, Warren Togami Jr. <wtogami@gmail•com> wrote:
Quoted text (56 lines)
> On Fri, Jun 19, 2015 at 12:24 AM, Mike Hearn <mike@plan99•net> wrote: > >> The new list currently has footers removed during testing. I am not >>> pleased with the need to remove the subject tag and footer to be more >>> compatible with DKIM users. >>> >> >> Lists can do what are effectively MITM attacks on people's messages in >> any way they like, if they resign for the messages themselves. That seems >> fair to me! :) >> > > Mailman isn't resigning it. Should it be? Does other mailing list > software? > > >> >> >>> I'm guessing DKIM enforcement is not very common because of issues like >>> this? >>> >> >> DKIM is used by most mail on the internet. DMARC rules that publish in >> DNS statements like "All mail from bitpay.com is signed correctly so >> trash any that isn't" are used on some of the worlds most heavily phished >> domains like google.com, PayPal, eBay, and indeed BitPay. >> >> These rules are understood and enforced by all major webmail providers >> including Gmail. It's actually only rusty geek infrastructure that has >> problems with this, I've never heard of DKIM/DMARC users having issues >> outside of dealing with mailman. The vast majority of email users who never >> post to technical mailing lists benefit from it significantly. >> >> Really everyone should use them. Adding cryptographic integrity to email >> is hardly a crazy idea :) >> > > I understand the reason to protect the "heavily phished" domains. I heard > that LKML does not modify the subject or add a footer, perhaps because it > would make it incompatible with DKIM of the several big corporate domains > who participate. > > I suppose it is somewhat acceptable for us to remove subject tags and > footers if we have no choice... > > Warren > > > ------------------------------------------------------------------------------ > > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge•net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > >
On Fri, Jun 19, 2015 at 12:47 PM, Adam Weiss <adam@signal11•com> wrote:
Quoted text (8 lines)
> Hi Warren, > > If you set dmarc_moderation_action to "Munge from", the list will detect > when someone posts from a domain that publishes a request for strict > signature checking for all mails originating from it (in DNS) and rewrite > the envelope-from to the list's address. Reply-to will be added and set to > the original sender. >
That seems to change Reply behavior for those recipients? It would seem to accidentally direct mail intended to DKIM-user + list to DKIM-user.
It changes the mechanics at least. A quick glance at RFC(2)822 makes it clear that this is a pretty weakly specified behavior and is somewhat of an edge case. However, rewriting the envelopes has become somewhat prevalent since strict DMARC has been adopted and I suspect that most recent MUAs will handle it well. I know that at least with gmail it works as I would expect. (Makes sense considering that this is how Google Groups handles the problem.)
In any event, I really think it's worth a shot since having the subject and footer tags is valuable. If it turns out to be problematic, it's not the end of the world and things could be easily switched to go the lkml route...
--adam
On Fri, Jun 19, 2015 at 4:44 PM, Jeff Garzik <jgarzik@bitpay•com> wrote:
Quoted text (19 lines)
> On Fri, Jun 19, 2015 at 12:47 PM, Adam Weiss <adam@signal11•com> wrote: > >> Hi Warren, >> >> If you set dmarc_moderation_action to "Munge from", the list will detect >> when someone posts from a domain that publishes a request for strict >> signature checking for all mails originating from it (in DNS) and rewrite >> the envelope-from to the list's address. Reply-to will be added and set to >> the original sender. >> > > That seems to change Reply behavior for those recipients? It would seem > to accidentally direct mail intended to DKIM-user + list to DKIM-user. > > -- > Jeff Garzik > Bitcoin core developer and open source evangelist > BitPay, Inc. https://bitpay.com/ >