11 ms·

Yahoo breaks every mailing list in the world including the IETF's

256 points by somerandomness 13y ago

JohnTHaller 13y ago
For this to fail, wouldn't the mailing list have to send the message on to its subscribers listing it as "From" the Yahoo email address? In that case, it's the mailing list doing it 'wrong' (in the eyes of SPF, DKIM, et al, anyway), as they should be sending it 'from' their mailing list email address, not the original person who sent the message. This isn't a new problem as SPF has required mailing lists to do this for years now. *UPDATE: Clarified 'wrong' wrt the various protocols.
- ChuckMcM 13y ago
  They often do that (resend with the receiver being the person). Mailing lists have intense debates over the correctness of replying to the sender by default or to the list by default (and whether or not you want to add cc or not). Many also munge the subject line with things like [The AWESOMEList] because forever and ever people's mail clients could only filter on subject lines or body text and rarely additional headers. I appreciate where Yahoo's heart is, but this wasn't really well thought out on their part.
- throwaway2048 13y ago
  Breaking the entire world and then complaining that "they are doing it wrong" goes over like a lead balloon. Its a messy state of affairs to have to workaround issues that dont match your particular models, but its often necessary to have things work at all. Do note that mailing lists and the way they send mail predate DMARC, DKIM and SPF by far, why don't they better account for this extremely prevalent model of email usage. Why is it that mailing lists that are "broken".
  - JohnTHaller 13y ago
    Mailing lists are a tiny, tiny fraction of email and used mostly by us techie types. So, I thinking 'breaking the entire world' is a bit hyperbolic. I'd meant 'broken' with respect to DKIM, SPF et al (and have updated the post to reflect that).
    - dfc 13y ago
      I am going to take a guess and say that you are not/have not been involved in any somewhat serious activism. Activists love mailing lists.
      - JohnTHaller 13y ago
        I'd wager the vast majority of folks on the internet don't even know what a participatory mailing list is. And I say that as someone who has been a part of multiple lists for nearly 20 years.
        comex 13y ago
        As an anecdote, both of my parents have dealt with (unrelated) private "listservs" associated with their work.
        tragic 13y ago
        I'm on ten or twelve different mailing lists. Only two have anything whatsoever to do with tech. The rest are - as per dfc above - politics-related. So I don't view lists as somehow a techie habit - quite the opposite, in fact. They work for politics because the technical 'barrier to entry' is very low. You don't need to know anything more about computers than how to send and receive email in a client of your choice. Given that in most political groups you're dealing with students, pensioners and everyone in between, this is a distinct evolutionary advantage. I have no idea how the total volume of list-mail breaks down between tech stuff and muggle concerns. But in any situation where you have to communicate between people of different generations with wildly different technical expertise, lists are an ideal lowest-common-denominator. Until, that is, Yahoo or whoever cut their users off, and muggins here has to explain to uncomprehending people why their messages aren't appearing. Guess it's going to be a long week for me. sigh
        chalst 13y ago
        With just my eldest, 10-yo daughter, I have subscribed to 9 mailing lists at 3 different schools in two countries, to discuss things with various groups of parents, with none of these lists being started by tech folk.
- dragonwriter 13y ago
  > For this to fail, wouldn't the mailing list have to send the message on to its subscribers listing it as "From" the Yahoo email address? Yes, which many mailing lists do. > In that case, it's the mailing list doing it wrong Is it? Not having the actual originator as the "From:" seems to be "doing it wrong".
  - deleted 13y ago
    [deleted]
  - vidarh 13y ago
    Exactly, and "From:" has "always" been expected to possibly be a different identity to the sender of the e-mail: We have "Sender:" to indicate where the message was sent from when "From:" does not.
- thaumaturgy 13y ago
  The mailing lists I am subscribed to -- openbsd-misc, roundcube-dev, and backuppc-users -- all do this. So does the mailing list I admin, for a local hackers group. As well as every other mailing list in recent memory. (Including the LKML. I just checked.) Whether it's "right" or "wrong" might be an argument that someone will find worth having, but I don't think it can be argued that this isn't common practice.
  - meepmeep 13y ago
    We shouldn't have to keep this giant security hole in email (spoofing) just because mailing lists, which predate the spam and spoofing issues, don't want to update how they operate. Accommodating =reject DMARC policy is simple for the mailing lists. A policy of "Because that's the way we always did it" is dumb. We didn't use to recycle, now we do...because it is better. DMARC is better than allowing spoofing. Update your mailing list. Stop forging sender. Move on.
- gkoz 13y ago
  I think SPF is happy if the bounce address (envelope from) is replaced but the From: header is intact.
  - toast0 13y ago
    To pass DMARC, you need to have either DKIM pass, or SPF pass when the envelope sender and the From: header are aligned (whatever aligned means... generally within the same domain). Because the envelope from headers are very likely changed to the mailing list, the SPF pass doesn't help with DMARC. And since the mailing lists very likely alter the messages (to put footers), the messages probably don't pass DKIM either.
- zhemao 13y ago
  What? Every mailing list I've ever subscribed to has the original sender in the "From" field and the mailing list address in the "To" field. Otherwise how would you know who sent the message to the list?
- PhasmaFelis 13y ago
  That's bizarre. Correct behavior used to be to send "from" the original sender, with the list address in the "reply-to" field. That way everyone knows who posted it, but discussion is still directed to the list.
  - zAy0LfpBZLC8mAC 13y ago
    No, it's not. A maling list should not ever touch the Reply-To: header field. See also: http://www.unicom.com/pw/reply-to-harmful.html http://www.unicom.com/pw/reply-to-harmful.html
    - comex 13y ago
      Note the (also very old) opposing argument: http://marc.merlins.org/netrants/reply-to-useful.html http://marc.merlins.org/netrants/reply-to-useful.html Personally, I've almost never wanted to privately reply to a mailing list posts (and I'm under the impression that doing so in most contexts is vaguely rude), but redundant CCs in reply-all chains are ugly, so I prefer munging Reply-To.
      - zAy0LfpBZLC8mAC 13y ago
        Except the opposing argument is severely broken, of course. It starts with not even comprehending the RFC he is quoting ("include the address of that service in the "Reply-To" field of all messages submitted to the teleconference", which clearly does not sanction the "teleconference" itself munging the header), and after that essentially turning logic on its head and appealing to ignorance ("People who don't know how what the working solution looks like are asking for a broken solution! If software does something else than it should do by definition, it's the user's mistake, and therefore the user's fault! [...]"). As for redundant Ccs: That is what Mail-Followup-To is there for, see also: http://cr.yp.to/proto/replyto.html http://cr.yp.to/proto/replyto.html edit: Oh, and also, in particular, he completely ignores the fact that Reply-To munging deletes existing Reply-To headers, and thus breaks things in a way that even the best MUA cannot possibly work around - the original Reply-To isn't there anymore, so it cannot possibly offer you an option to reply to the original Reply-To or to the munged Reply-To.
jmathai 13y ago
We noticed noticed last month that one of our Yahoo! Groups mailing lists would randomly drop emails[1]. We couldn't find any consistent behavior to it. Wonder if this is the culprit. [1] https://twitter.com/jmathai/status/440529845198790656 https://twitter.com/jmathai/status/440529845198790656
codinghorror 13y ago
We really want Discourse (http://www.discourse.org http://www.discourse.org) to get to a place where we have extremely robust 100% open source mailing list support -- we now have reply via email, notify via email, and community contributed submit topics via email. Working on attachments via email this week. It's interesting -- unlike forums, people really enjoy mailing lists. I don't think I've ever met anyone, ever, who said they liked forums. But mailing lists seem to inspire people. I want to see a long term hybrid model where you can interact nearly completely via email, or a good, modern web UI that YOUR org owns (not google groups or yahoo groups). This should be supported.
- comex 13y ago
  When I was younger, I loved hanging out on forums. There was perhaps a more robust community feeling, since forums can have multiple sections, and include off topic and chattier discussion without people complaining about being deluged with mail. Avatars and signatures let people express their personality and made it easy to tell the author of a post. Then again, to some extent I'm just nostalgic. What then was personalization I might now see as a rather low post text (signal) to metadata (noise) ratio...
  - TD-Linux 13y ago
    I spent a lot of time in high school on PHPBB systems. I find it less attractive now, using mostly IRC plus some forums, though I don't frequent them enough to be a "regular". I absolutely hate subreddits - I generally find specific enough ones to be of low quality compared to the equivalent forums. I also absolutely love Futaba-style imageboards - it's a great blend of anonymity, simplicity, and community. It's a shame 4chan's reputation is what it is.
    - krapp 13y ago
      Most of the problems you'd want to avoid could probably be solved if you get rid of the images, and required people to sign up but still post anonymously. Granted, then it's no longer strictly an anonymous imageboard but in a lot of cases, the images don't add to the discussion anyway.
kimonos 13y ago
I don't use Yahoo anymore because it runs very slow for me..
joemaller1 13y ago
This and SpamCop flagging a bunch of Google's mail server IPs broke a bunch of email last week.
throwwit 13y ago
So would this explain a spam email in my hotmail inbox that had the same From&To, with no mention of my actual email in the raw message source?
dredmorbius 13y ago
Having managed a number of largish email systems over the past few years, I've got to say that dealing with Yahoo -- DKIM and other anti-spam related issues -- is one of my larger ongoing headaches. The lack of response and transparency of Yahoo in general is a huge problem -- I managed to get resolution in one case via an executive email carpet bomb (this after repeated contacts with their support team and direct emails to the CTO/Postmaster's address). Sadly, as much of a fading giant as Yahoo are, their email presence remains huge.
- codinghorror 13y ago
  Yahoo may be bad, but Outlook/Hotmail are without a doubt worse. Exhibit A: http://serverfault.com/questions/434703/why-does-hotmail-still-reject-my-emails/452896#452896 http://serverfault.com/questions/434703/why-does-hotmail-sti...
  - kbuck 13y ago
    I disagree. I've had Yahoo reject my emails completely silently (didn't arrive in inbox OR 'spam' folder, even though Yahoo's MX said the message was accepted). Yahoo was entirely unresponsive about this issue. We didn't have the same problem with any other provider and it was resolved immediately upon switching to a third-party email delivery service. Additionally, Yahoo has a huge amount of abuse and doesn't seem to have an abuse handle either; you have to fill out some form buried deep on their site. On the other hand, I've reported abuse incidents to Hotmail before and have gotten an actual reply from a human (a rarity when submitting abuse reports; most places act on them but don't bother responding).
    - dredmorbius 13y ago
      Yeah, don't even get me started on Yahoo's spam emissions and lack of reporting.
  - dredmorbius 13y ago
    In terms of being responsive to issues, I've actually had far better response from Microsoft than Yahoo, as I described above. Call the switchboard, ask for the SVP in charge of the division, get transferred directly to him, he picks up the phone on the first ring and talks for ~10 minutes to hear my issue (spam transiting Microsoft's network). Tells me I'll hear from the manager in charge, who calls me 15 minutes later and works over the course of several months (these things do take time) to resolve the issue and improve their systems. As I wrote above: I'd written Raymie Stata repeatedly after getting a complete runaround from Yahoo's tech support (and diving into dead ends on their website), never hearing a peep from him. Not until I emailed pretty much the entire C-level suite and senior managers with a bit of data showing the nature of the problem (postfix delivery time stats) did I get a return response, from Yahoo's "concierge" service. That finally resolved the particular issue I was dealing with, but that's one appropriate response in years of dealing with the company. The resolution with Yahoo was essentially the one described in your Serverfault link: get explicitly whitelisted. That's not uncommon with top-tier email service providers.
billpg 13y ago
So what should Yahoo do? Change settings to say "Actually, anyone in the world can send emails from @yahoo.com now."? (Honest question.)
- mike-cardwell 13y ago
  Yes. If they want their users to be able to use mailing lists.
  - billpg 13y ago
    Is that the trade-off? Either we neuter SPF et al or we break mailing lists? I vote for SPF et al.
    - mike-cardwell 13y ago
      Yes, that is the tradeoff. SPF doesn't really come into it. Mailing lists use their own sender envelope. The problem is, when a mailing list makes changes to an email which breaks the DKIM signature. But the sender uses DMARC to say that DKIM must pass. Another fix would be for all mailing lists to be updated to not make any changes to messages which might break DKIM. E.g by adding [listname] to the subject line, or messing with other headers, or adding signatures to the body.
      - avz 13y ago
        The tradeoff isn't that bad: it's possible to make mailing lists work with DMARC, see for example their suggestions in the FAQ: http://dmarc.org/faq.html#s_3 http://dmarc.org/faq.html#s_3. It can be argued that the required changes are very burdensome and not mailing-list-friendly. The mail body modifications seem to me like something mailing lists could drop taking advantage of the list-* headers instead. The harder usability issue arises from the fact that DMARC imposes a different way of setting the from header potentially breaking all those filters we've set up. DMARC claim both issues can be solved using "Original Authentication Results" header but since it requires the receiving MTA to trust the mailing list the administrative overhead here just doesn't scale and will likely end up being pushed onto the list admins. Also, SPF does come into it since DMARC requires "alignment" between the from domains in the envelope and the header (see again the FAQ answer above).
mike-cardwell 13y ago
I played with DMARC about a year ago. I put it in monitoring mode so that I'd get email reports from systems to tell me when they received emails from my domain which failed DMARC. I started getting them from all over the place. Pretty much all related to mailing lists breaking the DKIM signature by rewriting headers or the body. My conclusion was: If any email address on your domain subscribes to one or more mailing lists, DMARC will break your email. I disabled it. I don't see myself enabling it again any time soon.
aendruk 13y ago
Please forgive my naivety—why are mailing lists forging from addresses in the first place? Have they just been fragilely dependent for years on the exploitation of an authentication vulnerability?
- bashcoder 13y ago
  Fair question. The basic internet email spec has virtually no security features whatsoever, and is completely unauthenticated. Mailing list management software doesn't forge sender information, but rather often retains it and generally trusts incoming headers. Back in the old days, some folks even referred to discussion lists as "reflectors." The proper usage of SMTP mail headers is outlined in RFC2822 (originally RFC822), and the definition of the headers From, Sender, Resent-From, etc. The rules for specifying sender information are spelled out in 3.6.2. [0] That said, system behavior also depends on if the MLM software is running behind a mail transport agent that enforces authentication protocols for incoming emails, scans for viruses, etc. When discussion list owners are concerned about receiving forged posts, they usually use list moderation features so they can ensure emails do not get distributed that haven't been reviewed first. But the biggest problem for MLMs isn't usually dealing with impostors, but rather blocking email-borne viruses and misconfigured auto-responders that could cause bogus emails to get reflected out to subscribers. The behavior of the outgoing From header from MLM software typically depends on the configuration of the list. Some lists (especially digests) are configured so outgoing messages are "From" the list itself. But most discussion lists are configured to retain the original From line, while clarifying their role as an email proxy through other headers. [0] http://tools.ietf.org/html/rfc2822#section-3.6.2 http://tools.ietf.org/html/rfc2822#section-3.6.2
syntheticnature 13y ago
I'm amused to note that this probably applies to Yahoo Groups as well, since it uses the yahoogroups.com domain, not a yahoo.com domain. Looking at the headers of a recent message from a Yahoo user over Yahoo groups it seems like it would be the case: dkim=pass header.i=@yahoogroups.com; dmarc=pass (p=REJECT dis=NONE) header.from=yahoo.com Of course, anyone who is using Yahoo Groups regularly has probably noticed that even with last year's redesign it's not getting much attention.
meepmeep 13y ago
DMARC.org has very clear remedies. Q: I operate a mailing list and I want to interoperate with DMARC, what should I do? A: DMARC introduces the concept of aligned identifiers. It means the domain in the from header must match the d= in the DKIM signature and the domain in the mail from envelope. You have a few solutions: - operate as a strict forwarder, where the message is not changed and the validity of the DKIM signature is preserved - introduce an "Original Authentication Results" header to indicate you have performed the authentication and you are validating it - take ownership of the email, by removing the DKIM signature and putting your own as well as changing the from header in the email to contain an email address within your mailing list domain. Spoofing is a huge issue for all email customers. DMARC was started, in part, to deal with the coming problems that were foreseen here. Mailing Lists don't have to forge or spoof to work. They can adjust and everyone is better off.
- avz 13y ago
  (the post above has been largely copy-pasted from http://dmarc.org/faq.html#s_3 http://dmarc.org/faq.html#s_3) Interesting point for the discussion on whether MLMs are allowed to modify the from header is in the section 3.6.2 of rfc 2822: http://tools.ietf.org/html/rfc2822#section-3.6.2 http://tools.ietf.org/html/rfc2822#section-3.6.2. The intended meaning of the from field is to indicate the author of a message which is explicitly allowed to be different than the sender. Thus list-originated communication like digest messages should be sent with the from header of the list, but messages forwarded by the MLM should be sent with the from header indicating the original author. In the absence of the sender header it can be assumed to be the same as the from header. Thus, DMARC could use the sender header instead of the from header and fall back to the from header only when sender is absent. This way MLMs would have a way of avoiding the issue by supplying the sender header. Unfortunately, DMARC chose not to use the sender header citing abuse and bugs in some MUAs which don't display the sender header to the user correctly: http://www.ietf.org/mail-archive/web/dmarc/current/msg00064.html http://www.ietf.org/mail-archive/web/dmarc/current/msg00064..... As for the "Original Authentication Results" it doesn't solve the problem for most lists since it requires the destination domain to explicitly trust the list, see http://www.dmarc.org/pipermail/dmarc-discuss/2012-February/000428.html http://www.dmarc.org/pipermail/dmarc-discuss/2012-February/0... and http://tools.ietf.org/id/draft-kucherawy-original-authres-00.txt http://tools.ietf.org/id/draft-kucherawy-original-authres-00.... Few list admins could afford getting a trust explicitly established with every domain where the members happen to have mailboxes.

256 points by somerandomness 13y ago

For this to fail, wouldn't the mailing list have to send the message on to its subscribers listing it as "From" the Yahoo email address? In that case, it's the mailing list doing it 'wrong' (in the eyes of SPF, DKIM, et al, anyway), as they should be sending it 'from' their mailing list email address, not the original person who sent the message. This isn't a new problem as SPF has required mailing lists to do this for years now. *UPDATE: Clarified 'wrong' wrt the various protocols.

ChuckMcM 13y ago
They often do that (resend with the receiver being the person). Mailing lists have intense debates over the correctness of replying to the sender by default or to the list by default (and whether or not you want to add cc or not). Many also munge the subject line with things like [The AWESOMEList] because forever and ever people's mail clients could only filter on subject lines or body text and rarely additional headers. I appreciate where Yahoo's heart is, but this wasn't really well thought out on their part.
throwaway2048 13y ago
Breaking the entire world and then complaining that "they are doing it wrong" goes over like a lead balloon. Its a messy state of affairs to have to workaround issues that dont match your particular models, but its often necessary to have things work at all. Do note that mailing lists and the way they send mail predate DMARC, DKIM and SPF by far, why don't they better account for this extremely prevalent model of email usage. Why is it that mailing lists that are "broken".
- JohnTHaller 13y ago
  Mailing lists are a tiny, tiny fraction of email and used mostly by us techie types. So, I thinking 'breaking the entire world' is a bit hyperbolic. I'd meant 'broken' with respect to DKIM, SPF et al (and have updated the post to reflect that).
  - dfc 13y ago
    I am going to take a guess and say that you are not/have not been involved in any somewhat serious activism. Activists love mailing lists.
    - JohnTHaller 13y ago
      I'd wager the vast majority of folks on the internet don't even know what a participatory mailing list is. And I say that as someone who has been a part of multiple lists for nearly 20 years.
      - comex 13y ago
        As an anecdote, both of my parents have dealt with (unrelated) private "listservs" associated with their work.
      - tragic 13y ago
        I'm on ten or twelve different mailing lists. Only two have anything whatsoever to do with tech. The rest are - as per dfc above - politics-related. So I don't view lists as somehow a techie habit - quite the opposite, in fact. They work for politics because the technical 'barrier to entry' is very low. You don't need to know anything more about computers than how to send and receive email in a client of your choice. Given that in most political groups you're dealing with students, pensioners and everyone in between, this is a distinct evolutionary advantage. I have no idea how the total volume of list-mail breaks down between tech stuff and muggle concerns. But in any situation where you have to communicate between people of different generations with wildly different technical expertise, lists are an ideal lowest-common-denominator. Until, that is, Yahoo or whoever cut their users off, and muggins here has to explain to uncomprehending people why their messages aren't appearing. Guess it's going to be a long week for me. sigh
      - chalst 13y ago
        With just my eldest, 10-yo daughter, I have subscribed to 9 mailing lists at 3 different schools in two countries, to discuss things with various groups of parents, with none of these lists being started by tech folk.
dragonwriter 13y ago
> For this to fail, wouldn't the mailing list have to send the message on to its subscribers listing it as "From" the Yahoo email address? Yes, which many mailing lists do. > In that case, it's the mailing list doing it wrong Is it? Not having the actual originator as the "From:" seems to be "doing it wrong".
- deleted 13y ago
  [deleted]
- vidarh 13y ago
  Exactly, and "From:" has "always" been expected to possibly be a different identity to the sender of the e-mail: We have "Sender:" to indicate where the message was sent from when "From:" does not.
thaumaturgy 13y ago
The mailing lists I am subscribed to -- openbsd-misc, roundcube-dev, and backuppc-users -- all do this. So does the mailing list I admin, for a local hackers group. As well as every other mailing list in recent memory. (Including the LKML. I just checked.) Whether it's "right" or "wrong" might be an argument that someone will find worth having, but I don't think it can be argued that this isn't common practice.
- meepmeep 13y ago
  We shouldn't have to keep this giant security hole in email (spoofing) just because mailing lists, which predate the spam and spoofing issues, don't want to update how they operate. Accommodating =reject DMARC policy is simple for the mailing lists. A policy of "Because that's the way we always did it" is dumb. We didn't use to recycle, now we do...because it is better. DMARC is better than allowing spoofing. Update your mailing list. Stop forging sender. Move on.
gkoz 13y ago
I think SPF is happy if the bounce address (envelope from) is replaced but the From: header is intact.
- toast0 13y ago
  To pass DMARC, you need to have either DKIM pass, or SPF pass when the envelope sender and the From: header are aligned (whatever aligned means... generally within the same domain). Because the envelope from headers are very likely changed to the mailing list, the SPF pass doesn't help with DMARC. And since the mailing lists very likely alter the messages (to put footers), the messages probably don't pass DKIM either.
zhemao 13y ago
What? Every mailing list I've ever subscribed to has the original sender in the "From" field and the mailing list address in the "To" field. Otherwise how would you know who sent the message to the list?
PhasmaFelis 13y ago
That's bizarre. Correct behavior used to be to send "from" the original sender, with the list address in the "reply-to" field. That way everyone knows who posted it, but discussion is still directed to the list.
- zAy0LfpBZLC8mAC 13y ago
  No, it's not. A maling list should not ever touch the Reply-To: header field. See also: http://www.unicom.com/pw/reply-to-harmful.html http://www.unicom.com/pw/reply-to-harmful.html
  - comex 13y ago
    Note the (also very old) opposing argument: http://marc.merlins.org/netrants/reply-to-useful.html http://marc.merlins.org/netrants/reply-to-useful.html Personally, I've almost never wanted to privately reply to a mailing list posts (and I'm under the impression that doing so in most contexts is vaguely rude), but redundant CCs in reply-all chains are ugly, so I prefer munging Reply-To.
    - zAy0LfpBZLC8mAC 13y ago
      Except the opposing argument is severely broken, of course. It starts with not even comprehending the RFC he is quoting ("include the address of that service in the "Reply-To" field of all messages submitted to the teleconference", which clearly does not sanction the "teleconference" itself munging the header), and after that essentially turning logic on its head and appealing to ignorance ("People who don't know how what the working solution looks like are asking for a broken solution! If software does something else than it should do by definition, it's the user's mistake, and therefore the user's fault! [...]"). As for redundant Ccs: That is what Mail-Followup-To is there for, see also: http://cr.yp.to/proto/replyto.html http://cr.yp.to/proto/replyto.html edit: Oh, and also, in particular, he completely ignores the fact that Reply-To munging deletes existing Reply-To headers, and thus breaks things in a way that even the best MUA cannot possibly work around - the original Reply-To isn't there anymore, so it cannot possibly offer you an option to reply to the original Reply-To or to the munged Reply-To.

jmathai 13y ago

We noticed noticed last month that one of our Yahoo! Groups mailing lists would randomly drop emails[1]. We couldn't find any consistent behavior to it. Wonder if this is the culprit. [1] https://twitter.com/jmathai/status/440529845198790656 https://twitter.com/jmathai/status/440529845198790656

codinghorror 13y ago

We really want Discourse (http://www.discourse.org http://www.discourse.org) to get to a place where we have extremely robust 100% open source mailing list support -- we now have reply via email, notify via email, and community contributed submit topics via email. Working on attachments via email this week. It's interesting -- unlike forums, people really enjoy mailing lists. I don't think I've ever met anyone, ever, who said they liked forums. But mailing lists seem to inspire people. I want to see a long term hybrid model where you can interact nearly completely via email, or a good, modern web UI that YOUR org owns (not google groups or yahoo groups). This should be supported.

comex 13y ago
When I was younger, I loved hanging out on forums. There was perhaps a more robust community feeling, since forums can have multiple sections, and include off topic and chattier discussion without people complaining about being deluged with mail. Avatars and signatures let people express their personality and made it easy to tell the author of a post. Then again, to some extent I'm just nostalgic. What then was personalization I might now see as a rather low post text (signal) to metadata (noise) ratio...
- TD-Linux 13y ago
  I spent a lot of time in high school on PHPBB systems. I find it less attractive now, using mostly IRC plus some forums, though I don't frequent them enough to be a "regular". I absolutely hate subreddits - I generally find specific enough ones to be of low quality compared to the equivalent forums. I also absolutely love Futaba-style imageboards - it's a great blend of anonymity, simplicity, and community. It's a shame 4chan's reputation is what it is.
  - krapp 13y ago
    Most of the problems you'd want to avoid could probably be solved if you get rid of the images, and required people to sign up but still post anonymously. Granted, then it's no longer strictly an anonymous imageboard but in a lot of cases, the images don't add to the discussion anyway.

kimonos 13y ago

I don't use Yahoo anymore because it runs very slow for me..

joemaller1 13y ago

This and SpamCop flagging a bunch of Google's mail server IPs broke a bunch of email last week.

throwwit 13y ago

So would this explain a spam email in my hotmail inbox that had the same From&To, with no mention of my actual email in the raw message source?

dredmorbius 13y ago

Having managed a number of largish email systems over the past few years, I've got to say that dealing with Yahoo -- DKIM and other anti-spam related issues -- is one of my larger ongoing headaches. The lack of response and transparency of Yahoo in general is a huge problem -- I managed to get resolution in one case via an executive email carpet bomb (this after repeated contacts with their support team and direct emails to the CTO/Postmaster's address). Sadly, as much of a fading giant as Yahoo are, their email presence remains huge.

codinghorror 13y ago
Yahoo may be bad, but Outlook/Hotmail are without a doubt worse. Exhibit A: http://serverfault.com/questions/434703/why-does-hotmail-still-reject-my-emails/452896#452896 http://serverfault.com/questions/434703/why-does-hotmail-sti...
- kbuck 13y ago
  I disagree. I've had Yahoo reject my emails completely silently (didn't arrive in inbox OR 'spam' folder, even though Yahoo's MX said the message was accepted). Yahoo was entirely unresponsive about this issue. We didn't have the same problem with any other provider and it was resolved immediately upon switching to a third-party email delivery service. Additionally, Yahoo has a huge amount of abuse and doesn't seem to have an abuse handle either; you have to fill out some form buried deep on their site. On the other hand, I've reported abuse incidents to Hotmail before and have gotten an actual reply from a human (a rarity when submitting abuse reports; most places act on them but don't bother responding).
  - dredmorbius 13y ago
    Yeah, don't even get me started on Yahoo's spam emissions and lack of reporting.
- dredmorbius 13y ago
  In terms of being responsive to issues, I've actually had far better response from Microsoft than Yahoo, as I described above. Call the switchboard, ask for the SVP in charge of the division, get transferred directly to him, he picks up the phone on the first ring and talks for ~10 minutes to hear my issue (spam transiting Microsoft's network). Tells me I'll hear from the manager in charge, who calls me 15 minutes later and works over the course of several months (these things do take time) to resolve the issue and improve their systems. As I wrote above: I'd written Raymie Stata repeatedly after getting a complete runaround from Yahoo's tech support (and diving into dead ends on their website), never hearing a peep from him. Not until I emailed pretty much the entire C-level suite and senior managers with a bit of data showing the nature of the problem (postfix delivery time stats) did I get a return response, from Yahoo's "concierge" service. That finally resolved the particular issue I was dealing with, but that's one appropriate response in years of dealing with the company. The resolution with Yahoo was essentially the one described in your Serverfault link: get explicitly whitelisted. That's not uncommon with top-tier email service providers.

billpg 13y ago

So what should Yahoo do? Change settings to say "Actually, anyone in the world can send emails from @yahoo.com now."? (Honest question.)

mike-cardwell 13y ago
Yes. If they want their users to be able to use mailing lists.
- billpg 13y ago
  Is that the trade-off? Either we neuter SPF et al or we break mailing lists? I vote for SPF et al.
  - mike-cardwell 13y ago
    Yes, that is the tradeoff. SPF doesn't really come into it. Mailing lists use their own sender envelope. The problem is, when a mailing list makes changes to an email which breaks the DKIM signature. But the sender uses DMARC to say that DKIM must pass. Another fix would be for all mailing lists to be updated to not make any changes to messages which might break DKIM. E.g by adding [listname] to the subject line, or messing with other headers, or adding signatures to the body.
    - avz 13y ago
      The tradeoff isn't that bad: it's possible to make mailing lists work with DMARC, see for example their suggestions in the FAQ: http://dmarc.org/faq.html#s_3 http://dmarc.org/faq.html#s_3. It can be argued that the required changes are very burdensome and not mailing-list-friendly. The mail body modifications seem to me like something mailing lists could drop taking advantage of the list-* headers instead. The harder usability issue arises from the fact that DMARC imposes a different way of setting the from header potentially breaking all those filters we've set up. DMARC claim both issues can be solved using "Original Authentication Results" header but since it requires the receiving MTA to trust the mailing list the administrative overhead here just doesn't scale and will likely end up being pushed onto the list admins. Also, SPF does come into it since DMARC requires "alignment" between the from domains in the envelope and the header (see again the FAQ answer above).

mike-cardwell 13y ago

I played with DMARC about a year ago. I put it in monitoring mode so that I'd get email reports from systems to tell me when they received emails from my domain which failed DMARC. I started getting them from all over the place. Pretty much all related to mailing lists breaking the DKIM signature by rewriting headers or the body. My conclusion was: If any email address on your domain subscribes to one or more mailing lists, DMARC will break your email. I disabled it. I don't see myself enabling it again any time soon.

aendruk 13y ago

Please forgive my naivety—why are mailing lists forging from addresses in the first place? Have they just been fragilely dependent for years on the exploitation of an authentication vulnerability?

bashcoder 13y ago
Fair question. The basic internet email spec has virtually no security features whatsoever, and is completely unauthenticated. Mailing list management software doesn't forge sender information, but rather often retains it and generally trusts incoming headers. Back in the old days, some folks even referred to discussion lists as "reflectors." The proper usage of SMTP mail headers is outlined in RFC2822 (originally RFC822), and the definition of the headers From, Sender, Resent-From, etc. The rules for specifying sender information are spelled out in 3.6.2. [0] That said, system behavior also depends on if the MLM software is running behind a mail transport agent that enforces authentication protocols for incoming emails, scans for viruses, etc. When discussion list owners are concerned about receiving forged posts, they usually use list moderation features so they can ensure emails do not get distributed that haven't been reviewed first. But the biggest problem for MLMs isn't usually dealing with impostors, but rather blocking email-borne viruses and misconfigured auto-responders that could cause bogus emails to get reflected out to subscribers. The behavior of the outgoing From header from MLM software typically depends on the configuration of the list. Some lists (especially digests) are configured so outgoing messages are "From" the list itself. But most discussion lists are configured to retain the original From line, while clarifying their role as an email proxy through other headers. [0] http://tools.ietf.org/html/rfc2822#section-3.6.2 http://tools.ietf.org/html/rfc2822#section-3.6.2

syntheticnature 13y ago

I'm amused to note that this probably applies to Yahoo Groups as well, since it uses the yahoogroups.com domain, not a yahoo.com domain. Looking at the headers of a recent message from a Yahoo user over Yahoo groups it seems like it would be the case: dkim=pass header.i=@yahoogroups.com; dmarc=pass (p=REJECT dis=NONE) header.from=yahoo.com Of course, anyone who is using Yahoo Groups regularly has probably noticed that even with last year's redesign it's not getting much attention.

meepmeep 13y ago

DMARC.org has very clear remedies. Q: I operate a mailing list and I want to interoperate with DMARC, what should I do? A: DMARC introduces the concept of aligned identifiers. It means the domain in the from header must match the d= in the DKIM signature and the domain in the mail from envelope. You have a few solutions: - operate as a strict forwarder, where the message is not changed and the validity of the DKIM signature is preserved - introduce an "Original Authentication Results" header to indicate you have performed the authentication and you are validating it - take ownership of the email, by removing the DKIM signature and putting your own as well as changing the from header in the email to contain an email address within your mailing list domain. Spoofing is a huge issue for all email customers. DMARC was started, in part, to deal with the coming problems that were foreseen here. Mailing Lists don't have to forge or spoof to work. They can adjust and everyone is better off.

avz 13y ago
(the post above has been largely copy-pasted from http://dmarc.org/faq.html#s_3 http://dmarc.org/faq.html#s_3) Interesting point for the discussion on whether MLMs are allowed to modify the from header is in the section 3.6.2 of rfc 2822: http://tools.ietf.org/html/rfc2822#section-3.6.2 http://tools.ietf.org/html/rfc2822#section-3.6.2. The intended meaning of the from field is to indicate the author of a message which is explicitly allowed to be different than the sender. Thus list-originated communication like digest messages should be sent with the from header of the list, but messages forwarded by the MLM should be sent with the from header indicating the original author. In the absence of the sender header it can be assumed to be the same as the from header. Thus, DMARC could use the sender header instead of the from header and fall back to the from header only when sender is absent. This way MLMs would have a way of avoiding the issue by supplying the sender header. Unfortunately, DMARC chose not to use the sender header citing abuse and bugs in some MUAs which don't display the sender header to the user correctly: http://www.ietf.org/mail-archive/web/dmarc/current/msg00064.html http://www.ietf.org/mail-archive/web/dmarc/current/msg00064..... As for the "Original Authentication Results" it doesn't solve the problem for most lists since it requires the destination domain to explicitly trust the list, see http://www.dmarc.org/pipermail/dmarc-discuss/2012-February/000428.html http://www.dmarc.org/pipermail/dmarc-discuss/2012-February/0... and http://tools.ietf.org/id/draft-kucherawy-original-authres-00.txt http://tools.ietf.org/id/draft-kucherawy-original-authres-00.... Few list admins could afford getting a trust explicitly established with every domain where the members happen to have mailboxes.