7 ms·

I have been managing email servers for over 10 years, and it has gotten to the point that I feel like blocking some of the most common ISPs. Seriously. I am fo

I have been managing email servers for over 10 years, and it has gotten to the point that I feel like blocking some of the most common ISPs. Seriously. I am following all the best practises, hell, I even advocate them. It is just that these days it seems not to matter if you have SPF, Sender ID, DomainKey and DKIM, PTR, proper MX and even a normal to good IP reputation. There is still no guarantee what you will be able to reach the inbox of the likes of Gmail, Yahoo, Hotmail, etc. I have been filling out huge forms for each and every major ISP for the past year because one or two users mark a newsletter as SPAM. Conclusion: There is no common standard because every major ISP can set their own standards. This will eventually force everyone to use the same services worldwide. Where's the freedom of choice here?

dminor 13y ago
Yeah, we just completed a switch to MailChimp after years of sending out our email ourselves. It's just too much hassle now for a smaller organization.
kaoD 13y ago
> one or two users mark a newsletter as SPAM. I'm one of them. Those newsletters are spam. I would never sign up for a newsletter and somehow I'm getting those too. If my intent was not to get the newsletter, it's unsolicited mail by definition, i.e. spam. Stop spamming me and I'll stop flagging you. Period. How not to be flagged as spam: - There should be a checkbox clearly visible and it shouldn't be pre-checked. - Your "kind" product reminders are obnoxious too and I'll flag them as spam as well. Did I ask you to remind me of your product? Nope. Unsolicited then. - If you ToS say I agree to receive mail, guess what? I don't agree, I just want to try your product. I'll flag you in a breeze. - Social reminders like Twitter's trending around me or people I might know? SPAM! I don't care if I can disable these, I didn't enable them. - You want to offer me discounts but I didn't ask for them? Flagged! - I submitted a paper to a conference and it got published? Dozens of "calls for papers" in my inbox. Flagged, flagged, flagged, flagged! - Calling it a newsletter or adding a tiny "unsubscribe" link won't hide the fact that it's still spam. I didn't click subscribe, I shouldn't have to unsubscribe. -- EDIT: Woah, this seems controversial. Lots of up- and down-votes. Dear product owners, downvoting me here won't change the fact that me (and your fellow users) will still flag the shit out of your unsolicited mail. I guess it pays if you keep doing it, but you should direct your energy far from that downvote button and closer to "ways not to annoy my users".
- _asciiker_ 13y ago
  You have all the right to mark it as SPAM of course. What I'm trying to get across here is that IP blocking is a bad solution. There are many cases where several domains share the same IP, specially with the shortage. So if domainA.com sends spam, why should domainB.com mail delivery on the same IP suffer from it?
  - kaoD 13y ago
    > So if domainA.com sends spam, why should domainB.com mail delivery on the same IP suffer from it? Because the mail servers both share seem to allow sending unsolicited email, i.e. spam. You're the one responsible for terminating the spamming user.
  - dredmorbius 13y ago
    Police the motherfuckingbejesus out of your users. If it costs you when your IP is blocked, insist on a deposit that's forfeit in the event of spamming. Sadly, I've worked for shops with poor practices. How bad? Sending thousands of emails to domains which no longer existed (let alone stale accounts at existing mail service providers). Simply not a priority.
  - DanBC 13y ago
    You would have hated SPEWS.
- pavel_lishin 13y ago
  I always click the "unsubscribe" link, first. If I get anything else after that, then I mark it as spam. (I don't buy into the 30-business-days bullshit, it's 2014.) I also have catch-all addresses at my domain, so I register for things as SERVICE@mydomain, so I know exactly where spam is coming from. Except I've been doing this for years, and 90% of garbage still just comes to my plain ol' gmail address that I've been using for over a decade, which suggests that businesses finally have wised up to the fact that sending garbage to everyone who puts their email address into a form is a bad, bad idea.
  - 13y ago
lazyant 13y ago
Yes, I found the same problem with email, it's not getting spam (relatively easy to deal with) but deliverability, that's why at the end we need to use a 3rd party service like sendgrid, mailgun, mailchimp etc
davideous 13y ago
The freedom of choice is actually in the hands of the end-users. The end-users choose which email providers (Yahoo, Gmail, Hotmail) they want to use. The email providers are motivated to provide a good user experience, which includes blocking unwanted email because they make their money through user engagement. The key to delivering into the inbox is sending mail that your recipients both want and expect. Provide a good user experience, and you'll build a good reputation. Push the limits (for example, use a "pre checked" checkbox on an order confirmation page to put people on your sales mailing list) and you'll be putting out email delivery fires all of the time. Most of the things that you describe (SPF, SenderID, DKIM, PTR, MX) are all technical requirements, which are just the baseline for delivery. These are required, but any sender of unsolicited email can configure them. They don't earn you access to the inbox. Just like properly formatted HTML does not earn you great SEO results. I like to break email delivery down into four areas: * who -- send to people who requested your email and are expecting it * what -- send something of value to these people * technical foundation -- (SPF, SenderID, DKIM, Feedback Loops, etc.) required, but having it does not give you any points * monitoring -- (open ratio, complaint ratio, ISP response codes) you need to know when something goes wrong (My company, www.drh.net, has been providing email server software, services, and deliverability consulting for over 10 years.) --- [edit; added the below] Another way the freedom-of-choice is in the hands of the end users is this: the big ISPs (yahoo, gmail, hotmail) make most of their filtering decisions based off of end-user behavioral data. For example: * what percentage of your email is opened * what percentage of your email is complained about (the "this is spam" button) * what percentage of your email is deleted without reading * how long is your email read * how much is forwarded * how much is replied to * what percentage of your email that was placed in the Spam folder when seen by the user received a click on the "this is not Spam" button. This is the end-users voting on if they want your email or not. This isn't the entire email deliverability equation, but it's a huge part of it. The ISPs treat this data so importantly because: (a) it's hard to game unlike content filtering, and (b) it directly correlates to good user experience which they want to provide. [edit to make bulleted list look right]
- jt2190 13y ago
  > Provide a good user experience, and you'll build a good > reputation. Push the limits (for example, use a "pre > checked" checkbox on an order confirmation page to put > people on your sales mailing list) and you'll be putting > out email delivery fires all of the time. Believe it or not, it's possible to be a good actor, follow all of the rules and best practices, and still get flagged as spam. Mail recipients are not perfect. They forget that they signed up for things. They accidentally click the "spam" flag on their messages. They get lazy and instead of unsubscribing they click the spam flag. The real culprit here is that email messages rely on blacklists and not whitelists, i.e. recipients are required to give all senders full access and then block them when they misbehave, instead of giving them no access and giving them more access as they build trust. So: What would it take to implement email whitelists across the industry?
bradknowles 13y ago
Speaking as the former Sr. Internet Mail Administrator for AOL, there is none. When I was there, we actively worked very, very hard to make sure that we did everything we should do to keep our stupid AOL users from inappropriately hurting people who contact them -- Like marking that message from your own mother as spam? Really? Problem is, there are plenty of large sites out there who aren't so conscientious, and you can keep up that level of commitment for only so long. There is no one on this entire PLANET that hates spam more than I do, and for a time I could rightfully claim that there was no other single person on this planet who had done more to fight spam than I had. But even I have my limits. I haven't been active in CAUCE or any other anti-spam effort for many years, and I haven't even been an active mail system administrator for a few years. As one of the members of the Postmaster Team for python.org (currently inactive), I have seriously hated some of the stupid shit that has been done by the likes of Google, hotmail, and Yahoo! I've been sorely tempted to just ban them outright, because of whatever might be their latest stupidity. But it hasn't happened -- At least, not yet.

dminor 13y ago

Yeah, we just completed a switch to MailChimp after years of sending out our email ourselves. It's just too much hassle now for a smaller organization.

kaoD 13y ago

> one or two users mark a newsletter as SPAM. I'm one of them. Those newsletters are spam. I would never sign up for a newsletter and somehow I'm getting those too. If my intent was not to get the newsletter, it's unsolicited mail by definition, i.e. spam. Stop spamming me and I'll stop flagging you. Period. How not to be flagged as spam: - There should be a checkbox clearly visible and it shouldn't be pre-checked. - Your "kind" product reminders are obnoxious too and I'll flag them as spam as well. Did I ask you to remind me of your product? Nope. Unsolicited then. - If you ToS say I agree to receive mail, guess what? I don't agree, I just want to try your product. I'll flag you in a breeze. - Social reminders like Twitter's trending around me or people I might know? SPAM! I don't care if I can disable these, I didn't enable them. - You want to offer me discounts but I didn't ask for them? Flagged! - I submitted a paper to a conference and it got published? Dozens of "calls for papers" in my inbox. Flagged, flagged, flagged, flagged! - Calling it a newsletter or adding a tiny "unsubscribe" link won't hide the fact that it's still spam. I didn't click subscribe, I shouldn't have to unsubscribe. -- EDIT: Woah, this seems controversial. Lots of up- and down-votes. Dear product owners, downvoting me here won't change the fact that me (and your fellow users) will still flag the shit out of your unsolicited mail. I guess it pays if you keep doing it, but you should direct your energy far from that downvote button and closer to "ways not to annoy my users".

_asciiker_ 13y ago
You have all the right to mark it as SPAM of course. What I'm trying to get across here is that IP blocking is a bad solution. There are many cases where several domains share the same IP, specially with the shortage. So if domainA.com sends spam, why should domainB.com mail delivery on the same IP suffer from it?
- kaoD 13y ago
  > So if domainA.com sends spam, why should domainB.com mail delivery on the same IP suffer from it? Because the mail servers both share seem to allow sending unsolicited email, i.e. spam. You're the one responsible for terminating the spamming user.
- dredmorbius 13y ago
  Police the motherfuckingbejesus out of your users. If it costs you when your IP is blocked, insist on a deposit that's forfeit in the event of spamming. Sadly, I've worked for shops with poor practices. How bad? Sending thousands of emails to domains which no longer existed (let alone stale accounts at existing mail service providers). Simply not a priority.
- DanBC 13y ago
  You would have hated SPEWS.
pavel_lishin 13y ago
I always click the "unsubscribe" link, first. If I get anything else after that, then I mark it as spam. (I don't buy into the 30-business-days bullshit, it's 2014.) I also have catch-all addresses at my domain, so I register for things as SERVICE@mydomain, so I know exactly where spam is coming from. Except I've been doing this for years, and 90% of garbage still just comes to my plain ol' gmail address that I've been using for over a decade, which suggests that businesses finally have wised up to the fact that sending garbage to everyone who puts their email address into a form is a bad, bad idea.
- 13y ago

lazyant 13y ago

Yes, I found the same problem with email, it's not getting spam (relatively easy to deal with) but deliverability, that's why at the end we need to use a 3rd party service like sendgrid, mailgun, mailchimp etc

davideous 13y ago

The freedom of choice is actually in the hands of the end-users. The end-users choose which email providers (Yahoo, Gmail, Hotmail) they want to use. The email providers are motivated to provide a good user experience, which includes blocking unwanted email because they make their money through user engagement. The key to delivering into the inbox is sending mail that your recipients both want and expect. Provide a good user experience, and you'll build a good reputation. Push the limits (for example, use a "pre checked" checkbox on an order confirmation page to put people on your sales mailing list) and you'll be putting out email delivery fires all of the time. Most of the things that you describe (SPF, SenderID, DKIM, PTR, MX) are all technical requirements, which are just the baseline for delivery. These are required, but any sender of unsolicited email can configure them. They don't earn you access to the inbox. Just like properly formatted HTML does not earn you great SEO results. I like to break email delivery down into four areas: * who -- send to people who requested your email and are expecting it * what -- send something of value to these people * technical foundation -- (SPF, SenderID, DKIM, Feedback Loops, etc.) required, but having it does not give you any points * monitoring -- (open ratio, complaint ratio, ISP response codes) you need to know when something goes wrong (My company, www.drh.net, has been providing email server software, services, and deliverability consulting for over 10 years.) --- [edit; added the below] Another way the freedom-of-choice is in the hands of the end users is this: the big ISPs (yahoo, gmail, hotmail) make most of their filtering decisions based off of end-user behavioral data. For example: * what percentage of your email is opened * what percentage of your email is complained about (the "this is spam" button) * what percentage of your email is deleted without reading * how long is your email read * how much is forwarded * how much is replied to * what percentage of your email that was placed in the Spam folder when seen by the user received a click on the "this is not Spam" button. This is the end-users voting on if they want your email or not. This isn't the entire email deliverability equation, but it's a huge part of it. The ISPs treat this data so importantly because: (a) it's hard to game unlike content filtering, and (b) it directly correlates to good user experience which they want to provide. [edit to make bulleted list look right]

jt2190 13y ago
> Provide a good user experience, and you'll build a good > reputation. Push the limits (for example, use a "pre > checked" checkbox on an order confirmation page to put > people on your sales mailing list) and you'll be putting > out email delivery fires all of the time. Believe it or not, it's possible to be a good actor, follow all of the rules and best practices, and still get flagged as spam. Mail recipients are not perfect. They forget that they signed up for things. They accidentally click the "spam" flag on their messages. They get lazy and instead of unsubscribing they click the spam flag. The real culprit here is that email messages rely on blacklists and not whitelists, i.e. recipients are required to give all senders full access and then block them when they misbehave, instead of giving them no access and giving them more access as they build trust. So: What would it take to implement email whitelists across the industry?

bradknowles 13y ago

Speaking as the former Sr. Internet Mail Administrator for AOL, there is none. When I was there, we actively worked very, very hard to make sure that we did everything we should do to keep our stupid AOL users from inappropriately hurting people who contact them -- Like marking that message from your own mother as spam? Really? Problem is, there are plenty of large sites out there who aren't so conscientious, and you can keep up that level of commitment for only so long. There is no one on this entire PLANET that hates spam more than I do, and for a time I could rightfully claim that there was no other single person on this planet who had done more to fight spam than I had. But even I have my limits. I haven't been active in CAUCE or any other anti-spam effort for many years, and I haven't even been an active mail system administrator for a few years. As one of the members of the Postmaster Team for python.org (currently inactive), I have seriously hated some of the stupid shit that has been done by the likes of Google, hotmail, and Yahoo! I've been sorely tempted to just ban them outright, because of whatever might be their latest stupidity. But it hasn't happened -- At least, not yet.