5 ms·

I really hate that the general wisdom is, "don't run your own mail". Email is such a beautiful thing; anybody can talk directly to (almost) anybody else anywher

I really hate that the general wisdom is, "don't run your own mail". Email is such a beautiful thing; anybody can talk directly to (almost) anybody else anywhere in the world, and it's been consistent and (mostly) compatible practically forever (much of what we call email was defined before it was even called "the internet"). And, that negative general wisdom is all because of abuse. Even if you figure out how to solve incoming spam, you might still have trouble sending mail to others because of their abuse prevention practices. But, it's really not as bad as the general wisdom would indicate. Even with the particularly severe challenges we deal with on behalf of our customers (we mostly interact with them while they're initially setting up a new server on an IP of indeterminate history), we almost always get things to a state where delivery is reliable. DKIM and SPF, while a little complicated, seem to be helping a lot...the big providers are seemingly relying less and less on IP address in determining spamminess and more on history of the sender. Anyway, I really hate that email is heavily consolidated into the hands of a very few very large corporations who have little respect for privacy. We still have some old-timers using QMail, and we keep supporting it in a sort of "we won't break it" way because we like our old-timers and they mostly take care of themselves and sometimes even submit patches. There's nothing really wrong with QMail. It's just quite dated in a number of areas. So, I don't mind supporting it. It's not bad software, it's just old software.

ktRolster 9y ago
How do you deal with the spam issue?
- gardnr 9y ago
  For sending or receiving?
  - ktRolster 9y ago
    How do you avoid receiving spam messages?
    - gardnr 9y ago
      I can't claim to be an expert in this field because I gave up years ago. Back when I was actively trying to prevent spam from entering my mail servers I would use: * Fail if no SPF record * Fail if DKIM record present in NS & not-present in message * blacklists (DNSBL) * heuristics (SpamAssassin) Back then, not everyone published both SPF and DKIM. I would only fail a message if it did have a DKIM record and the message did not have the matching DKIM header. If I recall correctly, I failed everything that didn't publish an SPF record. Blacklists are tricky because they are outside of your control. You are relying on a community of folks to tell you what is best. They are still extremely useful but use them with caution. Heuristics need to plug in to your mail clients' "Mark as Spam" feature. There has to be a feedback mechanism. There are a number of other configuration settings that control what can and cannot get through like requiring reverse DNS to accurately resolve sender ip = domain name. It is a wicked problem and a game of whack-a-mole. I found it better to pay dedicated experts to handle it.
- SwellJoe 9y ago
  SpamAssassin, greylisting with postgrey, DKIM/SPF checking in SpamAssassin (though DKIM is still configured to be somewhat forgiving). My email address has not changed in 12+ years and is widely available on the internet, and I get maybe a half dozen spam messages that make it to my inbox each day. That's more than I'd like, and slightly more than my GMail address receives. But, it's tolerable. I could probably tweak it a bit more to prevent more spam, but I don't really have time to micromanage it...I just let the filters do their thing. If I enforced DKIM I'd probably successfully block nearly everything, but I'd get a lot of false positives, too. A surprisingly large amount (like 5%) of the spam I get that successfully makes it through is not mass-email. It's people who have acquired a list of Y Combinator company founders and are sending out semi-personalized mail. A surprising amount of my spam mail name drops Y Combinator (and tries to imply they're already working with other YC companies). Which is really the weirdest damned thing, but it happens a couple times a week. But, I don't know that any spam filter could stop that. I get the same kind of thing (not YC related, usually, but things like recruiter emails) on my GMail account which is what I use in some developer communities.
gardnr 9y ago
> the big providers are seemingly relying less and less on IP address in determining spamminess... It's not the big providers that you need to worry about. Setup SPF and most of them are fine. Some will look at your DKIM record too. The problem is every mid-level smtp service that has been configured by a varying number of people with varying levels of policies and ideas around how to mitigate spam. If your system triggers their spam detection then sometimes you get an email saying it was returned to sender and marked as spam. Sometimes, it will quietly go into their spam folder. I only ever got DMARC reports from Google. One needs an IP registered to their own AS number owned by a real company to reliably deliver email to the entire web. You can't run an MTA on a VPS or most home internet connections without spending a lot of time monitoring and troubleshooting delivery. Using a 3rd party service for sending while hosting the MX at home is still the most secure and reliable setup. What would be really useful is a "bounce_forwarding_mta" where bounced emails would always attempt to redeliver through the 2nd MTA. Then you could send everything through your own MTA on your home internet connection and if one bounces then the system automatically attempts to send it through whatever smtp SaaS the user configures. Things on the internet usually suck because of corporate policy, not usually because of the technology.
- SwellJoe 9y ago
  Yeah, I agree with most of what you say, though I disagree about needing your own AS; we've never had one and almost none of our clients have one. Nonetheless, there's a lot of pressure toward using the big providers, and that's dangerous, IMHO. Also, the mail client landscape isn't great (lots of people are working on it, including my company, but a great Open Source webmail client comparable to GMail does not yet exist...and, the mobile client situation is probably even worse than the state of webmail). Nonetheless, we have never outsourced our mail, and we deliver several thousand emails a day (forum and tracker notifications, mostly, but also expiration notices and billing failure notices) to thousands of different domains from our own MTAs. We are on static IPs in a colocation provider on our own physical server. We lived on the same IP for about five years and then moved to a new one a couple of years ago (same colocation). We only had one deliverability problem on the fresh IP, which was with Hotmail/Outlook, who seem to block every IP by default and you have to ask to be unblocked. But, once we jumped through their hoops we've never had deliverability problems since. We may be dealing with less troublesome recipients. Our users are mostly Linux/UNIX users at small-ish companies. They probably care about people being able to send them mail reliably more than maybe bigger companies with layers of security theater to deal with. I like the idea of a bounce forwarding MTA. That's actually a pretty brilliant approach to the problem; having something like Sendgrid or Mailjet or whatever as the backup would be a pretty neat thing (and for such a low volume of mail it'd be cheap or free for the small companies we deal with). Briefly googling didn't really turn up anything about that approach to bounces (googling anything related to bounces is tough because it can be about backscatter, my mail bouncing, my mail server bouncing incoming mail, delivery of bounces, etc.). It's already possible to detect bounces in Postfix, but I dunno how one would turn the bounce message (which are infinitely varied in their contents) into a forward for the original email. Maybe just instant rejections would be handleable in such a case. That'd be a reasonable start on the problem. And, in the case of email, corporate policy sucks because abuse sucks (and people are dumb about security practices). Because somebody is gonna click on that virus attachment if it finds its way through, IT is expected to act like the TSA and engage in all manner of security theater to prevent it, at any cost.

ktRolster 9y ago

How do you deal with the spam issue?

gardnr 9y ago
For sending or receiving?
- ktRolster 9y ago
  How do you avoid receiving spam messages?
  - gardnr 9y ago
    I can't claim to be an expert in this field because I gave up years ago. Back when I was actively trying to prevent spam from entering my mail servers I would use: * Fail if no SPF record * Fail if DKIM record present in NS & not-present in message * blacklists (DNSBL) * heuristics (SpamAssassin) Back then, not everyone published both SPF and DKIM. I would only fail a message if it did have a DKIM record and the message did not have the matching DKIM header. If I recall correctly, I failed everything that didn't publish an SPF record. Blacklists are tricky because they are outside of your control. You are relying on a community of folks to tell you what is best. They are still extremely useful but use them with caution. Heuristics need to plug in to your mail clients' "Mark as Spam" feature. There has to be a feedback mechanism. There are a number of other configuration settings that control what can and cannot get through like requiring reverse DNS to accurately resolve sender ip = domain name. It is a wicked problem and a game of whack-a-mole. I found it better to pay dedicated experts to handle it.
SwellJoe 9y ago
SpamAssassin, greylisting with postgrey, DKIM/SPF checking in SpamAssassin (though DKIM is still configured to be somewhat forgiving). My email address has not changed in 12+ years and is widely available on the internet, and I get maybe a half dozen spam messages that make it to my inbox each day. That's more than I'd like, and slightly more than my GMail address receives. But, it's tolerable. I could probably tweak it a bit more to prevent more spam, but I don't really have time to micromanage it...I just let the filters do their thing. If I enforced DKIM I'd probably successfully block nearly everything, but I'd get a lot of false positives, too. A surprisingly large amount (like 5%) of the spam I get that successfully makes it through is not mass-email. It's people who have acquired a list of Y Combinator company founders and are sending out semi-personalized mail. A surprising amount of my spam mail name drops Y Combinator (and tries to imply they're already working with other YC companies). Which is really the weirdest damned thing, but it happens a couple times a week. But, I don't know that any spam filter could stop that. I get the same kind of thing (not YC related, usually, but things like recruiter emails) on my GMail account which is what I use in some developer communities.

gardnr 9y ago

> the big providers are seemingly relying less and less on IP address in determining spamminess... It's not the big providers that you need to worry about. Setup SPF and most of them are fine. Some will look at your DKIM record too. The problem is every mid-level smtp service that has been configured by a varying number of people with varying levels of policies and ideas around how to mitigate spam. If your system triggers their spam detection then sometimes you get an email saying it was returned to sender and marked as spam. Sometimes, it will quietly go into their spam folder. I only ever got DMARC reports from Google. One needs an IP registered to their own AS number owned by a real company to reliably deliver email to the entire web. You can't run an MTA on a VPS or most home internet connections without spending a lot of time monitoring and troubleshooting delivery. Using a 3rd party service for sending while hosting the MX at home is still the most secure and reliable setup. What would be really useful is a "bounce_forwarding_mta" where bounced emails would always attempt to redeliver through the 2nd MTA. Then you could send everything through your own MTA on your home internet connection and if one bounces then the system automatically attempts to send it through whatever smtp SaaS the user configures. Things on the internet usually suck because of corporate policy, not usually because of the technology.

SwellJoe 9y ago
Yeah, I agree with most of what you say, though I disagree about needing your own AS; we've never had one and almost none of our clients have one. Nonetheless, there's a lot of pressure toward using the big providers, and that's dangerous, IMHO. Also, the mail client landscape isn't great (lots of people are working on it, including my company, but a great Open Source webmail client comparable to GMail does not yet exist...and, the mobile client situation is probably even worse than the state of webmail). Nonetheless, we have never outsourced our mail, and we deliver several thousand emails a day (forum and tracker notifications, mostly, but also expiration notices and billing failure notices) to thousands of different domains from our own MTAs. We are on static IPs in a colocation provider on our own physical server. We lived on the same IP for about five years and then moved to a new one a couple of years ago (same colocation). We only had one deliverability problem on the fresh IP, which was with Hotmail/Outlook, who seem to block every IP by default and you have to ask to be unblocked. But, once we jumped through their hoops we've never had deliverability problems since. We may be dealing with less troublesome recipients. Our users are mostly Linux/UNIX users at small-ish companies. They probably care about people being able to send them mail reliably more than maybe bigger companies with layers of security theater to deal with. I like the idea of a bounce forwarding MTA. That's actually a pretty brilliant approach to the problem; having something like Sendgrid or Mailjet or whatever as the backup would be a pretty neat thing (and for such a low volume of mail it'd be cheap or free for the small companies we deal with). Briefly googling didn't really turn up anything about that approach to bounces (googling anything related to bounces is tough because it can be about backscatter, my mail bouncing, my mail server bouncing incoming mail, delivery of bounces, etc.). It's already possible to detect bounces in Postfix, but I dunno how one would turn the bounce message (which are infinitely varied in their contents) into a forward for the original email. Maybe just instant rejections would be handleable in such a case. That'd be a reasonable start on the problem. And, in the case of email, corporate policy sucks because abuse sucks (and people are dumb about security practices). Because somebody is gonna click on that virus attachment if it finds its way through, IT is expected to act like the TSA and engage in all manner of security theater to prevent it, at any cost.